Is It Possible to Provide a Customer-Specific Architecture Diagram for Deployed Models in Azure OpenAI?

Question

Hello Team,

One of our customers is using Azure OpenAI services and has asked for specific details on how their deployed models (fine-tuned models) are secured within their Azure environment. While we have shared the official Azure OpenAI Data Privacy and Security documentation (Microsoft Docs), they are looking for:

A detailed architecture diagram or documentation that explicitly shows how their deployed models are protected within a walled-off setup.

1. Confirmation on data isolation, specifically how Microsoft ensures that customer models, prompts, and completions remain completely isolated within their Azure subscription and are not accessible by Microsoft or OpenAI.

As per my understanding, Microsoft does not access, store, or retain customer-deployed models, and therefore, it may not be possible to provide a customer-specific architecture diagram. Could you confirm whether Microsoft offers such insights? Any official guidance, whitepapers, or architecture references that can help address this customer concern would be greatly appreciated.

Thank you in advance for your support!

Answer 1

Hi Sujit Pochampalli,

Microsoft does not provide customer-specific architecture diagrams for deployed Azure OpenAI models. However, it does offer general reference architectures that show how models are secured within an Azure Virtual Network (VNet) environment. Here’s what you need to know:

Data Isolation & Security:

Each customer's Azure OpenAI instance is logically isolated from others within the same service boundary.

Fine-tuned models are accessible only by the customer who deployed them.

Microsoft does not store, retain, or access customer data, including fine-tuned models, prompts, and completions.

Data usage policy - https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy

Network Security & Compliance

Customers can deploy models within a private VNet to ensure security and prevent public exposure.

Private Endpoints, Network Security Groups (NSGs), and Azure Firewall can be used to restrict access.

Azure Monitor and Log Analytics allow tracking of resource usage and potential security threats.

Encryption is enforced both at rest (via Azure Storage encryption) and in transit (TLS 1.2+).

Official References & Best Practices

Microsoft’s Azure OpenAI Data Privacy & Security documentation confirms that all customer data remains within their Azure subscription and is not accessible by Microsoft or OpenAI (Microsoft Docs).

Additional best practices for securing Azure OpenAI models are outlined, including DDoS protection, content filtering, and access control via RBAC and Azure AD.

Below are the reference docs:

Content Filtering - https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/content-filter

Follow best practices for securing Azure AI infrastructure with confidential data - https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/infrastructure/security?source=recommendations

While an exact customer-specific architecture diagram is not available, you can use the above best practices and reference documents to demonstrate how Azure ensures model security and data isolation. Let me know if you need further clarification!

Hope this helps. Do let us know if you any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thank you!