Is It Possible to Provide a Customer-Specific Architecture Diagram for Deployed Models in Azure OpenAI?

Sujit Pochampalli 90 Reputation points
2025-04-02T10:00:54.68+00:00

Hello Team,

One of our customers is using Azure OpenAI services and has asked for specific details on how their deployed models (fine-tuned models) are secured within their Azure environment. While we have shared the official Azure OpenAI Data Privacy and Security documentation (Microsoft Docs), they are looking for:

A detailed architecture diagram or documentation that explicitly shows how their deployed models are protected within a walled-off setup.

  1. Confirmation on data isolation, specifically how Microsoft ensures that customer models, prompts, and completions remain completely isolated within their Azure subscription and are not accessible by Microsoft or OpenAI.

As per my understanding, Microsoft does not access, store, or retain customer-deployed models, and therefore, it may not be possible to provide a customer-specific architecture diagram. Could you confirm whether Microsoft offers such insights? Any official guidance, whitepapers, or architecture references that can help address this customer concern would be greatly appreciated.

Thank you in advance for your support!

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,101 questions
{count} votes

Accepted answer
  1. Prashanth Veeragoni 5,325 Reputation points Microsoft External Staff Moderator
    2025-04-02T16:27:41.57+00:00

    Hi Sujit Pochampalli,

    Microsoft does not provide customer-specific architecture diagrams for deployed Azure OpenAI models. However, it does offer general reference architectures that show how models are secured within an Azure Virtual Network (VNet) environment. Here’s what you need to know:

    Data Isolation & Security:

    Each customer's Azure OpenAI instance is logically isolated from others within the same service boundary.

    Fine-tuned models are accessible only by the customer who deployed them.

    Microsoft does not store, retain, or access customer data, including fine-tuned models, prompts, and completions.

    Data usage policy - https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy

    Network Security & Compliance

    Customers can deploy models within a private VNet to ensure security and prevent public exposure.

    Private Endpoints, Network Security Groups (NSGs), and Azure Firewall can be used to restrict access.

    Azure Monitor and Log Analytics allow tracking of resource usage and potential security threats.

    Encryption is enforced both at rest (via Azure Storage encryption) and in transit (TLS 1.2+).

    Official References & Best Practices

    Microsoft’s Azure OpenAI Data Privacy & Security documentation confirms that all customer data remains within their Azure subscription and is not accessible by Microsoft or OpenAI (Microsoft Docs).

    Additional best practices for securing Azure OpenAI models are outlined, including DDoS protection, content filtering, and access control via RBAC and Azure AD.

    Below are the reference docs:

    Content Filtering - https://learn.microsoft.com/en-us/azure/cognitive-services/openai/concepts/content-filter

    Follow best practices for securing Azure AI infrastructure with confidential data - https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/scenarios/ai/infrastructure/security?source=recommendations

    While an exact customer-specific architecture diagram is not available, you can use the above best practices and reference documents to demonstrate how Azure ensures model security and data isolation. Let me know if you need further clarification!

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    Thank you!

    0 comments No comments

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.