Best Practices for Securing Azure OpenAI with Confidential Data

Question

Hi,

I am developing an internal application that leverages Azure OpenAI Service and interacts with confidential data, including internal documents, table metadata of highly sensitive data, and more. I need guidance on the following security and compliance aspects:

1. Infrastructure Security: Is setting up a private VNet (something like this) sufficient to secure the solution? What are the key factors and best practices to ensure security, considering the use of Azure Storage, Azure AI Search, Azure Monitor (logs), and Azure OpenAI?
2. Inference & Compliance: Can the application process confidential data, such as metadata, documents, or even highly confidential content, without violating Azure security and compliance policies? Are there specific restrictions or guidelines to follow?
3. Validation & POC: Is there a way to validate that my application complies with security and compliance policies before deployment? Are there Microsoft-recommended processes or tools for this?

Answer 1

Hello Saurabh Arjun Sawant,

Thank you for posting your question in the Microsoft Q&A forum.

Setting up a private Virtual Network (VNet) itself is a strong foundational step for securing your Azure OpenAI solution, however it is not sufficient on its own. To ensure comprehensive security and compliance, especially when using Azure Storage, Azure AI Search, Azure Monitor, and Azure OpenAI, you need to adopt a multi-layered security approach. Below, I’ll address your questions in detail and provide key factors, best practices, and validation methods to ensure your solution is secure and compliant.

1. Infrastructure Security: To answer your question, a private VNet is a critical component for securing your solution, but it must be complemented with additional security measures.

Key Factors for Infrastructure Security

Private Endpoints: Use private endpoints for Azure Storage, Azure AI Search, and Azure OpenAI to ensure all communication occurs within the VNet and is not exposed to the public internet.

Network Security Groups (NSGs): Configure NSGs to restrict traffic to and from your resources. For example, allow only specific IP ranges or subnets to access Azure OpenAI.

Azure Firewall: Implement Azure Firewall to filter and monitor traffic between your VNet and external networks.

Service Endpoints: Enable service endpoints for Azure Storage and Azure AI Search to restrict access to your VNet.

DDoS Protection: Enable Azure DDoS Protection to safeguard against distributed denial-of-service attacks.

Best Practices for Infrastructure Security

• Isolate Resources: Place Azure OpenAI, Azure Storage, and Azure AI Search in separate subnets within the VNet to minimize the attack surface.
• Zero Trust Architecture: Adopt a zero-trust approach by verifying every request, regardless of its origin.
• Encryption: Ensure all data is encrypted at rest (using Azure Storage encryption) and in transit (using TLS 1.2 or higher).
• Private DNS Zones: Use Azure Private DNS Zones to resolve private endpoints within your VNet securely.
• Logging and Monitoring: Enable Azure Monitor and Log Analytics to track and analyze network traffic and resource usage.

Few useful links to Microsoft docs:

• Deploy and Configure Azure Firewall
• Secure Azure Storage with Service Endpoints
• Enable DDoS Protection

2. Inference & Compliance: Yes, your application can process confidential data, but you must adhere to Azure security and compliance policies and follow specific guidelines to avoid violations.

Key Factors for Compliance

Data Classification: Classify your data (e.g., public, confidential, highly confidential) and apply appropriate security controls.

Data Residency: Ensure that data is stored and processed in regions compliant with your organization’s data residency requirements.

Regulatory Compliance: Verify that Azure OpenAI and related services comply with regulations like GDPR, HIPAA, ISO 27001, and SOC 2.

Best Practices for Processing Confidential Data

Data Anonymization: Anonymize or pseudonymize sensitive data before sending it to Azure OpenAI for inference.

Access Controls: Use RBAC and Azure AD to restrict access to confidential data.

Data Masking: Mask sensitive information in logs and outputs.

Audit Logs: Enable audit logs for Azure Storage, Azure AI Search, and Azure OpenAI to track data access and usage.

• Content Filtering: Use Azure OpenAI’s built-in content filtering to prevent the generation of inappropriate or harmful content.
• Define Security and Compliance Requirements:
  • Identify the regulations and standards your application must comply with (e.g., GDPR, HIPAA).
  • Define security controls (e.g., encryption, access controls).

Specific Restrictions and Guidelines

• Data Retention: Define and enforce data retention policies to ensure data is not stored longer than necessary.
• Data Deletion: Use Azure’s data deletion capabilities to permanently remove data when it is no longer needed.
• Third-Party Integrations: Ensure third-party services comply with the same security and compliance standards.

Some useful links to Microsoft docs:

• Microsoft Purview Data Classification
• Azure Data Residency Documentation
• Azure Compliance Documentation
• Azure OpenAI Content Filtering

1. Validation & POC: Microsoft provides several tools and processes to validate that your application complies with security and compliance policies before deployment.

Some useful links to Microsoft docs:

• Azure Penetration Testing Rules of Engagement
• Microsoft Compliance Manager Overview
• Secure Third-Party Integrations in Azure

If above answer helped, please do not forget to "Accept Answer" as this may help other community members to refer the info if facing similar issue.