Azure Open AI and Azure AI Search with Private EndPoint for EmbendingEndpoint in RAG

Question

Hello,

I'm trying to use text-embedding-ada-002 under embeddingEndpoint in RAG and recieve this error

{"error": {"requestid": "9c75e12a-ac82-4b........", "code": 400, "message": "Invalid Azure OpenAI configuration detected: Access denied: Server responded with status 403. Error message: {\"error\":{\"code\":\"403\",\"message\": \"Access denied due to Virtual Network/Firewall rules.\"}}"}

Done:

1. Private Endpoint are created for Azure Open AI and Azure AI Search
2. Public Access for both services are deactivated
   1. Allow Azure services on trusted services list is activated in both services
3. Managed Identity activated and RBAC setuped as folowed:
   1. |Role |Assignee |Resource|
   2. |Search Index Data Reader |Azure OpenAI |Azure AI Search|
   3. |Search Index Data Reader |Azure OpenAI |Azure AI Search|
   4. |Search Service Contributor |Azure OpenAI |Azure AI Search|
   5. |Cognitive Services OpenAI Contributor|Azure AI Search|Azure OpenAI|

I send this request in body to https://someopenaiservice.openai.azure.com/openai/deployments/gpt-35-turbo-16k/extensions/chat/completions?api-version=2023-06-01-preview:

{
"temperature": 0.3,
"max_tokens": 300,
"top_p": 1.0,
"dataSources": [
    {
        "type": "AzureCognitiveSearch",
        "parameters": {
            "endpoint": "https://someaisearch.search.windows.net",
            "indexName": "someindex",
 "topNDocuments": "3",
 "queryType": "vectorSemanticHybrid", 
 "semanticConfiguration": "semantic_config",
 "inScope": true,
 "embeddingEndpoint": "https://someopenaiservice.openai.azure.com/openai/deployments/text-embedding-ada-002/embeddings?api-version=2024-06-01", 
 "embeddingKey": "someopenaikey"
        }
    }
],
"messages": [
    {
        "role": "user",
        "content": "How to create ...."
    }
]
}

• https://someaisearch.search.windows.net (deployed in North Europe) has private Endpoint in West Europe
• https://someopenaiservice.openai.azure.com (deployed in West Europe) has private Endpoint in West Europe

If I remove embeddingEndpoint and embeddingKey, it works and I get an answer. But if I add both parameters, it doesn't work anymore.

Any ideas what I'm doing wrong?

Answer 1

Hello,

Thanks for reaching out to us. Based on the information provided and the error message you’re encountering (Access denied due to Virtual Network/Firewall rules), it seems like there might be an issue with how the private endpoints and network rules are configured for your Azure services. Please check on below items -

Verify Network Configuration

Private Endpoints: Ensure that the private endpoints for both Azure OpenAI and Azure Cognitive Search are properly configured and deployed in the same Virtual Network (VNet) or peered VNets. This ensures that the services can communicate with each other over the private IPs.

DNS Configuration: Check that your DNS settings are configured to resolve the private endpoint IPs correctly. Sometimes, issues with DNS resolution can lead to access issues.

Network Security Groups (NSGs): Ensure that there are no NSGs or firewall rules that might be blocking traffic between the services or from your VNet to the services.

Verify Service Access

Firewall Rules: Confirm that the firewall rules on both Azure Cognitive Search and Azure OpenAI are set up to allow traffic from the private endpoint's VNet.

Allow Azure Services: Double-check that the setting "Allow Azure services on trusted services list" is activated for both services, as this setting allows Azure resources to access each other.

Check RBAC Roles and Permissions

Ensure that the roles and permissions are correctly assigned:

Search Index Data Reader: Should be assigned to Azure OpenAI for Azure Cognitive Search.

Cognitive Services OpenAI Contributor: Should be assigned to Azure Cognitive Search for Azure OpenAI.

Make sure these permissions are correctly applied and propagated.

Validate API Request Configuration

Embedding Endpoint and Key: Ensure that the embeddingEndpoint and embeddingKey you are using are correct and have the right permissions.

API Version: Confirm that the API version specified in the embeddingEndpoint URL is correct and matches the version expected by your deployment.

Please take a look an let us know how it works. Let us know if you are still blocked by this issue.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.