Azure OpenAI Realtime WebSocket returns HTTP 500 (keyless + SDK + raw websocket)

Question

I’m seeing a consistent HTTP 500 when trying to open a Realtime WebSocket session for a deployed realtime model in Azure OpenAI / Foundry.

Scenario:

• Realtime model deployment exists (custom deployment name, e.g. <REALTIME_DEPLOYMENT_NAME>)
• Region is supported for realtime
• Endpoint format used: https://<RESOURCE_NAME>.openai.azure.com
• WebSocket path used via SDK and equivalent raw websocket approach

What I already validated:

1. Authentication was initially blocked by an expired local Azure CLI refresh token, but that is now fixed.
2. Entra token acquisition now succeeds.
3. Followed Microsoft Learn GA WebSocket guidance for Realtime: https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/realtime-audio-websockets
   • Keyless flow
   • DefaultAzureCredential
   • Scope: https://ai.azure.com/.default
   • OpenAI Realtime WebSocket client pattern
4. Also tested equivalent raw websocket connection attempts.

Current result:

• WebSocket connect/handshake consistently returns HTTP 500 before any audio output is produced.
• Same behavior across both SDK-based and raw websocket attempts.

Additional signal:

• Non-realtime calls and token issuance succeed, so the failure seems specific to Realtime WebSocket for this resource/deployment path.

Question:

Are there known service-side conditions, feature flags, deployment/region constraints, or endpoint requirements that can cause Realtime WebSocket HTTP 500 in this state? Any recommended diagnostics beyond the above?

Answer 1

The authentication scope must be corrected. A WebSocket endpoint on the Azure OpenAI resource domain uses the Azure Cognitive Services token audience, not the Azure AI Foundry audience. Request the Cognitive Services default scope and ensure the identity has Cognitive Services OpenAI User access on the Azure OpenAI resource. For the GA Realtime API, use the /openai/v1/realtime path and pass the custom deployment name in the model query parameter. Do not combine that path with the preview api-version and deployment query format. Also verify the deployment is a supported realtime model and deployment type in a supported region. A one-time server-side test with the resource key can isolate RBAC from endpoint or deployment problems. If the corrected handshake still returns HTTP 500 before a session is created, record the UTC time, response headers, request or correlation ID, deployment name, and region, then open Azure Support because service-side logs are required.

Answer 2

Hello @Sayan Ghosh ,

Welcome to Microsoft Q&A .Thank you for reaching out to us.

Based on the behavior observed, the failure is occurring during the initial Realtime WebSocket session establishment stage, before a Realtime session is successfully created.

Please check if the following steps are helpful-

1. Validating Realtime deployment and endpoint configuration Please check and verify the following:
   • The deployment is based on a supported Realtime model.
   • The deployment provisioning state is successful.
   • The deployment is healthy and has not reached applicable quota or capacity limits.
   • The value supplied in the model= parameter exactly matches the Azure OpenAI deployment name.
   • The resource is deployed in a region that supports the selected Realtime model.
   For the GA Realtime API, the documented WebSocket endpoint format is: wss://<resource-name>.openai.azure.com/openai/v1/realtime?model=<deployment-name> Validation points:
   • The GA endpoint uses model= with the deployment name.
   • Preview-specific parameters such as deployment= and api-version should not be combined with the GA endpoint format.
   • GA and preview endpoint contracts should be treated separately because the request formats are different.
2. Validating the authentication path Please verify that the following environment variables are not present in the application runtime environment: AZURE_OPENAI_API_KEY OPENAI_API_KEY Also validate:
   • DefaultAzureCredential is selecting the expected identity.
   • The identity has the required Cognitive Services OpenAI User role assignment.
   • The token audience (aud) matches the authentication flow being tested.
   Since both authentication patterns exist in official documentation for related Azure AI scenarios, testing both audiences while keeping all other settings unchanged can help determine whether authentication contributes to the handshake failure.
3. Performing isolation testing Test 1: Azure AI Foundry Realtime Playground Test the same resource and deployment through the Realtime Playground. Expected interpretation:
   • Playground also fails - investigation should focus on deployment, region, or backend service availability.
   • Playground succeeds - investigation should focus on client authentication, endpoint construction or network configuration.
   Test 2: Temporary API key authentication comparison Perform a one-time API key authentication test using the same endpoint and deployment. Expected interpretation:
   • API key succeeds while Entra authentication fails → investigate authentication, RBAC, or token configuration.
   • API key also fails with HTTP 500 → authentication becomes less likely; focus on deployment, networking, region, or backend service investigation.
   This test is intended only for troubleshooting isolation and should not replace Entra authentication as the recommended production authentication method.
4. Validating connectivity and network path Since both SDK and raw WebSocket approaches fail, the shared network path should also be reviewed. Please validate:
   • Outbound WebSocket traffic over port 443 is permitted.
   • Proxy servers, VPNs, TLS inspection, API Management, or Private Endpoint configurations are not modifying WebSocket upgrade requests.
   • The behavior can be compared from an alternate network path where possible.

The following references might be helpful , please check them out

• Use the GPT Realtime API via WebSockets - Microsoft Foundry | Microsoft Learn
• Use keyless connections with Azure OpenAI | Microsoft Learn
• Configure keyless authentication with Microsoft Entra ID - Microsoft Foundry | Microsoft Learn
• Role-based access control for Azure OpenAI (classic) - Microsoft Foundry (classic) portal | Microsoft Learn
• Use the GPT Realtime API for speech and audio with Azure OpenAI - Microsoft Foundry | Microsoft Learn
• Azure OpenAI in Microsoft Foundry Models Quotas and Limits - Microsoft Foundry | Microsoft Learn

Please let us know if the response was helpful

 

Thank you