Voice agent WebSocket API reference

Microsoft Foundry provides separate WebSocket APIs for Realtime and GPT-Live voice agents. This reference describes each connection route and the event stream exchanged after the WebSocket handshake. Use the linked event and model references for the complete schemas.

Connection routes

Both APIs use an HTTP GET request to establish a WebSocket connection. The successful response is 101 Switching Protocols and has no HTTP response body. WebSocket events begin after the upgrade.

Protocol Method and path Behavior
Realtime GET /agents/{agent_name}/endpoint/protocols/voice Uses WebSocket JSON events by default. You can select WebRTC with the transport query parameter.
GPT-Live GET /agents/{agent_name}/endpoint/protocols/voice/live/sessions Opens a Live session. Don't negotiate the realtime WebSocket subprotocol on this route.

Append the route to your Foundry project endpoint. The {agent_name} path parameter identifies the voice agent.

Authenticate the connection

Authenticate the WebSocket handshake with a Microsoft Entra access token. Request a token for the https://ai.azure.com/.default scope and send it in the Authorization header as Bearer <token>. Don't put the token in the WebSocket URL or use a resource API key in its place.

The Foundry-Features: VoiceAgents=V1Preview header is a separate preview opt-in; it doesn't authenticate the request. You can use the foundry_features query parameter when your WebSocket client can't set that header.

Browser WebSocket APIs can't set an arbitrary Authorization header. The browser-only token method sends a short-lived token as an additional Sec-WebSocket-Protocol value. Foundry voice-agent routes don't support this method: Realtime accepts only the realtime subprotocol, and GPT-Live doesn't negotiate a subprotocol. Connect through an authenticated server-side proxy or use a supported SDK. Don't expose a bearer token in browser code. For more information about Foundry endpoint authorization, see Configure agent endpoint authorization.

Handshake parameters

Parameter Realtime GPT-Live Description
foundry_features query parameter or Foundry-Features header Conditional Required Set to VoiceAgents=V1Preview. Realtime requires the opt-in when the agent uses a preview feature.
transport query parameter Optional Not supported For Realtime, omit it or set it to websocket for JSON events over WebSocket. Set it to webrtc for SDP signaling over WebSocket, with media and the data channel peer-to-peer.
store query parameter Optional Optional For Realtime, overrides whether the conversation is persisted. For GPT-Live, overrides conversation and audio persistence for the session.
structured_inputs query parameter Optional Optional For Realtime, supplies URL-encoded JSON values for the agent's declared structured inputs. For GPT-Live, supplies URL-encoded JSON values for instruction and greeting templates.
agent_version_override query parameter Optional Optional Selects a specific voice agent version for the session.
Sec-WebSocket-Protocol header Optional Don't send Token authentication through this header is for browser connections only: the browser sends a short-lived token as an additional subprotocol value because its WebSocket API can't set an Authorization header. Foundry doesn't support this method: Realtime accepts only realtime, and GPT-Live doesn't negotiate a subprotocol.

For Realtime WebRTC connections, the agent must support WebRTC. WebRTC isn't compatible with bring-your-own-model or hosted-agent voice agents.

Exchange Realtime events

With the default WebSocket transport, the service applies the agent definition, resolves its configured tools, and sends the initial session configuration. Client and server messages are JSON events selected by their type property. Audio data in JSON event fields is base64-encoded.

Use session.update to change supported session settings. Send conversation.item.create to add a conversation item, or send audio with the input-audio buffer events. Send response.create to request a response. Handle server events such as response.created, response.output_text.delta, response.output_audio.delta, response.done, and error as appropriate for the session.

For WebRTC transport, the WebSocket carries SDP signaling events; media and the data channel are peer-to-peer. See the Realtime client events, Realtime server events, and Realtime models for event and payload schemas.

Exchange GPT-Live events

Foundry sends session.start to start the Live session. Wait for session.started before sending session commands or audio. Send session.update to change supported delegation settings or transcript triggers. Send audio with session.input_audio.append; receive generated audio in session.output_audio.delta events. Audio data is base64-encoded.

Live event semantics differ from Realtime. For example, Live session.update doesn't change the Foundry delegation target, and response.create is available only for Responses delegation. See the Live client events, Live server events, and Live models for event and payload schemas.