Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
These models are referenced by realtime client events and server events.
Models
Azure.AI.ProjectsInternal.VoiceAgentClientEventTypeExtension
Type: "session.avatar.connect" or "rtc.call.sdp.create".
Azure.AI.ProjectsInternal.VoiceAgentServerEventTypeExtension
Type: "warning" or "session.subagent.started" or "session.subagent.completed" or "session.subagent.aborted" or "session.avatar.connecting" or "session.avatar.switch_to_speaking" or "session.avatar.switch_to_idle" or "rtc.call.sdp.created" or "rtc.call.error" or "response.audio_timestamp.delta" or "response.audio_timestamp.done" or "response.animation_blendshapes.delta" or "response.animation_blendshapes.done" or "response.animation_viseme.delta" or "response.animation_viseme.done" or "response.video.delta" or "response.system_tool_arguments.delta" or "response.system_tool_arguments.done" or "input_collection.status" or "session.handoff.started" or "session.handoff.completed" or "session.handoff.aborted".
Conditional preview opt-ins: VoiceAgents=V1Preview.
FoundryDurationMilliseconds
A duration encoded as an integer number of milliseconds.
Type: integer.
format: "int32".
FoundryDurationSeconds
A duration encoded as an integer number of seconds.
Type: integer.
format: "int32".
FoundryTimestamp
Type: integer.
format: "unixtime".
OpenAI.CallableToolAllowedCaller
Type: "direct" or "programmatic".
OpenAI.CreateTranscriptionResponseJsonUsage
Token usage statistics for the request.
Variants by type: "tokens": OpenAI.TranscriptTextUsageTokens; "duration": OpenAI.TranscriptTextUsageDuration.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.CreateTranscriptionResponseJsonUsageType | Yes |
OpenAI.CreateTranscriptionResponseJsonUsageType
Any of: string, "tokens" or "duration".
OpenAI.LogProbProperties
A log probability object.
| Property | Type | Required | Description |
|---|---|---|---|
token |
string |
Yes | The token that was used to generate the log probability. |
logprob |
OpenAI.numeric | Yes | The log probability of the token. |
bytes |
Array of OpenAI.integer | Yes | The bytes that were used to generate the log probability. |
OpenAI.MCPListToolsTool
A tool available on an MCP server.
| Property | Type | Required | Description |
|---|---|---|---|
name |
string |
Yes | The name of the tool. |
description |
Any of: string; null |
No | |
input_schema |
OpenAI.MCPListToolsToolInputSchema | Yes | The JSON schema describing the tool's input. |
annotations |
Any of: OpenAI.MCPListToolsToolAnnotations; null |
No |
OpenAI.MCPListToolsToolAnnotations
OpenAI.MCPListToolsToolInputSchema
OpenAI.MCPTool
Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp" |
Yes | The type of the MCP tool. Always mcp. |
server_label |
string |
Yes | A label for this MCP server, used to identify it in tool calls. |
server_url |
string |
No | The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided. format: "uri". |
connector_id |
"connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or "connector_googledrive" or "connector_microsoftteams" or "connector_outlookcalendar" or "connector_outlookemail" or "connector_sharepoint" |
No | Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here. This field is deprecated for models released after September 1, 2026. Use server_url to connect to a remote MCP server, or tunnel_id to connect through a Secure MCP Tunnel. Currently supported connector_id values are: - Dropbox: connector_dropbox - Gmail: connector_gmail - Google Calendar: connector_googlecalendar - Google Drive: connector_googledrive - Microsoft Teams: connector_microsoftteams - Outlook Calendar: connector_outlookcalendar - Outlook Email: connector_outlookemail - SharePoint: connector_sharepoint deprecated: true. |
tunnel_id |
string |
No | The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided. pattern: "^tunnel_[a-z0-9]{32}$". |
authorization |
string |
No | An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here. |
server_description |
string |
No | Optional description of the MCP server, used to provide more context. |
headers |
Any of: object with additional values of string; null |
No | |
allowed_tools |
Any of: Array of string; OpenAI.MCPToolFilter; null |
No | |
allowed_callers |
Any of: Array of OpenAI.CallableToolAllowedCaller; null |
No | |
require_approval |
Any of: OpenAI.MCPToolRequireApproval; "always" or "never"; null |
No | default: "always". |
defer_loading |
boolean |
No | Whether this MCP tool is deferred and discovered via tool search. |
project_connection_id |
string |
No | The connection ID in the project for the MCP server. The connection stores authentication and other connection details needed to connect to the MCP server. |
tool_configs |
object with additional values of ToolConfig |
No | Deprecated. This property is deprecated and will be removed in a future version. deprecated: true. |
OpenAI.MCPToolFilter
A filter object to specify which tools are allowed.
| Property | Type | Required | Description |
|---|---|---|---|
tool_names |
Array of string |
No | List of allowed tool names. |
read_only |
boolean |
No | Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter. |
OpenAI.MCPToolRequireApproval
| Property | Type | Required | Description |
|---|---|---|---|
always |
OpenAI.MCPToolFilter | No | |
never |
OpenAI.MCPToolFilter | No |
OpenAI.Metadata
Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.
Additional properties: string.
OpenAI.RealtimeAudioFormats
Variants by type: "audio/pcm": OpenAI.RealtimeAudioFormatsAudioPcm; "audio/pcmu": OpenAI.RealtimeAudioFormatsAudioPcmu; "audio/pcma": OpenAI.RealtimeAudioFormatsAudioPcma.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.RealtimeAudioFormatsType | Yes |
OpenAI.RealtimeAudioFormatsAudioPcm
All of: OpenAI.RealtimeAudioFormats.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"audio/pcm" |
Yes | |
rate |
24000 |
No |
OpenAI.RealtimeAudioFormatsAudioPcma
All of: OpenAI.RealtimeAudioFormats.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"audio/pcma" |
Yes |
OpenAI.RealtimeAudioFormatsAudioPcmu
All of: OpenAI.RealtimeAudioFormats.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"audio/pcmu" |
Yes |
OpenAI.RealtimeAudioFormatsType
Any of: string, "audio/pcm" or "audio/pcmu" or "audio/pcma".
OpenAI.RealtimeClientEvent
A realtime client event.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.RealtimeClientEventType | Yes |
OpenAI.RealtimeClientEventType
Any of: string, "conversation.item.create" or "conversation.item.delete" or "conversation.item.retrieve" or "conversation.item.truncate" or "input_audio_buffer.append" or "input_audio_buffer.clear" or "output_audio_buffer.clear" or "input_audio_buffer.commit" or "response.cancel" or "response.create" or "session.update", Azure.AI.ProjectsInternal.VoiceAgentClientEventTypeExtension.
OpenAI.RealtimeConversationItem
A single item within a Realtime conversation.
Variants by type: "system_tool": VoiceAgentSystemToolItem; "message": OpenAI.RealtimeConversationItemMessage; "function_call": OpenAI.RealtimeConversationItemFunctionCall; "function_call_output": OpenAI.RealtimeConversationItemFunctionCallOutput; "mcp_approval_response": OpenAI.RealtimeMCPApprovalResponse; "mcp_list_tools": OpenAI.RealtimeMCPListTools; "mcp_call": OpenAI.RealtimeMCPToolCall; "mcp_approval_request": OpenAI.RealtimeMCPApprovalRequest.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.RealtimeConversationItemType | Yes |
OpenAI.RealtimeConversationItemFunctionCall
A function call item in a Realtime conversation.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the item. This may be provided by the client or generated by the server. |
object |
"realtime.item" |
No | Identifier for the API object being returned - always realtime.item. Optional when creating a new item. |
type |
"function_call" |
Yes | The type of the item. Always function_call. |
status |
"completed" or "incomplete" or "in_progress" |
No | The status of the item. Has no effect on the conversation. |
call_id |
string |
No | The ID of the function call. |
name |
string |
Yes | The name of the function being called. |
arguments |
string |
Yes | The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeConversationItemFunctionCallOutput
A function call output item in a Realtime conversation.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the item. This may be provided by the client or generated by the server. |
object |
"realtime.item" |
No | Identifier for the API object being returned - always realtime.item. Optional when creating a new item. |
type |
"function_call_output" |
Yes | The type of the item. Always function_call_output. |
status |
"completed" or "incomplete" or "in_progress" |
No | The status of the item. Has no effect on the conversation. |
call_id |
string |
Yes | The ID of the function call this output is for. |
output |
string |
Yes | The output of the function call, this is free text and can contain any information or simply be empty. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
name |
string |
No | The name of the function that was called. A Foundry extension: OpenAI's function_call_output does not carry the function name, only call_id. |
OpenAI.RealtimeConversationItemMessage
All of: OpenAI.RealtimeConversationItem.
Variants by role: "system": OpenAI.RealtimeConversationItemMessageSystem; "user": OpenAI.RealtimeConversationItemMessageUser; "assistant": OpenAI.RealtimeConversationItemMessageAssistant.
| Property | Type | Required | Description |
|---|---|---|---|
role |
OpenAI.RealtimeConversationItemMessageType | Yes | |
type |
"message" |
Yes |
OpenAI.RealtimeConversationItemMessageAssistant
An assistant message item in a Realtime conversation.
All of: OpenAI.RealtimeConversationItemMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the item. This may be provided by the client or generated by the server. |
object |
"realtime.item" |
No | Identifier for the API object being returned - always realtime.item. Optional when creating a new item. |
type |
"message" |
Yes | The type of the item. Always message. |
status |
"completed" or "incomplete" or "in_progress" |
No | The status of the item. Has no effect on the conversation. |
role |
"assistant" |
Yes | The role of the message sender. Always assistant. |
content |
Array of OpenAI.RealtimeConversationItemMessageAssistantContent | Yes | The content of the message. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeConversationItemMessageAssistantContent
| Property | Type | Required | Description |
|---|---|---|---|
type |
"output_text" or "output_audio" |
No | |
text |
string |
No | |
audio |
string |
No | |
transcript |
string |
No |
OpenAI.RealtimeConversationItemMessageSystem
A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.
All of: OpenAI.RealtimeConversationItemMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the item. This may be provided by the client or generated by the server. |
object |
"realtime.item" |
No | Identifier for the API object being returned - always realtime.item. Optional when creating a new item. |
type |
"message" |
Yes | The type of the item. Always message. |
status |
"completed" or "incomplete" or "in_progress" |
No | The status of the item. Has no effect on the conversation. |
role |
"system" |
Yes | The role of the message sender. Always system. |
content |
Array of OpenAI.RealtimeConversationItemMessageSystemContent | Yes | The content of the message. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeConversationItemMessageSystemContent
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_text" |
No | |
text |
string |
No |
OpenAI.RealtimeConversationItemMessageType
Any of: string, "system" or "user" or "assistant".
OpenAI.RealtimeConversationItemMessageUser
A user message item in a Realtime conversation.
All of: OpenAI.RealtimeConversationItemMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the item. This may be provided by the client or generated by the server. |
object |
"realtime.item" |
No | Identifier for the API object being returned - always realtime.item. Optional when creating a new item. |
type |
"message" |
Yes | The type of the item. Always message. |
status |
"completed" or "incomplete" or "in_progress" |
No | The status of the item. Has no effect on the conversation. |
role |
"user" |
Yes | The role of the message sender. Always user. |
content |
Array of OpenAI.RealtimeConversationItemMessageUserContent | Yes | The content of the message. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeConversationItemMessageUserContent
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_text" or "input_audio" or "input_image" |
No | |
text |
string |
No | |
audio |
string |
No | |
image_url |
string |
No | format: "uri". |
detail |
"auto" or "low" or "high" |
No | default: "auto". |
transcript |
string |
No |
OpenAI.RealtimeConversationItemType
Any of: string, "function_call" or "function_call_output" or "mcp_approval_response" or "mcp_list_tools" or "mcp_call" or "mcp_approval_request" or "message", VoiceAgentConversationItemTypeExtension.
OpenAI.RealtimeFunctionTool
| Property | Type | Required | Description |
|---|---|---|---|
type |
"function" |
No | The type of the tool, i.e. function. |
name |
string |
No | The name of the function. |
description |
string |
No | The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything). |
parameters |
OpenAI.RealtimeFunctionToolParameters | No | Parameters of the function in JSON Schema. |
OpenAI.RealtimeFunctionToolParameters
OpenAI.RealtimeMCPApprovalRequest
A Realtime item requesting human approval of a tool invocation.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp_approval_request" |
Yes | The type of the item. Always mcp_approval_request. |
id |
string |
Yes | The unique ID of the approval request. |
server_label |
string |
Yes | The label of the MCP server making the request. |
name |
string |
Yes | The name of the tool to run. |
arguments |
string |
Yes | A JSON string of arguments for the tool. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeMCPApprovalResponse
A Realtime item responding to an MCP approval request.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp_approval_response" |
Yes | The type of the item. Always mcp_approval_response. |
id |
string |
Yes | The unique ID of the approval response. |
approval_request_id |
string |
Yes | The ID of the approval request being answered. |
approve |
boolean |
Yes | Whether the request was approved. |
reason |
Any of: string; null |
No | |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeMCPError
Variants by type: "protocol_error": OpenAI.RealtimeMCPProtocolError; "tool_execution_error": OpenAI.RealtimeMCPToolExecutionError; "http_error": OpenAI.RealtimeMCPHTTPError.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.RealtimeMcpErrorType | Yes |
OpenAI.RealtimeMCPHTTPError
All of: OpenAI.RealtimeMCPError.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"http_error" |
Yes | |
code |
OpenAI.integer | Yes | |
message |
string |
Yes |
OpenAI.RealtimeMCPListTools
A Realtime item listing tools available on an MCP server.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp_list_tools" |
Yes | The type of the item. Always mcp_list_tools. |
id |
string |
No | The unique ID of the list. |
server_label |
string |
Yes | The label of the MCP server. |
tools |
Array of OpenAI.MCPListToolsTool | Yes | The tools available on the server. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeMCPProtocolError
All of: OpenAI.RealtimeMCPError.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"protocol_error" |
Yes | |
code |
OpenAI.integer | Yes | |
message |
string |
Yes |
OpenAI.RealtimeMCPToolCall
A Realtime item representing an invocation of a tool on an MCP server.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp_call" |
Yes | The type of the item. Always mcp_call. |
id |
string |
Yes | The unique ID of the tool call. |
server_label |
string |
Yes | The label of the MCP server running the tool. |
name |
string |
Yes | The name of the tool that was run. |
arguments |
string |
Yes | A JSON string of the arguments passed to the tool. |
approval_request_id |
Any of: string; null |
No | |
output |
Any of: string; null |
No | |
error |
OpenAI.RealtimeMCPError | No | |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
OpenAI.RealtimeMCPToolExecutionError
All of: OpenAI.RealtimeMCPError.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"tool_execution_error" |
Yes | |
message |
string |
Yes |
OpenAI.RealtimeMcpErrorType
Any of: string, "protocol_error" or "tool_execution_error" or "http_error".
OpenAI.RealtimeReasoning
Configuration for reasoning-capable Realtime models such as gpt-realtime-2.
| Property | Type | Required | Description |
|---|---|---|---|
effort |
OpenAI.RealtimeReasoningEffort | No |
OpenAI.RealtimeReasoningEffort
Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.
Type: "minimal" or "low" or "medium" or "high" or "xhigh".
OpenAI.RealtimeResponseInputAudio
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_audio" |
Yes | |
audio |
string |
Yes | Base64 audio bytes in the session input audio format. minLength: 1. |
transcript |
Any of: string; null |
No | Optional text to associate with the input audio. |
OpenAI.RealtimeResponseInputFunctionCall
A function call supplied as input to a Realtime response.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"function_call" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | default: "completed". |
call_id |
string |
Yes | minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
name |
string |
Yes | minLength: 1. maxLength: 64. pattern: "^[a-zA-Z0-9_-]+$". |
arguments |
string |
Yes | maxLength: 1048576. |
OpenAI.RealtimeResponseInputFunctionCallOutput
The result of a function call supplied as input to a Realtime response.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"function_call_output" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | default: "completed". |
call_id |
string |
Yes | minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
output |
string |
Yes | maxLength: 1048576. |
OpenAI.RealtimeResponseInputImage
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_image" |
Yes | |
image_url |
string |
Yes | A base64 data URL with an image MIME type; use PNG or JPEG. format: "uri". pattern: "^data:image/[^;,]+;base64,.+". |
detail |
Any of: "low" or "high" or "auto"; null; null |
No | default: "auto". |
OpenAI.RealtimeResponseInputItem
An existing item reference or a new message or function call item for a Realtime response.
Any of: OpenAI.RealtimeResponseInputMessage, OpenAI.RealtimeResponseInputFunctionCall, OpenAI.RealtimeResponseInputFunctionCallOutput, OpenAI.RealtimeResponseItemReference.
OpenAI.RealtimeResponseInputMessage
A message supplied as input to a Realtime response. The role determines the allowed content.
Variants by role: "system": OpenAI.RealtimeResponseInputMessageSystem; "developer": OpenAI.RealtimeResponseInputMessageDeveloper; "user": OpenAI.RealtimeResponseInputMessageUser; "assistant": OpenAI.RealtimeResponseInputMessageAssistant.
| Property | Type | Required | Description |
|---|---|---|---|
role |
OpenAI.RealtimeResponseInputMessageType | Yes |
OpenAI.RealtimeResponseInputMessageAssistant
All of: OpenAI.RealtimeResponseInputMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"message" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | Optional input status. Omission or null defaults to completed. default: "completed". |
role |
"assistant" |
Yes | |
content |
Array of OpenAI.RealtimeResponseInputMessageAssistantContent | Yes | minItems: 1. |
OpenAI.RealtimeResponseInputMessageAssistantContent
| Property | Type | Required | Description |
|---|---|---|---|
type |
"output_text" |
Yes | |
text |
string |
Yes | maxLength: 256000. |
OpenAI.RealtimeResponseInputMessageDeveloper
All of: OpenAI.RealtimeResponseInputMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"message" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | Optional input status. Omission or null defaults to completed. default: "completed". |
role |
"developer" |
Yes | |
content |
Array of OpenAI.RealtimeResponseInputText | Yes | minItems: 1. maxItems: 1. |
OpenAI.RealtimeResponseInputMessageSystem
All of: OpenAI.RealtimeResponseInputMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"message" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | Optional input status. Omission or null defaults to completed. default: "completed". |
role |
"system" |
Yes | |
content |
Array of OpenAI.RealtimeResponseInputText | Yes | minItems: 1. maxItems: 1. |
OpenAI.RealtimeResponseInputMessageType
Any of: string, "system" or "developer" or "user" or "assistant".
OpenAI.RealtimeResponseInputMessageUser
All of: OpenAI.RealtimeResponseInputMessage.
| Property | Type | Required | Description |
|---|---|---|---|
id |
Any of: string; null |
No | An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
type |
"message" |
Yes | |
status |
Any of: "completed" or "incomplete"; null; null |
No | Optional input status. Omission or null defaults to completed. default: "completed". |
role |
"user" |
Yes | |
content |
Array of Any of: OpenAI.RealtimeResponseInputText; OpenAI.RealtimeResponseInputAudio; OpenAI.RealtimeResponseInputImage | Yes | minItems: 1. |
OpenAI.RealtimeResponseInputText
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_text" |
Yes | |
text |
string |
Yes | maxLength: 256000. |
OpenAI.RealtimeResponseItemReference
Reference an existing item in this Realtime session as response.create input. Item references are only accepted in the response input array, not as new conversation items.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"item_reference" |
Yes | The type of this item. Always item_reference. |
id |
string |
Yes | The ID of an existing item in the same Realtime session. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$". |
OpenAI.RealtimeResponseStatusDetails
| Property | Type | Required | Description |
|---|---|---|---|
type |
"completed" or "cancelled" or "failed" or "incomplete" |
No | |
reason |
"turn_detected" or "client_cancelled" or "max_output_tokens" or "content_filter" |
No | |
error |
OpenAI.RealtimeResponseStatusDetailsError | No |
OpenAI.RealtimeResponseStatusDetailsError
| Property | Type | Required | Description |
|---|---|---|---|
type |
string |
No | |
code |
string |
No | |
message |
string |
No |
OpenAI.RealtimeResponseUsage
| Property | Type | Required | Description |
|---|---|---|---|
total_tokens |
OpenAI.integer | No | |
input_tokens |
OpenAI.integer | No | |
output_tokens |
OpenAI.integer | No | |
input_token_details |
OpenAI.RealtimeResponseUsageInputTokenDetails | No | |
output_token_details |
OpenAI.RealtimeResponseUsageOutputTokenDetails | No |
OpenAI.RealtimeResponseUsageInputTokenDetails
| Property | Type | Required | Description |
|---|---|---|---|
cached_tokens |
OpenAI.integer | No | |
text_tokens |
OpenAI.integer | No | |
image_tokens |
OpenAI.integer | No | |
audio_tokens |
OpenAI.integer | No | |
cached_tokens_details |
OpenAI.RealtimeResponseUsageInputTokenDetailsCachedTokensDetails | No |
OpenAI.RealtimeResponseUsageInputTokenDetailsCachedTokensDetails
| Property | Type | Required | Description |
|---|---|---|---|
text_tokens |
OpenAI.integer | No | |
image_tokens |
OpenAI.integer | No | |
audio_tokens |
OpenAI.integer | No |
OpenAI.RealtimeResponseUsageOutputTokenDetails
| Property | Type | Required | Description |
|---|---|---|---|
text_tokens |
OpenAI.integer | No | |
audio_tokens |
OpenAI.integer | No |
OpenAI.RealtimeServerEvent
A realtime server event.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.RealtimeServerEventType | Yes |
OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionFailedError
| Property | Type | Required | Description |
|---|---|---|---|
type |
string |
No | |
code |
string |
No | |
message |
string |
No | |
param |
string |
No |
OpenAI.RealtimeServerEventErrorError
| Property | Type | Required | Description |
|---|---|---|---|
type |
string |
Yes | |
code |
Any of: string; null |
No | |
message |
string |
Yes | |
param |
Any of: string; null |
No | |
event_id |
Any of: string; null |
No |
OpenAI.RealtimeServerEventRateLimitsUpdatedRateLimits
| Property | Type | Required | Description |
|---|---|---|---|
name |
"requests" or "tokens" |
No | |
limit |
OpenAI.integer | No | |
remaining |
OpenAI.integer | No | |
reset_seconds |
OpenAI.numeric | No |
OpenAI.RealtimeServerEventResponseContentPartAddedPart
| Property | Type | Required | Description |
|---|---|---|---|
type |
"audio" or "text" |
No | |
text |
string |
No | |
audio |
string |
No | |
transcript |
string |
No |
OpenAI.RealtimeServerEventResponseContentPartDonePart
| Property | Type | Required | Description |
|---|---|---|---|
type |
"audio" or "text" |
No | |
text |
string |
No | |
audio |
string |
No | |
transcript |
string |
No | |
format |
OpenAI.RealtimeAudioFormats | No | The audio format, when this is an audio content part. |
OpenAI.RealtimeServerEventType
Any of: string, "conversation.created" or "conversation.item.created" or "conversation.item.deleted" or "conversation.item.input_audio_transcription.completed" or "conversation.item.input_audio_transcription.delta" or "conversation.item.input_audio_transcription.failed" or "conversation.item.retrieved" or "conversation.item.truncated" or "error" or "input_audio_buffer.cleared" or "input_audio_buffer.committed" or "input_audio_buffer.dtmf_event_received" or "input_audio_buffer.speech_started" or "input_audio_buffer.speech_stopped" or "rate_limits.updated" or "response.output_audio.delta" or "response.output_audio.done" or "response.output_audio_transcript.delta" or "response.output_audio_transcript.done" or "response.content_part.added" or "response.content_part.done" or "response.created" or "response.done" or "response.function_call_arguments.delta" or "response.function_call_arguments.done" or "response.output_item.added" or "response.output_item.done" or "response.output_text.delta" or "response.output_text.done" or "session.created" or "session.updated" or "output_audio_buffer.started" or "output_audio_buffer.stopped" or "output_audio_buffer.cleared" or "conversation.item.added" or "conversation.item.done" or "input_audio_buffer.timeout_triggered" or "conversation.item.input_audio_transcription.segment" or "mcp_list_tools.in_progress" or "mcp_list_tools.completed" or "mcp_list_tools.failed" or "response.mcp_call_arguments.delta" or "response.mcp_call_arguments.done" or "response.mcp_call.in_progress" or "response.mcp_call.completed" or "response.mcp_call.failed", Azure.AI.ProjectsInternal.VoiceAgentServerEventTypeExtension.
OpenAI.SpecificApplyPatchParam
Forces the model to call the apply_patch tool when executing a tool call.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"apply_patch" |
Yes | The tool to call. Always apply_patch. |
OpenAI.SpecificFunctionShellParam
Forces the model to call the shell tool when a tool call is required.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"shell" |
Yes | The tool to call. Always shell. |
OpenAI.SpecificProgrammaticToolCallingParam
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"programmatic_tool_calling" |
Yes | The tool to call. Always programmatic_tool_calling. |
OpenAI.ToolChoiceAllowed
Constrains the tools available to the model to a pre-defined set.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"allowed_tools" |
Yes | Allowed tool configuration type. Always allowed_tools. |
mode |
"auto" or "required" |
Yes | Constrains the tools available to the model to a pre-defined set. auto allows the model to pick from among the allowed tools and generate a message. required requires the model to call one or more of the allowed tools. |
tools |
Array of object with additional values of any |
Yes | A list of tool definitions that the model should be allowed to call. For the Responses API, the list of tool definitions might look like: json [ { "type": "function", "name": "get_weather" }, { "type": "mcp", "server_label": "deepwiki" }, { "type": "image_generation" } ] |
OpenAI.ToolChoiceCodeInterpreter
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"code_interpreter" |
Yes |
OpenAI.ToolChoiceComputer
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"computer" |
Yes |
OpenAI.ToolChoiceComputerUse
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"computer_use" |
Yes |
OpenAI.ToolChoiceComputerUsePreview
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"computer_use_preview" |
Yes |
OpenAI.ToolChoiceCustom
Use this option to force the model to call a specific custom tool.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"custom" |
Yes | For custom tool calling, the type is always custom. |
name |
string |
Yes | The name of the custom tool to call. |
OpenAI.ToolChoiceFileSearch
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"file_search" |
Yes |
OpenAI.ToolChoiceFunction
Use this option to force the model to call a specific function.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"function" |
Yes | For function calling, the type is always function. |
name |
string |
Yes | The name of the function to call. |
OpenAI.ToolChoiceImageGeneration
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"image_generation" |
Yes |
OpenAI.ToolChoiceMCP
Use this option to force the model to call a specific tool on a remote MCP server.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"mcp" |
Yes | For MCP tools, the type is always mcp. |
server_label |
string |
Yes | The label of the MCP server to use. |
name |
Any of: string; null |
No |
OpenAI.ToolChoiceOptions
Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools.
Type: "none" or "auto" or "required".
OpenAI.ToolChoiceParam
How the model should select which tool (or tools) to use when generating a response. See the tools parameter to see how to specify which tools the model can call.
Variants by type: "function": OpenAI.ToolChoiceFunction; "mcp": OpenAI.ToolChoiceMCP; "allowed_tools": OpenAI.ToolChoiceAllowed; "custom": OpenAI.ToolChoiceCustom; "programmatic_tool_calling": OpenAI.SpecificProgrammaticToolCallingParam; "apply_patch": OpenAI.SpecificApplyPatchParam; "shell": OpenAI.SpecificFunctionShellParam; "file_search": OpenAI.ToolChoiceFileSearch; "web_search_preview": OpenAI.ToolChoiceWebSearchPreview; "computer_use_preview": OpenAI.ToolChoiceComputerUsePreview; "web_search_preview_2025_03_11": OpenAI.ToolChoiceWebSearchPreview20250311; "image_generation": OpenAI.ToolChoiceImageGeneration; "code_interpreter": OpenAI.ToolChoiceCodeInterpreter; "computer": OpenAI.ToolChoiceComputer; "computer_use": OpenAI.ToolChoiceComputerUse.
| Property | Type | Required | Description |
|---|---|---|---|
type |
OpenAI.ToolChoiceParamType | Yes |
OpenAI.ToolChoiceParamType
Any of: string, "allowed_tools" or "function" or "mcp" or "custom" or "programmatic_tool_calling" or "apply_patch" or "shell" or "file_search" or "web_search_preview" or "computer_use_preview" or "web_search_preview_2025_03_11" or "image_generation" or "code_interpreter" or "computer" or "computer_use".
OpenAI.ToolChoiceWebSearchPreview
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"web_search_preview" |
Yes |
OpenAI.ToolChoiceWebSearchPreview20250311
Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.
All of: OpenAI.ToolChoiceParam.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"web_search_preview_2025_03_11" |
Yes |
OpenAI.TranscriptTextUsageDuration
Usage statistics for models billed by audio input duration.
All of: OpenAI.CreateTranscriptionResponseJsonUsage.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"duration" |
Yes | The type of the usage object. Always duration for this variant. |
seconds |
integer |
Yes | Duration of the input audio in seconds. format: "int64". |
OpenAI.TranscriptTextUsageTokens
Usage statistics for models billed by token usage.
All of: OpenAI.CreateTranscriptionResponseJsonUsage.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"tokens" |
Yes | The type of the usage object. Always tokens for this variant. |
input_tokens |
OpenAI.integer | Yes | Number of input tokens billed for this request. |
input_token_details |
OpenAI.TranscriptTextUsageTokensInputTokenDetails | No | Details about the input tokens billed for this request. |
output_tokens |
OpenAI.integer | Yes | Number of output tokens generated. |
total_tokens |
OpenAI.integer | Yes | Total number of tokens used (input + output). |
OpenAI.TranscriptTextUsageTokensInputTokenDetails
| Property | Type | Required | Description |
|---|---|---|---|
text_tokens |
OpenAI.integer | No | |
audio_tokens |
OpenAI.integer | No |
OpenAI.TranscriptionLanguage
A language detected in transcribed audio.
| Property | Type | Required | Description |
|---|---|---|---|
code |
string |
Yes | The code of a language detected in the audio. |
OpenAI.integer
Type: integer.
format: "int64".
OpenAI.numeric
Type: number.
format: "double".
ToolConfig
Per-tool configuration that controls tool visibility and search behavior.
| Property | Type | Required | Description |
|---|---|---|---|
pin |
boolean |
No | When true, the tool is always included in agent context and visible in tools/list. When false (default), the tool is hidden from tools/list and only discoverable via tool_search. |
additional_search_text |
string |
No | Additional text indexed for tool_search. Supplements the native tool description to improve discoverability. Does not alter tools/list output. maxLength: 1000. |
VoiceAgentAnimationConfig
Animation settings for a voice-agent session.
| Property | Type | Required | Description |
|---|---|---|---|
model_name |
string |
No | The animation model name. default: "default". |
outputs |
Array of VoiceAgentAnimationOutputType | No | The requested animation output kinds. default: ["blendshapes"]. |
VoiceAgentAnimationOutputType
An animation output produced by a voice-agent session.
Type: "blendshapes" or "viseme_id".
VoiceAgentAudioConfig
The audio configuration for a voice agent. These values are session defaults and may be overridden when connecting.
| Property | Type | Required | Description |
|---|---|---|---|
input |
VoiceAgentAudioInputConfig | No | Input (microphone) audio configuration. |
output |
VoiceAgentAudioOutputConfig | No | Output (agent speech) audio configuration. |
VoiceAgentAudioInputConfig
Input audio configuration for a voice agent.
| Property | Type | Required | Description |
|---|---|---|---|
format |
OpenAI.RealtimeAudioFormats | No | The input audio format. |
noise_reduction |
Any of: VoiceAgentNoiseReduction; null |
No | Input noise reduction. Set to null to disable. |
turn_detection |
Any of: VoiceAgentTurnDetectionConfig; null |
No | Turn (end-of-speech) detection. Server-side turn detection is enabled by default; set to null to disable it, in which case the client must trigger responses manually. |
echo_cancellation |
Any of: VoiceAgentEchoCancellation; null |
No | Optional server-side echo cancellation settings. |
transcription |
Any of: VoiceAgentInputTranscription; null |
No | Asynchronous input-audio transcription. Set to null to disable transcription. |
VoiceAgentAudioOutputConfig
Output audio configuration for a voice agent. Extends the voice configuration with the session-level format and output_audio_timestamp_types settings, which apply to every voice type.
All of: VoiceAgentVoiceConfig.
| Property | Type | Required | Description |
|---|---|---|---|
format |
OpenAI.RealtimeAudioFormats | No | The output audio format. Applies to every voice_type and defaults to 24 kHz PCM. |
output_audio_timestamp_types |
Array of VoiceAgentAudioTimestampType | No | Timestamp kinds to include with output audio. Applies to every voice_type. |
VoiceAgentAudioTimestampType
An output-audio timestamp kind supported by a voice agent.
Type: "word".
VoiceAgentAvatarConfig
Avatar configuration for a voice agent. These values are session defaults and may be overridden when connecting.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentAvatarType | Yes | The avatar type. |
character |
string |
Yes | The avatar character identifier, e.g. 'lisa'. |
style |
string |
No | The avatar style, e.g. 'casual-sitting'. |
customized |
boolean |
No | Whether the avatar is a customer-customized avatar. Defaults to false. default: false. |
output_protocol |
VoiceAgentAvatarOutputProtocol | No | The transport used to deliver the avatar video stream. |
model |
string |
No | The avatar model identifier. |
video |
VoiceAgentAvatarVideoParams | No | Avatar video encoder and presentation settings. |
scene |
VoiceAgentAvatarScene | No | Avatar placement and motion settings. |
output_audit_audio |
boolean |
No | Whether audit audio is emitted with avatar output. Defaults to false. default: false. |
VoiceAgentAvatarIceServer
An ICE server used for avatar WebRTC negotiation.
| Property | Type | Required | Description |
|---|---|---|---|
urls |
Array of string (format: "uri".) |
Yes | |
username |
Any of: string; null |
No | |
credential |
Any of: string; null |
No | format: "password". |
VoiceAgentAvatarOutputProtocol
The transport used to deliver the avatar video stream.
Any of: string, "webrtc" or "websocket".
VoiceAgentAvatarScene
Avatar placement and motion settings.
| Property | Type | Required | Description |
|---|---|---|---|
zoom |
number |
No | format: "float". default: 1. exclusiveMinimum: 0. |
position_x |
number |
No | format: "float". default: 0. minimum: -1. maximum: 1. |
position_y |
number |
No | format: "float". default: 0. minimum: -1. maximum: 1. |
rotation_x |
number |
No | format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793. |
rotation_y |
number |
No | format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793. |
rotation_z |
number |
No | format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793. |
amplitude |
number |
No | format: "float". default: 1. maximum: 1. exclusiveMinimum: 0. |
VoiceAgentAvatarType
The avatar type.
Any of: string, "video_avatar" or "photo_avatar".
VoiceAgentAvatarVideoBackground
The avatar video background.
| Property | Type | Required | Description |
|---|---|---|---|
image_url |
string |
No | format: "uri". |
color |
string |
No |
VoiceAgentAvatarVideoCrop
The rectangular crop applied to avatar video.
| Property | Type | Required | Description |
|---|---|---|---|
bottom_right |
Array of integer (format: "uint32".) |
Yes | minItems: 2. maxItems: 2. |
top_left |
Array of integer (format: "uint32".) |
Yes | minItems: 2. maxItems: 2. |
VoiceAgentAvatarVideoParams
Avatar video encoder and presentation settings.
| Property | Type | Required | Description |
|---|---|---|---|
bitrate |
integer |
No | The target video bitrate in bits per second. format: "int32". default: 2000000. |
crop |
VoiceAgentAvatarVideoCrop | No | |
resolution |
VoiceAgentAvatarVideoResolution | No | |
background |
VoiceAgentAvatarVideoBackground | No | |
gop_size |
integer |
No | format: "int32". default: 10. minimum: 1. maximum: 2000. |
VoiceAgentAvatarVideoResolution
The avatar video resolution.
| Property | Type | Required | Description |
|---|---|---|---|
width |
integer |
Yes | format: "int32". minimum: 1. |
height |
integer |
Yes | format: "int32". minimum: 1. |
VoiceAgentAzureSemanticVadEnTurnDetection
English-optimized Azure semantic voice activity detection.
All of: VoiceAgentTurnDetectionConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"azure_semantic_vad_en" |
Yes | |
threshold |
number |
No | Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1. |
prefix_padding_ms |
FoundryDurationMilliseconds | No | Audio to include before detected speech, in milliseconds. |
silence_duration_ms |
FoundryDurationMilliseconds | No | Silence required to end speech detection, in milliseconds. |
idle_timeout_ms |
FoundryDurationMilliseconds | No | Maximum idle time before the detector ends the turn, in milliseconds. |
end_of_utterance_detection |
Any of: VoiceAgentEndOfUtteranceDetection; null |
No | Semantic end-of-utterance detection configuration. Set to null to disable it. |
speech_duration_ms |
FoundryDurationMilliseconds | No | Minimum speech duration required to trigger detection, in milliseconds. |
remove_filler_words |
boolean |
No | Whether filler words are removed from transcription. default: false. |
create_response |
boolean |
No | Whether a response is created automatically when speech stops. default: true. |
interrupt_response |
boolean |
No | Whether user speech may interrupt the agent's response. default: true. |
VoiceAgentAzureSemanticVadMultilingualTurnDetection
Multilingual Azure semantic voice activity detection.
All of: VoiceAgentTurnDetectionConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"azure_semantic_vad_multilingual" |
Yes | |
threshold |
number |
No | Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1. |
prefix_padding_ms |
FoundryDurationMilliseconds | No | Audio to include before detected speech, in milliseconds. |
silence_duration_ms |
FoundryDurationMilliseconds | No | Silence required to end speech detection, in milliseconds. |
idle_timeout_ms |
FoundryDurationMilliseconds | No | Maximum idle time before the detector ends the turn, in milliseconds. |
end_of_utterance_detection |
Any of: VoiceAgentEndOfUtteranceDetection; null |
No | Semantic end-of-utterance detection configuration. Set to null to disable it. |
speech_duration_ms |
FoundryDurationMilliseconds | No | Minimum speech duration required to trigger detection, in milliseconds. |
remove_filler_words |
boolean |
No | Whether filler words are removed from transcription. default: false. |
create_response |
boolean |
No | Whether a response is created automatically when speech stops. default: true. |
interrupt_response |
boolean |
No | Whether user speech may interrupt the agent's response. default: true. |
languages |
Array of string |
No | BCP-47 language codes used for speech detection. |
VoiceAgentAzureSemanticVadTurnDetection
Azure semantic voice activity detection.
All of: VoiceAgentTurnDetectionConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"azure_semantic_vad" |
Yes | |
threshold |
number |
No | Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1. |
prefix_padding_ms |
FoundryDurationMilliseconds | No | Audio to include before detected speech, in milliseconds. |
silence_duration_ms |
FoundryDurationMilliseconds | No | Silence required to end speech detection, in milliseconds. |
idle_timeout_ms |
FoundryDurationMilliseconds | No | Maximum idle time before the detector ends the turn, in milliseconds. |
end_of_utterance_detection |
Any of: VoiceAgentEndOfUtteranceDetection; null |
No | Semantic end-of-utterance detection configuration. Set to null to disable it. |
speech_duration_ms |
FoundryDurationMilliseconds | No | Minimum speech duration required to trigger detection, in milliseconds. |
remove_filler_words |
boolean |
No | Whether filler words are removed from transcription. default: false. |
create_response |
boolean |
No | Whether a response is created automatically when speech stops. default: true. |
interrupt_response |
boolean |
No | Whether user speech may interrupt the agent's response. default: true. |
languages |
Array of string |
No | BCP-47 language codes used for speech detection. |
VoiceAgentCollectDtmfSystemTool
Collects keypad digits on the current call.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"collect_dtmf" |
Yes | The service-managed control action. Always collect_dtmf. |
VoiceAgentCollectSmsSystemTool
Collects one complete SMS from an explicitly selected sender.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"collect_sms" |
Yes | The service-managed control action. Always collect_sms. |
VoiceAgentConversationItemTypeExtension
Additional item kinds used by the Voice Agent service.
Type: "system_tool".
VoiceAgentDigitRecognitionSystemTool
Retrieves the unchanged transcript of the last user audio turn.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"digit_recognition" |
Yes | The service-managed control action. Always digit_recognition. |
VoiceAgentEchoCancellation
Server-side echo cancellation settings for input audio.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"server_echo_cancellation" |
Yes | The echo cancellation implementation. Always server_echo_cancellation. |
reference_source |
VoiceAgentEchoCancellationReferenceSource | No | Whether reference audio comes from server playback or a client-provided channel. default: "server". |
channels |
integer |
No | The number of input channels. Use two interleaved channels when reference_source is client. format: "int32". default: 1. minimum: 1. maximum: 2. |
VoiceAgentEchoCancellationReferenceSource
The source of reference audio used for echo cancellation.
Type: "server" or "client".
VoiceAgentEndConversationSystemTool
A service-managed control that ends the active conversation.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"end_conversation" |
Yes | The service-managed control action. Always end_conversation. |
VoiceAgentEndOfUtteranceDetection
Semantic end-of-utterance detection configuration.
| Property | Type | Required | Description |
|---|---|---|---|
model |
VoiceAgentEndOfUtteranceDetectionModel | Yes | The semantic detection model. |
threshold_level |
VoiceAgentEndOfUtteranceThresholdLevel | No | The sensitivity threshold. |
timeout_ms |
FoundryDurationMilliseconds | No | The detection timeout in milliseconds. |
VoiceAgentEndOfUtteranceDetectionModel
The semantic end-of-utterance detection model.
Any of: string, "semantic_detection_v1" or "semantic_detection_v1_en" or "semantic_detection_v1_multilingual" or "smart_end_of_turn_detection".
VoiceAgentEndOfUtteranceThresholdLevel
The sensitivity threshold for semantic end-of-utterance detection.
Any of: string, "low" or "medium" or "high" or "default".
VoiceAgentFunctionTool
A native function tool executed by the client.
All of: VoiceAgentTool.
| Property | Type | Required | Description |
|---|---|---|---|
description |
string |
No | The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything). |
parameters |
OpenAI.RealtimeFunctionToolParameters | No | Parameters of the function in JSON Schema. |
type |
"function" |
Yes | |
name |
string |
Yes | The function name. minLength: 1. |
VoiceAgentGreetingConfig
Session-start greeting configuration for a voice agent.
Variants by type: "template": VoiceAgentTemplateGreetingConfig; "llm_generated": VoiceAgentLlmGeneratedGreetingConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentGreetingMode | Yes | The greeting mode. |
VoiceAgentGreetingMode
The greeting modes supported by a voice agent.
Any of: string, "template" or "llm_generated".
VoiceAgentHandoffAbortReason
Why a server-managed handoff ended before target commit.
Type: "user_interruption" or "error".
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentHandoffEdgeState
Non-sensitive topology metadata for one compiled handoff edge.
| Property | Type | Required | Description |
|---|---|---|---|
id |
VoiceAgentHandoffId | Yes | The edge identifier. |
source |
VoiceAgentHandoffNodeReference | Yes | The source node identifier or $entrypoint. |
target |
VoiceAgentHandoffNodeReference | Yes | The target node identifier or $entrypoint. |
cancel_on_interruption |
boolean |
No | Whether user interruption before commit cancels the pending handoff. default: false. |
delay_ms |
FoundryDurationMilliseconds | No | The minimum delay before target activation, in milliseconds. default: 0. |
transfer_message |
Any of: string; null |
No | The deterministic source-voice announcement played before target activation. maxLength: 500. |
target_response |
VoiceAgentHandoffTargetResponse | No | Whether Voice Live automatically starts a response after target activation. default: "auto". |
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentHandoffId
A case-sensitive identifier for an inlined subagent or handoff condition.
Type: string.
pattern: "^[a-z][a-z0-9_-]{0,63}$".
VoiceAgentHandoffNodeReference
A handoff node reference: either $entrypoint or a configured node identifier.
Type: string.
pattern: "^(?:\\$entrypoint|[a-z][a-z0-9_-]{0,63})$".
VoiceAgentHandoffNodeState
Non-sensitive topology metadata for one compiled handoff node.
| Property | Type | Required | Description |
|---|---|---|---|
id |
VoiceAgentHandoffNodeReference | Yes | The node identifier or $entrypoint. |
description |
string |
Yes | The display-only node description. This value is not provided to the model. |
implicit |
boolean |
No | Whether this is the implicit $entrypoint node. default: false. |
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentHandoffPipelineFamily
The immutable pipeline family selected for a voice-agent handoff graph.
Type: "cascaded" or "realtime".
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentHandoffState
The effective topology and runtime state of a voice-agent handoff graph.
| Property | Type | Required | Description |
|---|---|---|---|
pipeline_family |
VoiceAgentHandoffPipelineFamily | Yes | The immutable execution pipeline family. |
active_node_id |
VoiceAgentHandoffNodeReference | Yes | The currently committed node identifier or $entrypoint. |
node_generation |
integer |
Yes | The monotonically increasing generation of the committed active-node configuration. format: "int32". minimum: 1. |
transfer_count |
integer |
Yes | The number of successfully completed handoffs. format: "int32". minimum: 0. |
attempt_count |
integer |
Yes | The number of unique reserved handoff attempts. format: "int32". minimum: 0. |
available_edge_ids |
Array of VoiceAgentHandoffId | Yes | The edge identifiers currently available from the active node. |
transfer_tool |
Any of: VoiceAgentFunctionTool; null |
Yes | The generated handoff function visible to the active model, or null when no route is available. |
nodes |
Array of VoiceAgentHandoffNodeState | Yes | The compiled handoff nodes, including the implicit or explicit $entrypoint. |
edges |
Array of VoiceAgentHandoffEdgeState | Yes | The compiled handoff edges. |
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentHandoffTargetResponse
Whether Voice Live automatically starts a response after a handoff target becomes active.
Type: "auto" or "none".
Required preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentInputCollectionChannel
The receiver used by an input collection. Voice collection is not supported.
Type: "sms" or "dtmf".
VoiceAgentInputCollectionStatus
Collection lifecycle and value-free telephony-operation outcomes. Additional outcomes may be introduced over time.
Any of: string, "accepted" or "listening" or "completed" or "expired" or "cancelled" or "failed" or "busy" or "invalid_input" or "invalid_arguments" or "stale_collection" or "unsupported_channel" or "dispatch_failed" or "provider_accepted" or "transfer_not_answered" or "provider_failed_or_unsupported" or "provider_result_uncertain" or "provider_not_executed".
VoiceAgentInputTranscription
Settings for transcribing input audio, including the transcription model, Custom Speech deployments, and phrase hints. When model is azure-speech-streaming, transcription is supported for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.
| Property | Type | Required | Description |
|---|---|---|---|
language |
string |
No | The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency. Optional for other transcription models; required and must be nonempty when model is azure-speech-streaming (such as en-US; comma-separated locales are supported). |
languages |
Array of string |
No | Possible languages of the input audio, in ISO-639-1 format. Supported by gpt-transcribe and gpt-live-transcribe. minItems: 1. |
keywords |
Array of string |
No | Words or phrases to guide transcription of the input audio. Supported by gpt-transcribe and gpt-live-transcribe. |
prompt |
string |
No | An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions. |
delay |
"minimal" or "low" or "medium" or "high" or "xhigh" |
No | Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions. |
model |
VoiceAgentInputTranscriptionModel | Yes | The transcription model identifier. Configure customer custom speech deployments in custom_speech. azure-speech-streaming is for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter. |
custom_speech |
object with additional values of string |
No | Optional customer custom speech deployment configuration, keyed by locale. |
phrase_list |
Array of string |
No | Optional phrase hints that bias recognition toward domain terms. |
profanity_filter |
VoiceAgentProfanityFilterMode | No | Controls how profanity is handled in input transcripts. Supported only with managed Azure Speech (azure-speech and azure-speech-streaming), including Custom Speech. Set this property before the session starts. If omitted, the transcription provider's default applies. You cannot change the mode during a session. Filtering affects transcript events and the text sent to cascaded models. It does not change input audio, voice activity detection, or assistant speech; audio-native models still receive the original audio. Mandatory service masking policies apply even when you select unfiltered. |
VoiceAgentInputTranscriptionModel
The input-audio transcription model identifier. This is a model name, not a Foundry deployment name. Mirrors the transcription models supported by the managed voice backend, covering the OpenAI Realtime transcription models plus the Azure and MAI models. azure-speech-streaming is supported for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter. Additional values may be added over time.
Any of: string, "whisper-1" or "gpt-realtime-whisper" or "gpt-4o-transcribe" or "gpt-4o-mini-transcribe" or "gpt-4o-transcribe-diarize" or "gpt-transcribe" or "gpt-live-transcribe" or "mai-transcribe" or "azure-speech" or "azure-speech-streaming".
VoiceAgentInterimResponseConfig
Fields shared by interim-response configurations.
Variants by type: "static_interim_response": VoiceAgentStaticInterimResponseConfig; "llm_interim_response": VoiceAgentLlmInterimResponseConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentInterimResponseType | Yes | The interim-response implementation. |
triggers |
Array of VoiceAgentInterimResponseTrigger | No | Conditions that may trigger one interim response. default: ["latency"]. |
latency_threshold_ms |
integer |
No | The latency threshold in milliseconds. format: "int32". default: 2000. |
VoiceAgentInterimResponseTrigger
A condition that may trigger an interim response.
Any of: string, "latency" or "tool".
VoiceAgentInterimResponseType
The interim-response implementations supported by a voice agent.
Any of: string, "static_interim_response" or "llm_interim_response".
VoiceAgentLanguageDetectionSystemTool
Switches supported conversation language settings.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"language_detection" |
Yes | The service-managed control action. Always language_detection. |
VoiceAgentLlmGeneratedGreetingConfig
A greeting authored by the session model from a scoped opening-turn prompt.
All of: VoiceAgentGreetingConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"llm_generated" |
Yes | |
prompt |
string |
Yes | The Handlebars prompt that guides the opening turn. minLength: 1. |
tool_choice |
VoiceAgentToolChoice | No | The tool-selection policy for the opening response. Defaults to none. default: "none". |
VoiceAgentLlmInterimResponseConfig
An interim response generated by a language model.
All of: VoiceAgentInterimResponseConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"llm_interim_response" |
Yes | |
model |
string |
No | The model used to generate interim responses. |
instructions |
string |
No | Optional instructions for generating interim responses. |
max_completion_tokens |
integer |
No | The maximum completion-token count for an interim response. format: "int32". default: 50. minimum: 1. |
VoiceAgentMaxOutputTokens
The maximum output-token count or the literal inf.
Any of: integer, "inf".
VoiceAgentMcpTool
An MCP tool available to a voice agent. Server endpoints use server_url; connector and tunnel identifiers are not supported.
All of: VoiceAgentTool.
| Property | Type | Required | Description |
|---|---|---|---|
server_label |
string |
Yes | A label for this MCP server, used to identify it in tool calls. |
server_url |
string |
No | The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided. format: "uri". |
authorization |
string |
No | An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here. |
server_description |
string |
No | Optional description of the MCP server, used to provide more context. |
headers |
Any of: object with additional values of string; null |
No | |
allowed_tools |
Any of: Array of string; OpenAI.MCPToolFilter; null |
No | |
allowed_callers |
Any of: Array of OpenAI.CallableToolAllowedCaller; null |
No | |
require_approval |
Any of: OpenAI.MCPToolRequireApproval; "always" or "never"; null |
No | default: "always". |
defer_loading |
boolean |
No | Whether this MCP tool is deferred and discovered via tool search. |
project_connection_id |
string |
No | The connection ID in the project for the MCP server. The connection stores authentication and other connection details needed to connect to the MCP server. |
tool_configs |
object with additional values of ToolConfig |
No | Deprecated. This property is deprecated and will be removed in a future version. deprecated: true. |
type |
"mcp" |
Yes | |
response_scheduling |
VoiceAgentToolResponseScheduling | No | When the MCP invocation creates a follow-up response. Defaults to when_idle. default: "when_idle". |
VoiceAgentNoiseReduction
Input audio noise reduction configuration. azure_deep_noise_suppression_headset and azure_personal_noise_suppression are modes for local voice activity detection that enhance detection audio without changing the audio used for recognition.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentNoiseReductionType | Yes | The noise reduction mode. azure_deep_noise_suppression_headset provides headset-optimized noise suppression for local voice activity detection, optimized for low-sample-rate audio including 8 kHz telephony audio without changing recognition audio (not restricted to headset devices). azure_personal_noise_suppression starts with headset suppression for local voice activity detection and personalizes detection after automatic speaker enrollment while keeping recognition audio unchanged and keeping headset suppression active when personalization is unavailable. |
VoiceAgentNoiseReductionType
The input audio noise reduction mode. - azure_deep_noise_suppression_headset: Azure headset-optimized noise suppression for local voice activity detection. Optimized for low-sample-rate audio, including 8 kHz telephony audio. Enhances detection audio without changing the audio used for recognition. The mode is not restricted to headset devices. - azure_personal_noise_suppression: Azure personalized noise suppression for local voice activity detection. Starts with headset suppression and personalizes detection after automatic speaker enrollment. Recognition audio is unchanged; headset suppression remains active when personalization is unavailable.
Any of: string, "near_field" or "far_field" or "azure_deep_noise_suppression" or "azure_deep_noise_suppression_headset" or "azure_personal_noise_suppression".
VoiceAgentOutputReference
Bare session variable name, such as verification_code, without a namespace or braces. Tool arguments reference it with {{name}}.
Type: string.
pattern: "^[A-Za-z_][A-Za-z0-9_]*$".
VoiceAgentProfanityFilterMode
The supported input-transcript profanity filtering modes.
Any of: string, "masked" or "unfiltered" or "removed".
VoiceAgentRealtimeResponse
A live realtime response returned by the voice-agent service in both response.created and response.done events.
| Property | Type | Required | Description |
|---|---|---|---|
id |
string |
No | The unique ID of the response, will look like resp_1234. |
object |
"realtime.response" |
No | The object type, must be realtime.response. |
status |
"completed" or "cancelled" or "failed" or "incomplete" or "in_progress" |
No | The final status of the response (completed, cancelled, failed, or incomplete, in_progress). |
status_details |
OpenAI.RealtimeResponseStatusDetails | No | Additional details about the status. |
output |
Array of OpenAI.RealtimeConversationItem | No | The list of output items generated by the response. |
metadata |
Any of: OpenAI.Metadata; null |
No | |
usage |
OpenAI.RealtimeResponseUsage | No | Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns. |
conversation_id |
string |
No | Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation |
output_modalities |
Array of "text" or "audio" |
No | The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model. |
max_output_tokens |
Exactly one of: OpenAI.integer; "inf" |
No | Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response. |
audio |
VoiceResponseAudio | No | The audio configuration used by the live response, including flat voice provider, locale, and format fields under output. |
VoiceAgentResponseCreateParams
Parameters accepted by a voice-agent response.create event.
| Property | Type | Required | Description |
|---|---|---|---|
instructions |
string |
No | The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior. Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session. |
tools |
Array of Any of: OpenAI.RealtimeFunctionTool; OpenAI.MCPTool | No | Tools available to the model. |
tool_choice |
Exactly one of: OpenAI.ToolChoiceOptions; OpenAI.ToolChoiceFunction; OpenAI.ToolChoiceMCP | No | How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool. default: "auto". |
parallel_tool_calls |
boolean |
No | Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2. |
reasoning |
OpenAI.RealtimeReasoning | No | |
max_output_tokens |
Exactly one of: OpenAI.integer; "inf" |
No | Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf. |
conversation |
Exactly one of: string; "auto" or "none" |
No | Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation. default: "auto". |
metadata |
Any of: OpenAI.Metadata; null |
No | |
input |
Array of OpenAI.RealtimeResponseInputItem | No | Input items to include in the prompt for the model. Using this field creates a new context for this Response instead of using the default conversation. An empty array [] will clear the context for this Response. Note that this can include references to items that previously appeared in the session using their id. |
output_modalities |
Array of VoiceOutputModality | No | Modalities that the response may return. |
audio |
object |
No | Response-specific audio settings. |
pre_generated_assistant_message |
Any of: OpenAI.RealtimeConversationItem; null |
No | A pre-generated assistant message used to begin the response. |
interim_response |
Any of: VoiceAgentInterimResponseConfig; null |
No | Interim-response settings for this response. |
audio properties
| Property | Type | Required | Description |
|---|---|---|---|
output |
VoiceAgentAudioOutputConfig | No | Output (agent speech) audio configuration. |
VoiceAgentRtcCallErrorDetails
Details of a WebRTC signaling error.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentRtcCallErrorType | Yes | The error category, following the VoiceLive wire contract: invalid_request_error for a client-side signaling fault (for example, a malformed SDP offer) or server_error for a service-side failure. Additional categories may be added over time. |
code |
string |
No | A machine-readable error code, when available. |
message |
string |
Yes | A human-readable error message. |
VoiceAgentRtcCallErrorType
The error categories reported for WebRTC signaling failures.
Any of: string, "invalid_request_error" or "server_error".
VoiceAgentSemanticVadTurnDetection
OpenAI semantic VAD turn-detection settings.
All of: VoiceAgentTurnDetectionConfig.
| Property | Type | Required | Description |
|---|---|---|---|
eagerness |
"low" or "medium" or "high" or "auto" |
No | default: "auto". |
create_response |
boolean |
No | default: true. |
interrupt_response |
boolean |
No | default: true. |
type |
"semantic_vad" |
Yes |
VoiceAgentSendDtmfSystemTool
Sends literal or templated keypad digits.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"send_dtmf" |
Yes | The service-managed control action. Always send_dtmf. |
VoiceAgentSendSmsSystemTool
Sends a text message from the Agent's SMS-capable number.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"send_sms" |
Yes | The service-managed control action. Always send_sms. |
VoiceAgentServerEventWarningDetails
Details of a non-fatal warning.
| Property | Type | Required | Description |
|---|---|---|---|
message |
string |
Yes | |
code |
string |
No | |
param |
string |
No |
VoiceAgentServerVadTurnDetection
Server-side voice activity detection.
All of: VoiceAgentTurnDetectionConfig.
| Property | Type | Required | Description |
|---|---|---|---|
threshold |
OpenAI.numeric | No | |
prefix_padding_ms |
OpenAI.integer | No | |
silence_duration_ms |
OpenAI.integer | No | |
create_response |
boolean |
No | default: true. |
interrupt_response |
boolean |
No | default: true. |
idle_timeout_ms |
Any of: OpenAI.integer; null |
No | |
type |
"server_vad" |
Yes | |
speech_duration_ms |
FoundryDurationMilliseconds | No | Minimum speech duration required to trigger detection, in milliseconds. |
end_of_utterance_detection |
Any of: VoiceAgentEndOfUtteranceDetection; null |
No | Semantic end-of-utterance detection configuration. Set to null to disable it. |
VoiceAgentSessionAvatarConfig
Avatar settings accepted by the stable voice-agent WebSocket contract.
All of: VoiceAgentAvatarConfig.
| Property | Type | Required | Description |
|---|---|---|---|
ice_servers |
Any of: Array of VoiceAgentAvatarIceServer; null |
No |
VoiceAgentSessionIncludeOption
Additional fields that a voice-agent session may include in service outputs.
Type: "item.input_audio_transcription.logprobs" or "item.input_audio_transcription.phrases" or "file_search_call.results".
VoiceAgentSessionResponse
The effective session configuration returned in voice-agent session lifecycle events.
VoiceAgentSessionResponseConfig
The effective stable realtime session settings returned by the voice-agent service.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"realtime" |
Yes | The session type. Always realtime. |
instructions |
string |
No | Instructions applied throughout the session. |
temperature |
number |
No | The sampling temperature for compatible cascaded pipelines. format: "float". minimum: 0. maximum: 2. |
max_output_tokens |
VoiceAgentMaxOutputTokens | No | The maximum output-token count for one response. |
output_modalities |
Array of VoiceOutputModality | No | The output modalities enabled for the session. |
audio |
VoiceAgentAudioConfig | No | The input- and output-audio settings for the session. |
avatar |
VoiceAgentSessionAvatarConfig | No | The avatar settings for the session. |
animation |
VoiceAgentAnimationConfig | No | Animation settings for the session. |
tools |
Array of VoiceAgentTool | No | Tools available to the session. |
tool_choice |
VoiceAgentToolChoice | No | Tool-selection behavior for the session. |
reasoning |
OpenAI.RealtimeReasoning | No | Reasoning settings for compatible realtime models. |
parallel_tool_calls |
boolean |
No | Whether the model may call multiple tools in parallel. |
include |
Array of VoiceAgentSessionIncludeOption | No | Additional fields to include in service outputs. |
metadata |
object with additional values of string |
No | Up to 16 string key-value pairs attached to the session. |
interim_response |
VoiceAgentInterimResponseConfig | No | Interim-response settings for latency and tool execution. |
greeting |
VoiceAgentGreetingConfig | No | A proactive assistant greeting started after session configuration. |
handoff |
Any of: VoiceAgentHandoffState; null |
No | The effective handoff topology and runtime state, when handoff is configured. |
object |
"realtime.session" |
Yes | The object type. Always realtime.session. |
id |
string |
Yes | The session identifier. |
model |
string |
Yes | The selected model. |
expires_at |
Any of: FoundryTimestamp; null |
No | The session expiration time as a Unix timestamp in seconds. |
VoiceAgentSessionUpdate
The session configuration accepted by a voice-agent session update event.
VoiceAgentSessionUpdateConfig
The stable realtime session settings accepted in a session.update client event.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"realtime" |
Yes | The session type. Always realtime. |
instructions |
string |
No | Instructions applied throughout the session. |
temperature |
number |
No | The sampling temperature for compatible cascaded pipelines. format: "float". minimum: 0. maximum: 2. |
max_output_tokens |
VoiceAgentMaxOutputTokens | No | The maximum output-token count for one response. |
output_modalities |
Array of VoiceOutputModality | No | The output modalities enabled for the session. |
audio |
VoiceAgentAudioConfig | No | The input- and output-audio settings for the session. |
avatar |
VoiceAgentSessionAvatarConfig | No | The avatar settings for the session. |
animation |
VoiceAgentAnimationConfig | No | Animation settings for the session. |
tools |
Array of VoiceAgentTool | No | Tools available to the session. |
tool_choice |
VoiceAgentToolChoice | No | Tool-selection behavior for the session. |
reasoning |
OpenAI.RealtimeReasoning | No | Reasoning settings for compatible realtime models. |
parallel_tool_calls |
boolean |
No | Whether the model may call multiple tools in parallel. |
include |
Array of VoiceAgentSessionIncludeOption | No | Additional fields to include in service outputs. |
metadata |
object with additional values of string |
No | Up to 16 string key-value pairs attached to the session. |
interim_response |
VoiceAgentInterimResponseConfig | No | Interim-response settings for latency and tool execution. |
greeting |
VoiceAgentGreetingConfig | No | A proactive assistant greeting started after session configuration. |
VoiceAgentStaticInterimResponseConfig
A static interim response selected from configured text.
All of: VoiceAgentInterimResponseConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"static_interim_response" |
Yes | |
texts |
Array of string |
No | Candidate text values for the interim response. |
VoiceAgentSubagentAbortReason
The reason a subagent consultation was aborted.
Any of: string, "unknown_target" or "timeout" or "cancelled" or "stopped_by_user" or "superseded" or "failed".
VoiceAgentSystemTool
A service-managed control that acts on the active voice session without customer code or external authentication.
All of: VoiceAgentTool.
Variants by name: "end_conversation": VoiceAgentEndConversationSystemTool; "language_detection": VoiceAgentLanguageDetectionSystemTool; "digit_recognition": VoiceAgentDigitRecognitionSystemTool; "collect_sms": VoiceAgentCollectSmsSystemTool; "collect_dtmf": VoiceAgentCollectDtmfSystemTool; "transfer": VoiceAgentTransferSystemTool; "send_dtmf": VoiceAgentSendDtmfSystemTool; "send_sms": VoiceAgentSendSmsSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"system" |
Yes | The type of the tool. Always system. |
name |
VoiceAgentSystemToolName | Yes | The service-managed control action. Known values are stable; additional values may be added over time. |
description |
string |
No | An optional description of the system tool. |
timeout_seconds |
FoundryDurationSeconds | No | The execution timeout in seconds, from 1 to 300. If omitted, the service defaults to 60 seconds. default: 60. minimum: 1. maximum: 300. |
VoiceAgentSystemToolItem
Service-generated system-tool intent, not a provider result. Clients cannot create it or complete it with function_call_output.
All of: OpenAI.RealtimeConversationItem.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"system_tool" |
Yes | The output-item kind. Always system_tool. |
id |
string |
No | The identifier of the generated item. |
object |
"realtime.item" |
No | The realtime item marker. |
status |
VoiceAgentSystemToolItemStatus | Yes | The generation status of the item, not the provider execution outcome. |
name |
VoiceAgentSystemToolName | Yes | The built-in capability invoked by the model. |
call_id |
string |
Yes | The invocation identifier used for correlation and duplicate suppression. |
arguments |
string |
Yes | Original JSON-encoded intent: literals remain visible and references stay unexpanded. Excludes private values, resolved arguments and executor-only settings. |
created_at |
FoundryTimestamp | No | The Unix timestamp (in seconds) for when the item was persisted. readOnly: true. |
response_id |
string |
No | The id of the response that produced this item, when applicable. readOnly: true. |
VoiceAgentSystemToolItemStatus
The generation status of a system-tool item. It does not indicate provider success.
Any of: string, "in_progress" or "completed" or "incomplete".
VoiceAgentSystemToolName
A service-managed voice-session control action. Known values are stable; additional values may be added over time.
Any of: string, "end_conversation" or "language_detection" or "digit_recognition" or "transfer" or "collect_sms" or "collect_dtmf" or "send_dtmf" or "send_sms".
VoiceAgentTemplateGreetingConfig
A deterministic greeting rendered with the voice agent's structured inputs and synthesized without model-authored generation.
All of: VoiceAgentGreetingConfig.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"template" |
Yes | |
text |
string |
Yes | The Handlebars text template spoken at session start. minLength: 1. |
VoiceAgentTool
A tool usable by a voice agent.
Variants by type: "function": VoiceAgentFunctionTool; "mcp": VoiceAgentMcpTool; "system": VoiceAgentSystemTool; "toolbox": VoiceAgentToolboxTool.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentToolType | Yes | The tool kind. |
VoiceAgentToolChoice
Tool-selection behavior for a voice agent.
One of: "none" or "auto" or "required", OpenAI.ToolChoiceFunction, OpenAI.ToolChoiceMCP.
VoiceAgentToolResponseScheduling
When a tool invocation creates a follow-up response. Additional values may be added over time.
Any of: string, "silent" or "when_idle" or "interrupt" or "skip_if_busy".
VoiceAgentToolType
The tool kinds supported by a voice agent.
Any of: string, "function" or "mcp" or "system" or "toolbox".
VoiceAgentToolboxTool
A reference to a Foundry toolbox, which is a versioned bundle of tools executed through its MCP endpoint.
All of: VoiceAgentTool.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"toolbox" |
Yes | The type of the tool. Always toolbox. |
toolbox_name |
string |
Yes | The name of the toolbox to attach. |
toolbox_version |
string |
Yes | The immutable version of the toolbox to attach. |
response_scheduling |
VoiceAgentToolResponseScheduling | No | When the toolbox invocation creates a follow-up response. Defaults to when_idle. default: "when_idle". |
VoiceAgentTranscriptionPhrase
A transcribed phrase with timing information.
| Property | Type | Required | Description |
|---|---|---|---|
offset_milliseconds |
FoundryDurationMilliseconds | Yes | The phrase offset from the beginning of the audio, in milliseconds. |
duration_milliseconds |
FoundryDurationMilliseconds | Yes | The phrase duration in milliseconds. |
text |
string |
Yes | The transcribed phrase text. |
words |
Any of: Array of VoiceAgentTranscriptionWord; null |
No | Word-level timing details, when available. |
locale |
Any of: string; null |
No | The detected locale. |
confidence |
Any of: number; null |
No | The transcription confidence score. |
VoiceAgentTranscriptionWord
A time-stamped word in an input-audio transcription.
| Property | Type | Required | Description |
|---|---|---|---|
text |
string |
Yes | The transcribed word text. |
offset_milliseconds |
FoundryDurationMilliseconds | Yes | The word offset from the beginning of the audio, in milliseconds. |
duration_milliseconds |
FoundryDurationMilliseconds | Yes | The word duration in milliseconds. |
VoiceAgentTransferDestination
The canonical system-tool transfer destination.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentTransferDestinationType | Yes | The destination format: phone_number for an E.164 number, teams for a Teams identifier, or sip for a SIP URI. |
value |
string |
Yes | The concrete address. Phone numbers use E.164; Teams and SIP identifiers are validated against the selected provider's capabilities. minLength: 1. |
VoiceAgentTransferDestinationType
The supported address formats for a system-tool transfer destination.
Type: "phone_number" or "teams" or "sip".
Conditional preview opt-ins: VoiceAgents=V1Preview.
VoiceAgentTransferRouting
The configured transfer routing policy. The model selects a target name, not destinations or routing settings.
| Property | Type | Required | Description |
|---|---|---|---|
strategy |
VoiceAgentTransferStrategy | Yes | The routing strategy. Current native Telephony transfer supports ACS sequential and random routing only. |
ring_timeout_seconds |
FoundryDurationSeconds | No | The per-destination ring timeout in seconds, from 1 to 300. If omitted, the service defaults to 20 seconds. Must not exceed the tool's execution timeout. ACS allows at most 180 seconds. default: 20. minimum: 1. maximum: 300. |
VoiceAgentTransferStrategy
Per-invocation transfer routing. Unsupported strategies are rejected before provider execution; schema membership does not imply support by every provider.
Any of: string, "sequential" or "random" or "parallel".
VoiceAgentTransferSystemTool
Requests a transfer to an explicitly configured logical target.
All of: VoiceAgentSystemTool.
| Property | Type | Required | Description |
|---|---|---|---|
name |
"transfer" |
Yes | The service-managed control action. Always transfer. |
require_handoff |
boolean |
No | Require a non-blank handoff summary in each model-generated transfer call. Defaults to false; does not guarantee delivery to the recipient. default: false. |
targets |
Array of VoiceAgentTransferTarget | Yes | The non-empty set of logical targets that the model may select. minItems: 1. |
VoiceAgentTransferTarget
A configured logical transfer target. Only its name and description are model-visible; destinations and routing are executor settings.
| Property | Type | Required | Description |
|---|---|---|---|
name |
string |
Yes | The non-blank logical target name. Names must be unique after trimming within the transfer tool. minLength: 1. pattern: "\\S". |
description |
string |
No | A description that helps the model select this target. |
destinations |
Array of VoiceAgentTransferDestination | Yes | The concrete destinations for this transfer target. minItems: 1. |
routing |
VoiceAgentTransferRouting | Yes | The trusted routing policy for this target. |
VoiceAgentTurnDetectionConfig
Turn-detection configuration for a voice agent.
Variants by type: "server_vad": VoiceAgentServerVadTurnDetection; "azure_semantic_vad": VoiceAgentAzureSemanticVadTurnDetection; "azure_semantic_vad_en": VoiceAgentAzureSemanticVadEnTurnDetection; "azure_semantic_vad_multilingual": VoiceAgentAzureSemanticVadMultilingualTurnDetection; "semantic_vad": VoiceAgentSemanticVadTurnDetection.
| Property | Type | Required | Description |
|---|---|---|---|
type |
VoiceAgentTurnDetectionType | Yes | The turn-detection strategy. |
auto_truncate |
boolean |
No | Whether the input audio buffer is truncated automatically when speech stops. default: false. |
VoiceAgentTurnDetectionType
The turn-detection strategy. Additional values may be added over time.
Any of: string, "server_vad" or "semantic_vad" or "azure_semantic_vad" or "azure_semantic_vad_en" or "azure_semantic_vad_multilingual".
VoiceAgentVoiceConfig
Voice output configuration. Inline subagents use it directly as their voice override, and the session-level output audio configuration extends it. Provider-specific fields are selected by voice_type: - openai: voice and speed. - azure-standard: voice, voice_locale, speed, voice_temperature, custom_lexicon_url, custom_text_normalization_url, prefer_locales, style, pitch, and volume. - azure-custom: all azure-standard fields except style, plus custom_voice_endpoint_id. - azure-personal: all azure-standard fields except style, plus personal_voice_model. - avatar-voice-sync: all azure-standard fields except voice and style, plus personal_voice_model; the voice name is derived from the avatar. - azure-realtime-native: voice and speed.
| Property | Type | Required | Description |
|---|---|---|---|
voice |
string |
No | The voice name or identifier. Applies to openai, azure-standard, azure-custom, azure-personal, and azure-realtime-native. It does not apply to avatar-voice-sync, which derives the voice name from the avatar. |
voice_type |
VoiceType | No | The voice implementation. |
voice_locale |
string |
No | The enforced BCP-47 output locale. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. |
speed |
number |
No | The numeric output speed multiplier. Applies to all known voice_type values and defaults to 1. format: "float". default: 1. minimum: 0.25. maximum: 1.5. |
voice_temperature |
number |
No | The voice variation temperature. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "float". minimum: 0. maximum: 1. |
custom_lexicon_url |
string |
No | The URL of a custom pronunciation lexicon. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "uri". |
custom_text_normalization_url |
string |
No | The URL of a custom text-normalization configuration. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "uri". |
prefer_locales |
Array of string |
No | Preferred BCP-47 locales for multilingual synthesis. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. |
style |
string |
No | The voice speaking style. Applies only when voice_type is azure-standard. |
pitch |
string |
No | The voice pitch adjustment. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. |
volume |
string |
No | The voice volume adjustment. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. |
custom_voice_endpoint_id |
string |
No | The Azure custom-voice deployment endpoint identifier. Applies only when voice_type is azure-custom. |
personal_voice_model |
string |
No | The Azure personal or avatar voice model. Applies only when voice_type is azure-personal or avatar-voice-sync. |
VoiceOutputModality
An output modality the agent may produce. animation and avatar are used when an avatar is configured.
Any of: string, "text" or "audio" or "animation" or "avatar".
VoiceResponseAudio
Audio configuration for a response. Follows the OpenAI Realtime GA audio object shape.
| Property | Type | Required | Description |
|---|---|---|---|
output |
VoiceResponseAudioOutput | No | The audio output configuration used for the response. |
VoiceResponseAudioOutput
The flat response audio-output projection, with optional voice, voice_type, voice_locale, and format fields.
| Property | Type | Required | Description |
|---|---|---|---|
voice |
string |
No | The voice name used for the response's audio output. |
voice_type |
VoiceType | No | The extensible provider/type of the voice used for the response's audio output. |
voice_locale |
string |
No | The BCP-47 locale of the voice used for the response's audio output. |
format |
OpenAI.RealtimeAudioFormats | No | The audio format used for the response's audio output. |
VoiceType
The voice implementation. Additional values may be added over time.
Any of: string, "openai" or "azure-standard" or "azure-custom" or "azure-personal" or "avatar-voice-sync" or "azure-realtime-native".