Voice agent realtime event models

These models are referenced by realtime client events and server events.

Models

Azure.AI.ProjectsInternal.VoiceAgentClientEventTypeExtension

Type: "session.avatar.connect" or "rtc.call.sdp.create".

Azure.AI.ProjectsInternal.VoiceAgentServerEventTypeExtension

Type: "warning" or "session.subagent.started" or "session.subagent.completed" or "session.subagent.aborted" or "session.avatar.connecting" or "session.avatar.switch_to_speaking" or "session.avatar.switch_to_idle" or "rtc.call.sdp.created" or "rtc.call.error" or "response.audio_timestamp.delta" or "response.audio_timestamp.done" or "response.animation_blendshapes.delta" or "response.animation_blendshapes.done" or "response.animation_viseme.delta" or "response.animation_viseme.done" or "response.video.delta" or "response.system_tool_arguments.delta" or "response.system_tool_arguments.done" or "input_collection.status" or "session.handoff.started" or "session.handoff.completed" or "session.handoff.aborted".

Conditional preview opt-ins: VoiceAgents=V1Preview.

FoundryDurationMilliseconds

A duration encoded as an integer number of milliseconds.

Type: integer.

format: "int32".

FoundryDurationSeconds

A duration encoded as an integer number of seconds.

Type: integer.

format: "int32".

FoundryTimestamp

Type: integer.

format: "unixtime".

OpenAI.CallableToolAllowedCaller

Type: "direct" or "programmatic".

OpenAI.CreateTranscriptionResponseJsonUsage

Token usage statistics for the request.

Variants by type: "tokens": OpenAI.TranscriptTextUsageTokens; "duration": OpenAI.TranscriptTextUsageDuration.

Property Type Required Description
type OpenAI.CreateTranscriptionResponseJsonUsageType Yes

OpenAI.CreateTranscriptionResponseJsonUsageType

Any of: string, "tokens" or "duration".

OpenAI.LogProbProperties

A log probability object.

Property Type Required Description
token string Yes The token that was used to generate the log probability.
logprob OpenAI.numeric Yes The log probability of the token.
bytes Array of OpenAI.integer Yes The bytes that were used to generate the log probability.

OpenAI.MCPListToolsTool

A tool available on an MCP server.

Property Type Required Description
name string Yes The name of the tool.
description Any of: string; null No
input_schema OpenAI.MCPListToolsToolInputSchema Yes The JSON schema describing the tool's input.
annotations Any of: OpenAI.MCPListToolsToolAnnotations; null No

OpenAI.MCPListToolsToolAnnotations

OpenAI.MCPListToolsToolInputSchema

OpenAI.MCPTool

Give the model access to additional tools via remote Model Context Protocol (MCP) servers. Learn more about MCP.

Property Type Required Description
type "mcp" Yes The type of the MCP tool. Always mcp.
server_label string Yes A label for this MCP server, used to identify it in tool calls.
server_url string No The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided. format: "uri".
connector_id "connector_dropbox" or "connector_gmail" or "connector_googlecalendar" or "connector_googledrive" or "connector_microsoftteams" or "connector_outlookcalendar" or "connector_outlookemail" or "connector_sharepoint" No Identifier for service connectors, like those available in ChatGPT. One of server_url, connector_id, or tunnel_id must be provided. Learn more about service connectors here. This field is deprecated for models released after September 1, 2026. Use server_url to connect to a remote MCP server, or tunnel_id to connect through a Secure MCP Tunnel. Currently supported connector_id values are: - Dropbox: connector_dropbox - Gmail: connector_gmail - Google Calendar: connector_googlecalendar - Google Drive: connector_googledrive - Microsoft Teams: connector_microsoftteams - Outlook Calendar: connector_outlookcalendar - Outlook Email: connector_outlookemail - SharePoint: connector_sharepoint deprecated: true.
tunnel_id string No The Secure MCP Tunnel ID to use instead of a direct server URL. One of server_url, connector_id, or tunnel_id must be provided. pattern: "^tunnel_[a-z0-9]{32}$".
authorization string No An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
server_description string No Optional description of the MCP server, used to provide more context.
headers Any of: object with additional values of string; null No
allowed_tools Any of: Array of string; OpenAI.MCPToolFilter; null No
allowed_callers Any of: Array of OpenAI.CallableToolAllowedCaller; null No
require_approval Any of: OpenAI.MCPToolRequireApproval; "always" or "never"; null No default: "always".
defer_loading boolean No Whether this MCP tool is deferred and discovered via tool search.
project_connection_id string No The connection ID in the project for the MCP server. The connection stores authentication and other connection details needed to connect to the MCP server.
tool_configs object with additional values of ToolConfig No Deprecated. This property is deprecated and will be removed in a future version. deprecated: true.

OpenAI.MCPToolFilter

A filter object to specify which tools are allowed.

Property Type Required Description
tool_names Array of string No List of allowed tool names.
read_only boolean No Indicates whether or not a tool modifies data or is read-only. If an MCP server is annotated with readOnlyHint, it will match this filter.

OpenAI.MCPToolRequireApproval

Property Type Required Description
always OpenAI.MCPToolFilter No
never OpenAI.MCPToolFilter No

OpenAI.Metadata

Set of 16 key-value pairs that can be attached to an object. This can be useful for storing additional information about the object in a structured format, and querying for objects via API or the dashboard. Keys are strings with a maximum length of 64 characters. Values are strings with a maximum length of 512 characters.

Additional properties: string.

OpenAI.RealtimeAudioFormats

Variants by type: "audio/pcm": OpenAI.RealtimeAudioFormatsAudioPcm; "audio/pcmu": OpenAI.RealtimeAudioFormatsAudioPcmu; "audio/pcma": OpenAI.RealtimeAudioFormatsAudioPcma.

Property Type Required Description
type OpenAI.RealtimeAudioFormatsType Yes

OpenAI.RealtimeAudioFormatsAudioPcm

All of: OpenAI.RealtimeAudioFormats.

Property Type Required Description
type "audio/pcm" Yes
rate 24000 No

OpenAI.RealtimeAudioFormatsAudioPcma

All of: OpenAI.RealtimeAudioFormats.

Property Type Required Description
type "audio/pcma" Yes

OpenAI.RealtimeAudioFormatsAudioPcmu

All of: OpenAI.RealtimeAudioFormats.

Property Type Required Description
type "audio/pcmu" Yes

OpenAI.RealtimeAudioFormatsType

Any of: string, "audio/pcm" or "audio/pcmu" or "audio/pcma".

OpenAI.RealtimeClientEvent

A realtime client event.

Property Type Required Description
type OpenAI.RealtimeClientEventType Yes

OpenAI.RealtimeClientEventType

Any of: string, "conversation.item.create" or "conversation.item.delete" or "conversation.item.retrieve" or "conversation.item.truncate" or "input_audio_buffer.append" or "input_audio_buffer.clear" or "output_audio_buffer.clear" or "input_audio_buffer.commit" or "response.cancel" or "response.create" or "session.update", Azure.AI.ProjectsInternal.VoiceAgentClientEventTypeExtension.

OpenAI.RealtimeConversationItem

A single item within a Realtime conversation.

Variants by type: "system_tool": VoiceAgentSystemToolItem; "message": OpenAI.RealtimeConversationItemMessage; "function_call": OpenAI.RealtimeConversationItemFunctionCall; "function_call_output": OpenAI.RealtimeConversationItemFunctionCallOutput; "mcp_approval_response": OpenAI.RealtimeMCPApprovalResponse; "mcp_list_tools": OpenAI.RealtimeMCPListTools; "mcp_call": OpenAI.RealtimeMCPToolCall; "mcp_approval_request": OpenAI.RealtimeMCPApprovalRequest.

Property Type Required Description
type OpenAI.RealtimeConversationItemType Yes

OpenAI.RealtimeConversationItemFunctionCall

A function call item in a Realtime conversation.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
id string No The unique ID of the item. This may be provided by the client or generated by the server.
object "realtime.item" No Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
type "function_call" Yes The type of the item. Always function_call.
status "completed" or "incomplete" or "in_progress" No The status of the item. Has no effect on the conversation.
call_id string No The ID of the function call.
name string Yes The name of the function being called.
arguments string Yes The arguments of the function call. This is a JSON-encoded string representing the arguments passed to the function, for example {"arg1": "value1", "arg2": 42}.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeConversationItemFunctionCallOutput

A function call output item in a Realtime conversation.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
id string No The unique ID of the item. This may be provided by the client or generated by the server.
object "realtime.item" No Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
type "function_call_output" Yes The type of the item. Always function_call_output.
status "completed" or "incomplete" or "in_progress" No The status of the item. Has no effect on the conversation.
call_id string Yes The ID of the function call this output is for.
output string Yes The output of the function call, this is free text and can contain any information or simply be empty.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.
name string No The name of the function that was called. A Foundry extension: OpenAI's function_call_output does not carry the function name, only call_id.

OpenAI.RealtimeConversationItemMessage

All of: OpenAI.RealtimeConversationItem.

Variants by role: "system": OpenAI.RealtimeConversationItemMessageSystem; "user": OpenAI.RealtimeConversationItemMessageUser; "assistant": OpenAI.RealtimeConversationItemMessageAssistant.

Property Type Required Description
role OpenAI.RealtimeConversationItemMessageType Yes
type "message" Yes

OpenAI.RealtimeConversationItemMessageAssistant

An assistant message item in a Realtime conversation.

All of: OpenAI.RealtimeConversationItemMessage.

Property Type Required Description
id string No The unique ID of the item. This may be provided by the client or generated by the server.
object "realtime.item" No Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
type "message" Yes The type of the item. Always message.
status "completed" or "incomplete" or "in_progress" No The status of the item. Has no effect on the conversation.
role "assistant" Yes The role of the message sender. Always assistant.
content Array of OpenAI.RealtimeConversationItemMessageAssistantContent Yes The content of the message.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeConversationItemMessageAssistantContent

Property Type Required Description
type "output_text" or "output_audio" No
text string No
audio string No
transcript string No

OpenAI.RealtimeConversationItemMessageSystem

A system message in a Realtime conversation can be used to provide additional context or instructions to the model. This is similar but distinct from the instruction prompt provided at the start of a conversation, as system messages can be added at any point in the conversation. For major changes to the conversation's behavior, use instructions, but for smaller updates (e.g. "the user is now asking about a different topic"), use system messages.

All of: OpenAI.RealtimeConversationItemMessage.

Property Type Required Description
id string No The unique ID of the item. This may be provided by the client or generated by the server.
object "realtime.item" No Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
type "message" Yes The type of the item. Always message.
status "completed" or "incomplete" or "in_progress" No The status of the item. Has no effect on the conversation.
role "system" Yes The role of the message sender. Always system.
content Array of OpenAI.RealtimeConversationItemMessageSystemContent Yes The content of the message.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeConversationItemMessageSystemContent

Property Type Required Description
type "input_text" No
text string No

OpenAI.RealtimeConversationItemMessageType

Any of: string, "system" or "user" or "assistant".

OpenAI.RealtimeConversationItemMessageUser

A user message item in a Realtime conversation.

All of: OpenAI.RealtimeConversationItemMessage.

Property Type Required Description
id string No The unique ID of the item. This may be provided by the client or generated by the server.
object "realtime.item" No Identifier for the API object being returned - always realtime.item. Optional when creating a new item.
type "message" Yes The type of the item. Always message.
status "completed" or "incomplete" or "in_progress" No The status of the item. Has no effect on the conversation.
role "user" Yes The role of the message sender. Always user.
content Array of OpenAI.RealtimeConversationItemMessageUserContent Yes The content of the message.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeConversationItemMessageUserContent

Property Type Required Description
type "input_text" or "input_audio" or "input_image" No
text string No
audio string No
image_url string No format: "uri".
detail "auto" or "low" or "high" No default: "auto".
transcript string No

OpenAI.RealtimeConversationItemType

Any of: string, "function_call" or "function_call_output" or "mcp_approval_response" or "mcp_list_tools" or "mcp_call" or "mcp_approval_request" or "message", VoiceAgentConversationItemTypeExtension.

OpenAI.RealtimeFunctionTool

Property Type Required Description
type "function" No The type of the tool, i.e. function.
name string No The name of the function.
description string No The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
parameters OpenAI.RealtimeFunctionToolParameters No Parameters of the function in JSON Schema.

OpenAI.RealtimeFunctionToolParameters

OpenAI.RealtimeMCPApprovalRequest

A Realtime item requesting human approval of a tool invocation.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
type "mcp_approval_request" Yes The type of the item. Always mcp_approval_request.
id string Yes The unique ID of the approval request.
server_label string Yes The label of the MCP server making the request.
name string Yes The name of the tool to run.
arguments string Yes A JSON string of arguments for the tool.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeMCPApprovalResponse

A Realtime item responding to an MCP approval request.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
type "mcp_approval_response" Yes The type of the item. Always mcp_approval_response.
id string Yes The unique ID of the approval response.
approval_request_id string Yes The ID of the approval request being answered.
approve boolean Yes Whether the request was approved.
reason Any of: string; null No
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeMCPError

Variants by type: "protocol_error": OpenAI.RealtimeMCPProtocolError; "tool_execution_error": OpenAI.RealtimeMCPToolExecutionError; "http_error": OpenAI.RealtimeMCPHTTPError.

Property Type Required Description
type OpenAI.RealtimeMcpErrorType Yes

OpenAI.RealtimeMCPHTTPError

All of: OpenAI.RealtimeMCPError.

Property Type Required Description
type "http_error" Yes
code OpenAI.integer Yes
message string Yes

OpenAI.RealtimeMCPListTools

A Realtime item listing tools available on an MCP server.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
type "mcp_list_tools" Yes The type of the item. Always mcp_list_tools.
id string No The unique ID of the list.
server_label string Yes The label of the MCP server.
tools Array of OpenAI.MCPListToolsTool Yes The tools available on the server.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeMCPProtocolError

All of: OpenAI.RealtimeMCPError.

Property Type Required Description
type "protocol_error" Yes
code OpenAI.integer Yes
message string Yes

OpenAI.RealtimeMCPToolCall

A Realtime item representing an invocation of a tool on an MCP server.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
type "mcp_call" Yes The type of the item. Always mcp_call.
id string Yes The unique ID of the tool call.
server_label string Yes The label of the MCP server running the tool.
name string Yes The name of the tool that was run.
arguments string Yes A JSON string of the arguments passed to the tool.
approval_request_id Any of: string; null No
output Any of: string; null No
error OpenAI.RealtimeMCPError No
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

OpenAI.RealtimeMCPToolExecutionError

All of: OpenAI.RealtimeMCPError.

Property Type Required Description
type "tool_execution_error" Yes
message string Yes

OpenAI.RealtimeMcpErrorType

Any of: string, "protocol_error" or "tool_execution_error" or "http_error".

OpenAI.RealtimeReasoning

Configuration for reasoning-capable Realtime models such as gpt-realtime-2.

Property Type Required Description
effort OpenAI.RealtimeReasoningEffort No

OpenAI.RealtimeReasoningEffort

Constrains effort on reasoning for reasoning-capable Realtime models such as gpt-realtime-2.

Type: "minimal" or "low" or "medium" or "high" or "xhigh".

OpenAI.RealtimeResponseInputAudio

Property Type Required Description
type "input_audio" Yes
audio string Yes Base64 audio bytes in the session input audio format. minLength: 1.
transcript Any of: string; null No Optional text to associate with the input audio.

OpenAI.RealtimeResponseInputFunctionCall

A function call supplied as input to a Realtime response.

Property Type Required Description
id Any of: string; null No minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "function_call" Yes
status Any of: "completed" or "incomplete"; null; null No default: "completed".
call_id string Yes minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
name string Yes minLength: 1. maxLength: 64. pattern: "^[a-zA-Z0-9_-]+$".
arguments string Yes maxLength: 1048576.

OpenAI.RealtimeResponseInputFunctionCallOutput

The result of a function call supplied as input to a Realtime response.

Property Type Required Description
id Any of: string; null No minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "function_call_output" Yes
status Any of: "completed" or "incomplete"; null; null No default: "completed".
call_id string Yes minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
output string Yes maxLength: 1048576.

OpenAI.RealtimeResponseInputImage

Property Type Required Description
type "input_image" Yes
image_url string Yes A base64 data URL with an image MIME type; use PNG or JPEG. format: "uri". pattern: "^data:image/[^;,]+;base64,.+".
detail Any of: "low" or "high" or "auto"; null; null No default: "auto".

OpenAI.RealtimeResponseInputItem

An existing item reference or a new message or function call item for a Realtime response.

Any of: OpenAI.RealtimeResponseInputMessage, OpenAI.RealtimeResponseInputFunctionCall, OpenAI.RealtimeResponseInputFunctionCallOutput, OpenAI.RealtimeResponseItemReference.

OpenAI.RealtimeResponseInputMessage

A message supplied as input to a Realtime response. The role determines the allowed content.

Variants by role: "system": OpenAI.RealtimeResponseInputMessageSystem; "developer": OpenAI.RealtimeResponseInputMessageDeveloper; "user": OpenAI.RealtimeResponseInputMessageUser; "assistant": OpenAI.RealtimeResponseInputMessageAssistant.

Property Type Required Description
role OpenAI.RealtimeResponseInputMessageType Yes

OpenAI.RealtimeResponseInputMessageAssistant

All of: OpenAI.RealtimeResponseInputMessage.

Property Type Required Description
id Any of: string; null No An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "message" Yes
status Any of: "completed" or "incomplete"; null; null No Optional input status. Omission or null defaults to completed. default: "completed".
role "assistant" Yes
content Array of OpenAI.RealtimeResponseInputMessageAssistantContent Yes minItems: 1.

OpenAI.RealtimeResponseInputMessageAssistantContent

Property Type Required Description
type "output_text" Yes
text string Yes maxLength: 256000.

OpenAI.RealtimeResponseInputMessageDeveloper

All of: OpenAI.RealtimeResponseInputMessage.

Property Type Required Description
id Any of: string; null No An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "message" Yes
status Any of: "completed" or "incomplete"; null; null No Optional input status. Omission or null defaults to completed. default: "completed".
role "developer" Yes
content Array of OpenAI.RealtimeResponseInputText Yes minItems: 1. maxItems: 1.

OpenAI.RealtimeResponseInputMessageSystem

All of: OpenAI.RealtimeResponseInputMessage.

Property Type Required Description
id Any of: string; null No An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "message" Yes
status Any of: "completed" or "incomplete"; null; null No Optional input status. Omission or null defaults to completed. default: "completed".
role "system" Yes
content Array of OpenAI.RealtimeResponseInputText Yes minItems: 1. maxItems: 1.

OpenAI.RealtimeResponseInputMessageType

Any of: string, "system" or "developer" or "user" or "assistant".

OpenAI.RealtimeResponseInputMessageUser

All of: OpenAI.RealtimeResponseInputMessage.

Property Type Required Description
id Any of: string; null No An optional client ID. The value root is reserved. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".
type "message" Yes
status Any of: "completed" or "incomplete"; null; null No Optional input status. Omission or null defaults to completed. default: "completed".
role "user" Yes
content Array of Any of: OpenAI.RealtimeResponseInputText; OpenAI.RealtimeResponseInputAudio; OpenAI.RealtimeResponseInputImage Yes minItems: 1.

OpenAI.RealtimeResponseInputText

Property Type Required Description
type "input_text" Yes
text string Yes maxLength: 256000.

OpenAI.RealtimeResponseItemReference

Reference an existing item in this Realtime session as response.create input. Item references are only accepted in the response input array, not as new conversation items.

Property Type Required Description
type "item_reference" Yes The type of this item. Always item_reference.
id string Yes The ID of an existing item in the same Realtime session. minLength: 1. maxLength: 32. pattern: "^[a-zA-Z0-9_-]+$".

OpenAI.RealtimeResponseStatusDetails

Property Type Required Description
type "completed" or "cancelled" or "failed" or "incomplete" No
reason "turn_detected" or "client_cancelled" or "max_output_tokens" or "content_filter" No
error OpenAI.RealtimeResponseStatusDetailsError No

OpenAI.RealtimeResponseStatusDetailsError

Property Type Required Description
type string No
code string No
message string No

OpenAI.RealtimeResponseUsage

Property Type Required Description
total_tokens OpenAI.integer No
input_tokens OpenAI.integer No
output_tokens OpenAI.integer No
input_token_details OpenAI.RealtimeResponseUsageInputTokenDetails No
output_token_details OpenAI.RealtimeResponseUsageOutputTokenDetails No

OpenAI.RealtimeResponseUsageInputTokenDetails

Property Type Required Description
cached_tokens OpenAI.integer No
text_tokens OpenAI.integer No
image_tokens OpenAI.integer No
audio_tokens OpenAI.integer No
cached_tokens_details OpenAI.RealtimeResponseUsageInputTokenDetailsCachedTokensDetails No

OpenAI.RealtimeResponseUsageInputTokenDetailsCachedTokensDetails

Property Type Required Description
text_tokens OpenAI.integer No
image_tokens OpenAI.integer No
audio_tokens OpenAI.integer No

OpenAI.RealtimeResponseUsageOutputTokenDetails

Property Type Required Description
text_tokens OpenAI.integer No
audio_tokens OpenAI.integer No

OpenAI.RealtimeServerEvent

A realtime server event.

Property Type Required Description
type OpenAI.RealtimeServerEventType Yes

OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionFailedError

Property Type Required Description
type string No
code string No
message string No
param string No

OpenAI.RealtimeServerEventErrorError

Property Type Required Description
type string Yes
code Any of: string; null No
message string Yes
param Any of: string; null No
event_id Any of: string; null No

OpenAI.RealtimeServerEventRateLimitsUpdatedRateLimits

Property Type Required Description
name "requests" or "tokens" No
limit OpenAI.integer No
remaining OpenAI.integer No
reset_seconds OpenAI.numeric No

OpenAI.RealtimeServerEventResponseContentPartAddedPart

Property Type Required Description
type "audio" or "text" No
text string No
audio string No
transcript string No

OpenAI.RealtimeServerEventResponseContentPartDonePart

Property Type Required Description
type "audio" or "text" No
text string No
audio string No
transcript string No
format OpenAI.RealtimeAudioFormats No The audio format, when this is an audio content part.

OpenAI.RealtimeServerEventType

Any of: string, "conversation.created" or "conversation.item.created" or "conversation.item.deleted" or "conversation.item.input_audio_transcription.completed" or "conversation.item.input_audio_transcription.delta" or "conversation.item.input_audio_transcription.failed" or "conversation.item.retrieved" or "conversation.item.truncated" or "error" or "input_audio_buffer.cleared" or "input_audio_buffer.committed" or "input_audio_buffer.dtmf_event_received" or "input_audio_buffer.speech_started" or "input_audio_buffer.speech_stopped" or "rate_limits.updated" or "response.output_audio.delta" or "response.output_audio.done" or "response.output_audio_transcript.delta" or "response.output_audio_transcript.done" or "response.content_part.added" or "response.content_part.done" or "response.created" or "response.done" or "response.function_call_arguments.delta" or "response.function_call_arguments.done" or "response.output_item.added" or "response.output_item.done" or "response.output_text.delta" or "response.output_text.done" or "session.created" or "session.updated" or "output_audio_buffer.started" or "output_audio_buffer.stopped" or "output_audio_buffer.cleared" or "conversation.item.added" or "conversation.item.done" or "input_audio_buffer.timeout_triggered" or "conversation.item.input_audio_transcription.segment" or "mcp_list_tools.in_progress" or "mcp_list_tools.completed" or "mcp_list_tools.failed" or "response.mcp_call_arguments.delta" or "response.mcp_call_arguments.done" or "response.mcp_call.in_progress" or "response.mcp_call.completed" or "response.mcp_call.failed", Azure.AI.ProjectsInternal.VoiceAgentServerEventTypeExtension.

OpenAI.SpecificApplyPatchParam

Forces the model to call the apply_patch tool when executing a tool call.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "apply_patch" Yes The tool to call. Always apply_patch.

OpenAI.SpecificFunctionShellParam

Forces the model to call the shell tool when a tool call is required.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "shell" Yes The tool to call. Always shell.

OpenAI.SpecificProgrammaticToolCallingParam

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "programmatic_tool_calling" Yes The tool to call. Always programmatic_tool_calling.

OpenAI.ToolChoiceAllowed

Constrains the tools available to the model to a pre-defined set.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "allowed_tools" Yes Allowed tool configuration type. Always allowed_tools.
mode "auto" or "required" Yes Constrains the tools available to the model to a pre-defined set. auto allows the model to pick from among the allowed tools and generate a message. required requires the model to call one or more of the allowed tools.
tools Array of object with additional values of any Yes A list of tool definitions that the model should be allowed to call. For the Responses API, the list of tool definitions might look like: json [ { "type": "function", "name": "get_weather" }, { "type": "mcp", "server_label": "deepwiki" }, { "type": "image_generation" } ]

OpenAI.ToolChoiceCodeInterpreter

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "code_interpreter" Yes

OpenAI.ToolChoiceComputer

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "computer" Yes

OpenAI.ToolChoiceComputerUse

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "computer_use" Yes

OpenAI.ToolChoiceComputerUsePreview

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "computer_use_preview" Yes

OpenAI.ToolChoiceCustom

Use this option to force the model to call a specific custom tool.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "custom" Yes For custom tool calling, the type is always custom.
name string Yes The name of the custom tool to call.

OpenAI.ToolChoiceFileSearch

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "file_search" Yes

OpenAI.ToolChoiceFunction

Use this option to force the model to call a specific function.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "function" Yes For function calling, the type is always function.
name string Yes The name of the function to call.

OpenAI.ToolChoiceImageGeneration

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "image_generation" Yes

OpenAI.ToolChoiceMCP

Use this option to force the model to call a specific tool on a remote MCP server.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "mcp" Yes For MCP tools, the type is always mcp.
server_label string Yes The label of the MCP server to use.
name Any of: string; null No

OpenAI.ToolChoiceOptions

Controls which (if any) tool is called by the model. none means the model will not call any tool and instead generates a message. auto means the model can pick between generating a message or calling one or more tools. required means the model must call one or more tools.

Type: "none" or "auto" or "required".

OpenAI.ToolChoiceParam

How the model should select which tool (or tools) to use when generating a response. See the tools parameter to see how to specify which tools the model can call.

Variants by type: "function": OpenAI.ToolChoiceFunction; "mcp": OpenAI.ToolChoiceMCP; "allowed_tools": OpenAI.ToolChoiceAllowed; "custom": OpenAI.ToolChoiceCustom; "programmatic_tool_calling": OpenAI.SpecificProgrammaticToolCallingParam; "apply_patch": OpenAI.SpecificApplyPatchParam; "shell": OpenAI.SpecificFunctionShellParam; "file_search": OpenAI.ToolChoiceFileSearch; "web_search_preview": OpenAI.ToolChoiceWebSearchPreview; "computer_use_preview": OpenAI.ToolChoiceComputerUsePreview; "web_search_preview_2025_03_11": OpenAI.ToolChoiceWebSearchPreview20250311; "image_generation": OpenAI.ToolChoiceImageGeneration; "code_interpreter": OpenAI.ToolChoiceCodeInterpreter; "computer": OpenAI.ToolChoiceComputer; "computer_use": OpenAI.ToolChoiceComputerUse.

Property Type Required Description
type OpenAI.ToolChoiceParamType Yes

OpenAI.ToolChoiceParamType

Any of: string, "allowed_tools" or "function" or "mcp" or "custom" or "programmatic_tool_calling" or "apply_patch" or "shell" or "file_search" or "web_search_preview" or "computer_use_preview" or "web_search_preview_2025_03_11" or "image_generation" or "code_interpreter" or "computer" or "computer_use".

OpenAI.ToolChoiceWebSearchPreview

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "web_search_preview" Yes

OpenAI.ToolChoiceWebSearchPreview20250311

Indicates that the model should use a built-in tool to generate a response. Learn more about built-in tools.

All of: OpenAI.ToolChoiceParam.

Property Type Required Description
type "web_search_preview_2025_03_11" Yes

OpenAI.TranscriptTextUsageDuration

Usage statistics for models billed by audio input duration.

All of: OpenAI.CreateTranscriptionResponseJsonUsage.

Property Type Required Description
type "duration" Yes The type of the usage object. Always duration for this variant.
seconds integer Yes Duration of the input audio in seconds. format: "int64".

OpenAI.TranscriptTextUsageTokens

Usage statistics for models billed by token usage.

All of: OpenAI.CreateTranscriptionResponseJsonUsage.

Property Type Required Description
type "tokens" Yes The type of the usage object. Always tokens for this variant.
input_tokens OpenAI.integer Yes Number of input tokens billed for this request.
input_token_details OpenAI.TranscriptTextUsageTokensInputTokenDetails No Details about the input tokens billed for this request.
output_tokens OpenAI.integer Yes Number of output tokens generated.
total_tokens OpenAI.integer Yes Total number of tokens used (input + output).

OpenAI.TranscriptTextUsageTokensInputTokenDetails

Property Type Required Description
text_tokens OpenAI.integer No
audio_tokens OpenAI.integer No

OpenAI.TranscriptionLanguage

A language detected in transcribed audio.

Property Type Required Description
code string Yes The code of a language detected in the audio.

OpenAI.integer

Type: integer.

format: "int64".

OpenAI.numeric

Type: number.

format: "double".

ToolConfig

Per-tool configuration that controls tool visibility and search behavior.

Property Type Required Description
pin boolean No When true, the tool is always included in agent context and visible in tools/list. When false (default), the tool is hidden from tools/list and only discoverable via tool_search.
additional_search_text string No Additional text indexed for tool_search. Supplements the native tool description to improve discoverability. Does not alter tools/list output. maxLength: 1000.

VoiceAgentAnimationConfig

Animation settings for a voice-agent session.

Property Type Required Description
model_name string No The animation model name. default: "default".
outputs Array of VoiceAgentAnimationOutputType No The requested animation output kinds. default: ["blendshapes"].

VoiceAgentAnimationOutputType

An animation output produced by a voice-agent session.

Type: "blendshapes" or "viseme_id".

VoiceAgentAudioConfig

The audio configuration for a voice agent. These values are session defaults and may be overridden when connecting.

Property Type Required Description
input VoiceAgentAudioInputConfig No Input (microphone) audio configuration.
output VoiceAgentAudioOutputConfig No Output (agent speech) audio configuration.

VoiceAgentAudioInputConfig

Input audio configuration for a voice agent.

Property Type Required Description
format OpenAI.RealtimeAudioFormats No The input audio format.
noise_reduction Any of: VoiceAgentNoiseReduction; null No Input noise reduction. Set to null to disable.
turn_detection Any of: VoiceAgentTurnDetectionConfig; null No Turn (end-of-speech) detection. Server-side turn detection is enabled by default; set to null to disable it, in which case the client must trigger responses manually.
echo_cancellation Any of: VoiceAgentEchoCancellation; null No Optional server-side echo cancellation settings.
transcription Any of: VoiceAgentInputTranscription; null No Asynchronous input-audio transcription. Set to null to disable transcription.

VoiceAgentAudioOutputConfig

Output audio configuration for a voice agent. Extends the voice configuration with the session-level format and output_audio_timestamp_types settings, which apply to every voice type.

All of: VoiceAgentVoiceConfig.

Property Type Required Description
format OpenAI.RealtimeAudioFormats No The output audio format. Applies to every voice_type and defaults to 24 kHz PCM.
output_audio_timestamp_types Array of VoiceAgentAudioTimestampType No Timestamp kinds to include with output audio. Applies to every voice_type.

VoiceAgentAudioTimestampType

An output-audio timestamp kind supported by a voice agent.

Type: "word".

VoiceAgentAvatarConfig

Avatar configuration for a voice agent. These values are session defaults and may be overridden when connecting.

Property Type Required Description
type VoiceAgentAvatarType Yes The avatar type.
character string Yes The avatar character identifier, e.g. 'lisa'.
style string No The avatar style, e.g. 'casual-sitting'.
customized boolean No Whether the avatar is a customer-customized avatar. Defaults to false. default: false.
output_protocol VoiceAgentAvatarOutputProtocol No The transport used to deliver the avatar video stream.
model string No The avatar model identifier.
video VoiceAgentAvatarVideoParams No Avatar video encoder and presentation settings.
scene VoiceAgentAvatarScene No Avatar placement and motion settings.
output_audit_audio boolean No Whether audit audio is emitted with avatar output. Defaults to false. default: false.

VoiceAgentAvatarIceServer

An ICE server used for avatar WebRTC negotiation.

Property Type Required Description
urls Array of string (format: "uri".) Yes
username Any of: string; null No
credential Any of: string; null No format: "password".

VoiceAgentAvatarOutputProtocol

The transport used to deliver the avatar video stream.

Any of: string, "webrtc" or "websocket".

VoiceAgentAvatarScene

Avatar placement and motion settings.

Property Type Required Description
zoom number No format: "float". default: 1. exclusiveMinimum: 0.
position_x number No format: "float". default: 0. minimum: -1. maximum: 1.
position_y number No format: "float". default: 0. minimum: -1. maximum: 1.
rotation_x number No format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793.
rotation_y number No format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793.
rotation_z number No format: "float". default: 0. minimum: -3.141592653589793. maximum: 3.141592653589793.
amplitude number No format: "float". default: 1. maximum: 1. exclusiveMinimum: 0.

VoiceAgentAvatarType

The avatar type.

Any of: string, "video_avatar" or "photo_avatar".

VoiceAgentAvatarVideoBackground

The avatar video background.

Property Type Required Description
image_url string No format: "uri".
color string No

VoiceAgentAvatarVideoCrop

The rectangular crop applied to avatar video.

Property Type Required Description
bottom_right Array of integer (format: "uint32".) Yes minItems: 2. maxItems: 2.
top_left Array of integer (format: "uint32".) Yes minItems: 2. maxItems: 2.

VoiceAgentAvatarVideoParams

Avatar video encoder and presentation settings.

Property Type Required Description
bitrate integer No The target video bitrate in bits per second. format: "int32". default: 2000000.
crop VoiceAgentAvatarVideoCrop No
resolution VoiceAgentAvatarVideoResolution No
background VoiceAgentAvatarVideoBackground No
gop_size integer No format: "int32". default: 10. minimum: 1. maximum: 2000.

VoiceAgentAvatarVideoResolution

The avatar video resolution.

Property Type Required Description
width integer Yes format: "int32". minimum: 1.
height integer Yes format: "int32". minimum: 1.

VoiceAgentAzureSemanticVadEnTurnDetection

English-optimized Azure semantic voice activity detection.

All of: VoiceAgentTurnDetectionConfig.

Property Type Required Description
type "azure_semantic_vad_en" Yes
threshold number No Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1.
prefix_padding_ms FoundryDurationMilliseconds No Audio to include before detected speech, in milliseconds.
silence_duration_ms FoundryDurationMilliseconds No Silence required to end speech detection, in milliseconds.
idle_timeout_ms FoundryDurationMilliseconds No Maximum idle time before the detector ends the turn, in milliseconds.
end_of_utterance_detection Any of: VoiceAgentEndOfUtteranceDetection; null No Semantic end-of-utterance detection configuration. Set to null to disable it.
speech_duration_ms FoundryDurationMilliseconds No Minimum speech duration required to trigger detection, in milliseconds.
remove_filler_words boolean No Whether filler words are removed from transcription. default: false.
create_response boolean No Whether a response is created automatically when speech stops. default: true.
interrupt_response boolean No Whether user speech may interrupt the agent's response. default: true.

VoiceAgentAzureSemanticVadMultilingualTurnDetection

Multilingual Azure semantic voice activity detection.

All of: VoiceAgentTurnDetectionConfig.

Property Type Required Description
type "azure_semantic_vad_multilingual" Yes
threshold number No Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1.
prefix_padding_ms FoundryDurationMilliseconds No Audio to include before detected speech, in milliseconds.
silence_duration_ms FoundryDurationMilliseconds No Silence required to end speech detection, in milliseconds.
idle_timeout_ms FoundryDurationMilliseconds No Maximum idle time before the detector ends the turn, in milliseconds.
end_of_utterance_detection Any of: VoiceAgentEndOfUtteranceDetection; null No Semantic end-of-utterance detection configuration. Set to null to disable it.
speech_duration_ms FoundryDurationMilliseconds No Minimum speech duration required to trigger detection, in milliseconds.
remove_filler_words boolean No Whether filler words are removed from transcription. default: false.
create_response boolean No Whether a response is created automatically when speech stops. default: true.
interrupt_response boolean No Whether user speech may interrupt the agent's response. default: true.
languages Array of string No BCP-47 language codes used for speech detection.

VoiceAgentAzureSemanticVadTurnDetection

Azure semantic voice activity detection.

All of: VoiceAgentTurnDetectionConfig.

Property Type Required Description
type "azure_semantic_vad" Yes
threshold number No Activation threshold for voice activity detection, from 0 to 1. format: "float". minimum: 0. maximum: 1.
prefix_padding_ms FoundryDurationMilliseconds No Audio to include before detected speech, in milliseconds.
silence_duration_ms FoundryDurationMilliseconds No Silence required to end speech detection, in milliseconds.
idle_timeout_ms FoundryDurationMilliseconds No Maximum idle time before the detector ends the turn, in milliseconds.
end_of_utterance_detection Any of: VoiceAgentEndOfUtteranceDetection; null No Semantic end-of-utterance detection configuration. Set to null to disable it.
speech_duration_ms FoundryDurationMilliseconds No Minimum speech duration required to trigger detection, in milliseconds.
remove_filler_words boolean No Whether filler words are removed from transcription. default: false.
create_response boolean No Whether a response is created automatically when speech stops. default: true.
interrupt_response boolean No Whether user speech may interrupt the agent's response. default: true.
languages Array of string No BCP-47 language codes used for speech detection.

VoiceAgentCollectDtmfSystemTool

Collects keypad digits on the current call.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "collect_dtmf" Yes The service-managed control action. Always collect_dtmf.

VoiceAgentCollectSmsSystemTool

Collects one complete SMS from an explicitly selected sender.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "collect_sms" Yes The service-managed control action. Always collect_sms.

VoiceAgentConversationItemTypeExtension

Additional item kinds used by the Voice Agent service.

Type: "system_tool".

VoiceAgentDigitRecognitionSystemTool

Retrieves the unchanged transcript of the last user audio turn.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "digit_recognition" Yes The service-managed control action. Always digit_recognition.

VoiceAgentEchoCancellation

Server-side echo cancellation settings for input audio.

Property Type Required Description
type "server_echo_cancellation" Yes The echo cancellation implementation. Always server_echo_cancellation.
reference_source VoiceAgentEchoCancellationReferenceSource No Whether reference audio comes from server playback or a client-provided channel. default: "server".
channels integer No The number of input channels. Use two interleaved channels when reference_source is client. format: "int32". default: 1. minimum: 1. maximum: 2.

VoiceAgentEchoCancellationReferenceSource

The source of reference audio used for echo cancellation.

Type: "server" or "client".

VoiceAgentEndConversationSystemTool

A service-managed control that ends the active conversation.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "end_conversation" Yes The service-managed control action. Always end_conversation.

VoiceAgentEndOfUtteranceDetection

Semantic end-of-utterance detection configuration.

Property Type Required Description
model VoiceAgentEndOfUtteranceDetectionModel Yes The semantic detection model.
threshold_level VoiceAgentEndOfUtteranceThresholdLevel No The sensitivity threshold.
timeout_ms FoundryDurationMilliseconds No The detection timeout in milliseconds.

VoiceAgentEndOfUtteranceDetectionModel

The semantic end-of-utterance detection model.

Any of: string, "semantic_detection_v1" or "semantic_detection_v1_en" or "semantic_detection_v1_multilingual" or "smart_end_of_turn_detection".

VoiceAgentEndOfUtteranceThresholdLevel

The sensitivity threshold for semantic end-of-utterance detection.

Any of: string, "low" or "medium" or "high" or "default".

VoiceAgentFunctionTool

A native function tool executed by the client.

All of: VoiceAgentTool.

Property Type Required Description
description string No The description of the function, including guidance on when and how to call it, and guidance about what to tell the user when calling (if anything).
parameters OpenAI.RealtimeFunctionToolParameters No Parameters of the function in JSON Schema.
type "function" Yes
name string Yes The function name. minLength: 1.

VoiceAgentGreetingConfig

Session-start greeting configuration for a voice agent.

Variants by type: "template": VoiceAgentTemplateGreetingConfig; "llm_generated": VoiceAgentLlmGeneratedGreetingConfig.

Property Type Required Description
type VoiceAgentGreetingMode Yes The greeting mode.

VoiceAgentGreetingMode

The greeting modes supported by a voice agent.

Any of: string, "template" or "llm_generated".

VoiceAgentHandoffAbortReason

Why a server-managed handoff ended before target commit.

Type: "user_interruption" or "error".

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentHandoffEdgeState

Non-sensitive topology metadata for one compiled handoff edge.

Property Type Required Description
id VoiceAgentHandoffId Yes The edge identifier.
source VoiceAgentHandoffNodeReference Yes The source node identifier or $entrypoint.
target VoiceAgentHandoffNodeReference Yes The target node identifier or $entrypoint.
cancel_on_interruption boolean No Whether user interruption before commit cancels the pending handoff. default: false.
delay_ms FoundryDurationMilliseconds No The minimum delay before target activation, in milliseconds. default: 0.
transfer_message Any of: string; null No The deterministic source-voice announcement played before target activation. maxLength: 500.
target_response VoiceAgentHandoffTargetResponse No Whether Voice Live automatically starts a response after target activation. default: "auto".

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentHandoffId

A case-sensitive identifier for an inlined subagent or handoff condition.

Type: string.

pattern: "^[a-z][a-z0-9_-]{0,63}$".

VoiceAgentHandoffNodeReference

A handoff node reference: either $entrypoint or a configured node identifier.

Type: string.

pattern: "^(?:\\$entrypoint|[a-z][a-z0-9_-]{0,63})$".

VoiceAgentHandoffNodeState

Non-sensitive topology metadata for one compiled handoff node.

Property Type Required Description
id VoiceAgentHandoffNodeReference Yes The node identifier or $entrypoint.
description string Yes The display-only node description. This value is not provided to the model.
implicit boolean No Whether this is the implicit $entrypoint node. default: false.

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentHandoffPipelineFamily

The immutable pipeline family selected for a voice-agent handoff graph.

Type: "cascaded" or "realtime".

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentHandoffState

The effective topology and runtime state of a voice-agent handoff graph.

Property Type Required Description
pipeline_family VoiceAgentHandoffPipelineFamily Yes The immutable execution pipeline family.
active_node_id VoiceAgentHandoffNodeReference Yes The currently committed node identifier or $entrypoint.
node_generation integer Yes The monotonically increasing generation of the committed active-node configuration. format: "int32". minimum: 1.
transfer_count integer Yes The number of successfully completed handoffs. format: "int32". minimum: 0.
attempt_count integer Yes The number of unique reserved handoff attempts. format: "int32". minimum: 0.
available_edge_ids Array of VoiceAgentHandoffId Yes The edge identifiers currently available from the active node.
transfer_tool Any of: VoiceAgentFunctionTool; null Yes The generated handoff function visible to the active model, or null when no route is available.
nodes Array of VoiceAgentHandoffNodeState Yes The compiled handoff nodes, including the implicit or explicit $entrypoint.
edges Array of VoiceAgentHandoffEdgeState Yes The compiled handoff edges.

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentHandoffTargetResponse

Whether Voice Live automatically starts a response after a handoff target becomes active.

Type: "auto" or "none".

Required preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentInputCollectionChannel

The receiver used by an input collection. Voice collection is not supported.

Type: "sms" or "dtmf".

VoiceAgentInputCollectionStatus

Collection lifecycle and value-free telephony-operation outcomes. Additional outcomes may be introduced over time.

Any of: string, "accepted" or "listening" or "completed" or "expired" or "cancelled" or "failed" or "busy" or "invalid_input" or "invalid_arguments" or "stale_collection" or "unsupported_channel" or "dispatch_failed" or "provider_accepted" or "transfer_not_answered" or "provider_failed_or_unsupported" or "provider_result_uncertain" or "provider_not_executed".

VoiceAgentInputTranscription

Settings for transcribing input audio, including the transcription model, Custom Speech deployments, and phrase hints. When model is azure-speech-streaming, transcription is supported for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.

Property Type Required Description
language string No The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency. Optional for other transcription models; required and must be nonempty when model is azure-speech-streaming (such as en-US; comma-separated locales are supported).
languages Array of string No Possible languages of the input audio, in ISO-639-1 format. Supported by gpt-transcribe and gpt-live-transcribe. minItems: 1.
keywords Array of string No Words or phrases to guide transcription of the input audio. Supported by gpt-transcribe and gpt-live-transcribe.
prompt string No An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.
delay "minimal" or "low" or "medium" or "high" or "xhigh" No Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
model VoiceAgentInputTranscriptionModel Yes The transcription model identifier. Configure customer custom speech deployments in custom_speech. azure-speech-streaming is for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.
custom_speech object with additional values of string No Optional customer custom speech deployment configuration, keyed by locale.
phrase_list Array of string No Optional phrase hints that bias recognition toward domain terms.
profanity_filter VoiceAgentProfanityFilterMode No Controls how profanity is handled in input transcripts. Supported only with managed Azure Speech (azure-speech and azure-speech-streaming), including Custom Speech. Set this property before the session starts. If omitted, the transcription provider's default applies. You cannot change the mode during a session. Filtering affects transcript events and the text sent to cascaded models. It does not change input audio, voice activity detection, or assistant speech; audio-native models still receive the original audio. Mandatory service masking policies apply even when you select unfiltered.

VoiceAgentInputTranscriptionModel

The input-audio transcription model identifier. This is a model name, not a Foundry deployment name. Mirrors the transcription models supported by the managed voice backend, covering the OpenAI Realtime transcription models plus the Azure and MAI models. azure-speech-streaming is supported for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter. Additional values may be added over time.

Any of: string, "whisper-1" or "gpt-realtime-whisper" or "gpt-4o-transcribe" or "gpt-4o-mini-transcribe" or "gpt-4o-transcribe-diarize" or "gpt-transcribe" or "gpt-live-transcribe" or "mai-transcribe" or "azure-speech" or "azure-speech-streaming".

VoiceAgentInterimResponseConfig

Fields shared by interim-response configurations.

Variants by type: "static_interim_response": VoiceAgentStaticInterimResponseConfig; "llm_interim_response": VoiceAgentLlmInterimResponseConfig.

Property Type Required Description
type VoiceAgentInterimResponseType Yes The interim-response implementation.
triggers Array of VoiceAgentInterimResponseTrigger No Conditions that may trigger one interim response. default: ["latency"].
latency_threshold_ms integer No The latency threshold in milliseconds. format: "int32". default: 2000.

VoiceAgentInterimResponseTrigger

A condition that may trigger an interim response.

Any of: string, "latency" or "tool".

VoiceAgentInterimResponseType

The interim-response implementations supported by a voice agent.

Any of: string, "static_interim_response" or "llm_interim_response".

VoiceAgentLanguageDetectionSystemTool

Switches supported conversation language settings.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "language_detection" Yes The service-managed control action. Always language_detection.

VoiceAgentLlmGeneratedGreetingConfig

A greeting authored by the session model from a scoped opening-turn prompt.

All of: VoiceAgentGreetingConfig.

Property Type Required Description
type "llm_generated" Yes
prompt string Yes The Handlebars prompt that guides the opening turn. minLength: 1.
tool_choice VoiceAgentToolChoice No The tool-selection policy for the opening response. Defaults to none. default: "none".

VoiceAgentLlmInterimResponseConfig

An interim response generated by a language model.

All of: VoiceAgentInterimResponseConfig.

Property Type Required Description
type "llm_interim_response" Yes
model string No The model used to generate interim responses.
instructions string No Optional instructions for generating interim responses.
max_completion_tokens integer No The maximum completion-token count for an interim response. format: "int32". default: 50. minimum: 1.

VoiceAgentMaxOutputTokens

The maximum output-token count or the literal inf.

Any of: integer, "inf".

VoiceAgentMcpTool

An MCP tool available to a voice agent. Server endpoints use server_url; connector and tunnel identifiers are not supported.

All of: VoiceAgentTool.

Property Type Required Description
server_label string Yes A label for this MCP server, used to identify it in tool calls.
server_url string No The URL for the MCP server. One of server_url, connector_id, or tunnel_id must be provided. format: "uri".
authorization string No An OAuth access token that can be used with a remote MCP server, either with a custom MCP server URL or a service connector. Your application must handle the OAuth authorization flow and provide the token here.
server_description string No Optional description of the MCP server, used to provide more context.
headers Any of: object with additional values of string; null No
allowed_tools Any of: Array of string; OpenAI.MCPToolFilter; null No
allowed_callers Any of: Array of OpenAI.CallableToolAllowedCaller; null No
require_approval Any of: OpenAI.MCPToolRequireApproval; "always" or "never"; null No default: "always".
defer_loading boolean No Whether this MCP tool is deferred and discovered via tool search.
project_connection_id string No The connection ID in the project for the MCP server. The connection stores authentication and other connection details needed to connect to the MCP server.
tool_configs object with additional values of ToolConfig No Deprecated. This property is deprecated and will be removed in a future version. deprecated: true.
type "mcp" Yes
response_scheduling VoiceAgentToolResponseScheduling No When the MCP invocation creates a follow-up response. Defaults to when_idle. default: "when_idle".

VoiceAgentNoiseReduction

Input audio noise reduction configuration. azure_deep_noise_suppression_headset and azure_personal_noise_suppression are modes for local voice activity detection that enhance detection audio without changing the audio used for recognition.

Property Type Required Description
type VoiceAgentNoiseReductionType Yes The noise reduction mode. azure_deep_noise_suppression_headset provides headset-optimized noise suppression for local voice activity detection, optimized for low-sample-rate audio including 8 kHz telephony audio without changing recognition audio (not restricted to headset devices). azure_personal_noise_suppression starts with headset suppression for local voice activity detection and personalizes detection after automatic speaker enrollment while keeping recognition audio unchanged and keeping headset suppression active when personalization is unavailable.

VoiceAgentNoiseReductionType

The input audio noise reduction mode. - azure_deep_noise_suppression_headset: Azure headset-optimized noise suppression for local voice activity detection. Optimized for low-sample-rate audio, including 8 kHz telephony audio. Enhances detection audio without changing the audio used for recognition. The mode is not restricted to headset devices. - azure_personal_noise_suppression: Azure personalized noise suppression for local voice activity detection. Starts with headset suppression and personalizes detection after automatic speaker enrollment. Recognition audio is unchanged; headset suppression remains active when personalization is unavailable.

Any of: string, "near_field" or "far_field" or "azure_deep_noise_suppression" or "azure_deep_noise_suppression_headset" or "azure_personal_noise_suppression".

VoiceAgentOutputReference

Bare session variable name, such as verification_code, without a namespace or braces. Tool arguments reference it with {{name}}.

Type: string.

pattern: "^[A-Za-z_][A-Za-z0-9_]*$".

VoiceAgentProfanityFilterMode

The supported input-transcript profanity filtering modes.

Any of: string, "masked" or "unfiltered" or "removed".

VoiceAgentRealtimeResponse

A live realtime response returned by the voice-agent service in both response.created and response.done events.

Property Type Required Description
id string No The unique ID of the response, will look like resp_1234.
object "realtime.response" No The object type, must be realtime.response.
status "completed" or "cancelled" or "failed" or "incomplete" or "in_progress" No The final status of the response (completed, cancelled, failed, or incomplete, in_progress).
status_details OpenAI.RealtimeResponseStatusDetails No Additional details about the status.
output Array of OpenAI.RealtimeConversationItem No The list of output items generated by the response.
metadata Any of: OpenAI.Metadata; null No
usage OpenAI.RealtimeResponseUsage No Usage statistics for the Response, this will correspond to billing. A Realtime API session will maintain a conversation context and append new Items to the Conversation, thus output from previous turns (text and audio tokens) will become the input for later turns.
conversation_id string No Which conversation the response is added to, determined by the conversation field in the response.create event. If auto, the response will be added to the default conversation and the value of conversation_id will be an id like conv_1234. If none, the response will not be added to any conversation and the value of conversation_id will be null. If responses are being triggered automatically by VAD the response will be added to the default conversation
output_modalities Array of "text" or "audio" No The set of modalities the model used to respond, currently the only possible values are [\"audio\"], [\"text\"]. Audio output always include a text transcript. Setting the output to mode text will disable audio output from the model.
max_output_tokens Exactly one of: OpenAI.integer; "inf" No Maximum number of output tokens for a single assistant response, inclusive of tool calls, that was used in this response.
audio VoiceResponseAudio No The audio configuration used by the live response, including flat voice provider, locale, and format fields under output.

VoiceAgentResponseCreateParams

Parameters accepted by a voice-agent response.create event.

Property Type Required Description
instructions string No The default system instructions (i.e. system message) prepended to model calls. This field allows the client to guide the model on desired responses. The model can be instructed on response content and format, (e.g. "be extremely succinct", "act friendly", "here are examples of good responses") and on audio behavior (e.g. "talk quickly", "inject emotion into your voice", "laugh frequently"). The instructions are not guaranteed to be followed by the model, but they provide guidance to the model on the desired behavior. Note that the server sets default instructions which will be used if this field is not set and are visible in the session.created event at the start of the session.
tools Array of Any of: OpenAI.RealtimeFunctionTool; OpenAI.MCPTool No Tools available to the model.
tool_choice Exactly one of: OpenAI.ToolChoiceOptions; OpenAI.ToolChoiceFunction; OpenAI.ToolChoiceMCP No How the model chooses tools. Provide one of the string modes or force a specific function/MCP tool. default: "auto".
parallel_tool_calls boolean No Whether the model may call multiple tools in parallel. Only supported by reasoning Realtime models such as gpt-realtime-2.
reasoning OpenAI.RealtimeReasoning No
max_output_tokens Exactly one of: OpenAI.integer; "inf" No Maximum number of output tokens for a single assistant response, inclusive of tool calls. Provide an integer between 1 and 4096 to limit output tokens, or inf for the maximum available tokens for a given model. Defaults to inf.
conversation Exactly one of: string; "auto" or "none" No Controls which conversation the response is added to. Currently supports auto and none, with auto as the default value. The auto value means that the contents of the response will be added to the default conversation. Set this to none to create an out-of-band response which will not add items to default conversation. default: "auto".
metadata Any of: OpenAI.Metadata; null No
input Array of OpenAI.RealtimeResponseInputItem No Input items to include in the prompt for the model. Using this field creates a new context for this Response instead of using the default conversation. An empty array [] will clear the context for this Response. Note that this can include references to items that previously appeared in the session using their id.
output_modalities Array of VoiceOutputModality No Modalities that the response may return.
audio object No Response-specific audio settings.
pre_generated_assistant_message Any of: OpenAI.RealtimeConversationItem; null No A pre-generated assistant message used to begin the response.
interim_response Any of: VoiceAgentInterimResponseConfig; null No Interim-response settings for this response.

audio properties

Property Type Required Description
output VoiceAgentAudioOutputConfig No Output (agent speech) audio configuration.

VoiceAgentRtcCallErrorDetails

Details of a WebRTC signaling error.

Property Type Required Description
type VoiceAgentRtcCallErrorType Yes The error category, following the VoiceLive wire contract: invalid_request_error for a client-side signaling fault (for example, a malformed SDP offer) or server_error for a service-side failure. Additional categories may be added over time.
code string No A machine-readable error code, when available.
message string Yes A human-readable error message.

VoiceAgentRtcCallErrorType

The error categories reported for WebRTC signaling failures.

Any of: string, "invalid_request_error" or "server_error".

VoiceAgentSemanticVadTurnDetection

OpenAI semantic VAD turn-detection settings.

All of: VoiceAgentTurnDetectionConfig.

Property Type Required Description
eagerness "low" or "medium" or "high" or "auto" No default: "auto".
create_response boolean No default: true.
interrupt_response boolean No default: true.
type "semantic_vad" Yes

VoiceAgentSendDtmfSystemTool

Sends literal or templated keypad digits.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "send_dtmf" Yes The service-managed control action. Always send_dtmf.

VoiceAgentSendSmsSystemTool

Sends a text message from the Agent's SMS-capable number.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "send_sms" Yes The service-managed control action. Always send_sms.

VoiceAgentServerEventWarningDetails

Details of a non-fatal warning.

Property Type Required Description
message string Yes
code string No
param string No

VoiceAgentServerVadTurnDetection

Server-side voice activity detection.

All of: VoiceAgentTurnDetectionConfig.

Property Type Required Description
threshold OpenAI.numeric No
prefix_padding_ms OpenAI.integer No
silence_duration_ms OpenAI.integer No
create_response boolean No default: true.
interrupt_response boolean No default: true.
idle_timeout_ms Any of: OpenAI.integer; null No
type "server_vad" Yes
speech_duration_ms FoundryDurationMilliseconds No Minimum speech duration required to trigger detection, in milliseconds.
end_of_utterance_detection Any of: VoiceAgentEndOfUtteranceDetection; null No Semantic end-of-utterance detection configuration. Set to null to disable it.

VoiceAgentSessionAvatarConfig

Avatar settings accepted by the stable voice-agent WebSocket contract.

All of: VoiceAgentAvatarConfig.

Property Type Required Description
ice_servers Any of: Array of VoiceAgentAvatarIceServer; null No

VoiceAgentSessionIncludeOption

Additional fields that a voice-agent session may include in service outputs.

Type: "item.input_audio_transcription.logprobs" or "item.input_audio_transcription.phrases" or "file_search_call.results".

VoiceAgentSessionResponse

The effective session configuration returned in voice-agent session lifecycle events.

VoiceAgentSessionResponseConfig

The effective stable realtime session settings returned by the voice-agent service.

Property Type Required Description
type "realtime" Yes The session type. Always realtime.
instructions string No Instructions applied throughout the session.
temperature number No The sampling temperature for compatible cascaded pipelines. format: "float". minimum: 0. maximum: 2.
max_output_tokens VoiceAgentMaxOutputTokens No The maximum output-token count for one response.
output_modalities Array of VoiceOutputModality No The output modalities enabled for the session.
audio VoiceAgentAudioConfig No The input- and output-audio settings for the session.
avatar VoiceAgentSessionAvatarConfig No The avatar settings for the session.
animation VoiceAgentAnimationConfig No Animation settings for the session.
tools Array of VoiceAgentTool No Tools available to the session.
tool_choice VoiceAgentToolChoice No Tool-selection behavior for the session.
reasoning OpenAI.RealtimeReasoning No Reasoning settings for compatible realtime models.
parallel_tool_calls boolean No Whether the model may call multiple tools in parallel.
include Array of VoiceAgentSessionIncludeOption No Additional fields to include in service outputs.
metadata object with additional values of string No Up to 16 string key-value pairs attached to the session.
interim_response VoiceAgentInterimResponseConfig No Interim-response settings for latency and tool execution.
greeting VoiceAgentGreetingConfig No A proactive assistant greeting started after session configuration.
handoff Any of: VoiceAgentHandoffState; null No The effective handoff topology and runtime state, when handoff is configured.
object "realtime.session" Yes The object type. Always realtime.session.
id string Yes The session identifier.
model string Yes The selected model.
expires_at Any of: FoundryTimestamp; null No The session expiration time as a Unix timestamp in seconds.

VoiceAgentSessionUpdate

The session configuration accepted by a voice-agent session update event.

VoiceAgentSessionUpdateConfig

The stable realtime session settings accepted in a session.update client event.

Property Type Required Description
type "realtime" Yes The session type. Always realtime.
instructions string No Instructions applied throughout the session.
temperature number No The sampling temperature for compatible cascaded pipelines. format: "float". minimum: 0. maximum: 2.
max_output_tokens VoiceAgentMaxOutputTokens No The maximum output-token count for one response.
output_modalities Array of VoiceOutputModality No The output modalities enabled for the session.
audio VoiceAgentAudioConfig No The input- and output-audio settings for the session.
avatar VoiceAgentSessionAvatarConfig No The avatar settings for the session.
animation VoiceAgentAnimationConfig No Animation settings for the session.
tools Array of VoiceAgentTool No Tools available to the session.
tool_choice VoiceAgentToolChoice No Tool-selection behavior for the session.
reasoning OpenAI.RealtimeReasoning No Reasoning settings for compatible realtime models.
parallel_tool_calls boolean No Whether the model may call multiple tools in parallel.
include Array of VoiceAgentSessionIncludeOption No Additional fields to include in service outputs.
metadata object with additional values of string No Up to 16 string key-value pairs attached to the session.
interim_response VoiceAgentInterimResponseConfig No Interim-response settings for latency and tool execution.
greeting VoiceAgentGreetingConfig No A proactive assistant greeting started after session configuration.

VoiceAgentStaticInterimResponseConfig

A static interim response selected from configured text.

All of: VoiceAgentInterimResponseConfig.

Property Type Required Description
type "static_interim_response" Yes
texts Array of string No Candidate text values for the interim response.

VoiceAgentSubagentAbortReason

The reason a subagent consultation was aborted.

Any of: string, "unknown_target" or "timeout" or "cancelled" or "stopped_by_user" or "superseded" or "failed".

VoiceAgentSystemTool

A service-managed control that acts on the active voice session without customer code or external authentication.

All of: VoiceAgentTool.

Variants by name: "end_conversation": VoiceAgentEndConversationSystemTool; "language_detection": VoiceAgentLanguageDetectionSystemTool; "digit_recognition": VoiceAgentDigitRecognitionSystemTool; "collect_sms": VoiceAgentCollectSmsSystemTool; "collect_dtmf": VoiceAgentCollectDtmfSystemTool; "transfer": VoiceAgentTransferSystemTool; "send_dtmf": VoiceAgentSendDtmfSystemTool; "send_sms": VoiceAgentSendSmsSystemTool.

Property Type Required Description
type "system" Yes The type of the tool. Always system.
name VoiceAgentSystemToolName Yes The service-managed control action. Known values are stable; additional values may be added over time.
description string No An optional description of the system tool.
timeout_seconds FoundryDurationSeconds No The execution timeout in seconds, from 1 to 300. If omitted, the service defaults to 60 seconds. default: 60. minimum: 1. maximum: 300.

VoiceAgentSystemToolItem

Service-generated system-tool intent, not a provider result. Clients cannot create it or complete it with function_call_output.

All of: OpenAI.RealtimeConversationItem.

Property Type Required Description
type "system_tool" Yes The output-item kind. Always system_tool.
id string No The identifier of the generated item.
object "realtime.item" No The realtime item marker.
status VoiceAgentSystemToolItemStatus Yes The generation status of the item, not the provider execution outcome.
name VoiceAgentSystemToolName Yes The built-in capability invoked by the model.
call_id string Yes The invocation identifier used for correlation and duplicate suppression.
arguments string Yes Original JSON-encoded intent: literals remain visible and references stay unexpanded. Excludes private values, resolved arguments and executor-only settings.
created_at FoundryTimestamp No The Unix timestamp (in seconds) for when the item was persisted. readOnly: true.
response_id string No The id of the response that produced this item, when applicable. readOnly: true.

VoiceAgentSystemToolItemStatus

The generation status of a system-tool item. It does not indicate provider success.

Any of: string, "in_progress" or "completed" or "incomplete".

VoiceAgentSystemToolName

A service-managed voice-session control action. Known values are stable; additional values may be added over time.

Any of: string, "end_conversation" or "language_detection" or "digit_recognition" or "transfer" or "collect_sms" or "collect_dtmf" or "send_dtmf" or "send_sms".

VoiceAgentTemplateGreetingConfig

A deterministic greeting rendered with the voice agent's structured inputs and synthesized without model-authored generation.

All of: VoiceAgentGreetingConfig.

Property Type Required Description
type "template" Yes
text string Yes The Handlebars text template spoken at session start. minLength: 1.

VoiceAgentTool

A tool usable by a voice agent.

Variants by type: "function": VoiceAgentFunctionTool; "mcp": VoiceAgentMcpTool; "system": VoiceAgentSystemTool; "toolbox": VoiceAgentToolboxTool.

Property Type Required Description
type VoiceAgentToolType Yes The tool kind.

VoiceAgentToolChoice

Tool-selection behavior for a voice agent.

One of: "none" or "auto" or "required", OpenAI.ToolChoiceFunction, OpenAI.ToolChoiceMCP.

VoiceAgentToolResponseScheduling

When a tool invocation creates a follow-up response. Additional values may be added over time.

Any of: string, "silent" or "when_idle" or "interrupt" or "skip_if_busy".

VoiceAgentToolType

The tool kinds supported by a voice agent.

Any of: string, "function" or "mcp" or "system" or "toolbox".

VoiceAgentToolboxTool

A reference to a Foundry toolbox, which is a versioned bundle of tools executed through its MCP endpoint.

All of: VoiceAgentTool.

Property Type Required Description
type "toolbox" Yes The type of the tool. Always toolbox.
toolbox_name string Yes The name of the toolbox to attach.
toolbox_version string Yes The immutable version of the toolbox to attach.
response_scheduling VoiceAgentToolResponseScheduling No When the toolbox invocation creates a follow-up response. Defaults to when_idle. default: "when_idle".

VoiceAgentTranscriptionPhrase

A transcribed phrase with timing information.

Property Type Required Description
offset_milliseconds FoundryDurationMilliseconds Yes The phrase offset from the beginning of the audio, in milliseconds.
duration_milliseconds FoundryDurationMilliseconds Yes The phrase duration in milliseconds.
text string Yes The transcribed phrase text.
words Any of: Array of VoiceAgentTranscriptionWord; null No Word-level timing details, when available.
locale Any of: string; null No The detected locale.
confidence Any of: number; null No The transcription confidence score.

VoiceAgentTranscriptionWord

A time-stamped word in an input-audio transcription.

Property Type Required Description
text string Yes The transcribed word text.
offset_milliseconds FoundryDurationMilliseconds Yes The word offset from the beginning of the audio, in milliseconds.
duration_milliseconds FoundryDurationMilliseconds Yes The word duration in milliseconds.

VoiceAgentTransferDestination

The canonical system-tool transfer destination.

Property Type Required Description
type VoiceAgentTransferDestinationType Yes The destination format: phone_number for an E.164 number, teams for a Teams identifier, or sip for a SIP URI.
value string Yes The concrete address. Phone numbers use E.164; Teams and SIP identifiers are validated against the selected provider's capabilities. minLength: 1.

VoiceAgentTransferDestinationType

The supported address formats for a system-tool transfer destination.

Type: "phone_number" or "teams" or "sip".

Conditional preview opt-ins: VoiceAgents=V1Preview.

VoiceAgentTransferRouting

The configured transfer routing policy. The model selects a target name, not destinations or routing settings.

Property Type Required Description
strategy VoiceAgentTransferStrategy Yes The routing strategy. Current native Telephony transfer supports ACS sequential and random routing only.
ring_timeout_seconds FoundryDurationSeconds No The per-destination ring timeout in seconds, from 1 to 300. If omitted, the service defaults to 20 seconds. Must not exceed the tool's execution timeout. ACS allows at most 180 seconds. default: 20. minimum: 1. maximum: 300.

VoiceAgentTransferStrategy

Per-invocation transfer routing. Unsupported strategies are rejected before provider execution; schema membership does not imply support by every provider.

Any of: string, "sequential" or "random" or "parallel".

VoiceAgentTransferSystemTool

Requests a transfer to an explicitly configured logical target.

All of: VoiceAgentSystemTool.

Property Type Required Description
name "transfer" Yes The service-managed control action. Always transfer.
require_handoff boolean No Require a non-blank handoff summary in each model-generated transfer call. Defaults to false; does not guarantee delivery to the recipient. default: false.
targets Array of VoiceAgentTransferTarget Yes The non-empty set of logical targets that the model may select. minItems: 1.

VoiceAgentTransferTarget

A configured logical transfer target. Only its name and description are model-visible; destinations and routing are executor settings.

Property Type Required Description
name string Yes The non-blank logical target name. Names must be unique after trimming within the transfer tool. minLength: 1. pattern: "\\S".
description string No A description that helps the model select this target.
destinations Array of VoiceAgentTransferDestination Yes The concrete destinations for this transfer target. minItems: 1.
routing VoiceAgentTransferRouting Yes The trusted routing policy for this target.

VoiceAgentTurnDetectionConfig

Turn-detection configuration for a voice agent.

Variants by type: "server_vad": VoiceAgentServerVadTurnDetection; "azure_semantic_vad": VoiceAgentAzureSemanticVadTurnDetection; "azure_semantic_vad_en": VoiceAgentAzureSemanticVadEnTurnDetection; "azure_semantic_vad_multilingual": VoiceAgentAzureSemanticVadMultilingualTurnDetection; "semantic_vad": VoiceAgentSemanticVadTurnDetection.

Property Type Required Description
type VoiceAgentTurnDetectionType Yes The turn-detection strategy.
auto_truncate boolean No Whether the input audio buffer is truncated automatically when speech stops. default: false.

VoiceAgentTurnDetectionType

The turn-detection strategy. Additional values may be added over time.

Any of: string, "server_vad" or "semantic_vad" or "azure_semantic_vad" or "azure_semantic_vad_en" or "azure_semantic_vad_multilingual".

VoiceAgentVoiceConfig

Voice output configuration. Inline subagents use it directly as their voice override, and the session-level output audio configuration extends it. Provider-specific fields are selected by voice_type: - openai: voice and speed. - azure-standard: voice, voice_locale, speed, voice_temperature, custom_lexicon_url, custom_text_normalization_url, prefer_locales, style, pitch, and volume. - azure-custom: all azure-standard fields except style, plus custom_voice_endpoint_id. - azure-personal: all azure-standard fields except style, plus personal_voice_model. - avatar-voice-sync: all azure-standard fields except voice and style, plus personal_voice_model; the voice name is derived from the avatar. - azure-realtime-native: voice and speed.

Property Type Required Description
voice string No The voice name or identifier. Applies to openai, azure-standard, azure-custom, azure-personal, and azure-realtime-native. It does not apply to avatar-voice-sync, which derives the voice name from the avatar.
voice_type VoiceType No The voice implementation.
voice_locale string No The enforced BCP-47 output locale. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync.
speed number No The numeric output speed multiplier. Applies to all known voice_type values and defaults to 1. format: "float". default: 1. minimum: 0.25. maximum: 1.5.
voice_temperature number No The voice variation temperature. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "float". minimum: 0. maximum: 1.
custom_lexicon_url string No The URL of a custom pronunciation lexicon. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "uri".
custom_text_normalization_url string No The URL of a custom text-normalization configuration. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync. format: "uri".
prefer_locales Array of string No Preferred BCP-47 locales for multilingual synthesis. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync.
style string No The voice speaking style. Applies only when voice_type is azure-standard.
pitch string No The voice pitch adjustment. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync.
volume string No The voice volume adjustment. Applies to azure-standard, azure-custom, azure-personal, and avatar-voice-sync.
custom_voice_endpoint_id string No The Azure custom-voice deployment endpoint identifier. Applies only when voice_type is azure-custom.
personal_voice_model string No The Azure personal or avatar voice model. Applies only when voice_type is azure-personal or avatar-voice-sync.

VoiceOutputModality

An output modality the agent may produce. animation and avatar are used when an avatar is configured.

Any of: string, "text" or "audio" or "animation" or "avatar".

VoiceResponseAudio

Audio configuration for a response. Follows the OpenAI Realtime GA audio object shape.

Property Type Required Description
output VoiceResponseAudioOutput No The audio output configuration used for the response.

VoiceResponseAudioOutput

The flat response audio-output projection, with optional voice, voice_type, voice_locale, and format fields.

Property Type Required Description
voice string No The voice name used for the response's audio output.
voice_type VoiceType No The extensible provider/type of the voice used for the response's audio output.
voice_locale string No The BCP-47 locale of the voice used for the response's audio output.
format OpenAI.RealtimeAudioFormats No The audio format used for the response's audio output.

VoiceType

The voice implementation. Additional values may be added over time.

Any of: string, "openai" or "azure-standard" or "azure-custom" or "azure-personal" or "avatar-voice-sync" or "azure-realtime-native".