VoiceAgentDefinition interface

Package:

@azure/ai-projects

The voice agent definition. Its configuration (model, instructions, audio, tools, and optional avatar) drives a managed speech-to-speech experience. Establish realtime voice sessions through GET /agents/{agent_name}/endpoint/protocols/voice. Every create or update produces a new immutable version.

Extends

AgentDefinition

Properties

audio	The audio configuration, including input and output formats, voice, turn detection, noise reduction, and transcription. These values are session defaults; a client may override supported fields when connecting.
avatar	Optional avatar configuration. These values are session defaults and may be overridden when connecting.
conversation_engine	The engine that owns conversation handling for this voice agent. Exactly one of this property and the model-backed configuration (model_type with model) must be provided. When this property is provided, model_type, model, instructions, tools, and tool_choice must be omitted, and greeting.tool_choice cannot be required, because the engine owns the conversation logic. The initial implementation supports a hosted-agent engine.
greeting	Optional session-start greeting. Template mode speaks exact rendered text; LLM-generated mode asks the session model to author the opening response and may use configured tools.
include	Additional fields to include in service outputs.
instructions	A system (or developer) message inserted into the model's context. Supports template substitution via structured_inputs, rendered per session before the live session starts.
interim_response	Interim-response settings for latency and tool execution.
kind	The kind discriminator for a voice agent definition. Always voice.
max_output_tokens	The maximum output-token count for one response.
model	The model to use for this agent. Required with model_type for a model-backed voice agent and omitted when conversation_engine is provided. The model must support realtime or cascaded voice.
model_type	How the model backing this voice agent is served. Required with model for a model-backed voice agent and omitted when conversation_engine is provided. This is independent of the architecture (realtime or cascaded), which the service derives from the selected model.
output_modalities	The output modalities the agent produces. Defaults to ["audio"]. animation and avatar are available when an avatar is configured.
parallel_tool_calls	Whether the model may call multiple tools in parallel.
store	Whether conversations with this agent are persisted. A single, all-or-nothing persistence switch that defaults to false (privacy-safe: off by default). When true, Foundry persists the full conversation — the transcript/event timeline and raw audio. When false, nothing is persisted and no conversation is surfaced. There is no separate audio-logging control; audio is persisted only as part of this switch. Latency/performance telemetry (e.g. time-to-first-audio, inter-token latency, interruption) is observability-only (customer trace / App Insights) and is not part of the persisted conversation content.
structured_inputs	Set of structured inputs that participate in prompt template substitution, rendered per session before the live session starts.
subagent_config	Optional configuration for sibling Foundry text agents that this voice agent may consult as background specialists.
tools	The tools the voice agent may use. Supported tool kinds are function (executed by the client), mcp, system (service-managed session controls), and toolbox. Server-side tools such as web_search, azure_ai_search, and openapi are provided through a toolbox rather than declared directly.
tool_choice	How the model chooses tools for generated responses. none prevents tool calls, auto lets the model decide, required requires at least one tool call, and a specific function or MCP tool can be selected with an object. Defaults to auto.

Inherited Properties

rai_config

Configuration for Responsible AI (RAI) content filtering and safety features.

Property Details

audio

The audio configuration, including input and output formats, voice, turn detection, noise reduction, and transcription. These values are session defaults; a client may override supported fields when connecting.

audio?: VoiceAgentAudioConfig

Property Value

VoiceAgentAudioConfig

avatar

Optional avatar configuration. These values are session defaults and may be overridden when connecting.

avatar?: VoiceAgentAvatarConfig

Property Value

VoiceAgentAvatarConfig

conversation_engine

The engine that owns conversation handling for this voice agent. Exactly one of this property and the model-backed configuration (model_type with model) must be provided. When this property is provided, model_type, model, instructions, tools, and tool_choice must be omitted, and greeting.tool_choice cannot be required, because the engine owns the conversation logic. The initial implementation supports a hosted-agent engine.

conversation_engine?: VoiceConversationEngineUnion

Property Value

VoiceConversationEngineUnion

greeting

Optional session-start greeting. Template mode speaks exact rendered text; LLM-generated mode asks the session model to author the opening response and may use configured tools.

greeting?: VoiceAgentGreetingConfigUnion

Property Value

VoiceAgentGreetingConfigUnion

include

Additional fields to include in service outputs.

include?: VoiceAgentSessionIncludeOption[]

Property Value

VoiceAgentSessionIncludeOption[]

instructions

A system (or developer) message inserted into the model's context. Supports template substitution via structured_inputs, rendered per session before the live session starts.

instructions?: string

Property Value

string

interim_response

Interim-response settings for latency and tool execution.

interim_response?: VoiceAgentInterimResponseConfigUnion

Property Value

VoiceAgentInterimResponseConfigUnion

kind

The kind discriminator for a voice agent definition. Always voice.

kind: "voice"

Property Value

"voice"

max_output_tokens

The maximum output-token count for one response.

max_output_tokens?: VoiceAgentMaxOutputTokens

Property Value

VoiceAgentMaxOutputTokens

model

The model to use for this agent. Required with model_type for a model-backed voice agent and omitted when conversation_engine is provided. The model must support realtime or cascaded voice.

model?: string

Property Value

string

model_type

How the model backing this voice agent is served. Required with model for a model-backed voice agent and omitted when conversation_engine is provided. This is independent of the architecture (realtime or cascaded), which the service derives from the selected model.

model_type?: VoiceModelType

Property Value

VoiceModelType

output_modalities

The output modalities the agent produces. Defaults to ["audio"]. animation and avatar are available when an avatar is configured.

output_modalities?: VoiceOutputModality[]

Property Value

VoiceOutputModality[]

parallel_tool_calls

Whether the model may call multiple tools in parallel.

parallel_tool_calls?: boolean

Property Value

boolean

store

Whether conversations with this agent are persisted. A single, all-or-nothing persistence switch that defaults to false (privacy-safe: off by default). When true, Foundry persists the full conversation — the transcript/event timeline and raw audio. When false, nothing is persisted and no conversation is surfaced. There is no separate audio-logging control; audio is persisted only as part of this switch. Latency/performance telemetry (e.g. time-to-first-audio, inter-token latency, interruption) is observability-only (customer trace / App Insights) and is not part of the persisted conversation content.

store?: boolean

Property Value

boolean

structured_inputs

Set of structured inputs that participate in prompt template substitution, rendered per session before the live session starts.

structured_inputs?: Record<string, StructuredInputDefinition>

Property Value

Record<string, StructuredInputDefinition>

subagent_config

Optional configuration for sibling Foundry text agents that this voice agent may consult as background specialists.

subagent_config?: VoiceAgentSubagentConfig

Property Value

VoiceAgentSubagentConfig

tools

The tools the voice agent may use. Supported tool kinds are function (executed by the client), mcp, system (service-managed session controls), and toolbox. Server-side tools such as web_search, azure_ai_search, and openapi are provided through a toolbox rather than declared directly.

tools?: VoiceAgentToolUnion[]

Property Value

VoiceAgentToolUnion[]

tool_choice

How the model chooses tools for generated responses. none prevents tool calls, auto lets the model decide, required requires at least one tool call, and a specific function or MCP tool can be selected with an object. Defaults to auto.

tool_choice?: VoiceAgentToolChoice

Property Value

VoiceAgentToolChoice

Inherited Property Details

rai_config

Configuration for Responsible AI (RAI) content filtering and safety features.

rai_config?: RaiConfig

Property Value

RaiConfig

Inherited From AgentDefinition.rai_config