VoiceAgentInputTranscription interface

Package:

@azure/ai-projects

Settings for transcribing input audio, including the transcription model, Custom Speech deployments, and phrase hints. When model is azure-speech-streaming, transcription is supported for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.

Properties

custom_speech	Optional customer custom speech deployment configuration, keyed by locale.
delay	Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.
keywords	Words or phrases to guide transcription of the input audio. Supported by gpt-transcribe and gpt-live-transcribe.
language	The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency. Optional for other transcription models; required and must be nonempty when model is azure-speech-streaming (such as en-US; comma-separated locales are supported).
languages	Possible languages of the input audio, in ISO-639-1 format. Supported by gpt-transcribe and gpt-live-transcribe.
model	The transcription model identifier. Configure customer custom speech deployments in custom_speech. azure-speech-streaming is for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.
phrase_list	Optional phrase hints that bias recognition toward domain terms.
profanity_filter	Controls how profanity is handled in input transcripts. Supported only with managed Azure Speech (azure-speech and azure-speech-streaming), including Custom Speech. Set this property before the session starts. If omitted, the transcription provider's default applies. You cannot change the mode during a session. Filtering affects transcript events and the text sent to cascaded models. It does not change input audio, voice activity detection, or assistant speech; audio-native models still receive the original audio. Mandatory service masking policies apply even when you select unfiltered.
prompt	An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.

Property Details

custom_speech

Optional customer custom speech deployment configuration, keyed by locale.

custom_speech?: Record<string, string>

Property Value

Record<string, string>

delay

Controls how long the model waits before emitting transcription text. Higher values can improve transcription accuracy at the cost of latency. Only supported with gpt-realtime-whisper in GA Realtime sessions.

delay?: "low" | "medium" | "high" | "xhigh" | "minimal"

Property Value

"low" | "medium" | "high" | "xhigh" | "minimal"

keywords

Words or phrases to guide transcription of the input audio. Supported by gpt-transcribe and gpt-live-transcribe.

keywords?: string[]

Property Value

string[]

language

The language of the input audio. Supplying the input language in ISO-639-1 (e.g. en) format will improve accuracy and latency. Optional for other transcription models; required and must be nonempty when model is azure-speech-streaming (such as en-US; comma-separated locales are supported).

language?: string

Property Value

string

languages

Possible languages of the input audio, in ISO-639-1 format. Supported by gpt-transcribe and gpt-live-transcribe.

languages?: string[]

Property Value

string[]

model

The transcription model identifier. Configure customer custom speech deployments in custom_speech. azure-speech-streaming is for cascaded text-model conversations only (not realtime speech-to-speech models), requires a nonempty language such as en-US (comma-separated locales are supported), and supports custom_speech, phrase_list, and profanity_filter.

model: VoiceAgentInputTranscriptionModel

Property Value

VoiceAgentInputTranscriptionModel

phrase_list

Optional phrase hints that bias recognition toward domain terms.

phrase_list?: string[]

Property Value

string[]

profanity_filter

Controls how profanity is handled in input transcripts. Supported only with managed Azure Speech (azure-speech and azure-speech-streaming), including Custom Speech. Set this property before the session starts. If omitted, the transcription provider's default applies. You cannot change the mode during a session. Filtering affects transcript events and the text sent to cascaded models. It does not change input audio, voice activity detection, or assistant speech; audio-native models still receive the original audio. Mandatory service masking policies apply even when you select unfiltered.

profanity_filter?: VoiceAgentProfanityFilterMode

Property Value

VoiceAgentProfanityFilterMode

prompt

An optional text to guide the model's style or continue a previous audio segment. For whisper-1, the prompt is a list of keywords. For gpt-4o-transcribe models (excluding gpt-4o-transcribe-diarize), the prompt is a free text string, for example "expect words related to technology". Prompt is not supported with gpt-realtime-whisper in GA Realtime sessions.

prompt?: string

Property Value

string