Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
These events are sent by the service on a voice-agent realtime connection.
These tables describe the event properties and link to their models.
Server events
- conversation.item.added
- conversation.item.created
- conversation.item.deleted
- conversation.item.done
- conversation.item.input_audio_transcription.completed
- conversation.item.input_audio_transcription.delta
- conversation.item.input_audio_transcription.failed
- conversation.item.input_audio_transcription.segment
- conversation.item.retrieved
- conversation.item.truncated
- input_audio_buffer.cleared
- input_audio_buffer.committed
- input_audio_buffer.speech_started
- input_audio_buffer.speech_stopped
- input_audio_buffer.timeout_triggered
- mcp_list_tools.completed
- mcp_list_tools.failed
- mcp_list_tools.in_progress
- output_audio_buffer.cleared
- rate_limits.updated
- response.output_audio.delta
- response.output_audio.done
- response.output_audio_transcript.delta
- response.output_audio_transcript.done
- response.content_part.added
- response.content_part.done
- response.created
- response.done
- response.function_call_arguments.delta
- response.function_call_arguments.done
- response.system_tool_arguments.delta
- response.system_tool_arguments.done
- input_collection.status
- response.mcp_call_arguments.delta
- response.mcp_call_arguments.done
- response.mcp_call.completed
- response.mcp_call.failed
- response.mcp_call.in_progress
- response.output_item.added
- response.output_item.done
- response.output_text.delta
- response.output_text.done
- session.created
- session.subagent.started
- session.subagent.completed
- session.subagent.aborted
- session.updated
- error
- warning
- session.avatar.connecting
- rtc.call.sdp.created
- rtc.call.error
- session.avatar.switch_to_speaking
- session.avatar.switch_to_idle
- response.audio_timestamp.delta
- response.audio_timestamp.done
- response.animation_blendshapes.delta
- response.animation_blendshapes.done
- response.animation_viseme.delta
- response.animation_viseme.done
- response.video.delta
- session.handoff.started
- session.handoff.completed
- session.handoff.aborted
conversation.item.added
Schema: OpenAI.RealtimeServerEventConversationItemAdded.
Sent by the server when an Item is added to the default Conversation. This can happen in several cases: - When the client sends a conversation.item.create event. - When the input audio buffer is committed. In this case the item will be a user message containing the audio from the buffer. - When the model is generating a Response. In this case the conversation.item.added event will be sent when the model starts generating a specific Item, and thus it will not yet have any content (and status will be in_progress). The event will include the full content of the Item (except when model is generating a Response) except for audio data, which can be retrieved separately with a conversation.item.retrieve event if necessary.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.added" |
Yes | The event type, must be conversation.item.added. |
previous_item_id |
Any of: string; null |
No | |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"type": "conversation.item.added",
"event_id": "event_C9G8pjSJCfRNEhMEnYAVy",
"previous_item_id": null,
"item": {
"id": "item_C9G8pGVKYnaZu8PH5YQ9O",
"type": "message",
"status": "completed",
"role": "user",
"content": [
{
"type": "input_text",
"text": "hi"
}
]
}
}
conversation.item.created
Schema: OpenAI.RealtimeServerEventConversationItemCreated.
Returned when a conversation item is created. There are several scenarios that produce this event: - The server is generating a Response, which if successful will produce either one or two Items, which will be of type message (role assistant) or type function_call. - The input audio buffer has been committed, either by the client or the server (in server_vad mode). The server will take the content of the input audio buffer and add it to a new user message Item. - The client has sent a conversation.item.create event to add a new Item to the Conversation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.created" |
Yes | The event type, must be conversation.item.created. |
previous_item_id |
Any of: string; null |
No | |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"event_id": "event_1920",
"type": "conversation.item.created",
"previous_item_id": "msg_002",
"item": {
"id": "msg_003",
"object": "realtime.item",
"type": "message",
"status": "completed",
"role": "user",
"content": []
}
}
conversation.item.deleted
Schema: OpenAI.RealtimeServerEventConversationItemDeleted.
Returned when an item in the conversation is deleted by the client with a conversation.item.delete event. This event is used to synchronize the server's understanding of the conversation history with the client's view.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.deleted" |
Yes | The event type, must be conversation.item.deleted. |
item_id |
string |
Yes | The ID of the item that was deleted. |
Example event:
{
"event_id": "event_2728",
"type": "conversation.item.deleted",
"item_id": "msg_005"
}
conversation.item.done
Schema: OpenAI.RealtimeServerEventConversationItemDone.
Returned when a conversation item is finalized. The event will include the full content of the Item except for audio data, which can be retrieved separately with a conversation.item.retrieve event if needed.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.done" |
Yes | The event type, must be conversation.item.done. |
previous_item_id |
Any of: string; null |
No | |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"type": "conversation.item.done",
"event_id": "event_CCXLgMZPo3qioWCeQa4WH",
"previous_item_id": "item_CCXLecNJVIVR2HUy3ABLj",
"item": {
"id": "item_CCXLfxmM5sXVJVz4mCa2S",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "output_audio",
"transcript": "Oh, I can hear you loud and clear! Sounds like we're connected just fine. What can I help you with today?"
}
]
}
}
conversation.item.input_audio_transcription.completed
Schema: OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionCompleted.
This event is the output of audio transcription for user audio written to the user audio buffer. Transcription begins when the input audio buffer is committed by the client or server (when VAD is enabled). Transcription runs asynchronously with Response creation, so this event may come before or after the Response events. Realtime API models accept audio natively, and thus input transcription is a separate process run on a separate ASR (Automatic Speech Recognition) model. The transcript may diverge somewhat from the model's interpretation, and should be treated as a rough guide.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.input_audio_transcription.completed" |
Yes | The event type, must be conversation.item.input_audio_transcription.completed. |
item_id |
string |
Yes | The ID of the item containing the audio that is being transcribed. |
content_index |
OpenAI.integer | Yes | The index of the content part containing the audio. |
transcript |
string |
Yes | The transcribed text. |
languages |
Array of OpenAI.TranscriptionLanguage | No | The languages detected in the audio. Returned by gpt-transcribe. An empty array indicates that no language could be reliably detected. |
logprobs |
Any of: Array of OpenAI.LogProbProperties; null |
No | |
usage |
Exactly one of: OpenAI.TranscriptTextUsageTokens; OpenAI.TranscriptTextUsageDuration | Yes | Usage statistics for the transcription, this is billed according to the ASR model's pricing rather than the realtime model's pricing. |
phrases |
Any of: Array of VoiceAgentTranscriptionPhrase; null |
No | Phrase-level transcription timing and confidence details. |
Example event:
{
"type": "conversation.item.input_audio_transcription.completed",
"event_id": "event_CCXGRvtUVrax5SJAnNOWZ",
"item_id": "item_CCXGQ4e1ht4cOraEYcuR2",
"content_index": 0,
"transcript": "Hey, can you hear me?",
"usage": {
"type": "tokens",
"total_tokens": 22,
"input_tokens": 13,
"input_token_details": {
"text_tokens": 0,
"audio_tokens": 13
},
"output_tokens": 9
}
}
conversation.item.input_audio_transcription.delta
Schema: OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionDelta.
Returned when the text value of an input audio transcription content part is updated with incremental transcription results.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.input_audio_transcription.delta" |
Yes | The event type, must be conversation.item.input_audio_transcription.delta. |
item_id |
string |
Yes | The ID of the item containing the audio that is being transcribed. |
content_index |
OpenAI.integer | No | The index of the content part in the item's content array. |
delta |
string |
No | The text delta. |
logprobs |
Any of: Array of OpenAI.LogProbProperties; null |
No |
Example event:
{
"type": "conversation.item.input_audio_transcription.delta",
"event_id": "event_CCXGRxsAimPAs8kS2Wc7Z",
"item_id": "item_CCXGQ4e1ht4cOraEYcuR2",
"content_index": 0,
"delta": "Hey",
"obfuscation": "aLxx0jTEciOGe"
}
conversation.item.input_audio_transcription.failed
Schema: OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionFailed.
Returned when input audio transcription is configured, and a transcription request for a user message failed. These events are separate from other error events so that the client can identify the related Item.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.input_audio_transcription.failed" |
Yes | The event type, must be conversation.item.input_audio_transcription.failed. |
item_id |
string |
Yes | The ID of the user message item. |
content_index |
OpenAI.integer | Yes | The index of the content part containing the audio. |
error |
OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionFailedError | Yes | Details of the transcription error. |
conversation.item.input_audio_transcription.segment
Schema: OpenAI.RealtimeServerEventConversationItemInputAudioTranscriptionSegment.
Returned when an input audio transcription segment is identified for an item.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.input_audio_transcription.segment" |
Yes | The event type, must be conversation.item.input_audio_transcription.segment. |
item_id |
string |
Yes | The ID of the item containing the input audio content. |
content_index |
OpenAI.integer | Yes | The index of the input audio content part within the item. |
text |
string |
Yes | The text for this segment. |
id |
string |
Yes | The segment identifier. |
speaker |
string |
Yes | The detected speaker label for this segment. |
start |
number |
Yes | Start time of the segment in seconds. format: "double". |
end |
number |
Yes | End time of the segment in seconds. format: "double". |
Example event:
{
"event_id": "event_6501",
"type": "conversation.item.input_audio_transcription.segment",
"item_id": "msg_011",
"content_index": 0,
"text": "hello",
"id": "seg_0001",
"speaker": "spk_1",
"start": 0,
"end": 0.4
}
conversation.item.retrieved
Schema: OpenAI.RealtimeServerEventConversationItemRetrieved.
Returned when a conversation item is retrieved with conversation.item.retrieve. This is provided as a way to fetch the server's representation of an item, for example to get access to the post-processed audio data after noise cancellation and VAD. It includes the full content of the Item, including audio data.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.retrieved" |
Yes | The event type, must be conversation.item.retrieved. |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"type": "conversation.item.retrieved",
"event_id": "event_CCXGSizgEppa2d4XbKA7K",
"item": {
"id": "item_CCXGRxbY0n6WE4EszhF5w",
"object": "realtime.item",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "output_audio",
"transcript": "Yes, I can hear you loud and clear. How can I help you today?",
"audio": "AAABAP//AAA="
}
]
}
}
conversation.item.truncated
Schema: OpenAI.RealtimeServerEventConversationItemTruncated.
Returned when an earlier assistant audio message item is truncated by the client with a conversation.item.truncate event. This event is used to synchronize the server's understanding of the audio with the client's playback. This action will truncate the audio and remove the server-side text transcript to ensure there is no text in the context that hasn't been heard by the user.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"conversation.item.truncated" |
Yes | The event type, must be conversation.item.truncated. |
item_id |
string |
Yes | The ID of the assistant message item that was truncated. |
content_index |
OpenAI.integer | Yes | The index of the content part that was truncated. |
audio_end_ms |
OpenAI.integer | Yes | The duration up to which the audio was truncated, in milliseconds. |
item |
OpenAI.RealtimeConversationItem | No | The assistant message after truncation, when the service returns the updated item. |
Example event:
{
"event_id": "event_2526",
"type": "conversation.item.truncated",
"item_id": "msg_004",
"content_index": 0,
"audio_end_ms": 1500
}
input_audio_buffer.cleared
Schema: OpenAI.RealtimeServerEventInputAudioBufferCleared.
Returned when the input audio buffer is cleared by the client with a input_audio_buffer.clear event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"input_audio_buffer.cleared" |
Yes | The event type, must be input_audio_buffer.cleared. |
Example event:
{
"event_id": "event_1314",
"type": "input_audio_buffer.cleared"
}
input_audio_buffer.committed
Schema: OpenAI.RealtimeServerEventInputAudioBufferCommitted.
Returned when an input audio buffer is committed, either by the client or automatically in server VAD mode. The item_id property is the ID of the user message item that will be created, thus a conversation.item.created event will also be sent to the client.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"input_audio_buffer.committed" |
Yes | The event type, must be input_audio_buffer.committed. |
previous_item_id |
Any of: string; null |
No | |
item_id |
string |
Yes | The ID of the user message item that will be created. |
Example event:
{
"event_id": "event_1121",
"type": "input_audio_buffer.committed",
"previous_item_id": "msg_001",
"item_id": "msg_002"
}
input_audio_buffer.speech_started
Schema: OpenAI.RealtimeServerEventInputAudioBufferSpeechStarted.
Sent by the server when in server_vad mode to indicate that speech has been detected in the audio buffer. This can happen any time audio is added to the buffer (unless speech is already detected). The client may want to use this event to interrupt audio playback or provide visual feedback to the user. The client should expect to receive a input_audio_buffer.speech_stopped event when speech stops. The item_id property is the ID of the user message item that will be created when speech stops and will also be included in the input_audio_buffer.speech_stopped event (unless the client manually commits the audio buffer during VAD activation).
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"input_audio_buffer.speech_started" |
Yes | The event type, must be input_audio_buffer.speech_started. |
audio_start_ms |
OpenAI.integer | Yes | Milliseconds from the start of all audio written to the buffer during the session when speech was first detected. This will correspond to the beginning of audio sent to the model, and thus includes the prefix_padding_ms configured in the Session. |
item_id |
string |
Yes | The ID of the user message item that will be created when speech stops. |
Example event:
{
"event_id": "event_1516",
"type": "input_audio_buffer.speech_started",
"audio_start_ms": 1000,
"item_id": "msg_003"
}
input_audio_buffer.speech_stopped
Schema: OpenAI.RealtimeServerEventInputAudioBufferSpeechStopped.
Returned in server_vad mode when the server detects the end of speech in the audio buffer. The server will also send an conversation.item.created event with the user message item that is created from the audio buffer.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"input_audio_buffer.speech_stopped" |
Yes | The event type, must be input_audio_buffer.speech_stopped. |
audio_end_ms |
OpenAI.integer | Yes | Milliseconds since the session started when speech stopped. This will correspond to the end of audio sent to the model, and thus includes the min_silence_duration_ms configured in the Session. |
item_id |
string |
Yes | The ID of the user message item that will be created. |
Example event:
{
"event_id": "event_1718",
"type": "input_audio_buffer.speech_stopped",
"audio_end_ms": 2000,
"item_id": "msg_003"
}
input_audio_buffer.timeout_triggered
Schema: OpenAI.RealtimeServerEventInputAudioBufferTimeoutTriggered.
Returned when the Server VAD timeout is triggered for the input audio buffer. This is configured with idle_timeout_ms in the turn_detection settings of the session, and it indicates that there hasn't been any speech detected for the configured duration. The audio_start_ms and audio_end_ms fields indicate the segment of audio after the last model response up to the triggering time, as an offset from the beginning of audio written to the input audio buffer. This means it demarcates the segment of audio that was silent and the difference between the start and end values will roughly match the configured timeout. The empty audio will be committed to the conversation as an input_audio item (there will be a input_audio_buffer.committed event) and a model response will be generated. There may be speech that didn't trigger VAD but is still detected by the model, so the model may respond with something relevant to the conversation or a prompt to continue speaking.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"input_audio_buffer.timeout_triggered" |
Yes | The event type, must be input_audio_buffer.timeout_triggered. |
audio_start_ms |
OpenAI.integer | Yes | Millisecond offset of audio written to the input audio buffer that was after the playback time of the last model response. |
audio_end_ms |
OpenAI.integer | Yes | Millisecond offset of audio written to the input audio buffer at the time the timeout was triggered. |
item_id |
string |
Yes | The ID of the item associated with this segment. |
Example event:
{
"type": "input_audio_buffer.timeout_triggered",
"event_id": "event_CEKKrf1KTGvemCPyiJTJ2",
"audio_start_ms": 13216,
"audio_end_ms": 19232,
"item_id": "item_CEKKrWH0GiwN0ET97NUZc"
}
mcp_list_tools.completed
Schema: OpenAI.RealtimeServerEventMCPListToolsCompleted.
Returned when listing MCP tools has completed for an item.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"mcp_list_tools.completed" |
Yes | The event type, must be mcp_list_tools.completed. |
item_id |
string |
Yes | The ID of the MCP list tools item. |
Example event:
{
"event_id": "event_6102",
"type": "mcp_list_tools.completed",
"item_id": "mcp_list_tools_001"
}
mcp_list_tools.failed
Schema: OpenAI.RealtimeServerEventMCPListToolsFailed.
Returned when listing MCP tools has failed for an item.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"mcp_list_tools.failed" |
Yes | The event type, must be mcp_list_tools.failed. |
item_id |
string |
Yes | The ID of the MCP list tools item. |
Example event:
{
"event_id": "event_6103",
"type": "mcp_list_tools.failed",
"item_id": "mcp_list_tools_001"
}
mcp_list_tools.in_progress
Schema: OpenAI.RealtimeServerEventMCPListToolsInProgress.
Returned when listing MCP tools is in progress for an item.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"mcp_list_tools.in_progress" |
Yes | The event type, must be mcp_list_tools.in_progress. |
item_id |
string |
Yes | The ID of the MCP list tools item. |
Example event:
{
"event_id": "event_6101",
"type": "mcp_list_tools.in_progress",
"item_id": "mcp_list_tools_001"
}
output_audio_buffer.cleared
Schema: OpenAI.RealtimeServerEventOutputAudioBufferCleared.
WebRTC/SIP Only: Emitted when the output audio buffer is cleared. This happens either in VAD mode when the user has interrupted (input_audio_buffer.speech_started), or when the client has emitted the output_audio_buffer.clear event to manually cut off the current audio response. Learn more.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"output_audio_buffer.cleared" |
Yes | The event type, must be output_audio_buffer.cleared. |
response_id |
string |
Yes | The unique ID of the response that produced the audio. |
Example event:
{
"event_id": "event_abc123",
"type": "output_audio_buffer.cleared",
"response_id": "resp_abc123"
}
rate_limits.updated
Schema: OpenAI.RealtimeServerEventRateLimitsUpdated.
Emitted at the beginning of a Response to indicate the updated rate limits. When a Response is created some tokens will be "reserved" for the output tokens, the rate limits shown here reflect that reservation, which is then adjusted accordingly once the Response is completed.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"rate_limits.updated" |
Yes | The event type, must be rate_limits.updated. |
rate_limits |
Array of OpenAI.RealtimeServerEventRateLimitsUpdatedRateLimits | Yes | List of rate limit information. |
Example event:
{
"event_id": "event_5758",
"type": "rate_limits.updated",
"rate_limits": [
{
"name": "requests",
"limit": 1000,
"remaining": 999,
"reset_seconds": 60
},
{
"name": "tokens",
"limit": 50000,
"remaining": 49950,
"reset_seconds": 60
}
]
}
response.output_audio.delta
Schema: OpenAI.RealtimeServerEventResponseAudioDelta.
Returned when the model-generated audio is updated.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_audio.delta" |
Yes | The event type, must be response.output_audio.delta. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
delta |
string |
Yes | Base64-encoded audio data delta. |
Example event:
{
"event_id": "event_4950",
"type": "response.output_audio.delta",
"response_id": "resp_001",
"item_id": "msg_008",
"output_index": 0,
"content_index": 0,
"delta": "Base64EncodedAudioDelta"
}
response.output_audio.done
Schema: OpenAI.RealtimeServerEventResponseAudioDone.
Returned when the model-generated audio is done. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_audio.done" |
Yes | The event type, must be response.output_audio.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
Example event:
{
"event_id": "event_5152",
"type": "response.output_audio.done",
"response_id": "resp_001",
"item_id": "msg_008",
"output_index": 0,
"content_index": 0
}
response.output_audio_transcript.delta
Schema: OpenAI.RealtimeServerEventResponseAudioTranscriptDelta.
Returned when the model-generated transcription of audio output is updated.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_audio_transcript.delta" |
Yes | The event type, must be response.output_audio_transcript.delta. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
delta |
string |
Yes | The transcript delta. |
Example event:
{
"event_id": "event_4546",
"type": "response.output_audio_transcript.delta",
"response_id": "resp_001",
"item_id": "msg_008",
"output_index": 0,
"content_index": 0,
"delta": "Hello, how can I a"
}
response.output_audio_transcript.done
Schema: OpenAI.RealtimeServerEventResponseAudioTranscriptDone.
Returned when the model-generated transcription of audio output is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_audio_transcript.done" |
Yes | The event type, must be response.output_audio_transcript.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
transcript |
string |
Yes | The final transcript of the audio. |
Example event:
{
"event_id": "event_4748",
"type": "response.output_audio_transcript.done",
"response_id": "resp_001",
"item_id": "msg_008",
"output_index": 0,
"content_index": 0,
"transcript": "Hello, how can I assist you today?"
}
response.content_part.added
Schema: OpenAI.RealtimeServerEventResponseContentPartAdded.
Returned when a new content part is added to an assistant message item during response generation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.content_part.added" |
Yes | The event type, must be response.content_part.added. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item to which the content part was added. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
part |
OpenAI.RealtimeServerEventResponseContentPartAddedPart | Yes | The content part that was added. |
Example event:
{
"event_id": "event_3738",
"type": "response.content_part.added",
"response_id": "resp_001",
"item_id": "msg_007",
"output_index": 0,
"content_index": 0,
"part": {
"type": "text",
"text": ""
}
}
response.content_part.done
Schema: OpenAI.RealtimeServerEventResponseContentPartDone.
Returned when a content part is done streaming in an assistant message item. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.content_part.done" |
Yes | The event type, must be response.content_part.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
part |
OpenAI.RealtimeServerEventResponseContentPartDonePart | Yes | The content part that is done. |
Example event:
{
"event_id": "event_3940",
"type": "response.content_part.done",
"response_id": "resp_001",
"item_id": "msg_007",
"output_index": 0,
"content_index": 0,
"part": {
"type": "text",
"text": "Sure, I can help with that."
}
}
response.created
Schema: OpenAI.RealtimeServerEventResponseCreated.
Returned when a new Response is created. The first event of response creation, where the response is in an initial state of in_progress.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.created" |
Yes | The event type, must be response.created. |
response |
VoiceAgentRealtimeResponse | Yes |
response.done
Schema: OpenAI.RealtimeServerEventResponseDone.
Returned when a Response is done streaming. Always emitted, no matter the final state. The Response object included in the response.done event will include all output Items in the Response but will omit the raw audio data. Clients should check the status field of the Response to determine if it was successful (completed) or if there was another outcome: cancelled, failed, or incomplete. A response will contain all output items that were generated during the response, excluding any audio content.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.done" |
Yes | The event type, must be response.done. |
response |
VoiceAgentRealtimeResponse | Yes |
response.function_call_arguments.delta
Schema: OpenAI.RealtimeServerEventResponseFunctionCallArgumentsDelta.
Returned when the model-generated function call arguments are updated.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.function_call_arguments.delta" |
Yes | The event type, must be response.function_call_arguments.delta. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the function call item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
call_id |
string |
Yes | The ID of the function call. |
delta |
string |
Yes | The arguments delta as a JSON string. |
response.function_call_arguments.done
Schema: OpenAI.RealtimeServerEventResponseFunctionCallArgumentsDone.
Returned when the model-generated function call arguments are done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.function_call_arguments.done" |
Yes | The event type, must be response.function_call_arguments.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the function call item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
call_id |
string |
Yes | The ID of the function call. |
name |
string |
Yes | The name of the function that was called. |
arguments |
string |
Yes | The final arguments as a JSON string. |
response.system_tool_arguments.delta
Schema: VoiceAgentServerEventSystemToolArgumentsDelta.
An incremental model-argument fragment for a service-managed system tool.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.system_tool_arguments.delta" |
Yes | The event type. Always response.system_tool_arguments.delta. |
event_id |
string |
Yes | The unique identifier of this server event. |
response_id |
string |
Yes | The response containing the tool invocation. |
item_id |
string |
Yes | The generated tool item's identifier. |
output_index |
integer |
Yes | The item's zero-based position in the response output. format: "int32". |
call_id |
string |
Yes | The invocation identifier. |
delta |
string |
Yes | An incremental JSON fragment of the intent arguments. |
obfuscation |
string |
No | Optional padding metadata with no application semantics. |
response.system_tool_arguments.done
Schema: VoiceAgentServerEventSystemToolArgumentsDone.
The completed intent arguments for a system tool, not provider success or a request for a customer function result.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.system_tool_arguments.done" |
Yes | The event type. Always response.system_tool_arguments.done. |
event_id |
string |
Yes | The unique identifier of this server event. |
response_id |
string |
Yes | The response containing the tool invocation. |
item_id |
string |
Yes | The generated tool item's identifier. |
output_index |
integer |
Yes | The item's zero-based position in the response output. format: "int32". |
call_id |
string |
Yes | The invocation identifier. |
name |
VoiceAgentSystemToolName | Yes | The invoked built-in capability. |
arguments |
string |
Yes | The complete, unexpanded model arguments as a JSON string. |
input_collection.status
Schema: VoiceAgentServerEventInputCollectionStatus.
Reports collection or telephony outcomes for a model call.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"input_collection.status" |
Yes | The event type. Always input_collection.status. |
event_id |
string |
No | An optional event identifier. |
call_id |
string |
Yes | The originating system-tool invocation. |
collection_id |
string |
No | The collection operation identifier, when this outcome concerns a collection. |
request_id |
string |
No | The current receiver request identifier, when applicable. |
channel |
VoiceAgentInputCollectionChannel | No | The collection receiver, omitted for telephony-operation feedback. |
status |
VoiceAgentInputCollectionStatus | Yes | The reported outcome. Generation completion of a system-tool item is separate. |
value |
string |
No | Completed ASCII digits, including leading zeros, only for secret: false; absent for private collection and telephony-operation feedback. |
output_ref |
VoiceAgentOutputReference | No | Actual bare variable name from successful private collection, not its value. May be suffixed on collision; reference it with {{name}}. |
response.mcp_call_arguments.delta
Schema: OpenAI.RealtimeServerEventResponseMCPCallArgumentsDelta.
Returned when MCP tool call arguments are updated during response generation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.mcp_call_arguments.delta" |
Yes | The event type, must be response.mcp_call_arguments.delta. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the MCP tool call item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
delta |
string |
Yes | The JSON-encoded arguments delta. |
obfuscation |
Any of: string; null |
No |
response.mcp_call_arguments.done
Schema: OpenAI.RealtimeServerEventResponseMCPCallArgumentsDone.
Returned when MCP tool call arguments are finalized during response generation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.mcp_call_arguments.done" |
Yes | The event type, must be response.mcp_call_arguments.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the MCP tool call item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
arguments |
string |
Yes | The final JSON-encoded arguments string. |
response.mcp_call.completed
Schema: OpenAI.RealtimeServerEventResponseMCPCallCompleted.
Returned when an MCP tool call has completed successfully.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.mcp_call.completed" |
Yes | The event type, must be response.mcp_call.completed. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
item_id |
string |
Yes | The ID of the MCP tool call item. |
Example event:
{
"event_id": "event_6302",
"type": "response.mcp_call.completed",
"output_index": 0,
"item_id": "mcp_call_001"
}
response.mcp_call.failed
Schema: OpenAI.RealtimeServerEventResponseMCPCallFailed.
Returned when an MCP tool call has failed.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.mcp_call.failed" |
Yes | The event type, must be response.mcp_call.failed. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
item_id |
string |
Yes | The ID of the MCP tool call item. |
Example event:
{
"event_id": "event_6303",
"type": "response.mcp_call.failed",
"output_index": 0,
"item_id": "mcp_call_001"
}
response.mcp_call.in_progress
Schema: OpenAI.RealtimeServerEventResponseMCPCallInProgress.
Returned when an MCP tool call has started and is in progress.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.mcp_call.in_progress" |
Yes | The event type, must be response.mcp_call.in_progress. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
item_id |
string |
Yes | The ID of the MCP tool call item. |
Example event:
{
"event_id": "event_6301",
"type": "response.mcp_call.in_progress",
"output_index": 0,
"item_id": "mcp_call_001"
}
response.output_item.added
Schema: OpenAI.RealtimeServerEventResponseOutputItemAdded.
Returned when a new Item is created during Response generation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_item.added" |
Yes | The event type, must be response.output_item.added. |
response_id |
string |
Yes | The ID of the Response to which the item belongs. |
output_index |
OpenAI.integer | Yes | The index of the output item in the Response. |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"event_id": "event_3334",
"type": "response.output_item.added",
"response_id": "resp_001",
"output_index": 0,
"item": {
"id": "msg_007",
"object": "realtime.item",
"type": "message",
"status": "in_progress",
"role": "assistant",
"content": []
}
}
response.output_item.done
Schema: OpenAI.RealtimeServerEventResponseOutputItemDone.
Returned when an Item is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_item.done" |
Yes | The event type, must be response.output_item.done. |
response_id |
string |
Yes | The ID of the Response to which the item belongs. |
output_index |
OpenAI.integer | Yes | The index of the output item in the Response. |
item |
OpenAI.RealtimeConversationItem | Yes |
Example event:
{
"event_id": "event_3536",
"type": "response.output_item.done",
"response_id": "resp_001",
"output_index": 0,
"item": {
"id": "msg_007",
"object": "realtime.item",
"type": "message",
"status": "completed",
"role": "assistant",
"content": [
{
"type": "output_text",
"text": "Sure, I can help with that."
}
]
}
}
response.output_text.delta
Schema: OpenAI.RealtimeServerEventResponseTextDelta.
Returned when the text value of an "output_text" content part is updated.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_text.delta" |
Yes | The event type, must be response.output_text.delta. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
delta |
string |
Yes | The text delta. |
Example event:
{
"event_id": "event_4142",
"type": "response.output_text.delta",
"response_id": "resp_001",
"item_id": "msg_007",
"output_index": 0,
"content_index": 0,
"delta": "Sure, I can h"
}
response.output_text.done
Schema: OpenAI.RealtimeServerEventResponseTextDone.
Returned when the text value of an "output_text" content part is done streaming. Also emitted when a Response is interrupted, incomplete, or cancelled.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"response.output_text.done" |
Yes | The event type, must be response.output_text.done. |
response_id |
string |
Yes | The ID of the response. |
item_id |
string |
Yes | The ID of the item. |
output_index |
OpenAI.integer | Yes | The index of the output item in the response. |
content_index |
OpenAI.integer | Yes | The index of the content part in the item's content array. |
text |
string |
Yes | The final text content. |
Example event:
{
"event_id": "event_4344",
"type": "response.output_text.done",
"response_id": "resp_001",
"item_id": "msg_007",
"output_index": 0,
"content_index": 0,
"text": "Sure, I can help with that."
}
session.created
Schema: OpenAI.RealtimeServerEventSessionCreated.
Returned when a Session is created. Emitted automatically when a new connection is established as the first server event. This event will contain the default Session configuration.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"session.created" |
Yes | The event type, must be session.created. |
session |
VoiceAgentSessionResponse | Yes | The session configuration. |
conversation_id |
string |
No | The session-scoped conversation id. When present, responses attached to the session conversation use the same value in response.created and response.done. |
session.subagent.started
Schema: VoiceAgentServerEventSessionSubagentStarted.
The session.subagent.started server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.subagent.started" |
Yes | The event type. Always session.subagent.started. |
event_id |
string |
Yes | The server-generated event identifier. |
consultation_id |
string |
Yes | The identifier of the subagent consultation. |
call_id |
string |
Yes | The identifier of the function call that initiated the consultation. |
subagent_name |
string |
Yes | The name of the consulted subagent. |
session.subagent.completed
Schema: VoiceAgentServerEventSessionSubagentCompleted.
The session.subagent.completed server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.subagent.completed" |
Yes | The event type. Always session.subagent.completed. |
event_id |
string |
Yes | The server-generated event identifier. |
consultation_id |
string |
Yes | The identifier of the subagent consultation. |
call_id |
string |
Yes | The identifier of the function call that initiated the consultation. |
subagent_name |
string |
Yes | The name of the consulted subagent. |
session.subagent.aborted
Schema: VoiceAgentServerEventSessionSubagentAborted.
The session.subagent.aborted server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.subagent.aborted" |
Yes | The event type. Always session.subagent.aborted. |
event_id |
string |
Yes | The server-generated event identifier. |
consultation_id |
string |
Yes | The identifier of the subagent consultation. |
call_id |
string |
Yes | The identifier of the function call that initiated the consultation. |
subagent_name |
string |
Yes | The name of the consulted subagent. |
reason |
VoiceAgentSubagentAbortReason | Yes | The reason the consultation was aborted. |
session.updated
Schema: OpenAI.RealtimeServerEventSessionUpdated.
Returned when a session is updated with a session.update event, unless there is an error.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"session.updated" |
Yes | The event type, must be session.updated. |
session |
VoiceAgentSessionResponse | Yes | The session configuration. |
error
Schema: OpenAI.RealtimeServerEventError.
Returned when an error occurs, which could be a client problem or a server problem. Most errors are recoverable and the session will stay open, we recommend to implementers to monitor and log error messages by default.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The unique ID of the server event. |
type |
"error" |
Yes | The event type, must be error. |
error |
OpenAI.RealtimeServerEventErrorError | Yes | Details of the error. |
Example event:
{
"event_id": "event_890",
"type": "error",
"error": {
"type": "invalid_request_error",
"code": "invalid_event",
"message": "The 'type' field is missing.",
"param": null,
"event_id": "event_567"
}
}
warning
Schema: VoiceAgentServerEventWarning.
The warning server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"warning" |
Yes | |
event_id |
string |
Yes | |
warning |
VoiceAgentServerEventWarningDetails | Yes |
session.avatar.connecting
Schema: VoiceAgentServerEventSessionAvatarConnecting.
The session.avatar.connecting server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.avatar.connecting" |
Yes | |
event_id |
string |
Yes | |
server_sdp |
string |
Yes | The server's SDP answer for avatar media negotiation. |
rtc.call.sdp.created
Schema: VoiceAgentServerEventRtcCallSdpCreated.
The rtc.call.sdp.created server event: the SDP answer that completes WebRTC negotiation.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"rtc.call.sdp.created" |
Yes | The event type. Always rtc.call.sdp.created. |
event_id |
string |
Yes | The server-generated event identifier. |
rtc_call_id |
string |
Yes | The identifier of the established WebRTC call. |
sdp_answer |
string |
Yes | The server's SDP answer for the WebRTC connection. |
rtc.call.error
Schema: VoiceAgentServerEventRtcCallError.
The rtc.call.error server event: a WebRTC signaling failure.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"rtc.call.error" |
Yes | The event type. Always rtc.call.error. |
event_id |
string |
No | An optional server-generated event identifier. |
operation |
string |
No | The signaling operation that failed, when known. |
rtc_call_id |
string |
No | The identifier of the WebRTC call, when known. |
error |
VoiceAgentRtcCallErrorDetails | Yes | The error detail. |
session.avatar.switch_to_speaking
Schema: VoiceAgentServerEventSessionAvatarSwitchToSpeaking.
The session.avatar.switch_to_speaking server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.avatar.switch_to_speaking" |
Yes | |
event_id |
string |
Yes | |
turn_id |
string |
No |
session.avatar.switch_to_idle
Schema: VoiceAgentServerEventSessionAvatarSwitchToIdle.
The session.avatar.switch_to_idle server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"session.avatar.switch_to_idle" |
Yes | |
event_id |
string |
Yes | |
turn_id |
string |
No |
response.audio_timestamp.delta
Schema: VoiceAgentServerEventResponseAudioTimestampDelta.
The response.audio_timestamp.delta server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.audio_timestamp.delta" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
content_index |
integer |
Yes | format: "int32". |
audio_offset_ms |
FoundryDurationMilliseconds | Yes | |
audio_duration_ms |
FoundryDurationMilliseconds | Yes | |
text |
string |
Yes | |
timestamp_type |
"word" |
Yes |
response.audio_timestamp.done
Schema: VoiceAgentServerEventResponseAudioTimestampDone.
The response.audio_timestamp.done server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.audio_timestamp.done" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
content_index |
integer |
Yes | format: "int32". |
response.animation_blendshapes.delta
Schema: VoiceAgentServerEventResponseAnimationBlendshapesDelta.
The response.animation_blendshapes.delta server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.animation_blendshapes.delta" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
content_index |
integer |
Yes | format: "int32". |
frames |
Array of Array of number (format: "float".) |
Yes | Animation frames as numeric blendshape weights. |
frame_index |
integer |
Yes | The index of the first frame in this delta. format: "int32". |
response.animation_blendshapes.done
Schema: VoiceAgentServerEventResponseAnimationBlendshapesDone.
The response.animation_blendshapes.done server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.animation_blendshapes.done" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
response.animation_viseme.delta
Schema: VoiceAgentServerEventResponseAnimationVisemeDelta.
The response.animation_viseme.delta server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.animation_viseme.delta" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
content_index |
integer |
Yes | format: "int32". |
audio_offset_ms |
FoundryDurationMilliseconds | Yes | |
viseme_id |
integer |
Yes | format: "int32". |
response.animation_viseme.done
Schema: VoiceAgentServerEventResponseAnimationVisemeDone.
The response.animation_viseme.done server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.animation_viseme.done" |
Yes | |
event_id |
string |
Yes | |
response_id |
string |
Yes | |
item_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
content_index |
integer |
Yes | format: "int32". |
response.video.delta
Schema: VoiceAgentServerEventResponseVideoDelta.
The response.video.delta server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
type |
"response.video.delta" |
Yes | |
event_id |
string |
Yes | |
output_index |
integer |
Yes | format: "int32". |
codec |
string |
Yes | |
delta |
string |
Yes | The base64-encoded video frame data. |
session.handoff.started
Schema: VoiceAgentServerEventSessionHandoffStarted.
The session.handoff.started server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The server-generated event identifier. |
handoff_id |
string |
Yes | The server-generated handoff attempt identifier. |
edge_id |
VoiceAgentHandoffId | Yes | The configured edge selected for this handoff. |
from_node_id |
VoiceAgentHandoffNodeReference | Yes | The source node identifier or $entrypoint. |
to_node_id |
VoiceAgentHandoffNodeReference | Yes | The target node identifier or $entrypoint. |
from_model |
string |
Yes | The model active before the handoff. |
to_model |
string |
Yes | The model prepared for the target behavior. |
tool_call_id |
string |
Yes | The function-call identifier that initiated the handoff. |
node_generation |
integer |
Yes | The source generation for started or aborted events, and the committed target generation for completed events. format: "int32". minimum: 1. |
type |
"session.handoff.started" |
Yes | The event type. Always session.handoff.started. |
Required preview opt-ins: VoiceAgents=V1Preview.
session.handoff.completed
Schema: VoiceAgentServerEventSessionHandoffCompleted.
The session.handoff.completed server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The server-generated event identifier. |
handoff_id |
string |
Yes | The server-generated handoff attempt identifier. |
edge_id |
VoiceAgentHandoffId | Yes | The configured edge selected for this handoff. |
from_node_id |
VoiceAgentHandoffNodeReference | Yes | The source node identifier or $entrypoint. |
to_node_id |
VoiceAgentHandoffNodeReference | Yes | The target node identifier or $entrypoint. |
from_model |
string |
Yes | The model active before the handoff. |
to_model |
string |
Yes | The model prepared for the target behavior. |
tool_call_id |
string |
Yes | The function-call identifier that initiated the handoff. |
node_generation |
integer |
Yes | The source generation for started or aborted events, and the committed target generation for completed events. format: "int32". minimum: 1. |
prepare_duration_ms |
FoundryDurationMilliseconds | Yes | The target-preparation duration. |
duration_ms |
FoundryDurationMilliseconds | Yes | The total handoff duration. |
type |
"session.handoff.completed" |
Yes | The event type. Always session.handoff.completed. |
Required preview opt-ins: VoiceAgents=V1Preview.
session.handoff.aborted
Schema: VoiceAgentServerEventSessionHandoffAborted.
The session.handoff.aborted server event.
All of: OpenAI.RealtimeServerEvent.
| Property | Type | Required | Description |
|---|---|---|---|
event_id |
string |
Yes | The server-generated event identifier. |
handoff_id |
string |
Yes | The server-generated handoff attempt identifier. |
edge_id |
VoiceAgentHandoffId | Yes | The configured edge selected for this handoff. |
from_node_id |
VoiceAgentHandoffNodeReference | Yes | The source node identifier or $entrypoint. |
to_node_id |
VoiceAgentHandoffNodeReference | Yes | The target node identifier or $entrypoint. |
from_model |
string |
Yes | The model active before the handoff. |
to_model |
string |
Yes | The model prepared for the target behavior. |
tool_call_id |
string |
Yes | The function-call identifier that initiated the handoff. |
node_generation |
integer |
Yes | The source generation for started or aborted events, and the committed target generation for completed events. format: "int32". minimum: 1. |
reason |
VoiceAgentHandoffAbortReason | Yes | Why the handoff ended before target commit. |
error |
Any of: OpenAI.RealtimeServerEventErrorError; null |
No | The handoff failure detail when reason is error. |
type |
"session.handoff.aborted" |
Yes | The event type. Always session.handoff.aborted. |
Required preview opt-ins: VoiceAgents=V1Preview.