Word count limit for system message in gpt-4o-realtime API? Can gpt-4o-realtime API handle video?

Question

Word count limit for system message in gpt-4o-realtime API? Can gpt-4o-realtime API handle video?

GenixPRO 116

Hi Team,

Is there a word count limit for system message in gpt-4o-realtime API? When we pass a system message with about 2500 words -> API responds and user can have a conversation. However, when system message word count exceeds 6500 words -> API does not initiate a conversation. We don't receive any error message so don't know what may be the issue and how to troubleshoot.
Can gpt-4o-realtime API handle video?
We've tried audio and it works intermittently. How can we troubleshoot to find error message and the AI does not converse?

Thanks & appreciate your help.

1 answer

Your answer

Answer 1

Pavankumar Purilla 8,570 Microsoft External Staff Moderator

Hi GenixPRO,
Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

I understand that you are facing an issue with the gpt-4o-realtime API where system messages exceeding a certain word count (~6500 words) fail to initiate a conversation, and you're also exploring if the API can handle video.

The gpt-4o-realtime API has a token limit of 8,192 or 32,768 tokens, which includes the system message, user inputs, and model responses. If your system message exceeds this limit, the API may silently fail. For a system message exceeding ~6500 words (likely over 32,000 tokens), consider reducing its length or splitting the conversation into smaller parts. Use tools like OpenAI's Tokenizer to calculate token usage and ensure you're within limits.

Regarding video, the API does not support direct video processing. However, you can use Azure services like Video Indexer to extract metadata and transcription or Speech Services for audio-to-text conversion, then pass the processed text to the API. For intermittent audio issues, ensure you log errors, verify rate limits, and test in different Azure regions to troubleshoot latency or service degradation. Let us know if you need further assistance!

Hope this helps. Do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

GenixPRO 116 Reputation points

2024-12-13T05:22:18.1166667+00:00

@Pavankumar Purilla thanks for the prompt response. For video our use case is to build audio-video conversations like this video from Open AI -> https://www.youtube.com/watch?v=vgYi3Wr7v_g

Open Ai documentation suggests that gpt-4o-realtime may be handling video too? But we don't see any API documentation for audio+video. pls. help clarify. thanks.
GenixPRO 116 Reputation points

2024-12-13T05:34:28.8366667+00:00

Hi. Is it possible to increase token limit for gpt-4o-realtime API? If yes, how?
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-12-13T16:21:33.1933333+00:00

Hi GenixPRO,
Hope you are doing well.

Regarding your query about video support, at this time it does not provide direct support for processing video or combined audio-video inputs.
However, you can keep an eye on the Azure OpenAI Service release notes for updates on What's new in Azure OpenAI Service page will be updated accordingly with more details.

As for the token limit, the gpt-4o-realtime API has a fixed limit of 8,192 or 32,768 tokens, depending on your deployment type. Unfortunately, this limit cannot be increased as it is tied to the model’s architecture.
For more information, please follow the: Azure OpenAI Service quotas and limits

I hope this information helps. Thank you!
GenixPRO 116 Reputation points

2024-12-14T23:34:38.47+00:00

@Pavankumar Purilla you mentioned "the gpt-4o-realtime API has a fixed limit of 8,192 or 32,768 tokens, depending on your deployment type." We want 32,768 tokens and we're currently at 6000 token limit. How do we increase this limit? What deployment type can we choose. East US2 region indicates 6k tokens max.
GenixPRO 116 Reputation points

2024-12-16T08:39:32.73+00:00

@Pavankumar Purilla Open AI documentation here https://platform.openai.com/docs/models suggests that gpt-4o-realtime-preview Context window allows 128k tokens. The system message is part of Context window. Correct? If so, why is our msg being restricted to 8k tokens? How can we increase limit to 32,768 tokens or more? Thanks.
Pavankumar Purilla 8,570 Reputation points Microsoft External Staff Moderator

2024-12-17T16:10:50.7166667+00:00

Hi GenixPRO,
Unfortunately, the token limit for the gpt-4o-realtime API in the East US 2 region is fixed at 6k tokens, and it cannot be increased at this time. I apologize for any inconvenience this may cause.
However, you can keep an eye on the Azure OpenAI Service release notes for updates on What's new in Azure OpenAI Service page for updates, as token limits and model availability may change in the future.
Thank you for your understanding!

Share via

Word count limit for system message in gpt-4o-realtime API? Can gpt-4o-realtime API handle video?

1 answer

Your answer