Azure AI - Error 429 GPT4o Chat Playground

Question

Dear Team,

I have configured in Azure AI Chat Playground the "Add your data" (Preview) with GPT4o deployment.

It is connected to my Azure AI Search Index which has 9 PDF documents.

Whenever I chat with the bot, it successfully respond to the first question, however if I ask another question related to the documents, its easily throws an error which states

"Server responded with status 429. Error message: {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 27 seconds.'}"

I have tested it on the other Chat Playground where I do not configure enterprise data, and it seems to work quite well. Would like to know what could be the reason for this? does this have to do with TPM quota limits?

Answer 1

The error 429 means that you have submitted too many tokens or requests in a short period of time and have exceeded the number of requests allowed.

Azure services often enforce quotas on transactions per minute (TPM). If your application or service exceeds this quota, it will start receiving 429 errors.

So you need to check your Azure portal for the specific TPM limits for your Azure AI service (in this case, Azure AI Chat Playground with GPT4o deployment and Azure AI Search). If you're hitting the TPM limits, you may need to adjust your application logic to spread out requests more evenly over time or consider increasing your TPM quota.

https://help.openai.com/en/articles/6891829-error-code-429-rate-limit-reached-for-requests

https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits