Getting rate limited on Azure OpenAI 429 even with low usage and extremely high TPM

Question

We are getting rate limited randomly with 429s even though we have very high TPM and very low usage.

We do not experience this if we use Open AI directly instead of via azure

Answer 1

Hello Lars,

Thanks for reaching out to Microsoft Q&A.

A 429 response in Azure OpenAI means you’ve hit a rate limit, even if your overall usage looks low compared to your quota. This happens because Azure enforces limits in short time slices (like 1- or 10-second windows). If too many requests bunch up inside one of those windows, the service may return 429 until the next window opens.

It’s also important to keep in mind that tokens-per-minute (TPM) and requests-per-minute (RPM) are tracked separately. A high TPM quota doesn’t prevent rate limiting if your RPM bursts are too high or unevenly distributed. This is one of the main differences between Azure’s enforcement and OpenAI’s direct API, which is why you may not see the same issue there.

To reduce 429s, try spreading requests more evenly over time, implementing retry logic with exponential backoff, and checking both your TPM and RPM limits in the portal. If your workload is bursty, distributing traffic across multiple deployments or regions can also help. And if the issue persists for production workloads, opening a support request is the best way to confirm quotas and request adjustments.Please find the attached document for your reference:

• Manage Azure OpenAI in Azure AI Foundry Models quota
• Azure OpenAI frequently asked questions

I Hope this helps. Do let me know if you have any further queries.

---

If this answers your query, please do click Accept Answer and Yes for was this answer helpful.

Thank you!