How to increase the higher TPM more than exceeding limit i.e 200K.

Question

Customer need help on increasing the TPM more than 200K exceeding limit for the OpenAI GPT-4 Turbo model. Customer already requested for the increasing limit, but it is not approved. Customer is facing latency issue and requested for increasing the TPM.
Shared the document below for reference.
https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits#how-to-request-increases-to-the-default-quotas-and-limits
Kindly need confirmation and steps for increasing the TPM more than 200K.

Answer 1

Hey there Paluri Krishnaji (MINDTREE LIMITED)

Thats a good question and thanks for using QandA platform

The rate limits are based on a sliding time window, when you make too many requests within that timeframe, you’ll be unable to make new request’s until that usage has slit it’s way out of the time window.

Have a look at these docs

https://platform.openai.com/docs/guides/rate-limits/error-mitigation?context=tier-free

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota?tabs=rest

If this helps kindly accept the answer thanks much.