Trying to determine what is considered reasonable request for a increase in TPM quota on Azure OpenAI Studio.

Question

We are trying to determine what a reasonable request for an increase in TPM quota would be for Azure OpenAI studio. Based on the below.

use case:

We convert PPTX slides to images and then use GPT-4-Vision to help make sense of the text. This is needed because PPTXs rely a lot on visual association of elements.

Using vision during the ingestion process is very challenging right now with the 30K tokens per minute rate limit.

model details to start with would be:

Deployment : gpt-4-v

Region: West-US

Current Rate Limit: 30K TPM

Requested: I just requested 60K

Answer 1

Al_AA23 Greetings & Welcome to Microsoft Q&A forum!

To determine a reasonable request for an increase in TPM quota, it depends on your specific requirements. For example, expected number of transactions per minute.

To give more context, When a deployment is created, the assigned TPM will directly map to the tokens-per-minute rate limit enforced on its inferencing requests. A Requests-Per-Minute (RPM) rate limit will also be enforced whose value is set proportionally to the TPM assignment using the following ratio: 6 RPM per 1000 TPM.

You have mentioned that you have requested an increase to 60K TPM. This seems like a reasonable request, but it ultimately depends on the specific needs of your use case.

If you find that 60K TPM is still not enough to meet your needs, you can always apply for another quota increase in the future.

I hope this helps. Do let me know if you have any specific queries.