OpenAI GPT-4 TPM cannot be set to higher than 8K

Question

OpenAI GPT-4 TPM cannot be set to higher than 8K

Morten Johnsen 5

A customer of ours needs an OpenAI GPT-4 large language model (LLM) for an application they have purchased from us. They started out with a Pay-As-You-Go (0003P) subscription that, as stated in quotas-limits, has a maximum quota of 8K tokens per minute (TPM). This is too little and for that reason they obtained a new subscription of the type Azure Plan (0017G). However, even after swapping to that new subscription, the 8K TPM limit remains in place. They have tried different regions (including for instance "resource rich" Sweden Central) as not all regions have the same offerings. However, in no region was it possible to increase the configurable TPM beyond 8K.

What do the customer have to do to increase their GPT-4 TPM quota to let say 150K TPM, which is what other customers of ours have with the very same 0017G type subscription?

1 answer

Your answer

Answer 1

YutongTie-MSFT 53,966 Moderator

Hello @Morten

Thanks for reaching out to us, you can definitely request for more quota if your business need it. The gating team will review on your usage and meet your need as much as possible according to the region capacity.

You can do it from Azure OpenAI Studio as below screenshot, and you need to fill a quick form then.

User's image

At the begin, users will start from a reasonable quota, but as business need, users can request more.

I hope this helps.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.

Morten Johnsen 5 Reputation points

2024-07-31T05:48:10.6166667+00:00

Thanks Yotong for your response. I will advice the customer to request a larger TPM quota.

At the begin, users will start from a reasonable quota, but as business need, users can request more

What determines what a reasonable quota is?

My question is motivated by the fact that only one of our customers seems to ever having run into an initial GPT-4 8K TPM limit. We need to address this in our installation guide and in that regard I want to understand why all customers but one end up with the same much larger quota.

Regards,

Morten
YutongTie-MSFT 53,966 Reputation points Moderator

2024-08-04T06:10:23.96+00:00

Hello Morten, do you mean you want to increase the default quota for all your customers? If this is the case, I would suggest you raise a support ticket for this issue to see if there is anything support team can help.
Morten Johnsen 5 Reputation points

2024-08-05T08:08:27.7166667+00:00

Hi Yutong, each of our customers has their own Microsoft OpenAI LLM setup via their own subscription from which they get the API key and API address to configure our application that is running on a server in their environment. My contribution in this regard is to provide our customers with instructions on how they should go about configuring their MS OpenAI instance.

This is a single individual case of a customer following our instructions but ending up with a different maximum TPM for GPT-4 (just 8K) than all our other customers (80K, the same as the Microsoft documentation states). The only thing different with this customer is that they started out with another subscription type (Pay-as-you-go) with an 8K TPM limit before creating a new subscription of type Azure Plan. However, the old limit seems to have followed them to the new subscription.

Following your initial reply, the customer has already applied for a larger quota.
Srinu Tamada 45 Reputation points

2024-11-29T09:12:00.8833333+00:00

As part quota request there is no option for GPT-4 Turbo Vision. So, I clearly mentioned I wanted GPT-4 V quota to increase but it got increased to GPT-4 Turbo. Is there any way to specify GPT-4 V?

Share via

OpenAI GPT-4 TPM cannot be set to higher than 8K

1 answer

Your answer