Why am I being throttled, or rate limited on Azure OpenAI API

Question

Why am I being throttled, or rate limited on Azure OpenAI API

TDerenthal 30

I am able to submit a prompt and get a completion once. Then I get an error "We're experiencing heavy traffic right now. Please try again at a later time." I have to about 1 minute before I can submit and get a completion. I do not have this problem at all when I call the OpenAI API. This behavior is consistent. So, I am ready to give up on Azure OpenAI entirely and just call the OpenAI API and build my application with that.

YutongTie-MSFT 53,941 Reputation points

2023-08-07T20:49:35.2933333+00:00

Hello @TDerenthal

Sorry to hear your experience, may I know which region and which model you are using? Thanks.
TDerenthal 30 Reputation points

2023-08-07T20:52:08.88+00:00

I have tried it in eastus2 for 3.5 and 4, and I have tried it in eastus for 3.5. I have the same problem regardless of region or model
TDerenthal 30 Reputation points

2023-08-08T14:06:06.92+00:00

No resolution? I am a partner, and I have access through my Partner IUR benefit. However, it looks like support for this would be perhaps handled differently than a typical Azure support or billing issue. If that is indeed the case, can you direct me to the proper support channel?

I really need more than one completion per minute. This is completely unusable, so I am writing my code to go directly against OpenAI's API. But I wlil need the features of Azure OpenAI at some point very soon. What do I need to do to get this working?
TDerenthal 30 Reputation points

2023-08-08T15:55:52.82+00:00

Hello? Where do I get support for this?
TDerenthal 30 Reputation points

2023-08-08T16:03:42.8333333+00:00

Yes, please do that. Thank you very much!
YutongTie-MSFT 53,941 Reputation points

2023-08-08T16:07:36.5833333+00:00

Hello @TDerenthal

Please send me an email with your Azure Subscription ID to Azcommunity@microsoft.com and title it as ATTN:Yutong

I can take action than.

Regards,

Yutong
TDerenthal 30 Reputation points

2023-08-08T18:44:53.0566667+00:00

Done.
YutongTie-MSFT 53,941 Reputation points

2023-08-14T04:22:54.37+00:00

Hello @TDerenthal

Thanks for your confirmation from email side that recreation of a new deployment works, could you please accept the answer if you feel helpful?

Regards,

Yutong

Accepted answer

1 additional answer

Your answer

YutongTie-MSFT 53,941 Reputation points

2023-08-07T20:49:35.2933333+00:00

Hello @TDerenthal

Sorry to hear your experience, may I know which region and which model you are using? Thanks.
TDerenthal 30 Reputation points

2023-08-07T20:52:08.88+00:00

I have tried it in eastus2 for 3.5 and 4, and I have tried it in eastus for 3.5. I have the same problem regardless of region or model
TDerenthal 30 Reputation points

2023-08-08T14:06:06.92+00:00

No resolution? I am a partner, and I have access through my Partner IUR benefit. However, it looks like support for this would be perhaps handled differently than a typical Azure support or billing issue. If that is indeed the case, can you direct me to the proper support channel?

I really need more than one completion per minute. This is completely unusable, so I am writing my code to go directly against OpenAI's API. But I wlil need the features of Azure OpenAI at some point very soon. What do I need to do to get this working?
TDerenthal 30 Reputation points

2023-08-08T15:55:52.82+00:00

Hello? Where do I get support for this?
TDerenthal 30 Reputation points

2023-08-08T16:03:42.8333333+00:00

Yes, please do that. Thank you very much!
YutongTie-MSFT 53,941 Reputation points

2023-08-08T16:07:36.5833333+00:00

Hello @TDerenthal

Please send me an email with your Azure Subscription ID to Azcommunity@microsoft.com and title it as ATTN:Yutong

I can take action than.

Regards,

Yutong
TDerenthal 30 Reputation points

2023-08-08T18:44:53.0566667+00:00

Done.
YutongTie-MSFT 53,941 Reputation points

2023-08-14T04:22:54.37+00:00

Hello @TDerenthal

Thanks for your confirmation from email side that recreation of a new deployment works, could you please accept the answer if you feel helpful?

Regards,

Yutong

Answer 1

Hello @TDerenthal

For the support channel - I can enable you a free support ticket on your case to investigate your backend data and also your quota, please let me know if you need that.

If you have a support plan, you can raise a support ticket directly and share the ticket number with me so that I can track it.

We will close this case here, support side will investigate it on the backend.

Regards,

Yutong

-Please kindly accept the answer and vote 'Yes' if you feel helpful to support the community, thanks a lot.

Answer 2

TDerenthal 30

I deleted my model deployments and then re-created them. That fixed the problem.

Share via

Why am I being throttled, or rate limited on Azure OpenAI API

1 additional answer

Your answer