OpenAI Service Assistant Playground Rate Limit Error

Kuroodo 0

I'm brand new to Azure, and trying to assess Azure for a prototype/demo of an app I am working on using Azure OpenAI Services, leveraging the assistants feature. As such, I made a new account to take advantage of the $200 credits I am given.

I set up a subscription, a resource group, and then added the OpenAI Service to the resource group.
When I went to make an assistant, I needed to make a deployment. However, I kept getting a warning about insufficient quota. I was still able to deploy a model in regular chat, and I could chat with it. However, I could not make an assistant through most deployment types. For some deployment types I was actually able to make an assistant and not have that quota error, however when trying to chat with it I kept getting a "rate_limit_exceeded: Rate limit is exceeded. Try again in 86400 seconds." error, when all I sent was a single message with the text "Hello".

I eventually learned that I may need to upgrade my account. I upgraded my account to the "Standard" tier.

Since doing that, the quotas warning is no longer there when making a deployment when making an assistant. I can even set the TPM now for the assistant. Clearly progress was made.

I now make a deployment for my assistant and send a single "Hello" message. I am once again met with the "rate_limit_exceeded: Rate limit is exceeded. Try again in 86400 seconds." error. I tried increasing the TPM from 2k to 11k and still get the same exact error.

The regions are all the same "East US". My quotas in the OpenAI Service are there and look fine as far as I can tell. In my subscription, when I go to Usage + Quotas, it's empty. There is also a warning at the top that says "The selected provider is not registered for some of the selected subscriptions. To access your quotas, register the resource provider.". As far as I can tell, Microsoft.CognitiveServices is already registered. I also already assigned my account the "Cognitive Services Usages Reader" permission.

How do I solve this?

kothapally Snigdha 505 Reputation points Microsoft Vendor

2024-11-28T23:50:24.22+00:00

Hi @Kuroodo

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

The error message is related to rate limits, which is a common practice in APIs to prevent abuse and ensure fair usage.

In your case, the error message indicates that you’ve exceeded the token rate limit of your current AI Services pricing tier. If you are on free tier, please upgrade to standard tier.

More info here.

Azure OpenAI’s quota feature enables assignment of rate limits to your deployments, up-to a global limit called your “quota.” Quota is assigned to your subscription on a per-region, per-model basis in units of Tokens-per-Minute (TPM).

You can check this documentation for more details.

To give more context, Tokens-Per-Minute (TPM) and Requests-Per-Minute (RPM) rate limits for the deployment.

TPM rate limits are based on the maximum number of tokens that are estimated to be processed by a request at the time the request is received.

RPM rate limits are based on the number of requests received over time. The rate limit expects that requests be evenly distributed over a one-minute period. If this average flow isn't maintained, then requests may receive a 429 response even though the limit isn't met when measured over the course of a minute.

Please see Manage Azure OpenAI Service quota for more details.

To view your quota allocations across deployments in a given region, select Shared Resources> Quota in Azure OpenAI studio and click on the link to increase the quota*.*

Hope this helps. Do let us know if you any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Kuroodo 0 Reputation points

2024-11-29T01:01:34.1766667+00:00

Hello,

Thank you for the reply.

Everything you have mentioned I have already addressed in my original post as something I have already tried.

The only thing I did not try is setting the TPM to the max value (nothing told me it was required for it to be set to max either).

After raising the TPM to the max value, 30k, I am able to send a message to the assistant via the playground. However, I'm only able to make one request (input/output) every 60 seconds. I have set the TPM of my deployment to 30k TPM, and it tells me the RPM is 300. However, after I send "Hello", get the assistant response, and then send "How are you", I get a response "rate_limit_exceeded: Rate limit is exceeded. Try again in 50 seconds.". I have to essentially wait 60 seconds per message. Why does the RPM appear to be 1 and not 300?
If I did not specify, the model I am using is gpt-4o-mini.
Kuroodo 0 Reputation points

2024-11-29T01:03:32.4333333+00:00

This was a duplicate comment made in error. Perhaps a mod can delete this one
kothapally Snigdha 505 Reputation points Microsoft Vendor

2024-11-29T23:35:17.5133333+00:00

Hi @Kuroodo

I recommend considering the Pay-As-You-Go subscription model. This model provides greater flexibility and control over your resource usage, as you’ll only pay for what you use. With Pay-As-You-Go, you can adjust the TPM and RPM limits more freely, potentially resolving the "rate limit exceeded" errors you're experiencing. This approach can help you avoid the limitations of fixed pricing tiers and allow you to scale your resources according to your actual needs.

If the issue persists, could you please file a support ticket for deeper investigation and do share the SR# with us? In case if you don't have a support plan, please let us know here.https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request.
Kuroodo 0 Reputation points

2024-11-30T00:45:24.08+00:00

Hello,

Upon testing the assistant once again today, it began working. The rate limit errors are gone.

I do not know what the issue was, but it appears that, for now, things are working fine.
Kuroodo 0 Reputation points

2024-12-01T13:23:59.25+00:00

Hello again,

I attempted to try the assistant yesterday and was once again hit with the rate limit issues. I am trying again today, and the issue persists. Specifically, I can only send one request per minute despite having an RPM of 300. I am sending "Hello", the assistant responds, then I reply with "How are you", and then get rate limit error. I've tried making various deployments with various deployment types. Same problem.

As I mentioned before, I have upgraded to the Basic account (I previously y said Standard, but this was a typo), therefore I cannot file support tickets. I am still assessing Azure for my product; therefore, I do not yet have a need to upgrade my account to a support plan.
My subscription is already pay-as-you go. I tried making a new subscription as a test, and I still get the same errors.

I would also like to add that if I try to make a new deployment through the assistant playground (Assistants -> Create New Deployment), if I select Global Basic I get a warning "Insufficient quota for selection options". This error used to be there several days ago before upgrading my account. It went away after upgrading my account and upgrading my subscription to pay-as-you-go. But looks like this issue has returned.

I am running out of patience. It has been four days and I have been unable to properly assess Azure for commercial use of my product. I have been using OpenAI's own offerings in the meantime without issue, however I will have to drop all of this for a different LLM offering if I am unable to use Azure. I need to work on this prototype soon, so I am eager to hear back so that we can get this resolved.

Thank you
kothapally Snigdha 505 Reputation points Microsoft Vendor

2024-12-02T11:40:47.1433333+00:00

Hi @Kuroodo

Could you please clear your cache and try again. If you're still facing the same issue, please log out and log back in. Also, if the problem persists, kindly share your region. we are tried to repro the scenario without any issues.

Thank you!

Share via

OpenAI Service Assistant Playground Rate Limit Error

Your answer