Getting error about Rate Limit Exceeded

Question

Getting error about Rate Limit Exceeded

Linda Manganaro 40

I am doing a simple Learning exercise on Generative AI in Azure Portal and when I try the chat exercise I keep getting the message "Rate Limit Exceeded. Adjust your tokens per minute rate limit in Models + endpoints or try again later." The initial prompt works fine, but all subsequent ones fail with that message. Currently the TPS shows as 6. Shouldn't that be enough for a simple Learning task? What do I need to do for this to work?

VarunTha 14,880 Reputation points Microsoft External Staff Moderator

2025-05-08T11:40:34.83+00:00

Hi Linda Manganaro,
Could you please specify the Learn Path/Module you're following and share a screenshot of the error for better clarity?
Linda Manganaro 40 Reputation points

2025-05-08T14:56:43.1233333+00:00

Hi VarunTha - I am on Unit 9 (of 11) in the Introduction to Generative AI module of Azure Learning. The name of the exercise is "Explore generative AI in Azure AI Foundry portal". It is a short exercise about using the Chat playground. It goes through the steps of Creating a Deployment, choosing the gpt4 model. That was all successful, so I started the exercise, which was about planning a trip to Paris and interacting with the model through a series of prompts. The first prompt it asks to write worked well, but when I gave the second prompt and subsequent prompts it says Rate Limit Exceeded. Adjust your tokens per minute rate limit in Models + endpoints or try again later." Here is a screen shot. How do I resolve this? Thank you!
Linda Manganaro 40 Reputation points

2025-05-08T15:06:39.7033333+00:00

Hi VarunTha - I am on Unit 9 (of 11) in the Introduction to Generative AI module of Azure Learning. The name of the exercise is "Explore generative AI in Azure AI Foundry portal". It is a short exercise about using the Chat playground. It goes through the steps of Creating a Deployment, choosing the gpt4 model. That was all successful, so I started the exercise, which was about planning a trip to Paris and interacting with the model through a series of prompts. The first prompt it asks to write worked well, but when I gave the second prompt and subsequent prompts it says Rate Limit Exceeded. Adjust your tokens per minute rate limit in Models + endpoints or try again later." Here is a screen shot. How do I resolve this? Thank you!

1 answer

Your answer

VarunTha 14,880 Reputation points Microsoft External Staff Moderator

2025-05-08T11:40:34.83+00:00

Hi Linda Manganaro,
Could you please specify the Learn Path/Module you're following and share a screenshot of the error for better clarity?
Linda Manganaro 40 Reputation points

2025-05-08T14:56:43.1233333+00:00

Hi VarunTha - I am on Unit 9 (of 11) in the Introduction to Generative AI module of Azure Learning. The name of the exercise is "Explore generative AI in Azure AI Foundry portal". It is a short exercise about using the Chat playground. It goes through the steps of Creating a Deployment, choosing the gpt4 model. That was all successful, so I started the exercise, which was about planning a trip to Paris and interacting with the model through a series of prompts. The first prompt it asks to write worked well, but when I gave the second prompt and subsequent prompts it says Rate Limit Exceeded. Adjust your tokens per minute rate limit in Models + endpoints or try again later." Here is a screen shot. How do I resolve this? Thank you!
Linda Manganaro 40 Reputation points

2025-05-08T15:06:39.7033333+00:00

Hi VarunTha - I am on Unit 9 (of 11) in the Introduction to Generative AI module of Azure Learning. The name of the exercise is "Explore generative AI in Azure AI Foundry portal". It is a short exercise about using the Chat playground. It goes through the steps of Creating a Deployment, choosing the gpt4 model. That was all successful, so I started the exercise, which was about planning a trip to Paris and interacting with the model through a series of prompts. The first prompt it asks to write worked well, but when I gave the second prompt and subsequent prompts it says Rate Limit Exceeded. Adjust your tokens per minute rate limit in Models + endpoints or try again later." Here is a screen shot. How do I resolve this? Thank you!

Answer 1

VarunTha 14,880 Microsoft External Staff Moderator

Hi Linda Manganaro,

Thank you for providing the additional details.

We attempted to reproduce the issue using the exercise and obtained the following outcome, as shown in the screenshot below:

User's image

Could you please confirm whether you have selected the GPT-4 model? Selecting a different model may lead to a "rate limit exceeded" error. We have followed the documentation using GPT-4 and were able to achieve successful results.

If you have any further questions or need additional assistance, feel free to ask.

Linda Manganaro 40 Reputation points

2025-05-08T22:15:18.2833333+00:00

Thanks for getting back to me VarunTha. Yes, that is exactly the model that I deployed, as shown below. I don't know why it exceeds the Max Tokens per Second after only one prompt!
I just now ended up trying a new deploy, specifying US West instead of US East, because I found that sometimes behaviors are different when different locations are selected. But unfortunately, I got the SAME 'Rate Limit Exceeded' error.
What can I do about this? It is very frustrating, and I don't see any documentation that explains what to do anywhere. Can you help me with this, or otherwise, can you point me to someone who can help? Thanks.
VarunTha 14,880 Reputation points Microsoft External Staff Moderator

2025-05-09T05:41:48.5466667+00:00
Hi Linda Manganaro,

Thank you for the detailed information. As per the screenshot, I noticed that your "Model instructions and context" box is empty. To help improve the model’s performance and reduce the chance of issues like "Rate Limit Exceeded," please try the following steps:

Start with a specific goal for what you want the assistant to do

Iterate based on previous prompts and responses to refine the result

Provide a source to ground the response in a specific scope of information

Add context to maximize response appropriateness and relevance

Set clear expectations for the response

Please enter these five points in the "Model instructions and context" box in the Chat Playground, and then click on “Apply changes.” This will help the model respond more effectively and may help avoid rate limiting.

Let me know how it goes, and I’ll be happy to assist further if needed!
Linda Manganaro 40 Reputation points

2025-05-09T16:44:09.17+00:00

Hi VarunTha - I added some instructions in that box, as you suggested - but unfortunately, it looks like the issue is something else. What else can I try?
VarunTha 14,880 Reputation points Microsoft External Staff Moderator

2025-05-12T19:12:23.3266667+00:00

Hi Linda Manganaro,
Could you please confirm whether you're using an Azure Free Trial account or if you've upgraded to a Pay-As-You-Go subscription? If you're using the Free Trial, please try reducing the TPM (tokens per minute) from 250 to 200 and proceed with the remaining steps. If the issue persists, try lowering it further to 150 TPM.

Share via

Getting error about Rate Limit Exceeded

1 answer

Your answer