Getting Rate Limit Exceeded when testing AI Agent in AI Foundry

Question

Getting Rate Limit Exceeded when testing AI Agent in AI Foundry

Wenjun Che 140

Hello

I am following this tutorial https://microsoftlearning.github.io/mslearn-ai-agents/Instructions/02-build-ai-agent.html to create an AI Agent with Grounding with Bing Search. When I test it in Playground with a simple prompt, such as "what is the temperature of New York city today", it is getting Rate Limit Exceeded. It is using a deployed gpt-4 model with default rate limit of 8000 as tokens per minute and 48 as requests per minute.

Thanks

Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T16:27:37.93+00:00

Hi Wenjun Che,

It seems that there might be a rate limit issue with the AI Agent. Could you please provide a screenshot of the error message you are receiving? This will help us diagnose the problem more effectively.

Thank you.
Wenjun Che 140 Reputation points

2025-03-21T16:52:27.8533333+00:00

Attached is the error message I received
Wenjun Che 140 Reputation points

2025-03-21T16:56:31.5466667+00:00

AgentError.png
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T18:19:09.58+00:00

Hi Wenjun Che,

We attempted to reproduce the issue using the exercise and obtained the following outcome, as shown in the screenshot below:

It appears that the rate limit for your deployed GPT-4 model may have been exceeded. Azure enforces strict limits on token consumption per minute, hour, and day, as well as the rate of API calls per second or minute. In some cases, underlying configuration issues can also trigger rate limit errors.

To resolve this, we recommend deleting the model, refreshing the environment, and redeploying it to reset any potential configuration issues. Please let us know if you need further assistance.
Wenjun Che 140 Reputation points

2025-03-21T18:31:16.1633333+00:00

When I try to deploy gpt-4, the only version available is turbo-2024-04-09 (attached screenshot). How did you deploy version 0613 ? I want to select the same version to see if it works.AgentError.png
Wenjun Che 140 Reputation points

2025-03-21T18:36:16.21+00:00

I just checked your screenshot again. Based on the answer, it does not look like you added Grounding with Bing Search in the Knowledge section. If you did, it should have search for the temperature with Bing Search for real time information, instead of saying not able to provide real-time updates.
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T18:41:57.6666667+00:00

I followed the instructions provided in the exercise document.
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T19:07:57.75+00:00

As per the document, Grounding with Bing Search cannot be added. It is only supported with the following Azure OpenAI models: gpt-3.5-turbo-0125, gpt-4-0125-preview, gpt-4-turbo-2024-04-09, gpt-4o-0513.

To address this issue, we have reached out to the content author for the earliest possible resolution.
Wenjun Che 140 Reputation points

2025-03-21T19:22:12.53+00:00

I am actually using gpt-4-turbo-2024-04-09 when getting the error
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T19:35:20.7933333+00:00

However, Grounding with Bing Search cannot be added with the GPT-4 model, as stated in the exercise instructions. We have reached out to the internal team to request updates to the exercise document accordingly.
Wenjun Che 140 Reputation points

2025-03-21T21:45:59.2866667+00:00

can you try one of gpt-3.5-turbo-0125, gpt-4-0125-preview, gpt-4-turbo-2024-04-09, gpt-4o-0513 to see if any works for you ?
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-22T07:59:48.2066667+00:00

We attempted to add multiple GPT models but encountered a "rate limit exceeded" error.
Wenjun Che 140 Reputation points

2025-03-22T15:18:32.0633333+00:00

so you can reproduce the issue. What is the next step ?

Accepted answer

0 additional answers

Your answer

Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T16:27:37.93+00:00

Hi Wenjun Che,

It seems that there might be a rate limit issue with the AI Agent. Could you please provide a screenshot of the error message you are receiving? This will help us diagnose the problem more effectively.

Thank you.
Wenjun Che 140 Reputation points

2025-03-21T16:52:27.8533333+00:00

Attached is the error message I received
Wenjun Che 140 Reputation points

2025-03-21T16:56:31.5466667+00:00

AgentError.png
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T18:19:09.58+00:00

Hi Wenjun Che,

We attempted to reproduce the issue using the exercise and obtained the following outcome, as shown in the screenshot below:

It appears that the rate limit for your deployed GPT-4 model may have been exceeded. Azure enforces strict limits on token consumption per minute, hour, and day, as well as the rate of API calls per second or minute. In some cases, underlying configuration issues can also trigger rate limit errors.

To resolve this, we recommend deleting the model, refreshing the environment, and redeploying it to reset any potential configuration issues. Please let us know if you need further assistance.
Wenjun Che 140 Reputation points

2025-03-21T18:31:16.1633333+00:00

When I try to deploy gpt-4, the only version available is turbo-2024-04-09 (attached screenshot). How did you deploy version 0613 ? I want to select the same version to see if it works.AgentError.png
Wenjun Che 140 Reputation points

2025-03-21T18:36:16.21+00:00

I just checked your screenshot again. Based on the answer, it does not look like you added Grounding with Bing Search in the Knowledge section. If you did, it should have search for the temperature with Bing Search for real time information, instead of saying not able to provide real-time updates.
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T18:41:57.6666667+00:00

I followed the instructions provided in the exercise document.
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T19:07:57.75+00:00

As per the document, Grounding with Bing Search cannot be added. It is only supported with the following Azure OpenAI models: gpt-3.5-turbo-0125, gpt-4-0125-preview, gpt-4-turbo-2024-04-09, gpt-4o-0513.

To address this issue, we have reached out to the content author for the earliest possible resolution.
Wenjun Che 140 Reputation points

2025-03-21T19:22:12.53+00:00

I am actually using gpt-4-turbo-2024-04-09 when getting the error
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-21T19:35:20.7933333+00:00

However, Grounding with Bing Search cannot be added with the GPT-4 model, as stated in the exercise instructions. We have reached out to the internal team to request updates to the exercise document accordingly.
Wenjun Che 140 Reputation points

2025-03-21T21:45:59.2866667+00:00

can you try one of gpt-3.5-turbo-0125, gpt-4-0125-preview, gpt-4-turbo-2024-04-09, gpt-4o-0513 to see if any works for you ?
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-22T07:59:48.2066667+00:00

We attempted to add multiple GPT models but encountered a "rate limit exceeded" error.
Wenjun Che 140 Reputation points

2025-03-22T15:18:32.0633333+00:00

so you can reproduce the issue. What is the next step ?

Answer 1

Rakesh Gurram 15,700 Microsoft External Staff Moderator

Hi Wenjun Che,

We reached out to the content author regarding the issue, and they confirmed that it has been resolved. According to their response, the GPT model has been updated, and Grounding with Bing Search is now available as part of the model. Additionally, the rate-limiting issue is a known product-related limitation. If you encounter it, the recommended approach is to wait, as it should resolve itself over time.

Please reach out to us if you have any other queries.

If the information is helpful, please Accept Answer & Upvote so that it would be helpful to other community members.

Wenjun Che 140 Reputation points

2025-03-25T13:45:59.78+00:00
Hello Rakesh

I just tried again as following:

sent "what is temperature of New York today", got rate-limit error

waited for 20 minutes

sent the same prompt and got rate-limit error

This test has never worked for me, so I don't think it is related to "wait".
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-27T08:52:42.6833333+00:00

Hi Wenjun Che,

We just tried again to repro the issue by keeping the Tokens per Minute Rate Limit to 20K and it worked. We have seen minimum required TPM to use this model is 20k TPM.

Thank you.
Wenjun Che 140 Reputation points

2025-03-27T13:50:19.25+00:00

Hello Rakesh

Which model did you test with? I just checked gpt-4 (v613) and the max TPM is 8k
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-27T14:43:04.0533333+00:00

Model- GPT-4

Model version: turbo-2024-04-09

TPM- 20k
Wenjun Che 140 Reputation points

2025-03-27T14:57:40.5466667+00:00

Attached is the deployment page for turbo-2024-04-09 that I am seeing. The max TPM allowed is 8K.
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-03-27T16:30:55.2533333+00:00

Hi Wenjun Che,

It appears that your issue may be related to Azure OpenAI’s quota limits. Even after upgrading to a Pay-As-You-Go model, your subscription may still have usage restrictions for GPT-4.

Azure OpenAI applies rate limits per model and region, measured in Tokens-Per-Minute (TPM) and Requests-Per-Minute (RPM).

For more info, please refer to Manage Azure OpenAI Service quota.

Below is a screenshot showing where we were able to set the maximum TPM to 35k.
Wenjun Che 140 Reputation points

2025-03-31T13:39:47.7433333+00:00

Hello Rakesh

I just got an email for my application for increasing quota being rejected. Here is the detail:

At this time your quota request cannot be processed.`` ``The region East US you requested a quota increase in is not available to expand capacity at this time.`` ``Please expand your regional choices when submitting a new request or wait 30+ days before resubmitting for a quota increase within the same region again.

Which region should I apply for quota increase?
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-04-01T17:02:09.2+00:00

Hi Wenjun Che,

Your request for a quota increases in the East US region was unfortunately denied due to capacity limitations. To proceed, we recommend selecting an alternative region with available capacity, such as Australia East or East US 2. You may refer to document for region availability details. To improve the chances of approval, please consider submitting a new request for one of these regions.
Wenjun Che 140 Reputation points

2025-04-08T17:46:05.1933333+00:00

Hello Rakesh

My 2nd request got rejected as well: The region East US 2 you requested a quota increase in is not available to expand capacity at this time. I even checked Grant me quota in an alternate region and selected Anywhere in the USA.

What else can I do?
Wenjun Che 140 Reputation points

2025-04-10T15:27:48.65+00:00

Finally I got it working by deploying gpt-4o to US West 3 with Global Standard.
Wenjun Che 140 Reputation points

2025-04-10T15:28:03.21+00:00

Finally I got it working by deploying gpt-4o to US West 3 with Global Standard
Rakesh Gurram 15,700 Reputation points Microsoft External Staff Moderator

2025-04-10T16:46:38.5333333+00:00

I'm glad to hear that your issue has been resolved. As mentioned earlier, it may have been related to a product limitation based on the selected model.

Thank you.

Share via

Getting Rate Limit Exceeded when testing AI Agent in AI Foundry

0 additional answers

Your answer