What is causing a 429 rate Limit issue in Azure AI Studio Chat Completions

Question

When I use the chat playground in Azure OpenAI studio I'm frequently encountering 429 rate limit issues. This only happens when I connect a Azure AI search service. This is happening after two messages to the service.

the first message is -

What are my password complexity requirements?

The playground responds with the correct message:

The password complexity requirements for the ___________________________________ are as follows:

Passwords must be at least 8 characters long.

Passwords must contain characters from three of the following categories:

English uppercase characters (A through Z)

English lowercase characters (a through z)

Base 10 digits (0 through 9)

Non-alphabetic characters (for example, !, $, #, %)

Passwords must not contain the user’s account name or parts of the user’s full name that exceed two consecutive characters.

Users are prohibited from resetting their password to the 24 previous passwords 1 2 .

The second request is -

What is my teams recoding policy?

The playground responds with:

Server responded with status 429. Error message: {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 30 seconds.'}}

What rate limit is causing this?

English uppercase characters (A through Z)

What rate limit is causing this?

Answer 1

@Max Lacy we are sorry to hear you're facing this issue. Based on what you shared it seems like this problem is more related to Azure AI Search service rate limits rather than Azure OpenAI's rate limits. When you integrate Azure AI Search with Azure OpenAI in the playground, each query to generate a response typically involves multiple searches against your Azure AI Search index to retrieve relevant context.

There are a few potential rate limits that could be causing this:

1. Queries Per Second (QPS) limit: Azure AI Search has different QPS limits based on your service tier:
   • Free tier: 3 QPS | Basic tier: 15 QPS | Standard S1 tier: 25 QPS
   • Higher tiers have progressively higher limits
2. Also, there might be a limit on how many documents can be extracted per query.

To troubleshoot you should:

1. Check your Azure AI Search service tier. If you're on a lower tier, you might need to upgrade to a higher tier with better rate limits.
2. Look at the Search service metrics in Azure Portal to see if you're hitting QPS limits.
3. You could implement retry logic with exponential backoff in a production environment, though this isn't applicable in the playground.
4. Consider adjusting the number of documents retrieved per search or the number of searches performed per query if possible.
5. You might also want to check if there are other processes or users querying the same Search service

Hope that helps.

-Grace

Answer 2

I have the same problem.

I have a search service, standard tier.

When I attach a data source in the chat playground, after two or three prompts we get

Rate Limit Exceeded. Adjust your tokens per minute rate limit in Deployments (link) or try again later.

Now that message suggests the deployment - gtp-4o-mini which is set to 25k tokens per minute.

No other services/apps using this deployed model.

I do not see the error until the AI Search service is connected. The AI Search service metrics do not show anything like heavy usage and nothing to suggest throttling.