Why I always have the error "exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds" when using chat playground with my own data via file upload in Foundry in my free subscription?

Question

Why I always have the error "exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds" when using chat playground with my own data via file upload in Foundry in my free subscription?

Hung Nguyen 5

With my trial subscription, in Foundry, I

created a project
deployed either gpt-4o or gpt-4o-mini in either Sweden Central or East US 2.
uploaded a simple file in "Add my own data".
tried chatting.

I never succeeded chatting with my data. 100% of the times I hit this error: "Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-05-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds. Please contact Azure support service if you would like to further increase the default rate limit."

This error happened even it's the first chat request after the data is added. For one whole week I've been trying deleting and recreating projects, change all parameters that I can find but nothing helps.

AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-02T10:51:55.19+00:00
Hung Nguyen Greetings & Welcome to Microsoft Q&A forum!> Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-05-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds. Please contact Azure support service if you would like to further increase the default rate limit."

The error message is related to rate limits, which is a common practice in APIs to prevent abuse and ensure fair usage.

Did you check if you have exceeded the quota limit for your Azure OpenAI resources?

To give more context, As requests come into the deployment endpoint, the estimated max-processed-token count is added to a running token count of all requests that is reset each minute. If at any time during that minute, the TPM rate limit value is reached, then further requests will receive a 429 response code until the counter resets. For more details, see Understanding rate limits.

I understand your concern that you are getting an issue with first chat.

You could also try increasing the limit on your deployment.

Please see Manage Azure OpenAI Service quota for more details.

Also, To minimize issues related to rate limits, it's a good idea to use the following techniques:

Set max_tokens and best_of to the minimum values that serve the needs of your scenario. For example, don’t set a large max-tokens value if you expect your responses to be small.

Use quota management to increase TPM on deployments with high traffic, and to reduce TPM on deployments with limited needs.

Implement retry logic in your application.

Avoid sharp changes in the workload. Increase the workload gradually.

Test different load increase patterns.

Hope this helps. Do let me know if you have any further queries.
Hung Nguyen 5 Reputation points

2025-01-02T11:57:31.7666667+00:00

Hi,

I'm using Azure Free Subscription to try out the feature. With that subscription, the quote is limited to 1K token per minute and that cannot be changed.

As the service is designed to be tried out for free, I would expect to be able to at least send a question successfully. But as I mentioned in my question, I've been trying this for a week and ALL of my chat requests resulted in this error.
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-04T02:09:53.7033333+00:00

Just to ensure if it is working, can you try in different region other than Sweden Central/East US 2?
Hung Nguyen 5 Reputation points

2025-01-05T08:16:23.8866667+00:00

Other regions have no quota, which doesn't allow deploying the model. It can only be deployed in Sweden central or East US 2.

Out of hundreds of tries, a request seemed to pass "ChatCompletions_Create Operation under Azure OpenAI API" but then hit another limiter as the error message is different:

"An error occurred when calling Azure OpenAI: Server responded with status 429. Error message: {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 24 seconds.'}}".

Why is it designed for free trial but made so difficult to have one successful request?
Hung Nguyen 5 Reputation points

2025-01-05T08:56:27.8866667+00:00

Update:

I just tried my luck again and this time I had another error:

"Validation error at #/data_sources/0/azure_search/parameters/endpoint: Field required".

I'm not sure how this could happen, as an AI search service was provided otherwise data could not be uploaded (followed steps described here:

https://learn.microsoft.com/en-us/azure/ai-services/openai/use-your-data-quickstart?tabs=command-line%2Cjavascript-keyless%2Ctypescript-keyless%2Cpython-new&pivots=ai-foundry-portal ).
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-06T11:47:22.4466667+00:00

Understood. I tried multiple times in the mentioned regions and not able to reproduce the issue. Since you are using trial subscription, this could also be a cause of this.

Please see my private message.

Your answer

AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-02T10:51:55.19+00:00

Hung Nguyen Greetings & Welcome to Microsoft Q&A forum!> Requests to the ChatCompletions_Create Operation under Azure OpenAI API version 2024-05-01-preview have exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds. Please contact Azure support service if you would like to further increase the default rate limit."

The error message is related to rate limits, which is a common practice in APIs to prevent abuse and ensure fair usage.

Did you check if you have exceeded the quota limit for your Azure OpenAI resources?

To give more context, As requests come into the deployment endpoint, the estimated max-processed-token count is added to a running token count of all requests that is reset each minute. If at any time during that minute, the TPM rate limit value is reached, then further requests will receive a 429 response code until the counter resets. For more details, see Understanding rate limits.

I understand your concern that you are getting an issue with first chat.

You could also try increasing the limit on your deployment.

Please see Manage Azure OpenAI Service quota for more details.

Also, To minimize issues related to rate limits, it's a good idea to use the following techniques:

Set max_tokens and best_of to the minimum values that serve the needs of your scenario. For example, don’t set a large max-tokens value if you expect your responses to be small.

Use quota management to increase TPM on deployments with high traffic, and to reduce TPM on deployments with limited needs.

Implement retry logic in your application.

Avoid sharp changes in the workload. Increase the workload gradually.

Test different load increase patterns.

Hope this helps. Do let me know if you have any further queries.
Hung Nguyen 5 Reputation points

2025-01-02T11:57:31.7666667+00:00

Hi,

I'm using Azure Free Subscription to try out the feature. With that subscription, the quote is limited to 1K token per minute and that cannot be changed.

As the service is designed to be tried out for free, I would expect to be able to at least send a question successfully. But as I mentioned in my question, I've been trying this for a week and ALL of my chat requests resulted in this error.
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-04T02:09:53.7033333+00:00

Just to ensure if it is working, can you try in different region other than Sweden Central/East US 2?
Hung Nguyen 5 Reputation points

2025-01-05T08:16:23.8866667+00:00

Other regions have no quota, which doesn't allow deploying the model. It can only be deployed in Sweden central or East US 2.

Out of hundreds of tries, a request seemed to pass "ChatCompletions_Create Operation under Azure OpenAI API" but then hit another limiter as the error message is different:

"An error occurred when calling Azure OpenAI: Server responded with status 429. Error message: {'error': {'code': '429', 'message': 'Rate limit is exceeded. Try again in 24 seconds.'}}".

Why is it designed for free trial but made so difficult to have one successful request?
Hung Nguyen 5 Reputation points

2025-01-05T08:56:27.8866667+00:00

Update:

I just tried my luck again and this time I had another error:

"Validation error at #/data_sources/0/azure_search/parameters/endpoint: Field required".

I'm not sure how this could happen, as an AI search service was provided otherwise data could not be uploaded (followed steps described here:

https://learn.microsoft.com/en-us/azure/ai-services/openai/use-your-data-quickstart?tabs=command-line%2Cjavascript-keyless%2Ctypescript-keyless%2Cpython-new&pivots=ai-foundry-portal ).
AshokPeddakotla-MSFT 35,971 Reputation points Moderator

2025-01-06T11:47:22.4466667+00:00

Understood. I tried multiple times in the mentioned regions and not able to reproduce the issue. Since you are using trial subscription, this could also be a cause of this.

Please see my private message.

Share via

Why I always have the error "exceeded token rate limit of your current AIServices S0 pricing tier. Please retry after 86400 seconds" when using chat playground with my own data via file upload in Foundry in my free subscription?

Your answer