Latency and Timeout Errors for Azure OpenAI o1/o3mini API request

Question

Latency and Timeout Errors for Azure OpenAI o1/o3mini API request

Simon Huang 15

Azure openai o1/o3mini APIs are barely usable now. Most requests result in timeout errors. it takes over 1 minute to respond a "Hi" even if you set reasoning effort to low.

Region: eastus2

kothapally Snigdha 2,095 Reputation points Microsoft External Staff

2025-02-20T18:40:29.67+00:00

Hi Simon Huang,

There is an ongoing issue that our product team is working on.

Thank you!
Pedro Castelo Branco Lourenço 41 Reputation points

2025-02-21T07:44:30.28+00:00

Is there an ETA? Why if we go to Service Status it always says it is Healthy? I am facing similar issues since 3-4 days on Sweden with o1 and o3-mini. The lack of transparency and clear communication is depressing.
Sven Leidenbach 10 Reputation points

2025-02-21T07:51:44.57+00:00

It would be nice if microsoft could actually inform us about the issue at hand with key information instead of forcing users to discover the latency is an issue for everyone on the forums.

In the current status, the o3-mini models is simply unusable due to timeouts and variability in response time. I simply can't reliably use azure AI services if left in the dark like this.

Some status updates, a working status site with current expected tokens/s etc would be a good start.
YutongTie-MSFT 53,941 Reputation points

2025-02-21T17:59:59.1666667+00:00

@Simon Huang @Pedro Castelo Branco Lourenço @Sven Leidenbach Thanks for reaching out to us again, I will update you the status here daily until this issue has been mitigated, as of today, there is no ETA, but there are several tickets opening to product team and product team is investigating this issue.

For lack of announcement, I will share the feedback to the team.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-23T04:50:49.0333333+00:00

@Simon Huang As today's update, engineering team is working on a hotfix, however, the ETA is still not clear to us. Hope to see more progress soon.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-24T05:55:48.72+00:00

@Simon Huang No new update during the weekend, we are looking forward to seeing more update on Monday, will keep you posted. Thanks.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-25T01:27:21.0433333+00:00

@Simon Huang and everyone, I just got an update that the fix for East US 2 region will be released by tomorrow (or later today), so please check tomorrow to see if your issue has been resolved or not.

Sweden Central regio has already got the fix.

I hope this helps.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-26T02:30:46.1066667+00:00

@Simon Huang Thank you for your patience. Latest update and for EastUS2 region -

Since this issue was caused by limit capacity, the product team update the design to enable dynamic routing which will solve time out issues.

The dynamic routing is built and will be release step by step, hope to cover most of the customers by the end of this week.

There are some options for now, one is Provisioned Managed sku for latency sensitive workload, and you may want to switch to other regions as alternative plan for this week.

Please refer to this document to Provisioned Managed sku - https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/provisioned-throughput-onboarding

Other regions/version for o1 -

https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#global-standard-model-availability

I will update here again once this issue confirmed resolved, will let you know more, I hope this helps.

Regards,

Yutong

1 answer

Your answer

kothapally Snigdha 2,095 Reputation points Microsoft External Staff

2025-02-20T18:40:29.67+00:00

Hi Simon Huang,

There is an ongoing issue that our product team is working on.

Thank you!
Pedro Castelo Branco Lourenço 41 Reputation points

2025-02-21T07:44:30.28+00:00

Is there an ETA? Why if we go to Service Status it always says it is Healthy? I am facing similar issues since 3-4 days on Sweden with o1 and o3-mini. The lack of transparency and clear communication is depressing.
Sven Leidenbach 10 Reputation points

2025-02-21T07:51:44.57+00:00

It would be nice if microsoft could actually inform us about the issue at hand with key information instead of forcing users to discover the latency is an issue for everyone on the forums.

In the current status, the o3-mini models is simply unusable due to timeouts and variability in response time. I simply can't reliably use azure AI services if left in the dark like this.

Some status updates, a working status site with current expected tokens/s etc would be a good start.
YutongTie-MSFT 53,941 Reputation points

2025-02-21T17:59:59.1666667+00:00

@Simon Huang @Pedro Castelo Branco Lourenço @Sven Leidenbach Thanks for reaching out to us again, I will update you the status here daily until this issue has been mitigated, as of today, there is no ETA, but there are several tickets opening to product team and product team is investigating this issue.

For lack of announcement, I will share the feedback to the team.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-23T04:50:49.0333333+00:00

@Simon Huang As today's update, engineering team is working on a hotfix, however, the ETA is still not clear to us. Hope to see more progress soon.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-24T05:55:48.72+00:00

@Simon Huang No new update during the weekend, we are looking forward to seeing more update on Monday, will keep you posted. Thanks.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-25T01:27:21.0433333+00:00

@Simon Huang and everyone, I just got an update that the fix for East US 2 region will be released by tomorrow (or later today), so please check tomorrow to see if your issue has been resolved or not.

Sweden Central regio has already got the fix.

I hope this helps.

Regards,

Yutong
YutongTie-MSFT 53,941 Reputation points

2025-02-26T02:30:46.1066667+00:00

@Simon Huang Thank you for your patience. Latest update and for EastUS2 region -

Since this issue was caused by limit capacity, the product team update the design to enable dynamic routing which will solve time out issues.

The dynamic routing is built and will be release step by step, hope to cover most of the customers by the end of this week.

There are some options for now, one is Provisioned Managed sku for latency sensitive workload, and you may want to switch to other regions as alternative plan for this week.

Please refer to this document to Provisioned Managed sku - https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/provisioned-throughput-onboarding

Other regions/version for o1 -

https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models?tabs=global-standard%2Cstandard-chat-completions#global-standard-model-availability

I will update here again once this issue confirmed resolved, will let you know more, I hope this helps.

Regards,

Yutong

Answer 1

@Simon Huang Thanks for reporting this issue again, the latency issue should be mitigated right now. Please check and let us know if you are still facing this issue.

As product team confirmed that this issue has been mitigated, I will convert this comment to answer so that we can close this case.

Appreciate the reporting and your patience again.

I hope this helps!

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, feel free to reach out to us if you see further issues.

Share via

Latency and Timeout Errors for Azure OpenAI o1/o3mini API request

1 answer

Your answer