Why are calls to OpenAI API timing out?

James O 50

I have an Azure app service that utilizes the OpenAI GPT API and the requests frequently timeout; however, the API calls work fine outside of Azure. I have many apps outside of Azure that utilize the exact same OpenAI API and they have NEVER timed out, yet whenever I run the same calls from Azure, they timeout.

I have seen other users are having the same issue so it has nothing to do with my app/code.

Is there anyone that can offer advice?

Grmacjon-MSFT 16,191 Reputation points

2023-05-17T19:01:55.5766667+00:00

Hi @James O thanks for bringing this to our attention. to clarify you're not using the Azure OpenAI API correct? when did you start getting the timeout message. can you please post the full error message and make sure you didn't exceed any rate limits?
Nick Cochrane 30 Reputation points

2023-06-07T04:30:06.3933333+00:00

I am having the same issue. OpenAI API calls using langchain work great (they take several minutes of chewing through documents), but Azure OpenAI calls timeout after 60 seconds. Reason I wish to try Azure OpenAI is data privacy (think it's the main use case for most), but this is preventing us from scaling up usage.
Nick Cochrane 30 Reputation points

2023-06-07T04:32:25.74+00:00

Duplicate response.
Vikas Rijsinghani 25 Reputation points

2023-06-09T13:40:52.3533333+00:00

Having same issue. Seems to be happening more recently. The profile is everything is working fine, but roughly 5% of requests get either timed out or give a response minutes later (2-4 min). Very annoying in an interactive application and not a good look for us.
Pierangelo Calanna 10 Reputation points

2023-06-09T17:17:30.38+00:00

I am also facing the same issue, is this currently being handled by Microsoft?
Vikas Rijsinghani 25 Reputation points

2023-06-09T21:51:18.2833333+00:00

Some timeframes of the issue evolving.

Up until May 16, our max latency times were 2 minutes, <1% error rates.

From May 16 to June 6 max latencies went up to 5 minutes, <2% error rates

From June 6 to now max latencies are 10 minutes and error rates jumped to 5%.
Burlik, Vlad V 5 Reputation points

2023-06-21T17:42:58.9433333+00:00

This issue is still outstanding and causes a lot of pain.
05074117 0 Reputation points

2023-06-21T18:34:23.7966667+00:00

Experiencing a similar issue over here. Is there a solution yet?
xuanw 80 Reputation points

2023-07-11T04:37:05.5333333+00:00

Experiencing a similar issue over here. Is there a solution yet?
Sương Hoàng 5 Reputation points

2023-07-12T09:59:06.6433333+00:00

I am experiencing a similar issue with Azure OpenAI API calls that use langchain, as they frequently time out.
fatih 0 Reputation points

2023-07-23T12:36:19.51+00:00

Hello, I am getting timeout exceptions with Azure.AI.OpenAI 1.0.0-beta 6. Model is Azure Studio 35-turbo-0613. Is there a way to solve this or change timeout duration @Grmacjon-MSFT Timeout duration is too long. Thank you.
Joshua 0 Reputation points

2023-08-01T00:00:53.2033333+00:00

Same issue. Isn't this an Azure live service that we can expect an SLA for @Grmacjon-MSFT ?

The experience is not the same as this one: https://learn.microsoft.com/en-us/answers/questions/1193741/how-to-fix-regular-timeout-errors-when-using-chatg

There's no client or server error thrown or rate-limit breached.
Stefan 20 Reputation points

2023-08-09T09:02:24.3033333+00:00

Is there any news on this?

2 answers

Tom-4269 5 Reputation points

2023-08-09T05:28:56.7366667+00:00

Try switching region, looks like East-US is problematic and has high rate of API timeouts.
Please sign in to rate this answer.

1 person found this answer helpful.

0 comments No comments
Sign in to comment
Usman Ahmad 0 Reputation points

2023-08-07T14:52:37.1466667+00:00

Same issue. Sometimes the request gets stuck for 10 minutes which results in really poor user experience.
Moreover, there is no way we are near the rate-limit as per our deployment.
Please sign in to rate this answer.
Tom-4269 5 Reputation points

2023-08-09T04:58:28.2033333+00:00

Try switching to a different region, from my tests in the last days east-us region is experiencing a lot of response timeouts.
Sign in to comment