Share via

Azure OpenAI SDK - Channel response timed out after 60000 milliseconds

Dat Hoang 20 Reputation points
2024-04-02T04:42:24.2933333+00:00

Hi all,

I am using Java 17 + Quarkus framework with the Azure OpenAI SDK version beta 2.

Currently, our application use the OpenAIAsyncClient.getChatCompletionsStream to send the request & receive response from OpenAI https://api.openai.com/v1 (not Azure OpenAI)

We also use Mutiny to handle the response.

We are facing the time out issue continuously but after receive the issue, we can call and receive the response for some upcoming requests and the issue happen again.

The request to OpenAI is really simple and the success calls we received just in less than 10 seconds so the request time from OpenAI to our request more than 60 seconds is impossible from my point of view.

I did search for the information and saw some places where people say that there is the time out 60 seconds on the Netty Http Client if the client has been idle for a while but our case is that we calling to OpenAI continuously and randomly we get the issues.

I hope that any expert can explain and show us the way to solve.

Thanks and this is the log:

2024-04-01 08:44:32,602 DEBUG [com.kat.gpt.ope.AzureOpenAiClient] (vert.x-eventloop-thread-0) Invoking Chat Completion REST endpoint 2024-04-01 08:45:32,661 WARN [rea.net.htt.cli.HttpClientConnect] (reactor-http-nio-4) [c8fe8556-7, L:/10.10.7.79:60520 - R:api.openai.com/104.18.6.192:443] The connection observed an error: java.util.concurrent.TimeoutException: Channel response timed out after 60000 milliseconds. at com.azure.core.http.netty.implementation.AzureSdkHandler.responseTimedOut(AzureSdkHandler.java:200) at com.azure.core.http.netty.implementation.AzureSdkHandler.lambda$startResponseTracking$2(AzureSdkHandler.java:184) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833)

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,645 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. AshokPeddakotla-MSFT 30,511 Reputation points
    2024-04-02T10:05:57.4733333+00:00

    Dat Hoang Greetings!

    Based on the error log, it seems that the issue is related to the Netty Http Client, which has a timeout of 60 seconds if the client has been idle for a while. However, you mentioned that you are calling the OpenAI API continuously and randomly getting the issue.

    One possible solution is to increase the timeout value for the Netty Http Client.

    Another possible solution is to check if there are any network issues or latency problems between your application and the OpenAI API.

    Please go through Operation Timeouts to know more about the timeouts.

    You can also report an issue here for further help.

    0 comments