Azure OpenAI SDK - Channel response timed out after 60000 milliseconds

Dat Hoang 20 Reputation points
2024-04-02T04:42:24.2933333+00:00

Hi all,

I am using Java 17 + Quarkus framework with the Azure OpenAI SDK version beta 2.

Currently, our application use the OpenAIAsyncClient.getChatCompletionsStream to send the request & receive response from OpenAI https://api.openai.com/v1 (not Azure OpenAI)

We also use Mutiny to handle the response.

We are facing the time out issue continuously but after receive the issue, we can call and receive the response for some upcoming requests and the issue happen again.

The request to OpenAI is really simple and the success calls we received just in less than 10 seconds so the request time from OpenAI to our request more than 60 seconds is impossible from my point of view.

I did search for the information and saw some places where people say that there is the time out 60 seconds on the Netty Http Client if the client has been idle for a while but our case is that we calling to OpenAI continuously and randomly we get the issues.

I hope that any expert can explain and show us the way to solve.

Thanks and this is the log:

2024-04-01 08:44:32,602 DEBUG [com.kat.gpt.ope.AzureOpenAiClient] (vert.x-eventloop-thread-0) Invoking Chat Completion REST endpoint 2024-04-01 08:45:32,661 WARN [rea.net.htt.cli.HttpClientConnect] (reactor-http-nio-4) [c8fe8556-7, L:/10.10.7.79:60520 - R:api.openai.com/104.18.6.192:443] The connection observed an error: java.util.concurrent.TimeoutException: Channel response timed out after 60000 milliseconds. at com.azure.core.http.netty.implementation.AzureSdkHandler.responseTimedOut(AzureSdkHandler.java:200) at com.azure.core.http.netty.implementation.AzureSdkHandler.lambda$startResponseTracking$2(AzureSdkHandler.java:184) at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:833)

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,101 questions
{count} votes