gpt-4-1106-preview API is very very slow, waiting for more than 60 seconds

xiansen wang 80

I am using the gpt-4-1106-preview API with regions set at Norway East, South India, and UK South. However, the response time in each region is extremely slow, even exceeding 120 seconds, rendering the system unusable. My Max tokens is set to 2048, and I am using stream streaming transmission. I have just tried 3 times, and all the results have timed out.

AshokPeddakotla-MSFT 30,511 Reputation points

2024-01-23T09:34:12.61+00:00

xiansen wang Greetings!

I am using the gpt-4-1106-preview API with regions set at Norway East, South India, and UK South. However, the response time in each region is extremely slow, even exceeding 120 seconds, rendering the system unusable.

Thanks for sharing the details. Similar issue has been discussed here see if that helps. Please note that GPT 4 is much slower than 3. If you look at the docs on latency, 35-turbo is the fastest model. Also, refer to the same documentation which provides different factors which you can control to improve the performance like Model selection, Generation size and Max tokens, streaming etc.
AshokPeddakotla-MSFT 30,511 Reputation points

2024-01-24T12:37:57.0933333+00:00

xiansen wang Just checking to see if you had a chance to review my response. Do let me know if that helps or have any other queries.

Share via

gpt-4-1106-preview API is very very slow, waiting for more than 60 seconds