Azure Open AI API Timeouts

Cesar DS 20 Reputation points
2023-11-19T14:48:45.9433333+00:00

Hello.

I'm having random timeouts when accessing the chat completion API in Azure.

Lastnight I ran several tests and it worked perfect. Some prompts replied in 10 seconds, other more complex prompts in 20 seconds.

But this morning I'm running again the tests and sometimes it replies OK, but many times just return timeout.

Also, the timeout reply is received in less than 15 seconds (makes no sense as yesterday some prompts replied correctly in 20 seconds).

So, my questions:

  1. Is this normal behavior to receive timeouts using the Azure Open AI API? (I was using openAI directly and the delay was huge, this is why I switched to Azure, as the response time is average half compared to openAI)
  2. Why I'm getting timeouts even when the request is running for under 15 seconds (my postman is configured to 2 minutes timeouts, so the issue is not in client-side)

Anyone familiar with this issues that could help will be appreciated.

Thanks!

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,606 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Cesar DS 20 Reputation points
    2023-11-19T15:26:22.26+00:00

    Nevermind. I realized there was a misconfiguration in postman.

    0 comments No comments

  2. AshokPeddakotla-MSFT 30,086 Reputation points
    2023-11-20T06:38:36.98+00:00

    Cesar DS Greetings!

    I'm glad that your issue is resolved and thank you for posting your solution so that others experiencing the same thing can easily reference this!

    Since the Microsoft Q&A community has a policy that the question author cannot accept their own answer, they can only accept answers by others, I'll repost your solution in case you'd like to Accept the answer.

    Problem:

    I'm having random timeouts when accessing the chat completion API in Azure.

    Lastnight I ran several tests and it worked perfect. Some prompts replied in 10 seconds, other more complex prompts in 20 seconds.

    But this morning I'm running again the tests and sometimes it replies OK, but many times just return timeout.

    Also, the timeout reply is received in less than 15 seconds (makes no sense as yesterday some prompts replied correctly in 20 seconds).

    So, my questions:

    1. Is this normal behavior to receive timeouts using the Azure Open AI API? (I was using openAI directly and the delay was huge, this is why I switched to Azure, as the response time is average half compared to openAI)
    2. Why I'm getting timeouts even when the request is running for under 15 seconds (my postman is configured to 2 minutes timeouts, so the issue is not in client-side)

    Anyone familiar with this issues that could help will be appreciated.

    Solution:

    There was a misconfiguration in postman. After modifying the config issue is resolved.

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    0 comments No comments