Share via

Latency 2 to 3 times higher than openAI

myrdeb 20 Reputation points
2024-03-04T14:41:32.2433333+00:00

Hi,

I am using gpt 4 1106 model and the latency I get is huge (between 1 minute to 3 minutes). It is also way higher than the latency with the gpt 4 model of open AI.

  • What explains this huge difference in latency between Open AI and Azure Open AI
  • Does the region of the resource have an impact. I have choses southUK because our physical location is in the UK. Better to choose US ?
  • Any tips to improve the latency ?

Thanks!

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
4,083 questions
0 comments
Accepted answer
  1. Charlie Wei 3,335 Reputation points
    2024-03-04T15:42:58.75+00:00

    Hello myrdeb,

    Regarding latency, you may refer to this Microsoft Learn document for relevant adjustments.

    Another suggestion is to consider upgrading to the gpt-4-0125 version. It is hoped that these measures will offer improvements.

    Best regards,
    Charlie

    If you find my response helpful, please consider accepting this answer and voting 'yes' to support the community. Thank you!

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.