response times of chatgpt3.5 increase

Question

Good morning,

I have noticed that, on average, on the same input, the response times of chatgpt3.5-16k doubled at the end of July compared to the beginning of summer and are now even quadrupled on average. Sometimes we even started getting more errors than usual in response (always on the same inputs that, at the beginning of the summer, seem to give no problems).

Is this a problem that Microsoft knows? Are they going to mitigate it? Is there something I can do about it without changing the prompt (that at the beginning of the summer was perfect both in terms of quality and response time)?

Thank you very much!

Answer 1

@Emanuele You might want to enable monitoring on your azure OpenAI resource to check for latency. You can also report the behavior by providing details about your resource like model, region, version, scenario and no. of tokens used. This would help the team to check if there is any service issues that could increase your deployment response times. Thanks!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.