gpt-o1 performance

Question

gpt-o1 performance

Mohamed Hussein 715

Good Day

Is that normal to have 180 seconds avg delay at gpt-o1 general available model?

Sweden Zone

Reason Effort - low

Prompts like -

How are you ?

Knowledge cutoff

Using the Playground

SRILAKSHMI C 19,195 Reputation points Microsoft External Staff Moderator

2025-01-16T19:54:04.9866667+00:00
Hello Mohamed Hussein,

Greetings and Welcome to Microsoft Q&A! Thanks for posting the question.

An average delay of 180 seconds for the GPT-01 model in the Sweden zone is unusually high, particularly for simple prompts like "How are you?".

Here are some of the best practices to lower latency:

Evaluating the model's latency by testing with optimized configurations Improve performance.

The fewer tokens generated the faster the overall response will be. Remember this is like having a for loop with n tokens = n iterations. Lower the number of tokens generated and overall response time will improve accordingly.

OpenAI has found that even in cases where the total number of tokens generated is similar the request with the higher value set for the max token parameter will have more latency.

Please check this Performance and latency.

I hope you understand. And, if you have any further query do let us know.

Thank you!
SRILAKSHMI C 19,195 Reputation points Microsoft External Staff Moderator

2025-01-17T20:35:27.8266667+00:00

Hi Mohamed Hussein,

Did you get any chance to check the above response. Thank you!
SRILAKSHMI C 19,195 Reputation points Microsoft External Staff Moderator

2025-01-20T21:44:13.0333333+00:00

Hi Mohamed Hussein,

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Thank you!