Any recent regression on GPT-4o?

Question

Any recent regression on GPT-4o?

George Meng 20

Hi,

My team has been using OpenAI GPT-4o models for a while. We observed that since April 17 or 18, GPT-4o has trouble following the XML response format that we have been using, with varying probability 3%~30% depending on the application. This never happened in the past. I wonder if Azure AI Services are aware of this regression?

For context,

We're using GPT-4o model version: 2024-11-20, API version: 2024-10-21. Both these are the latest official versions.

kothapally Snigdha 2,590 Reputation points Microsoft External Staff Moderator

2025-04-22T08:50:15.4533333+00:00

Hi @George Meng

Could you please share the code you're currently using and describe the issue you're encountering? Specifically, it would be helpful to know the exact XML output you're expecting, as well as the current output that's being generated. This will allow us to better understand the problem and identify any discrepancies.
kothapally Snigdha 2,590 Reputation points Microsoft External Staff Moderator

2025-04-23T05:28:45.1466667+00:00

Hi @George Meng

Could you please provide the APIM Request ID and more details. This will allow us to better understand the problem and identify any discrepancies.
George Meng 20 Reputation points

2025-04-24T02:39:19.3366667+00:00

@kothapally Snigdha , thank you for reply!
The regression we observed is between using gpt-4o (version: 2024-11-20) before and after last week. The symptom is that the gpt-4o model output is not following certain structure. It's not happening every time but with some probably, so hard to re-produce. I'm still trying to find out a way to re-produce it.

I wonder if you're aware of any change to the gpt-4o model itself last week? From outside, I can see that gpt-4o begins to support image generation. Not sure if that's relevant.
SriLakshmi C 5,125 Reputation points Microsoft External Staff Moderator

2025-04-24T16:35:43.14+00:00

Hi @George Meng,

Thanks for the follow-up. Please share more details whenever you notice this behavior, and we’ll investigate to see if there are any issues with the model impacting certain responses.

Your answer

kothapally Snigdha 2,590 Reputation points Microsoft External Staff Moderator

2025-04-22T08:50:15.4533333+00:00

Hi @George Meng

Could you please share the code you're currently using and describe the issue you're encountering? Specifically, it would be helpful to know the exact XML output you're expecting, as well as the current output that's being generated. This will allow us to better understand the problem and identify any discrepancies.
kothapally Snigdha 2,590 Reputation points Microsoft External Staff Moderator

2025-04-23T05:28:45.1466667+00:00

Hi @George Meng

Could you please provide the APIM Request ID and more details. This will allow us to better understand the problem and identify any discrepancies.
George Meng 20 Reputation points

2025-04-24T02:39:19.3366667+00:00

@kothapally Snigdha , thank you for reply!
The regression we observed is between using gpt-4o (version: 2024-11-20) before and after last week. The symptom is that the gpt-4o model output is not following certain structure. It's not happening every time but with some probably, so hard to re-produce. I'm still trying to find out a way to re-produce it.

I wonder if you're aware of any change to the gpt-4o model itself last week? From outside, I can see that gpt-4o begins to support image generation. Not sure if that's relevant.
SriLakshmi C 5,125 Reputation points Microsoft External Staff Moderator

2025-04-24T16:35:43.14+00:00

Hi @George Meng,

Thanks for the follow-up. Please share more details whenever you notice this behavior, and we’ll investigate to see if there are any issues with the model impacting certain responses.

Share via

Any recent regression on GPT-4o?

Your answer