Azure OpenAI Responses API - o4-mini - A lot of Hallucinations

Question

I am using the o4-mini model with the Responses API and I am getting an incredible amount of hallucinations.
Here is an example below. Basically my prompt is a math pronblem but the response that I am getting is a workout plan ..

I understand that the Responses API in in preview right now, but is this a model problem when reasoning is enabled or a Responses API problem. Using the ChatCompletions API with reasoning, everything is fine.

This seems unusable to me

Answer 1

Hi Siry, Gaetan

The issue you're experiencing with the o4-mini model generating hallucinations, such as providing a workout plan in response to a math problem, could be attributed to several factors. Since the Responses API is currently in preview, it may not yet be fully optimized for all types of queries, especially complex reasoning tasks.

The hallucinations could be a result of the model's limitations in understanding the context or the specific nature of the task when reasoning is enabled. In contrast, the ChatCompletions API might be more robust for certain types of queries, as you noted that it performs well with reasoning.

It's important to consider that the Responses API may have different performance characteristics compared to the ChatCompletions API.

Kindly refer below link: responses

transparency-note

Thank You.