GPT-4o mini raising errors when using with Assistants with Functions & Vector Stores

Question

I am doing experiments combining GPT-4o-mini, Assistants, Vector Stores and Functions and we have noticed that with GPT-4o-mini there appears to be bugs that are often resulting in functions being invoked incorrectly / randomly, resulting in subsequent exceptions:

In the above example, I have 3 functions linked to the assistant, and they seem to be getting invoked randomly. 

1. Force changing the "strict" parameter to true did not change the behaviour. 
2. This same problem is not happening for GPT-4o, it's happening for GPT-4o-mini. 
3. The problem is reproducible when using OpenAI directly as well, so I have good reason to believe this is not an Azure issue, but rather OpenAI. As this functionality is formally in beta mode, I am reporting this problem as this is after all the point of beta functionality.

Thanks,

David

Answer 1

Hi David Captur,

Thank you for reaching out to Microsoft Q&A forum!

I understood that Your issue with GPT-4o-mini in Assistants Playground appears to be related to function calling behaviour, where functions are being invoked incorrectly/randomly, even when setting "strict": true. Since this issue is not occurring with GPT-4o, it is likely a bug in the GPT-4o-mini model rather than an Azure-specific issue.

Potential Causes:

Bugs in GPT-4o-mini's function calling logic – Since GPT-4o-mini is still in beta, its function execution might not be fully optimized.

Function Calling Behaviour Differences – GPT-4o-mini may be handling tool responses differently compared to GPT-4o.

Overlapping Function Names or Parameters – If your function names or parameters are too generic, the model might be incorrectly triggering functions.

Vector Store Interaction Conflicts – If the Assistant is also retrieving embeddings from a Vector Store, it may be misinterpreting retrieved data and triggering function calls incorrectly.

API Response 400 Error (Bad Request):

 Indicates incorrect function parameters are being passed.

 This might be due to an incorrect schema or missing required fields.

Possible Solutions:

GPT-4o-mini may be interpreting certain prompts or responses as needing function calls when it should not. It randomly calls functions that are not strictly relevant to the conversation.

Manually Handle Function Calls (Avoid Auto Invocation)

Instead of allowing the model to auto-invoke functions, you can manually parse its intent before making a function call.

Step 1__:__  Disable auto function calling.

Step 2__:__  First, let GPT-4o-mini only return JSON intent instead of executing a function.

Step 3:  Then, based on the output, you can manually trigger the function.

Validate Function Schemas Carefully:

Ensure that all your functions are well-defined with required parameters. If GPT-4o-mini is invoking functions incorrectly, it might be due to loose parameter matching.

Make sure that:

All required fields are listed under "required".

The data types are correctly defined.

please refer to the below document - GPT-4o mini which can help you further.
https://platform.openai.com/docs/models#gpt-4o-mini

Hope this helps. Do let us know if you any further queries.  

 

------------- 

If this answers your query, do click Accept Answer and Yes for was this answer helpful.

Thank you.