Azure OpenAI Throwing an unhandled Error when submitting a Thread Run with parameter tool_choice set to "type":"file_search"

Question

Hi there,

I am currently using Assistants/Threads/Runs to chat with a configured assistant in Azure OpenAI which has File Search enabled and some configured functions. I want it as much as possible to use the file search and get its answers from there, so when I post the Run, I am setting this body:

{
   "assistant_id":"{{AssistantID}}",
   "tool_choice":{
      "type":"file_search"
   },
   "parallel_tool_calls":true,
}

The tool_choice parameter works quite well and is yielding more accurate results rather than when its set to auto. However, I've hit a random, intermittent issue when it comes to function invocation. There are cases in which when I provide parameters for a function, when I check the run state, OpenAI is throwing an unhandled error as below:

Error text:

"last_error": {

    "code": "server_error",

    "message": "Sorry, something went wrong."

}

Repeating the run a few times will ultimately make it work, and the run state will go into requires_action. Alternatively, when I get the above error, I am posting the same run again, but removing the tool_choice parameter and it works fine.

However, why does this error happen in the first place?

Thanks!

Answer 1

Hello David Captur,

Thank you for posting your question in the Microsoft Q&A forum. 

The intermittent "server_error" you're encountering when forcing tool_choice: "file_search" while invoking functions in Azure OpenAI Assistants likely stems from a conflict between tool prioritization and function execution logic. When you explicitly demand file search, the model may struggle to reconcile this directive with function-calling requirements, especially if the function parameters or context don’t align cleanly with the file-based knowledge. 

This issue often arises when: 

• The function’s input schema clashes with file search results, causing the model to fail silently. 
• Rate limits or backend throttling disrupt the run’s state transitions. 
• The assistant’s instructions don’t gracefully handle hybrid scenarios (file search + functions). 

The fact that retries or removing tool_choice resolves it suggests race conditions in Azure’s tool-routing logic.

For reliability you may want to look below: 

• Audit function parameters for compatibility with file search outputs. 
• Implement retry logic (exponential backoff) for transient errors. 
• Use auto mode unless file search is absolutely critical, as it allows the model to dynamically balance tools. 

If the problem persists, log the full run steps and share them with Microsoft support to identify backend bottlenecks. 

If the above answer helped, please do not forget to "Accept Answer" as this may help other community members to refer the info if facing a similar issue. Your contribution to the Microsoft Q&A community is highly appreciated. 

Answer 2

Hi David Captur,

We understand you are encountering intermittent errors when submitting a Thread Run with the tool_choice parameter set to "type": "file_search" in Azure OpenAI, and you are seeing a server_error with the message “Sorry, something went wrong.” This type of error can happen due to several reasons, such as temporary service disruptions on the Azure side, momentary network issues, or specific configuration or timing conditions when using the file_search tool alongside functions. Since retrying the request a few times eventually allows it to succeed, this suggests the issue is likely transient rather than a persistent misconfiguration. To help address this, we recommend verifying the service status for any ongoing maintenance, reviewing the tool_choice configuration to ensure it’s correctly set, and implementing a retry mechanism in your code to gracefully handle occasional failures. Additionally, enabling logging and monitoring can help you track the frequency and patterns of these errors, providing valuable insights for further troubleshooting. As an extra recommendation, you may consider allowing the assistant to operate in “auto” mode (by not forcing file_search), as this gives it more flexibility to choose the most appropriate tool, reducing the likelihood of hitting this error.

I hope this information helps. Thank you!