500 Errors when using Foundry with OpenAI Models

Question

I have a SaaS that uses Azure Foundry with OpenAI models.

I have been receiving very frequent 500 errors with the message:

The server had an error processing your request. Sorry about that! You can retry your request, or contact us through an Azure support request at: https://go.microsoft.com/fwlink/?linkid=2213926 if you keep seeing this error. (Please include the request ID 562d905d-ceea-4417-995b-36e78bf2b10a in your email.)

Please advise, my clients are furious.

Answer 1

Welcome to Microsoft Q&A

Hello @João Abrantes I hope you are doing well.

A 500 Internal Server Error returned by the Azure AI Foundry/OpenAI endpoint generally indicates that the request reached the service but could not be processed successfully. This is typically different from authentication, quota, or content filtering errors, which usually return 4xx status codes.

To help determine whether this is a transient service issue or specific to your deployment, could you please confirm:

Does the issue occur with all deployed models or only a specific model?

Is it affecting every request, or only a subset of requests?

Are the failures isolated to a single Azure region?

Have you noticed any correlation with increased traffic or specific request sizes?

In the meantime, I recommend checking:

Azure AI Foundry metrics, to identify spikes in failed requests, latency, or throttling around the time the errors occur.

Activity Log, to determine whether there were any deployment changes or service events that coincide with the start of the failures.

Your application logs, using the Request ID returned with each 500 response. This can help correlate failed requests with Azure Monitor telemetry and determine whether the failures are intermittent or follow a specific pattern.

If the errors are intermittent, implementing retry logic with exponential backoff is also recommended, as transient 5xx responses can occasionally occur during service operations.

References:

Monitor Azure AI Foundry resources

Azure AI Foundry quotas and limits

If my answer helped you, please consider marking it as accepted. This helps others in the community find similar solutions.

Answer 2

Frequent 500 errors with that message are typically service-side or model-serving failures, not authentication failures or deployment-name mismatches.

For Foundry/OpenAI models, the supported mitigations are:

1. Retry the request with retry logic
   • This error can be transient.
   • Use exponential backoff and retry the same request.
   • Reattempting the request often succeeds.
2. Reduce temperature to less than 1
   • This is the documented workaround for known 500-class issues such as:
     • Server error (500): Unexpected special token
     • Failed to create completion as the model generated invalid Unicode output
   • If the workload currently uses a higher temperature, lowering it can reduce the error rate.
3. Reset sampling-related parameters to defaults if the issue continues
   • Set presence penalty, frequency penalty, and logit bias back to default values.
   • In some cases, lowering top_p from its default can help reduce these failures.
4. Check whether large tool/function definitions are being sent
   • A known issue affects GPT-4.1 series models: large tool or function call definitions that exceed 300,000 tokens can fail even though the model’s overall context limit is higher.
   • Depending on payload shape, this can surface as either:
     • 400 context_length_exceeded
     • 400 string_above_max_length
     • or a 500 server error on the Responses API
   • If tools/functions are large, reduce their size.
5. If the failures persist, open an Azure support request and include the request ID
   • The documented server error message explicitly instructs including the request ID when contacting support.
   • In this case, include request ID 562d905d-ceea-4417-995b-36e78bf2b10a.

If this started suddenly and is happening frequently across otherwise unchanged requests, that aligns with a backend/service issue. The immediate practical path is: add retries, lower temperature below 1, simplify any large tool definitions, and escalate with the request ID if the issue continues.

---

References:

• Azure OpenAI frequently asked questions
• Foundry Models sold by Azure (azure-openai)
• Work with chat completion models (programming-language-dotnet)