Could not finish the message because max_tokens or model output limit was reached.

Question

Could not finish the message because max_tokens or model output limit was reached.

Alex Cavanaugh 0

400_Count

Starting around midnight Central Time on February 23, 2025, we began experiencing intermittent 400 errors for requests to GPT-5 across all US regions where we are deployed. The error rate increased throughout the day, peaked mid-afternoon, and then subsided around 6:00 PM CT without any changes on our end. We have seen a few brief spikes since then, but none as large or sustained as the initial incident. Here's the error message we saw:

Could not finish the message because max_tokens or model output limit was reached. Please try again with a higher max_tokens.

This was unexpected because:

Our prompts are relatively small. Total token consumption for affected requests was approximately 12k tokens, well within the model's context window limit.
We do not explicitly set the max_tokens parameter in our requests, so we are relying on the model default.
Retries on failed requests would typically succeed after a few attempts, suggesting this was not a consistent input or configuration issue.
The errors resolved on their own without any changes to our code or configuration.

Our questions for the Microsoft team:

What caused this behavior, and was there a known incident or platform issue on February 23?
Should we expect similar occurrences in the future?

Also note we did try setting max_completion_tokens on our requests to the largest value for GPT-5 (128,000) and this seemed to have no impact on the error rate.

Sina Salam 27,971 Reputation points Volunteer Moderator

2026-03-02T15:02:44.29+00:00

Hello Alex Cavanaugh,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you could not finish the message because max_tokens or model output limit was reached.

To avoid token‑limit failures, always set both max_tokens and max_completion_tokens explicitly, since Azure may otherwise fall back to restrictive defaults. Use a safe upper bound like:

{ "max_tokens": 128000,

"max_completion_tokens": 128000

}

This link below is one of the accepted answers that addressed the above:

https://learn.microsoft.com/en-us/answers/questions/2115658/max-tokens-in-azure-open-ai-not-serving-its-purpos

Keep your prompt size below ~85% of the model’s window to accommodate hidden processing overhead, which can cause early termination under load. This aligns with guidance on Azure’s token‑budget estimation and overflow behavior. Optimizing Azure OpenAI Limits

Implement resilient exponential‑backoff retry logic, as transient platform pressure can trigger intermittent 400/429 failures even when requests are valid. A fallback strategy is recommended for consistent availability. PTU Retry Mechanism

Monitor Azure OpenAI performance and quotas to detect unexpected rate‑limit spikes or backend throttling that can mimic token‑limit errors. Adjusting TPM/RPM allocations can prevent unpredictable dropouts during peak activity. Azure OpenAI Quotas & Monitoring

If issues persist, validate token consumption through Azure’s token‑limit policies and log each request’s processed‑token estimate to identify silent overruns. These diagnostics help pinpoint misalignment between expected and actual token usage. Azure Token‑Limit Policy

I hope this is helpful! Do not hesitate to let me know if you have any other questions or clarifications.

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.
Anshika Varshney 7,970 Reputation points Microsoft External Staff Moderator

2026-03-03T19:34:27.9033333+00:00

Hi Alex Cavanaugh,

Thanks for sharing all the details. What you described actually matches something a few other people ran into around that same time.

Here’s the simple version of what likely happened.

Why this error can show up even when your token usage looks fine

The message saying the model could not finish because max tokens or the output limit was reached does not always mean your prompt was too large.

In situations like yours:

The requests were clearly within the documented context limits.

You did not explicitly set max tokens, or even setting it high did not fix it.

Retrying the exact same request shortly after worked.

The issue went away without any code or configuration changes.

That pattern strongly suggests a temporary service side issue rather than something wrong in your code. Under load or during backend throttling, the service can sometimes fail mid generation and return a generic token limit style error, even though the real cause is not your prompt size.

It will work because this was not caused by your input, retrying later often succeeds once things stabilize on the backend. That behavior is typical of short-lived capacity or execution hiccups rather than malformed requests.

What you can do going forward

Set an explicit output limit

Even though defaults usually work, it is still a good idea to explicitly set max tokens or max output tokens depending on the API you use. It gives the generation a clear boundary and removes ambiguity in normal situations.

Add retry with backoff

Since these errors can be temporary, having automatic retries with a small exponential backoff is important. Your experience already showed that retries resolved it, so this is a practical safeguard.

Log request IDs

Keep capturing request IDs from failed responses. If you ever notice a pattern or repeated issues, those IDs are useful for correlating what happened on the backend.

Watch for patterns, not one off failures

A short spike that resolves on its own usually does not require any action. What matters more is whether it becomes sustained or frequent.

This does not mean your prompts are too large. It is not caused by relying on default max tokens. It is not something you can fully prevent on the client side if it is a temporary service issue.

I Hope this helps. Do let me know if you have any further queries.

This really sounds like a transient platform hiccup rather than a configuration mistake in your application.

Thankyou!
Anshika Varshney 7,970 Reputation points Microsoft External Staff Moderator

2026-03-09T19:25:20.0566667+00:00

Hi Alex Cavanaugh,

Just checking back to see if you’re still facing the same issue. If the problem persists, please share a few more details and we’ll be happy to help you further.

Thankyou!