gpt 5.5 - terrible performance in azure EAST US2 region

gpt-5.5 is available only in specific regions and, like other Foundry models, its latency and perceived “slowness” are primarily driven by workload characteristics and deployment configuration rather than the model version alone.

Key points to check to improve performance relative to gpt-5.4:

1. Region and deployment type
   Ensure the gpt-5.5 deployment is in a supported region and, if possible, compare in the same region and deployment type (for example, provisioned throughput vs. shared). gpt-5.5 regional provisioned throughput is currently available only in eastus.
2. Throughput vs. latency (PTUs)
   If using provisioned throughput, verify that enough PTUs are allocated for the call pattern. Throughput (tokens per minute) and calls per minute scale roughly linearly with PTUs. For example, a workload of ~500k input + 150k output tokens per minute requires around 30 PTUs, and heavier workloads (millions of tokens per minute) require proportionally more. Under-provisioning can increase queueing and latency.
3. Prompt and generation sizes
   Latency is mainly affected by:
   • Model type
   • Number of prompt tokens
   • Number of generated tokens
   • Overall load on the deployment
   Each additional generated token adds more latency than each prompt token. If gpt-5.5 prompts or max_tokens are larger than what was used with gpt-5.4, responses will feel slower even with the same infrastructure.
4. Streaming for perceived responsiveness
   Enable streaming for interactive/chat scenarios. Streaming returns tokens incrementally as soon as they are available, improving perceived latency even if total completion time is similar. This is particularly useful for chatbots and conversational interfaces.
5. Content filtering configuration
   Azure OpenAI content filters add safety checks but also add latency. For low‑risk scenarios, consider whether a modified content filtering policy is appropriate to reduce overhead. This requires going through the documented process to request changes to default filters.
6. Separate workloads and batching
   • Avoid mixing very short and very long requests on the same deployment; mixed workloads can cause short calls to wait behind long ones and reduce cache efficiency. Use separate deployments for distinct workload patterns where possible.
   • If sending many small requests, consider batching multiple prompts into a single call and test whether this improves overall throughput and effective latency.
7. Measure, don’t guess
   Use Azure Monitor metrics on the deployment to understand:
   • Calls per minute (Requests metric split by ModelDeploymentName)
   • Total tokens per minute (Processed Inference tokens)
   • Per‑call latency (end‑to‑end request time for non‑streaming, or time to first token and average token generation rate for streaming)
   Comparing these metrics between gpt-5.4 and gpt-5.5 under the same load, region, and PTU allocation will show whether the slowdown is due to model behavior, under‑provisioning, or contention.

If, after matching region, PTUs, prompt/generation sizes, and workload separation, gpt-5.5 is still significantly slower than gpt-5.4 for the same measured tokens and call rate, collect the above metrics and open a support case so the service team can check for region‑specific or deployment‑specific issues.

References:

• What is provisioned throughput for Foundry Models?
• Performance and latency