GPT-5.6 cached-input telemetry and billing issue

Question

We have isolated a potential GPT-5.6 cached-input telemetry and billing issue.

Reproduction:

Three identical calls were sent through the Azure API Management gateway.

Each response reported:

• prompt_tokens: 1,942
• completion_tokens: 16

The third response additionally reported:

• prompt_tokens_details.cached_tokens: 1,792

Expected aggregate telemetry:

• total prompt tokens: 5,826
• cached prompt tokens: 1,792
• output tokens: 48

Observed Azure Monitor metrics:

• Input Tokens: 5.83k
• Output Tokens: 48
• Prompt tokens read from cache: no value ("--")
• Prompt tokens written to cache: no value ("--")

The model response therefore confirms a physical cache read of 1,792 tokens,

while the corresponding Azure Monitor cache-read metric remains unavailable.

All 5,826 prompt tokens appear in the Input Tokens metric.

We also observe that the later Cost Management data for larger workloads does

not appear to reflect the expected discounted cached-input volume. The

discrepancy remains visible after the normal reporting delay.

Answer 1

Hi @Nagrath, Richa

Thank you for reaching out to Microsoft Q&A. I'm sorry you are going through this issue and hope I can point you in the right direction.

The cached_tokens: 1,792 value in the third response confirms that the request had a GPT-5.6 prompt-cache hit. For GPT-5.6 Standard pay-as-you-go deployments, the supported per-request cache signals are cached_tokens for reads and cache_write_tokens for writes. Prompt caching

The Azure Monitor tiles named Prompt tokens read from cache and Prompt tokens written to cache are documented for Anthropic model deployments. Therefore, their blank values do not show that the GPT-5.6 cache read failed. The 5.83k Input Tokens value is consistent with the metric definition: it reports total prompt tokens processed, rather than uncached or full-price input tokens only. Azure OpenAI monitoring data reference

I would use the response-level cached_tokens and cache_write_tokens fields for GPT-5.6 cache accounting. The public documentation confirms that cache reads receive discounted input pricing, but it does not provide a Cost Management field that reconciles cached input and cache-write volume separately. Prompt caching

For the Cost Management discrepancy, please create an Azure support request and include the subscription/resource, region, deployment name and model version, deployment type, UTC timestamps, request IDs, redacted usage objects, and the relevant Cost Management export. Ask for confirmation that the 1,792 cached tokens were rated at the applicable cache-read rate and that the meter records reconcile to the request-level usage. This requires access to service-side metering data.

Please 'Upvote' (Thumbs-up) and 'Accept' as an answer if the response was helpful. This will benefit other community members who face the same issue.

Best regards, Andrew S Taylor

Answer 2

Hello @Nagrath, Richa

Thank you for providing a detailed reproduction scenario.

Based on your observations, it does appear that the model response indicates a cache hit (prompt_tokens_details.cached_tokens), while the corresponding Azure Monitor cache metrics and Cost Management data don't appear to reflect that cached usage.

Before concluding that this is a billing issue, I'd recommend verifying a few points:

• Confirm that all three requests were truly identical (prompt, system prompt, parameters, deployment, API version, and any tools/functions), as prompt caching requires an exact cache match.
• Allow sufficient time for Azure Monitor and Cost Management to complete data ingestion. While token usage is typically available sooner, billing and cost analytics may be delayed.
• Compare the Usage or Consumption records after the reporting delay to determine whether cached input pricing has been applied, as Azure Monitor metrics and billing data are generated through different telemetry pipelines.

If, after the expected reporting window, you continue to observe:

• prompt_tokens_details.cached_tokens populated in the API response,
• no values for Prompt tokens read from cache in Azure Monitor, and
• billing that reflects the full prompt token count rather than the discounted cached input,

then this would warrant a support investigation. Please capture the following when opening a support request:

• Request IDs (x-ms-request-id or apim-request-id)
• Timestamps (UTC)
• Deployment name
• API version
• Model name
• Region
• A sample of the API responses showing cached_tokens

These details will help Microsoft correlate the inference telemetry with Azure Monitor and Cost Management records to determine whether the discrepancy is due to reporting latency or an issue in the telemetry or billing pipeline.

If you've already waited beyond the normal reporting delay and can consistently reproduce this behavior, it would be helpful to know whether you're seeing the same results when calling the Azure OpenAI endpoint directly (bypassing Azure API Management). That would help determine whether the issue is specific to the telemetry pipeline or unrelated to APIM.

Please "Accept the Answer" if this information helped you. This will help us and others in the community as well.