Share via

Azure OpenAI PTU utilization

김세형 0 Reputation points
2024-06-19T03:36:05.9166667+00:00

I want to check PTU utilization in API management policy.

In OpenAI monitor dashboard, PTU utilization of model is shown.

When chat completion API is requested, response header has 'azure-openai-deployment-utilization' in PTU. But, it's not the total PTU utilization of the model.

In PAYGO, response header has 'x-ratelimit-remaining-tokens', so it seems that the utilization of the model can be checked.

Is there any way to check the current PTU utilization?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
2,521 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Adharsh Santhanam 2,320 Reputation points
    2024-06-19T04:56:51.3966667+00:00

    Hello 김세형, I believe you should be looking at the azure-openai-emit-token-metric policy in Azure API Management. You can find the details here -- https://learn.microsoft.com/en-us/azure/api-management/azure-openai-emit-token-metric-policy and https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/best-practice-guidance-for-ptu/ba-p/4152133

    Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments