Azure OpenAI PTU utilization

Question

I want to check PTU utilization in API management policy.

In OpenAI monitor dashboard, PTU utilization of model is shown.

When chat completion API is requested, response header has 'azure-openai-deployment-utilization' in PTU. But, it's not the total PTU utilization of the model.

In PAYGO, response header has 'x-ratelimit-remaining-tokens', so it seems that the utilization of the model can be checked.

Is there any way to check the current PTU utilization?

Answer 1

Hello 김세형, I believe you should be looking at the azure-openai-emit-token-metric policy in Azure API Management. You can find the details here -- https://learn.microsoft.com/en-us/azure/api-management/azure-openai-emit-token-metric-policy and https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/best-practice-guidance-for-ptu/ba-p/4152133

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.