What is 'azure-openai-deployment-utilization' of response header on PTU?

김세형 105 Reputation points
2024-07-24T04:17:16.3333333+00:00

I use both PAYGO and PTU on Azure OpenAI.

From PTU model, it responds 'azure-openai-deployment-utilization' header.

I couldn't find the reference about this.

I thought this is PTU utilization, but it doesn't seem match with PTU utilization.

What is 'azure-openai-deployment-utilization' of response header on PTU?

Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
3,244 questions
{count} votes

1 answer

Sort by: Most helpful
  1. navba-MSFT 24,985 Reputation points Microsoft Employee
    2024-07-24T05:49:56.7666667+00:00

    @김세형 Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

    .

    I believe, we might remove the azure-openai-deployment-utilization header as it is not reliable.

    .

    You can leverage the Provisioned Throughput Unit (PTU) deployment utilization metrics as the source of truth, which is always accurate. See the below screenshot:

    User's image

    Alternatively, you can also navigate to the Azure Ai Studio and follow the below steps:

    User's image

    User's image

    Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

    **

    Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.