Azure OpenAI Usage API?

Question

Azure OpenAI Usage API?

Andres da Silva Santos 195

Hi,

What is the ETA for the addition of the OpenAI Usage API to the OpenAI Services on Azure?

I have several OpenAI instances but can't manage costs beyond instance/model costs.

Considering enterprise scenarios, it's highly important for effective cost management by project/application.

References:

https://platform.openai.com/docs/api-reference/usage

https://platform.openai.com/docs/api-reference/usage/costs

0 comments

Answer accepted by question author

Vikram Singh 2,590 Microsoft Employee Moderator

Hi @Andres da Silva Santos

Thanks for posting your question in Microsoft Q&A.

As of now, Azure OpenAI Service does not provide a dedicated Usage API like OpenAI’s platform. However, you can manage costs using Azure Cost Management and Billing, Provisioned Throughput Units (PTUs) for predictable pricing, and Azure Monitor for tracking usage and setting alerts.

For more detailed information, please refer to the official Azure OpenAI Service documentation:

If the response helped, please do click Accept Answer and Yes for was this answer helpful.

Andres da Silva Santos 195 Reputation points

2025-01-31T12:41:35.98+00:00
Hello Vikram, thank you for your answer.

However, I still have doubts about how to monitor the cost at the application level, not at the resource level.

I know that the Azure resource has the usage cost of all deployed models individually, but how can I determine within a deployed model what percentage of consumption is from one client or another?

For example, OpenAI allows me to send project_id and provides the endpoint https://api.openai.com/v1/organization/costs which returns the following JSON with the project_id information:

{ "object": "page", "data": [ { "object": "bucket", "start_time": 1730419200, "end_time": 1730505600, "results": [ { "object": "organization.costs.result", "amount": { "value": 0.06, "currency": "usd" }, "line_item": null, "project_id": null } ] } ], "has_more": false, "next_page": null }

How can I perform this monitoring using Azure OpenAI Services?
Vikram Singh 2,590 Reputation points Microsoft Employee Moderator

2025-01-31T13:17:10.1066667+00:00
Hi @Andres da Silva Santos ,

Thanks for getting back.

Azure OpenAI does not directly support sending a project_id to determine the consumption percentage from different clients within a deployed model. However, you can achieve this by using Azure API Management. By setting up API Management in front of your Azure OpenAI service, you can track and manage token usage across multiple applications or clients. You can configure policies in Azure API Management to set token limits and emit token metrics based on various keys, such as subscription key or originating IP address. This setup helps you monitor and control the consumption of your Azure OpenAI service effectively.

For more details, you can refer:

Azure OpenAI Service REST API reference

Import an Azure OpenAI API

Overview of generative AI gateway capabilities in Azure API Management

If the reply was helpful please don't forget to upvote and/or accept as answer.

1 additional answer

Your answer

Andres da Silva Santos 195 Reputation points

2025-01-31T12:41:35.98+00:00

Hello Vikram, thank you for your answer.

However, I still have doubts about how to monitor the cost at the application level, not at the resource level.

I know that the Azure resource has the usage cost of all deployed models individually, but how can I determine within a deployed model what percentage of consumption is from one client or another?

For example, OpenAI allows me to send project_id and provides the endpoint https://api.openai.com/v1/organization/costs which returns the following JSON with the project_id information:

{ "object": "page", "data": [ { "object": "bucket", "start_time": 1730419200, "end_time": 1730505600, "results": [ { "object": "organization.costs.result", "amount": { "value": 0.06, "currency": "usd" }, "line_item": null, "project_id": null } ] } ], "has_more": false, "next_page": null }

How can I perform this monitoring using Azure OpenAI Services?
Vikram Singh 2,590 Reputation points Microsoft Employee Moderator

2025-01-31T13:17:10.1066667+00:00

Hi @Andres da Silva Santos ,

Thanks for getting back.

Azure OpenAI does not directly support sending a project_id to determine the consumption percentage from different clients within a deployed model. However, you can achieve this by using Azure API Management. By setting up API Management in front of your Azure OpenAI service, you can track and manage token usage across multiple applications or clients. You can configure policies in Azure API Management to set token limits and emit token metrics based on various keys, such as subscription key or originating IP address. This setup helps you monitor and control the consumption of your Azure OpenAI service effectively.

For more details, you can refer:

Azure OpenAI Service REST API reference

Import an Azure OpenAI API

Overview of generative AI gateway capabilities in Azure API Management

If the reply was helpful please don't forget to upvote and/or accept as answer.

Answer 1

Chiugo Okpala 1,925 MVP

@Andres da Silva Santos

The ETA for the addition of the OpenAI Usage API to the OpenAI Services on Azure is currently set for October 2024. This API will provide detailed usage metrics, which will be crucial for effective cost management by project or application

See:

https://learn.microsoft.com/en-us/azure/ai-services/openai/monitor-openai-reference

https://learn.microsoft.com/en-us/azure/ai-services/openai/

0 comments

Share via

Azure OpenAI Usage API?

1 additional answer

Your answer