How to fix Token usage monitoring in Azure AI Foundry?

Question

Hi!

After a redeploy of my .Net application which uses Azure OpenAI, I stopped seeing Token usage metrics:

I can see correct requests count and I see that token are used from changing costs, but I can not effectively monitor how much. It is happening second time to me - previous time it returned after about a week without my direct interference, but now it is gone again and I don't see how I could affect the configuration. Also it seems that switch to another region helps but it is not what I want to do.

Answer 1

Hi Astap Kuryan,
Greetings & Welcome to the Microsoft Q&A forum! Thank you for sharing your query.
I'm sorry to hear you're experiencing issues with monitoring token usage metrics in your Azure OpenAI integration.

Here are some steps and considerations that might help resolve the issue:

Ensure Diagnostic Settings Configuration is enabled for your Azure OpenAI resource and includes relevant metrics like "Processed Inference Tokens" in the logs.

Leverage Log Analytics Integration by connecting your Azure OpenAI resource to Log Analytics. This allows you to capture detailed logs and use custom queries to track token usage.

Implement API-Level Monitoring by capturing request and response payloads when using the API and calculating token usage locally using OpenAI’s token estimation methods.

I hope this information helps.