Tracking token usage for service calls on the server side

Question

Is there a way to monitor the usage of tokens for each call into our service on the server side? We need to accurately track token usage for billing purposes, but the client-side usage doesn't provide a complete picture. we wants to be able to query the service or Azure cost management to see the actual usage with an API call. Are there any possible solutions?

Answer 1

Thanks for the question, The following table summarizes the current subset of metrics available in Azure OpenAI.

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/monitoring#azure-openai-metrics