Azure OpenAI application usage

Question

We have a RAG app deployed in Azure. It's the one that comes "out of the box" after grounding it to our data. We have over 200 people using the application and would like to track usage such as the following:

• User visits per day
• Number of queries per user
• Number of tokens per user
• etc.

Please tell me how or what service must be setup in Azure to make this happen.

Thanks

Answer 1

Hi Pockey, Philip

The costs associated with using Azure OpenAI Service can vary based on the type and amount of resources you utilize, which are dependent on the specific models you choose.

1. Model Inference Charges: You will be charged per 1,000 tokens processed by the models, with different rates depending on the model and deployment type. Both input and output tokens are counted towards this charge.
2. Fine-Tuning Costs: If you use fine-tuned models, you will incur charges based on the number of tokens in your training file, as well as hosting hours and inference costs per 1,000 tokens.
3. Hosting Costs: Deployed fine-tuned models incur hourly hosting costs regardless of usage. If a model remains inactive for more than 15 days, it will be deleted, but the underlying model can still be redeployed.
4. Additional Costs: Enabling features like Azure Monitor Logs may incur extra costs. Also, if the service processes a request, you will be charged even if the response is not successful.
5. Cost Management Tools: You can utilize tools like the Azure pricing calculator to estimate costs and Microsoft Cost Management to monitor and control spending, set budgets, and track usage.

Kindly refer below link: manage-costs

Thank You.