Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
To see how often the agents and models in an Azure subscription run, and how reliably, go to Manage > Operations in the Microsoft Foundry portal. The Operations page shows agent runs, model requests, tool calls, tokens, errors, and latency for the scope and date range that you select. It also ranks the agents and models behind each number.
Platform admins and operators use Operations to find what changed, for example an agent whose runs jumped, a model with more errors, or slower agent runs. The page doesn't show cost. To see what usage costs, see View subscription costs in the Foundry portal. To see activity, cost, and policy violations together, start on Fleet. For more information, see Monitor your agent fleet.
Important
Items marked preview in this article are currently in preview. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.
Prerequisites
An Azure subscription with at least one Foundry resource and project that has agent or model usage.
Permission to read usage metrics (
Microsoft.Insights/metrics/read) for the resources in the scope. All Foundry roles, Monitoring Reader, and Reader include it.Optional: to see Agent quality metrics, a recurring evaluation in the project. See Create a recurring evaluation.
For the permission behind each page, see Permissions for cost data and cost controls. To assign roles, see Assign Azure roles using the Azure portal.
Important
The Foundry RBAC roles were recently renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously named Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. You might still see the previous names in some places while the rename rolls out. The role IDs and core permissions are unchanged by the rename.
Open the Operations page
- Sign in to Microsoft Foundry. Make sure the New Foundry toggle is on. These steps refer to Foundry (new).
- Select Manage in the upper-right navigation.
- In the left pane, select Operations. You can also select See Operations on the Fleet page, which keeps the date range that you selected there.
- In Subscription, select the subscription that you want to view. Optionally, to focus on part of it, select resource groups in Resource group and projects in Project.
- Select a date range: Last Day, 7D, 1M, or 3M. To set custom dates, select the date range picker. A custom range can cover up to three months.
Read the summary
The summary card shows four totals for the selected scope and date range. Each total shows the change from the previous period of the same length. Custom ranges don't show a comparison.
| Total | What it counts |
|---|---|
| Agent runs | Runs that agents started. |
| Model requests | Requests to model deployments, including the model calls that agents make. |
| Tool calls | Tool calls that agents made. |
| Total tokens | Tokens from agents and from direct model calls. |
Explore trends over time
The chart next to the summary shows a metric over time, for the total and for each asset.
Select a tab:
- Invocations: agent runs, model requests, or tool calls. To choose which metric to view, use the dropdown under the chart title.
- Tokens: token use.
- Errors: error incidents.
- Latency: agent run latency.
To see a value, hover over a point in the chart. The legend shows the total and each asset.
To enlarge the chart, select the full-screen button in its upper-right corner.
Find the agents and models behind a change
Three lists below the chart rank assets for the selected scope and date range:
- Top invocations: the agents and models with the most invocations, with the count and the change from the previous period.
- Highest error incidents: the agents and models with the most error incidents, with the error rate and the number of incidents.
- Highest agent run latency: the agents whose runs take the longest, in seconds.
To compare every asset, use Assets with invocations:
- Scroll to Assets with invocations.
- Select Agents or Models.
- Review each asset's invocations, token usage, errors, and latency.
- To investigate an asset, select its name.
Check agent quality
Agent quality metrics shows quality trends from recurring evaluations. It stays empty until a project in the scope has a recurring evaluation. To set one up, select Open project monitor. For the steps, see Create a recurring evaluation.
Connect usage to cost
Operations shows usage, not cost. Usage appears within minutes, and its cost appears on the Cost page 8 to 24 hours later. To find what a change in usage costs:
- Note the agents or models whose usage changed, and the scope and date range.
- In the left pane, select Cost. Select the same scope and date range.
- Compare Highest spend assets, and the cost table grouped by Agent or Model. If the change is recent, check again after the cost arrives. For more information, see View subscription costs in the Foundry portal.
Troubleshoot
| Symptom | Cause | Fix |
|---|---|---|
| The page says "You need the Monitoring Reader permission to view these metrics." | You don't have permission to read usage metrics in the scope. | Ask an admin for a Foundry role, or Monitoring Reader on the subscription, and then refresh the page. |
| Totals look lower than expected for a scope with many resources. | You can't read metrics for some resources in the scope, so the totals cover only the resources that you can access. | Narrow the scope, or ask an admin for Monitoring Reader on the subscription. |
| Agent quality metrics is empty. | No project in the scope has a recurring evaluation. | Select Open project monitor, and create a recurring evaluation. |
| Charts and lists are empty. | There was no agent or model usage in the scope and date range. | Select a longer date range or a different scope. |
Related content
- To see activity, cost, and policy violations together, see Monitor your agent fleet.
- To see what usage costs, see View subscription costs in the Foundry portal.
- To monitor one project's agents in detail, see Monitor agents with the Agent Monitoring Dashboard.
- To alert on or block spending, see Create and manage cost controls.