Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Important
Items marked preview in this article are currently in preview. This preview is provided without a service-level agreement, and Microsoft doesn't recommend it for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.
Use Monitor in Microsoft Foundry to understand how the agents and models in your project are performing. Start with agent activity, agent and model duration, and token consumption, then investigate the agents, models, and traces behind those numbers. You can also review evaluation results and explore estimated costs at the project, agent, and model levels.
Monitor provides a project-level starting point. You don't need to open each agent or model individually to begin investigating changes in your project's activity.
Monitor is available to all users in public preview. This article describes project-level Monitor within Build. It is separate from the subscription-level Manage experience.
Prerequisites
- A Microsoft Foundry project with agents, models, or both.
- Access to the Foundry portal and permission to view your project's monitoring data.
- Agent or model activity in the time range you want to monitor.
Agent errors, project-wide traces, and quality views use the Application Insights resource connected to the Foundry project you're monitoring. The additional setup depends on which data you want to view.
| Data | Application Insights requirement |
|---|---|
| Core usage and performance metrics for non-voice agents and models, including agent activity, duration, and token usage | Application Insights isn't required. |
| Agent errors | Your project's connected Application Insights resource must contain the relevant agent traces, and you need permission to read that data. |
| Project-wide traces | Your project's connected Application Insights resource must contain the relevant trace data, and you need permission to read that data. |
| Quality signals | Your project's connected Application Insights resource must contain the relevant evaluation data, and you need permission to read that data. |
Voice agents don't support the platform metrics used by the core usage and performance charts. This exception doesn't change the Application Insights requirements for traces, agent errors, or quality views.
Open project monitoring
- Sign in to Microsoft Foundry.
- Open the project you want to monitor in the new Foundry experience.
- Go to Build, and open the project-level Monitor experience.
- Select the time range you want to investigate.
You can look back up to three months. For traces, agent errors, and quality views, the available history also depends on the retention policy configured for your project's connected Application Insights resource.
Start at the project level to understand overall activity before narrowing your investigation to an individual agent or model.
Monitor usage and performance
Review usage and performance together to understand what ran in your project and where behavior changed.
| Signal | What it tells you | How to use it |
|---|---|---|
| Agent activity | How much agent activity occurred in the selected time range. | Identify changes in agent usage and the agents involved. |
| Model duration | How long model requests take in the selected time range. | Review model duration over time to identify changes in latency. |
| Agent errors | Errors reported in agent traces sent to your project's connected Application Insights resource. | Identify affected agents and inspect their traces to investigate individual errors. |
| Agent latency | How long agent runs take. | Identify slower agent runs and investigate the relevant agent or trace. |
| Token usage | Tokens consumed by agent and model activity. | Identify changes in consumption and compare them with activity and estimated costs. |
Investigate project-wide traces
Project-wide traces let you begin an investigation across the project rather than starting with a single agent. A trace records the steps in an execution. Each recorded step is a span, which can describe an operation such as a model request or tool call.
Set up trace access
- Use the Application Insights resource connected to your Foundry project. If your project doesn't have one connected, follow Set up tracing in Microsoft Foundry to connect a resource.
- Confirm that the activity you want to investigate sends traces to that resource. Connecting Application Insights doesn't replace any instrumentation required for your application.
- Confirm that you have permission to read the telemetry. For log-based views, this permission includes the Log Analytics Reader role on the connected Application Insights resource and, for workspace-scoped queries, its linked Log Analytics workspace. Protected trace content requires additional permissions. See Permissions for trace-based workflows.
Inspect a trace
- From project monitoring, open the project-wide traces view.
- Find a trace for the time period and activity you're investigating.
- Open the trace and review its recorded operations, timing, and errors.
- Inspect the relevant spans to understand where a failure or delay occurred.
The traces you see depend on the telemetry available in the connected Application Insights resource and your permissions to read it. An empty trace view doesn't necessarily mean that the project had no agent or model activity.
Review quality signals
Quality views show evaluation results when the relevant evaluation data is available in your project's connected Application Insights resource.
Use these results alongside usage and performance metrics. For example, an agent can complete a run without an operational error but still receive a low evaluation score for response quality.
- Confirm that the evaluations you want to review ran and that their results are available in the connected Application Insights resource.
- Open the quality view in project monitoring.
- Review the available evaluation results for the activity you're investigating.
- Interpret each score using the definition of its evaluator. Different evaluators measure different aspects of quality.
Monitor displays available evaluation data. Opening Monitor doesn't automatically run evaluations or create quality scores for all activity in the project.
For agent evaluation setup, see Monitor agents with the Agent Monitoring Dashboard.
Understand estimated costs
Use cost views to understand what drives consumption across your project.
| Cost view | What to investigate |
|---|---|
| Project | Estimated costs across the project. |
| Agent | Estimated costs associated with individual agents. |
| Model | Estimated costs associated with individual models. |
Start with the project cost view, and then explore agent and model costs to identify contributors. Compare costs with token usage and, for agents, activity over the same period to understand what changed.
For example, if the estimated project cost increases, review which agents and models contributed to the increase. Then review token usage and available duration data. For agents, also check activity before deciding what to investigate further.
Note
Cost views show estimates to help you understand consumption. For Azure billing and charge analysis, use Azure Cost Management.
Troubleshoot missing data
| Symptom | What to check |
|---|---|
| Agent activity, duration, or token usage charts are empty | Confirm that you're viewing the correct project and that the selected time range includes agent or model activity. Core usage and performance charts don't support voice agents. For other agents and models, generate new activity if needed, allow time for data to appear, and refresh. |
| Core metrics appear, but agent errors or traces don't | Confirm the Application Insights connection, trace collection for the relevant activity, and your permission to read the telemetry. Agent errors are reported from agent traces. |
| Quality results don't appear | Confirm that evaluations have run, that the relevant evaluation data is available in the connected Application Insights resource, and that you have read access. |
| You see authorization errors in trace or quality views | Check access to the connected Application Insights resource and, where required, its Log Analytics workspace. See Permissions for trace-based workflows. |