Monitor an agent overview

The Monitor tab in the new agent experience gives you visibility into your agent's performance and activity after deployment. Use it to track conversation sessions, review analytics, identify errors, and understand how users interact with your agent.

Note

Features in this article are powered by the GitHub Copilot harness, which uses Copilot Credits for usage-based billing. Learn more about Copilot Credits in Overview of billing for agents powered by the GitHub Copilot harness.

Usage-based billing applies to using, building, testing, and evaluating agents. These actions might consume Copilot Credits.

Note

The Monitor tab isn't available until you publish your agent. To make it available, select Publish.

What can you monitor?

The Monitor tab gives a high-level view of your agent's performance. Key metrics include:

  • Conversation sessions: The total number of conversations your agent handles.
  • User engagement: How actively users interact with the agent, including message counts and session duration.
  • Cost and billing tracking: Copilot Credit consumption over the selected period and savings estimates.
  • Total reactions: User responses, including thumbs-up and thumbs-down feedback and comments on specific agent responses.
  • Response quality: Indicators of how effectively the agent responds to user requests over time.

Filter the dashboard by time range to focus on recent activity or longer-term trends.

Summary

The Summary area uses Copilot to generate an AI summary of key analytics insights about your agent. Select View More to see the full list of summarized insights.

It helps you quickly identify common themes, key metrics, and areas for improvement for the selected reporting period. Insights can include engagement metrics, customer sentiment and feedback, and trends since the previous reporting period.

Cost and billing

The cost and billing metrics show the number of Copilot Credits billed for the selected period and provide savings estimates for time and cost.

Learn more in Billing rates and management.

Reactions

The Reactions section shows user feedback on agent responses. The chart counts how often users select thumbs up (positive) or thumbs down (negative) on each response they receive from your agent.

Agent capabilities

The Capabilities section helps you understand how users interact with your agent and where answer reliability can improve. It includes:

  • Tool use: Which tools the agent invoked during the session.

Session tracking

The session list shows individual conversation sessions with your agent. Each session includes:

  • Status: Whether the session is completed, failed, or in progress.
  • Duration: How long the session lasted.
  • User messages: The number of messages exchanged.
  • Tools used: Which tools the agent invoked during the session.

Filter sessions by status, such as failed, completed, or in progress, to quickly find sessions that need attention.

Select a session to view its full transcript, including the user's messages, the agent's responses, and details about tool invocations and knowledge source usage.

Error monitoring

The Monitor tab helps you identify and diagnose problems:

  • Filter sessions by Failed or Rejected status to find problematic interactions.
  • Review session transcripts to understand what went wrong.
  • Check tool connection status and authentication errors.
  • Look for patterns in failed sessions that might indicate a configuration problem.

For step-by-step guidance on using the Monitor tab, see Monitor an agent.

Download conversational transcripts

You can download conversation transcripts a few minutes after the conversation times out. You can download any time period within the last 29 days. You can download them in Dataverse using the Power Apps portal and as session chat transcripts using the Copilot Studio app. It can take up to an hour after the analytic session ends before the related data appears on the Monitor tab.

Note

You can't download conversation transcripts in Dataverse on the Copilot Studio app in Teams. To review and export transcripts in Dataverse, you need to sign up for the Copilot Studio web app. You can download session chat transcripts by using the Copilot Studio app. For more information, see Download agent session transcripts.

  • Last updated on