Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
[This article is prerelease documentation and is subject to change.]
The Monitor tab for agents powered by the Customer Conversation harness gives you visibility into your agent's performance and activity after deployment. Use it to track conversation sessions, review effectiveness and usage analytics, and understand how users interact with your agent.
Overview
The Overview panel gives a high-level view of your agent's performance for the selected time range. Key metrics include:
- Conversation sessions: The total number of conversations your agent handled.
- Engagement: The percentage of sessions with meaningful user engagement.
- Successful runs: The percentage of runs that completed successfully.
- Total reactions: Thumbs-up and thumbs-down feedback on agent responses.
- Runs: The total number of times the agent ran.
- Average duration: The average length of a session.
Summary
The Summary area uses Copilot to generate an AI summary of key analytics insights about your agent, such as changes in session volume and engagement. Select View More to see the full list of summarized insights.
Billing
The Billing panel shows the number of Copilot credits billed for the selected time period. Select See billing for more detail.
Savings
The Savings panel estimates the time and money your agent saves. Select Calculate savings to set up savings tracking if it's not already configured.
Custom metrics
The Custom metrics area (currently in preview) lets you track metrics specific to your agent, beyond the built-in ones.
Conversation outcomes
The Conversation outcomes chart shows how sessions were resolved over time, broken out by Abandoned, Escalated, Resolved, and Unengaged. Select See sessions or See details to drill in further.
Customer satisfaction
The Customer satisfaction panel shows survey-based feedback from users, once enough responses are collected.
Sentiment
The Sentiment panel analyzes the sentiment of user messages during the selected period, once enough session data is available.
Generated answer rate and quality
This section shows the total number of questions asked, the percentage answered and unanswered, and the quality of generated answers (rated as Looks good, Incomplete, Irrelevant, or Not fully grounded). Select See questions or See details to drill in further.
Tool use
The Tool use panel shows the total number of tool invocations, their success rate, and the top tools used. Select See details for more.
Knowledge source use
The Knowledge source use table lists each knowledge source connected to your agent, its type (such as SharePoint, Dataverse, or public website), total questions answered from it, response quality, and reactions. Select See questions on a row, or See all to view every knowledge source.
Download conversation transcripts
You can download conversation transcripts a few minutes after the conversation times out. You can download them in Dataverse using the Power Apps portal and as session chat transcripts using the Copilot Studio app.