Analyze agent performance

  • 5 minutes

After an agent is deployed and customers are interacting with it, statistics that are related to the agent will become available. You can access this information through the Analytics tab in the side navigation pane. On this tab, you can find key performance indicators (KPIs) that show the volume of sessions that your agent has handled, how effectively your agent was able to engage users and resolve issues, escalation rates to live agents, and abandonment rates during conversations. You'll find customer satisfaction information at the KPI level and on the Customer Satisfaction tab.

You can also view detailed session history and transcripts by selecting Sessions from the Analytics tab. On the Sessions page, you can download a file with the full session transcript, which can be a helpful way for you to adjust the performance of your agent and change the content in your topics to improve your agent's efficiency.

Analyze agent performance and usage

The Summary page gives you a broad overview of your agent's performance. It uses AI technology to show you which topics have the greatest impact on escalation rate, abandon rate, and resolution rate. For more information, see the table under Summary charts.

Screenshot of analytics.

The Summary page includes various charts with graphical views of your agent's KPIs:

For more information, see Analyze agent performance and usage.

The Engagement rate drivers, Abandon rate drivers, and Resolution rate drivers charts use natural language understanding to group issues as topics. These charts show you the topics that have the most impact on the performance of your agent.

Analyze customer satisfaction for Microsoft Copilot Studio agents

The Customer Satisfaction page provides a detailed view of customer satisfaction (CSAT) survey data, including the average CSAT score over time and the topics that have the most impact on the CSAT score. The Customer Satisfaction page includes various charts with graphical views of your agent's customer satisfaction indicators:

  • Customer satisfaction drivers chart - Uses AI to group related support cases as topics and then displays topics in order of their impact on customer satisfaction over the specified time period.

  • Scores over time chart - Provides a graphical view of the average CSAT score over the specified time period.

  • Average CSAT score - Provides a graphical view of the average of CSAT scores for sessions in which customers respond to an end-of-session request to take the survey.

  • CSAT survey response rate - Shows the number of CSAT surveys that were presented and the percentage of surveys that were completed.

For more information, see Analyze customer satisfaction.

Analyze topic usage in Microsoft Copilot Studio

The topic details page provides a view into the performance of individual topics and how they can be improved. You can display the topic details page by selecting the Detail link in one of the following charts on the Summary and Customer Satisfaction pages:

The topic details page can also be displayed by opening an individual topic from the Topics page and selecting Analytics at the top of the page.

Screenshot of the analytics button.

The topic details page includes various charts with graphical views of a topic's KPIs:

  • Topic Summary charts - Summarize topic performance indicators for the specified time period and the percent change over the period.

  • Impact Summary charts - Summarize the impact of the topic on KPIs for the specified time period.

  • Topic Volume by Day chart - Provides a graphical view of the number of sessions for the topic over the specified time period.

For more information, see Analyze topics usage.

Analyze session information in Microsoft Copilot Studio

Having access to session information that is related to your agent can help you identify potential changes that need to be made. By default, you can download up to seven days of agent conversation transcript sessions from the past 30 days directly from the Microsoft Copilot Studio portal.

Session transcripts are available by going to Analytics > Sessions tab. On the Sessions tab, you'll be able to see all the different sessions that have been run for the agent. A session represents a conversation that someone had with the agent. If your agent had a high number of sessions, they'll be broken down into multiple rows to help make managing the sessions easier. Each row will contain 2,500 sessions. You can select each row to download the session transcripts for the specified time frame.

The downloaded file contains the following information:

  • SessionID - A unique identifier for each session.

  • StartDateTime - Time at which the session started. Entries are sorted by this column in descending order.

  • InitialUserMessage - First message that is entered by the user.

  • TopicName - Name of the last authored topic that was triggered in this session.

  • ChatTranscript - Transcript of the session in the following format:

    • User says:" "; Agent says:" "; structure

    • Conversation turns are separated by semicolons

    • Agent says doesn't include the options that are presented to the user

  • SessionOutcome - Outcome of the session (Resolved, Escalated, Abandoned, Unengaged).

  • TopicId - A unique identifier of the last authored topic that was triggered in this session.

For more information, see Analyze session information.