Analytics considerations for Copilot Studio vs Dynamics 365 Omnichannel

Analytics is a top priority for Dynamics 365 Omnichannel and Copilot Studio customers tracking the performance of their agents and calls center representatives. There are major efforts currently underway to improve the analytics across both these products. While the analytics continue to improve, there are several considerations you can adopt as an agent builder or customer service expert to improve your understanding of analytics across Copilot Studio and Dynamics 365 Omnichannel to enhance agent performance.

This article touches on concepts that help you understand the difference between Dynamics 365 Omnichannel analytics and Copilot Studio analytics, and the purpose behind the analytics tools in each of them. This helps you make informed decisions about the agent performance for your implementation.

Using Copilot Studio Analytics

The standard analytics tools in Copilot Studio agents are focused mainly on providing insights about agent topics performance through during chat sessions. They don't provide an end-to-end insight that includes any human-representative conversation. But they can be used to improve the topic performance inside the agent.

The insights you can get from Copilot Studio analytics include metrics for questions like:

• How engaged are users with the agents?
• Which topics are driving resolutions, escalations, and abandonment?

In general, Copilot Studio analytics tools are useful to enhance the agent performance in terms of the topics. This gives insights about what new topics need to be created, and what topics need to be updated to achieve better resolution rates for future user queries.

To learn more, see Key concepts – Analytics

Understanding Copilot Studio analytics definitions

It's important to understand the way Copilot Studio calculates the various metrics made available to Analyze agent performance and usage in Copilot Studio. These calculations are based on the agent sessions. A session can be an unbilled or a billed session. A billed session is a continuous interaction between an agent and a user to address all of user’s question within a 60-minute time window.

A billed session can include multiple user authored topics and can't last longer than 60 minutes or 100 turns.

A session is interaction between an agent and a user to address or assist a user with one query or intent (one user authored topic). Most sessions are engaged, which means that a user-authored topic or an escalation was triggered. There are three possible outcomes for an engaged session:

• Resolved: Customers are offered an end-of-conversation survey and either responds yes or leaves without a response.
• Escalated: Customers indicate that they need a customer service representative to help resolve the issue.
• Abandoned: An engaged session that isn't resolved or escalated and is inactive after 60 minutes from the beginning of the session. This would happen, for example, if the user leaves.

A topic is tied to a dialog flow built in Copilot Studio. One topic contains one dialog flow.

Using Dynamics 365 Omnichannel chatbot analytics

The standard bot analytics tools in Dynamics 365 Omnichannel are focused on providing insights about how the overall agent deflection is performing through the conversations. These analytics are optimized to give insight into the overall interaction/conversation across both agent and human representative.

The insights you can get from Dynamics 365 Omnichannel include:

• Bot resolution time
• Bot escalation time
• Conversation escalation

Copilot Studio vs Omnichannel analytics comparison

Analytics calculations differ between Copilot Studio and Dynamics 365 Omnichannel for metrics with similar names, such as resolution rate, escalation rate, and abandonment rate.

For example, Copilot Studio analytics might consider an ongoing interaction with a user that would be considered one conversation, and one agent session in Dynamics 365 Omnichannel analytics as multiple separate analytic sessions. As a result of this, the resolution rates, escalation rates, and abandonment rates can be different across Copilot Studio analytics and Dynamics 365 Omnichannel analytics.

Here's an example showing the differences for analytics between the two for a single conversation:

These key differences in the calculation of resolution rates and escalation rates need to be considered when consuming the Copilot Studio and Dynamics 365 Omnichannel analytics to determine key metrics like deflection rate, resolution rate, and so on. The general recommendation is that you rely on Dynamics 365 Omnichannel analytics for tracking the overall agent deflection rate for the organization and use the Copilot Studio analytics for enhancing the topic performance.