Edit

Share via

Manage Quality Evaluation Agent

Conversations: Applies to: Dynamics 365 Contact center only

Cases: Applies to: Dynamics 365 Customer Service only

Quality Evaluation Agent is an AI agent that assesses customer engagement using an evaluation framework defined by supervisors. The quality evaluation framework comprises evaluation criteria, evaluation plan, and evaluations that are essential for the AI Agent to work.

Quality Evaluation Agent autonomously scores cases and conversations, delivering actionable insights to help supervisors improve interaction quality. It evaluates cases and closed conversations to make sure they comply with required standards. When standards aren't met, the AI Agent recommends actions to enhance future interactions.

Evaluation criteria:

Create a form with questions, answer choices, scoring metrics, and detailed instructions for the Quality Evaluation Agent. The AI agent uses this form to assess interactions.

Evaluation plan:

Set up plans to schedule when to evaluate interactions. You can select interactions based on specific conditions and use the right evaluation criteria to review interactions systematically. You can create, activate, and manage evaluation plans, request on-demand evaluations, and enable bulk evaluations to streamline your review process.

Evaluations:

The Quality Evaluation Agent evaluates case and conversations, provides summaries of the interactions, with insights and recommendations that help improve cutomer interactions.

Important

  • This feature is intended to help customer service managers or supervisors enhance their team's performance and improve customer satisfaction. It isn't intended to be used, and should not be used, to make decisions that affect the employment of an employee or group of employees, including compensation, rewards, seniority, or other rights or entitlements.
    Customers are solely responsible for using Dynamics 365, this feature, and any associated feature or service in compliance with all applicable laws, including laws that are related to accessing individual employee analytics, and monitoring, recording, and storing communications with users. As part of this compliance, customers must adequately notify users that their communications with customer service representatives (service representatives or representatives) might be monitored, recorded, or stored. As required by applicable laws, customers must also obtain consent from users before they use this feature with them. In addition, customers are encouraged to have a mechanism in place to inform their service representatives that their communications with users might be monitored, recorded, or stored.
  • Evaluations for conversations is a preview feature.
  • Preview features aren’t meant for production use and might have restricted functionality. These features are subject to supplemental terms of use, and are available before an official release so that customers can get early access and provide feedback.

Configure connection references

You need to configure connection references for Quality Evaluation Agent flow to integrate with Microsoft services. These connections link flows to essential data sources such as Microsoft Dataverse and Copilot Studio, ensuring smooth operation and enhanced functionality. Learn how to configure connection references for the Quality Evaluation Agent flow.

Prerequisites

Data movement across regions

For optimal performance, service calls might be routed outside the customer's regional boundary if local capacity is temporarily unavailable. To participate, customers are requested to provide consent for potential data movement across regions. This enables us to deliver a seamless experience leveraging the full capabilities of Copilot and generative AI features within the Power Platform.

If a customer prefers to keep data strictly within their region, we recommend waiting for broader availability before enabling the preview.

Learn more in Move data across regions for Copilots and generative AI features – Power Platform.

Role and privileges

Persona Role Privileges
Administrator Quality Administrator - Configure Quality Evaluation Agent.
- Create evaluation criteria and evaluation plan.
- Complete or assign an evaluation.
Supervisor Quality Manager - Create evaluation criteria and evaluation plan.
- Complete or assign an evaluation.
Supervisor Quality Evaluator Complete or assign an evaluation.

Enable Quality Evaluation Agent

You need to enable Quality Evaluation Agent for your supervisors in Copilot Service admin center. You can select cases or conversations record type. You can also set a scoring and threshold value. Each criteria and question in the evaluation is scored out of 100 points. You can set thresholds to define good and poor-quality metrics.

  1. In Copilot Service admin center, go to Customer Support > Quality management. The Quality management page appears.

  2. Select Manage for Quality Evaluation Agent. The Quality Evaluation Agent page appears.

  3. In the Enablement by record type section:

  4. Select Case for case evaluation or Conversation for conversation evaluation.

  5. Select the Enable bulk evaluations (preview) checkbox to perform bulk evaluation for cases. In Specify data, you can see the default entities that are added.

    1. Select Manage data to go to the Specify data dialog, where you can:
      • Delete data types or clear a row to temporarily exclude the data type from being summarized.
      • Add more data type fields for Quality Evaluation Agent input configuration by selecting Add data.

  6. In the Evaluation criteria score section, select Enable scoring for criteria. The Evaluation criteria scoring turned on dialog appears.

  7. Select Turn on. Once turned on, it can’t be turned off.

  8. From the Set threshold value out of 100 dropdown list, select a threshold value.

  9. Select Save.

Use evaluation plan
Use on-demand evaluation
Use evaluation criteria
Use evaluations

  • Last updated on