Configure quality assurance (preview)

[This article is prerelease documentation and is subject to change.]

Quality assurance uses AI-powered evaluations to help organizations consistently monitor, measure, and improve conversations handled by self-service agents. Create quality indicators that define good performance, add them to evaluation plans, and review quality scores and AI-generated insights for ongoing and closed conversations.

Quality assurance evaluates only the portion of a conversation handled by the self-service agent. If the conversation is escalated to a human representative, the representative-handled portion isn't included in the self-service agent's quality score.

Important

  • This is a preview feature.
  • Preview features aren’t meant for production use and might have restricted functionality. These features are subject to supplemental terms of use, and are available before an official release so that customers can get early access and provide feedback.

Prepare for quality assurance

Before configuring and using quality assurance, ensure that:

  • At least one self-service agent is configured.
  • Conversation data is available in Contact Center Intelligence.

Quality assurance uses the following building blocks:

  • Quality indicators define measurable aspects of conversation quality, such as:
    • Acknowledging the customer's concern.
    • Showing empathy.
    • Using an appropriate greeting.
    • Avoiding rude or inappropriate language.
    • Closing the conversation appropriately.
  • Evaluation plans define:
    • When conversations are evaluated.
    • Which quality indicators are used.
    • Which conversations are included.
    • What percentage of matching conversations is evaluated.

A quality indicator contains one or more questions. Each question has answer options with assigned scores. The AI evaluator selects an answer based on the conversation and uses its score to calculate the conversation's overall quality score.

Quality indicators are reusable and can be added to more than one evaluation plan. An evaluation plan must be active before it evaluates conversations.

Turn on quality evaluation

  1. In Contact Center Intelligence, open Insights and quality.
  2. Select Quality assurance.
  3. Turn on Quality evaluation.

After quality evaluation is turned on, configure quality indicators and evaluation plans on the same page.

Create quality indicators

Create the quality indicators that you need before creating an evaluation plan.

  1. On the Quality assurance page, select Quality indicators.
  2. Select Add indicator.
    • Enter an Indicator name.
    • Optionally, enter a description that explains what the indicator measures.
  3. Select Add questions.
    • Enter one or more questions that the AI evaluator uses to assess the conversation.
    • Select an answer type.
    • Configure the answer options and score for each option.
    • Optionally, add AI instructions.
  4. Add the indicator.
  5. Turn on the indicator when it's ready for use.

Select an answer type

Answer type Description
Yes/No Provides Yes and No answers. By default, Yes has a score of 100 and No has a score of 0. You can adjust the scores to match the question.
Multiple choice Lets the evaluator select all answers that apply. Use this type when more than one answer can be true. For example, for "Which compliance issues occurred?", the evaluator can select each issue found in the conversation.
Choose from list Lets the evaluator select exactly one best answer. Use this type when the options are mutually exclusive. For example, for "How well was the issue resolved?", the evaluator selects one option such as Excellent, Good, Needs improvement, or Poor.

Multiple choice is a multi-select answer type, while Choose from list is a single-select answer type.

For multiple-choice and list questions, enter a label and score for each answer option. The evaluator uses the scores associated with the selected answers when calculating the quality result. Higher answer scores should represent better performance for the quality dimension being measured. Keep the scoring direction consistent across the indicator.

Add AI instructions

AI instructions add guidance about how a question should be evaluated. Use them to:

  • Explain terms that have a specific meaning for your organization.
  • Describe acceptable and unacceptable behavior.
  • Clarify exceptions.
  • Help the evaluator distinguish between similar answer options.

Keep instructions specific to the question. Avoid including unrelated policies or asking the evaluator to assess several quality dimensions in one question.

Manage built-in and custom indicators

Contact Center Intelligence can include built-in quality indicators.

  • Built-in indicators are read-only. You can review their questions and scores, but you can't edit or delete them.
  • Custom indicators can be edited, duplicated, deactivated, or deleted when they aren't required.

The quality indicators list also shows how many evaluation plans use each indicator and when it was last modified.

View indicator version history

When you save changes to a custom quality indicator, it creates a new version.

Use Version history to review earlier versions of the indicator, including its questions and answer options. The latest saved version is used for future evaluations.

Create evaluation plans

An evaluation plan controls when evaluations run, the conversations evaluated, and the quality indicators used.

  1. On the Quality assurance page, select Evaluation plans.
  2. Select Add plan.
    • Enter a Plan name.
    • Optionally, enter a description.
    • Select a frequency:
      • Real-time: Evaluate a conversation while it is in progress. The score can change as the conversation continues.
      • On conversation close: Evaluates the conversation after it ends. The result is available in the closed-conversations experience. It includes the overall score, evaluation summary, suggested actions, and indicator-level results.
    • Review the score bands. Supervisors can review the latest score, score trend, evaluation timeline, AI summary, suggested actions, and indicator results for the ongoing conversation.
    • Enter the sampling rate to specify the percentage of matching conversations that the plan evaluates.
    • Select one or more quality indicators.
  3. Select Set condition. You are presented with options to set conditions on agent and channel.
  4. Add conditions to control the conversations the plan evaluates. When you add multiple conditions, a conversation must match all conditions to be evaluated by the plan. If you leave the conditions empty, the plan evaluates all matching conversations.
  5. Add the plan.
  6. Turn on the plan to begin evaluations.

Understand score bands

Quality scores use a scale from 0 through 100.

Score Status
0-40 Critical
41-70 Warning
71-100 Normal

Score bands help supervisors filter and prioritize conversations. The bands aren't editable in this release.

Manage evaluation plans

The Evaluation plans list shows each plan's frequency, conditions, last-modified date, and active state.

  • Turn on a plan to begin evaluations.
  • Turn off a plan to stop new evaluations.
  • Edit an existing plan to change its indicators, frequency, or conditions.
  • An active plan can't be deleted. Turn it off before deleting it.

Only one evaluation plan can score a conversation. If a closed conversation matches more than one on-close evaluation plan, the plan that was created first is used.

Understand quality scoring

Quality scores are calculated from the answers selected for the questions in the evaluation plan's quality indicators.

For each evaluated question:

  1. The AI evaluator reviews the conversation.
  2. It selects the answer that best matches the conversation.
  3. The score assigned to that answer contributes to the indicator score.

Question scores contribute to each quality indicator. When a plan contains multiple indicators, each indicator has equal weight. The indicator results are summed and normalized to produce an overall conversation score from 0 through 100.

Quality details show:

  • The overall score.
  • The evaluation plan that produced the score.
  • Each quality indicator.
  • The question evaluated for the indicator.
  • The selected answer and its score.
  • An explanation of why the answer was selected.

Review ongoing conversation quality

Real-time quality results are available for ongoing conversations covered by an active real-time evaluation plan.

  1. In Insights and quality, select Real-time.
  2. Find the Ongoing conversations table.
  3. Review the Status and Quality columns.
  4. Optionally, filter conversations by Normal, Warning, or Critical.
  5. Select View details for a conversation.

Select a point in the conversation timeline to review the score and indicator results from that stage of the conversation.

Real-time scores can improve or decline as new messages are added to the conversation. Use the latest result to understand the current conversation state.

Review closed conversation quality

On-close quality results are available after an eligible conversation ends.

  1. In Insights and quality, select Historic.
  2. Select Closed conversations.
  3. Review the quality score for a conversation.
  4. Select the conversation to open its details.

Troubleshoot missing quality scores

A conversation might not have a quality score when:

  • It isn't eligible under an active evaluation plan, such as when it doesn't match the plan's conditions.
  • The matching evaluation plan is activated only after the conversation is processed. Plans don't score conversations retroactively.
  • It isn't selected for evaluation based on the plan's sampling rate.

Sampling applies to both real-time and on-close evaluation plans. For example, if a plan has a sampling rate of 50%, only a subset representing 50% of the matching conversations is selected for evaluation.

Frequently asked questions

Can more than one evaluation plan score the same conversation?

No. A conversation can be scored by only one evaluation plan. If a closed conversation matches more than one on-close plan, the plan that was created first is used.

How are multiple quality indicators combined?

When an evaluation plan contains multiple quality indicators, each indicator has equal weight. The indicator results are summed and normalized to calculate the final conversation score from 0 through 100.

Can a conversation be scored again?

No. After a conversation has been scored, it can't be re-scored. Changes to an evaluation plan or quality indicator don't recalculate an existing score.

Do evaluation plans apply to earlier conversations?

No. Evaluation plans are forward-looking. Activating or changing a plan affects eligible conversations evaluated after the change. It doesn't retroactively evaluate earlier conversations.

Use Contact Center Intelligence

  • Last updated on