Configure voice settings (Customer Conversation) (preview)

[This article is prerelease documentation and is subject to change.]

Every agent you create with the Customer Conversation harness supports Speech conversations by default, and Voice is a configurable feature of that Speech modality. Use the steps in this article to adjust voice-specific settings for your agent.

Note

Voice authentication (caller verification on voice channels) is configured on the Security tab of the Shared Settings panel. For those steps, see Configure agent details and security (Customer Conversation). Voice authentication works together with the Biometric Authentication capability to identify and verify callers, and unlocks three extra events related to biometric authentication enrollment and consent. Unenrolling from biometric authentication is available as a skill instead of an event. Biometric authentication and fraud detection policies for voice authentication are managed in the Microsoft Contact Center Intelligence portal.

Configure voice settings

  1. Open your agent in Copilot Studio, and select the Build tab.

  2. Select the primary agent's card to open its configuration panel. Sub-agents inherit the voice settings configured for the primary agent. To update these settings, select the primary agent's card.

  3. Select Voice Settings to open the Voice Settings dialog. The dialog header shows the agent's currently selected model as a badge.

  4. Use the tabs on the left to configure each area, then select Save.

Voice font

Auto — optimized for cost

Agents that use the text model generate speech responses through Microsoft Neural Text-to-Speech (TTS). The following voice font settings apply to agents using the optimized for cost mode:

  1. Review the Primary language for the agent (for example, English (United States) (en-US)).

  2. Select a Voice name from the dropdown (for example, Adam Dragon HD Latest).

  3. Adjust Speaking speed on the Slow-Normal-Fast slider.

  4. Adjust Voice pitch on the Low-Medium-High slider.

  5. Select Try it out to preview the configured voice.

Auto — optimized for latency

Agents that use a built-in speech-to-speech model support the following voice fonts:

  • Alloy
  • Ash
  • Ballad
  • Coral
  • Echo
  • Sage
  • Shimmer
  • Verse
  • Marin
  • Cedar

Note

  • For agents connected to Dynamics Contact Center, the voice selected in the Customer Conversation harness (Settings > Voice) is the voice used by the agent. Voice settings configured in the Dynamics Copilot Service admin center don't override the agent voice.
  • To align Dynamics system message voices with your agent voice, use one of the shared supported voices: Alloy, Ash, Echo, or Shimmer.

DTMF behavior

  1. Turn the Allow DTMF toggle on or off to control whether the agent accepts touch-tone (dual-tone multi-frequency) input on phone-based conversations.

  2. Set the Interdigit timeout (ms), the time the agent waits between key presses before treating the entry as complete (default: 3000).

  3. Select a Termination key from the dropdown, the key that signals the end of DTMF input (default: # (pound)).

Speech sensitivity

For Auto — optimized for cost mode

  1. Under Speech sensitivity, adjust the Sensitivity slider to control how sensitive the agent is to audio input. Lower settings filter out more background noise; higher settings work best in quiet environments. The slider shows a numeric value (for example, 0.60).

For Auto — optimized for latency mode

  1. Voice activity detection (VAD) controls how your agent determines when a caller starts and finishes speaking. The Customer Conversation harness supports two VAD modes:
VAD mode How it works Best for
Server VAD Detects speech based on the audio signal and configurable silence thresholds. The agent considers the caller's turn complete after the configured period of silence. Structured interactions with short or discrete responses, such as entering an order number or answering a yes-or-no question.
Semantic VAD Uses the meaning and context of what the caller is saying to determine whether they have finished speaking, rather than relying only on silence. Conversational or open-ended interactions where callers might pause mid-sentence or provide longer responses.

Configure Server VAD

Server VAD provides the following settings:

Setting Description Suggested starting point
Threshold Controls the sensitivity of speech detection to voice versus background noise. 0.6
Prefix padding Specifies how much audio from immediately before detected speech is included, which helps prevent the beginning of an utterance from being cut off. 300 ms
Silence duration Specifies how long the caller must remain silent before the agent considers the turn complete. 750 ms

Threshold

  • Lower values, such as 0.3–0.4, make speech detection more sensitive to quieter speech but can also detect background noise.
  • Higher values, such as 0.7–0.9, require a stronger speech signal and can reduce false triggers from background noise.
  • Start with 0.5 and increase the value if background noise frequently triggers speech detection.

Prefix padding

  • Lower values, such as 200 ms, include less audio from before speech is detected.
  • Higher values, such as 500 ms, preserve more audio before speech detection and can help avoid cutting off the beginning of an utterance.
  • Start with 300 ms.

Silence duration

  • Lower values, such as 500 ms, allow faster turn-taking but can cause the agent to respond when a caller pauses mid-thought.
  • Higher values, such as 1,000 ms, give callers more time to pause but can make the interaction feel slower.
  • Start with 750 ms.

Configure Semantic VAD

Semantic VAD uses the context of the caller's speech to determine when their turn is complete.

Use Eagerness to control how quickly the agent responds after determining that the caller has finished speaking.

Eagerness Behavior Best for
Low Waits longer before responding. Callers who frequently pause or think out loud.
Medium Balances waiting and responding. General conversations.
High Responds more quickly when the caller's thought is complete. Fast-paced interactions with shorter or simpler requests.

Speech Collection

Note

Speech collection settings are available for text models (Auto — optimized for cost mode) only.

Speech collection controls when the agent considers a caller's spoken input complete and finalizes their turn.

Under Speech collection, configure Utterance end timeout. This setting determines how long the agent waits after the caller stops speaking before finalizing the utterance.

The default is 700 ms.

  • A shorter timeout (for example, 300 ms) can make conversations feel more responsive, but the agent might finalize the turn if the caller pauses mid-sentence.
  • A longer timeout (for example, 1,500 ms) gives callers more time to pause, but can make the conversation feel slower.

Start with the default of 700 ms and adjust based on your testing.

Silence detection

  1. Under Silence detection timeout, choose one:
    • No silence timeout (default): The agent doesn't time out during silence.
    • Silence detection timeout: The agent applies a timeout during silence. This setting works together with the Silence Detection event.

Important

  • Silence detection isn't turned on by default. If the user doesn't speak, the agent waits indefinitely without prompting. Explicitly turn on silence detection and configure a reprompt message to handle silent callers.
  • The default silence detection timeout is 7,000 ms (7 seconds). Validate this value against your specific use case and caller environment before deploying to production. Seven seconds might feel too long for some callers or too short for others depending on the nature of the interaction, for example, complex questions or noisy environments. Test with real-world call data to determine the appropriate threshold for your scenario.
  • Before enabling silence detection, ensure that the behavior you configure in your silence detection topic (for example, Escalate, Hang Up, Reprompt) is intentional and appropriate for your use case. Misconfigured fallback behavior, such as inadvertently setting the fallback to Escalate when the intent is to hang up, or vice versa, can result in unexpected call outcomes.

Latency messaging

  1. Under Latency options, choose what the agent does when there's latency:

    • Play a message (default): The agent plays a spoken message.
    • Play an audio file: The agent plays an audio file instead.

  2. If you chose Play a message, enter the Latency message text (default: "Hmm...Hold on a moment...").

  3. Set the Send message delay (ms), how long the agent waits before playing the latency message (default: 500 ms).

  4. Set the Minimum playback time (ms), the minimum duration the message plays (default: 5000 ms).

Note

Latency messaging settings only apply to telephony channels.

  • Last updated on