Handle microphone button events

Dragon Copilot SDK for JavaScript supports Nuance PowerMic and Philips SpeechMike button presses (and button releases) using the Mozilla WebHID API.

The Dragon Copilot SDK offers two models for handling microphone button events; you can choose one or the other:

  1. The SDK handles button events internally; you can configure the button actions (except the record button) when you initialize the SDK or after initialization. Some button actions still need to be executed by your integration as the SDK can't execute them internally. In this case the SDK raises an event with information about the expected action.

  2. Your app handles button events (except the record button) entirely by setting event listeners for the buttonPressed and buttonReleased events.

Both scenarios need user permission to handle microphone buttons in the browser.

Prerequisites

  • You've added the Dragon Copilot SDK to your page.

  • The browser supports the WebHID API.

  • WebHID events are enabled in your app.

  • The connected device is a Nuance PowerMic or Philips SpeechMike.

Get user permission

To get user permission for button actions, follow this sequence of API calls:

  1. Register buttonDevicePermissionRequired.

  2. Initialize the Dragon Copilot SDK.

  3. Call showMicButtonPermissionDialog in response to a user action, such as a button click or microphone button press. This function can't be called programmatically, it can only be triggered by a user action.

buttonDevicePermissionRequired

Register the buttonDevicePermissionRequired callback event before you call DragonCopilotSDK.dragon.initialize(). The Dragon Copilot SDK raises this event in the following scenarios:

  • The current input device is a PowerMic or SpeechMike, and the browser needs WebHID permission to listen to button events.

  • The user has changed their preferred device to a PowerMic or SpeechMike after initializing the Dragon Copilot SDK, and the browser needs WebHID permission to listen to button events.

DragonCopilotSDK.dragon.buttonDevice.events.addEventListener("buttonDevicePermissionRequired", (event) => {
  console.log("API.buttonDevice.events.buttonDevicePermissionRequired:", event);
  const isRequired = event?.detail?.isRequired === true;
  if (isRequired) {
    document.getElementById("enable-mic-button-tooltip").style.display = "block";
  }
});

showMicButtonPermissionDialog

To open a confirmation box in the browser to request permission from the user, call DragonCopilotSDK.dragon.buttonDevice.showMicButtonPermissionDialog(). This function must be triggered by a user action, such as a microphone button press. Returns a fulfilled promise on confirmation from the user, otherwise it returns a rejected promise.

Enable and disable microphone buttons

Microphone buttons are enabled by default. To turn button handling on and off, use DragonCopilotSDK.dragon.buttonDevice.enableButtons(). This can be useful to handle situations where the device focus is on another app.

DragonCopilotSDK.dragon.buttonDevice.enableButtons(true); // Enable button reactions

DragonCopilotSDK.dragon.buttonDevice.enableButtons(false); // Disable button reactions

Default button actions

This table provides the key and deviceType values for supported microphones, and their default actions in Dragon Copilot. The Handled by column indicates if Dragon Copilot handles the action internally or if the Dragon Copilot SDK sends an integratorActionRequiredOnButtonPress event to your app.

Nuance PowerMic 3

deviceType value:
"PowerMicII-NS"		 Nuance PowerMic 4

deviceType value:
"PowerMic 4"		 Philips SpeechMike

deviceType value:
"SpeechMike III"		 Default Dragon Copilot action Description Handled by
RECORD RECORD RECORD PressAndHold Press and hold to dictate audio. Dragon Copilot SDK
TRANSCRIBE TRANSCRIBE EOL_PRIO TransferText Transfer text from the dictation clipboard to the app. Integration
ENTER_SELECT ENTER_SELECT COMMAND AcceptDefaults Accept default values of fields in texts. Dragon Copilot SDK
CUSTOM_RIGHT F4_D INS_OVR toggleDictationClipboard Show and hide the dictation clipboard. Integration
TAB_BACKWARD TAB_BACKWARD F1_A PreviousEditControl Move text cursor to the previous edit control. Dragon Copilot SDK
TAB_FORWARD TAB_FORWARD F2_B NextEditControl Move text cursor to the next edit control. Dragon Copilot SDK
REWIND REWIND REWIND PreviousField Select the previous text field. Dragon Copilot SDK
FORWARD FORWARD FORWARD NextField Select the next text field. Dragon Copilot SDK
PLAY PLAY PLAY AmbientToggle Toggle ambient recording on and off. Dragon Copilot SDK
CUSTOM_LEFT F2_B F4_D None No default. N/A
- F1_A - None No default. N/A
- F3_C F3_C None No default. N/A

Configure button actions

You can configure button actions either during initialization or afterwards.

  • During initialization: Include a buttonPreferences object in your initialization options.

  • After initialization: call DragonCopilotSDK.dragon.buttonDevice.updateSettings(buttonPreferences: Partial<MicrophoneButtonPreferences>)

type MicrophoneButtonPreferences = {
  record: MicrophoneButtonAction;
  transcribe: MicrophoneButtonAction;
  enterSelect: MicrophoneButtonAction;
  customRight: MicrophoneButtonAction;
  tabBackward: MicrophoneButtonAction;
  tabForward: MicrophoneButtonAction;
  rewind: MicrophoneButtonAction;
  fastForward: MicrophoneButtonAction;
  playStop: MicrophoneButtonAction;
  customLeft: MicrophoneButtonAction;
  functionA: MicrophoneButtonAction;
  functionC: MicrophoneButtonAction;
};

type MicrophoneButtonAction = MicrophoneButtonPressActions | HotkeyPressAction;

enum MicrophoneButtonPressActions {
  Unknown = "None",
  PressAndHold = "PressAndHold", // Handled by Browser SDK
  MicOnOffToggle = "MicOnOffToggle", // Handled by Browser SDK
  AmbientToggle = "AmbientToggle", // Handled by Browser SDK
  TransferText = "TransferText", // Handled by integrator
  AnchorReleaseSpeechFocus = "AnchorReleaseSpeechFocus", // Handled by integrator
  ShowHideCompactView = "ShowHideDragonBar", // Handled by Integrator
  ShowHideDictationClipboard = "ShowHideDictationBox", // Handled by Integrator
  NextField = "NextField", // Handled by Browser SDK
  PreviousField = "PreviousField", // Handled by Browser SDK
  NextEditControl = "NextEditControl", // Handled by Browser SDK
  PreviousEditControl = "PreviousEditControl", // Handled by Browser SDK
  AcceptDefaults = "AcceptDefaults", // Handled by Browser SDK
  FieldComplete = "FieldComplete", // Handled by Browser SDK
}

Actions that must be performed by the integration

When the configured button action needs to be performed by the integrating app, the Dragon Copilot SDK raises an integratorActionRequiredOnButtonPress event. The event has the following properties:

  • action - The specific action required from the integration. This can be one of the following:

    transferText - Transfer text from the dictation clipboard to the app.
    anchorReleaseSpeechFocus - Release the speech focus from the anchor.
    toggleMinimizeRestore - Toggle the window between minimized and restored states.
    toggleDictationClipboard - Open or close the dictation clipboard.
    executeHotKeyPress - Execute a hotkey press.

  • arguments - (Optional) additional information relevant to the action. This can be an array of strings or an array of KeyboardKey objects.

    KeyboardKey has the following properties:

    key - The printable character or key value as reported by the browser (e.g., "a", "Enter", " ").
    code - The physical key code representing the key's position on the keyboard (e.g., "KeyA", "Enter", "Space").
    keyCode - The legacy numeric key code associated with the key (e.g., 65 for "a", 13 for "Enter").

Usage example

DragonCopilotSDK.dragon.buttonDevice.events.addEventListener("integratorActionRequiredOnButtonPress", (event) => {
   const action = event.detail.action;
   if (action === "transferText") {
      executeTransferText(event.detail.arguments[0]); // Could have value "transferAll" or "transferSelected"
  } else if (action === "anchorReleaseSpeechFocus") {
      executeAnchroReleaseSpeechFocus();
  } else if (action === "toggleDictationClipboard") {
      executeToggleDictationClipboard();
  } else if (action === "toggleMinimizeRestore") {
      executeToggleMinimizeRestore();
  } else if (action === "executeHotKeyPress") {
    executeHotkeyAction(event.detail.arguments as KeyboardKey[]);
  }
});

Handle button press and release events in your app

Instead of using the default button actions or configuring the button actions using the buttonPreferences options, you can choose to handle button press and release events entirely in your app.

Note

This section doesn't apply to the record button. The record button is exclusively for starting and stopping recording; other actions can't be configured by the integration.

buttonPressed

When the user presses a microphone button, the Dragon Copilot SDK raises the buttonPressed callback event with information on the device and the button name.

  • This event is only raised when WebHID permissions are in place.

  • This event is not raised when the user presses the record button. The record button is exclusively for starting and stopping recording; other actions can't be configured by the integration.

  • All other button presses are configurable by the integration.

  • If you don't handle button presses in your integration, Dragon Copilot provides a set of default actions. See Default button actions.

  • If you add a buttonPressed event listener, Dragon Copilot assumes that your integration handles all button presses except for the record button, so it won't execute default actions.

buttonPressed usage example

DragonCopilotSDK.dragon.buttonDevice.events.addEventListener("buttonPressed", (event) => {
    const { key, deviceType } = event.detail;
    const message = `Device: ${deviceType}, Key: ${key}`;
    console.log("API.buttonDevice.events.buttonPressed:", message);
  });

buttonReleased

When the user releases a microphone button, the Dragon Copilot SDK raises the buttonReleased callback event with information on the device and the button name.

  • This event is only raised when WebHID permissions are in place.

  • This event is not raised when the user releases the record button. The record button is exclusively for starting and stopping recording; other actions can't be configured by the integration.

  • All other button releases are configurable by the integration.

  • If you don't handle button releases in your integration, Dragon Copilot provides a set of default actions. See Default button actions.

  • If you add a buttonReleased event listener, Dragon Copilot assumes that your integration handles all button releases except for the record button, so it won't execute default actions.

buttonReleased usage example

DragonCopilotSDK.dragon.buttonDevice.events.addEventListener("buttonReleased", (event) => {
    const { key, deviceType } = event.detail;
    const message = `Device: ${deviceType}, Key: ${key}`;
    console.log("API.buttonDevice.events.buttonReleased:", message);
});

See also

API reference

  • Last updated on