Dictation and ambient recording modes

Dragon Copilot SDK supports two distinct recording modes for capturing clinical documentation: dictation and ambient recording. Each mode serves different clinical workflows and features entirely different architectures. This article explains the differences between these modes and SDK implementation considerations for each.

You can implement both modes by creating a custom recording UI. The SDK provides event listeners for both modes, allowing you to monitor recording state changes, volume levels, and processing events.

Dictation mode

In dictation mode, the clinician dictates notes directly into the microphone and the system processes the audio in real-time, providing immediate speech-to-text output in the app. The clinician controls the recording session through voice commands or UI interactions.

Characteristics:

  • The clinician speaks directly into the microphone.
  • The recognition results appear immediately in the app.
  • The event model focuses on recording state changes, volume levels, and dictation processing events.

In dictation mode you can provide features such as:

Dictation workflow

  1. Add the SDK script from the CDN.
  2. Enable speech in your app by defining the set of speech-enabled controls.
  3. Set up authentication.
  4. Initialize the SDK. In addition to the mandatory initialization options, set InitializationOptions.isDictationEnabled to true to enable dictation mode.
  5. Dictation is now available in standard controls.

Ambient recording mode

Ambient recording mode continuously and automatically captures ambient audio during patient encounters. The app uploads the output to the server for transcription and note generation, and you retrieve results through Dragon data exchange APIs. You configure a session with patient context, and the clinician can start and stop recording as needed. A session can contain multiple ambient recordings.

Characteristics:

  • Requires a configured ambient session with patient context.
  • Captures ambient audio for an entire patient encounter, including multiturn conversations.
  • Asynchronously processes and uploads output to the server for transcription.
  • Uses Dragon data exchange plus AI note generation services.
  • Requires result retrieval through Dragon data exchange APIs.
  • Event model includes recording state changes, recording upload, and processing events such as ambientRecordingUploadStatusChanged.

Ambient recording workflow

  1. Set up Dragon data exchange webhooks for upload status and result retrieval.
  2. Add the SDK script from the CDN.
  3. Set up authentication.
  4. Initialize the SDK. In addition to the mandatory initialization options, set InitializationOptions.isAmbientEnabled to true to enable ambient recording mode.
  5. Configure an ambient session with patient context.
  6. Start/stop ambient recording as needed.
  7. Retrieve results through Dragon data exchange APIs.

Key differences

The following table summarizes the implementation and workflow differences between dictation and ambient recording modes:

Aspect Dictation Ambient
Purpose Convert a clinician's speech directly to text Capture a clinician-patient conversation and generate clinical documentation
Prerequisites Set up your authentication method, define the set of speech-enabled controls, initialize the SDK Set up back-end, Dragon data exchange webhooks, authentication and initialize the SDK
Session setup None Uses the setSessionData() function with patient/EHR context
Input Usually one speaker dictating Multiparty conversation (typically clinician and patient)
Output Real-time text at the cursor A Dragon standard payload object that you can process further, including a transcript of the conversation, AI generated note sections based on the conversation, orders and conditions
Result delivery Immediate in-page Webhook or polling

Event handling

Both modes support event listeners for monitoring recording state:

Event Dictation Ambient
recordingStarted ✓ ✓
recordingStopped ✓ ✓
recordingVolumeChanged ✓ ✓
dictationProcessingStarted ✓ —
dictationProcessingFinished ✓ —
dictationProcessingStartedForElement ✓ —
dictationProcessingFinishedForElement ✓ —
ambientRecordingUploadStatusChanged — ✓
ambientRecordingLimitApproaching — ✓
ambientRecordingLimitReached — ✓

Implementation examples

Support for both modes

Many healthcare apps benefit from supporting both modes, so clinicians can choose the best option for their immediate needs. For example, a clinician might use dictation mode for quick notes and ambient recording mode for capturing a full patient encounter. Or they might want to edit a note from an ambient recording session. The following example shows how to implement both modes in a single app:

// Allow user to select recording mode
async function handleRecordingModeSelection(mode: "dictation" | "ambient") {
  if (mode === "ambient") {
    // Set up ambient session data
    await setupAmbientSession();
  }
  
  // Start recording in selected mode
  await DragonCopilotSDK.dragon.recording.startRecording({ 
    recordingMode: mode 
  });
}

Dictation-focused app

async function initiateDictationSession() {
  try {
    // Start dictation recording
    await DragonCopilotSDK.dragon.recording.startRecording({ 
      recordingMode: "dictation" 
    });
    
    // Update UI to show recording state
    updateUIForRecordingState(true);
    
  } catch (error) {
    console.error("Failed to start dictation:", error);
    showNotification("Unable to start recording", "error");
  }
}

Ambient-focused app

async function initializeAmbientSession(patientContext) {
  try {
    // Configure session data
    await DragonCopilotSDK.dragon.recording.ambient.setSessionData({
      correlationId: generateUUID(),
      clientApplicationStableId: "my-app",
      clientApplicationVersion: "1.0.0",
      localeInfo: {
        recordingLocales: ["en-US"],
        encounterReportLocale: "en-US",
        encounterUxLocale: "en-US"
      },
      ehrData: {
        patientId: patientContext.id,
        appointmentId: patientContext.appointmentId,
        physicianName: patientContext.providerName,
        patient: {
          id: patientContext.id,
          firstName: patientContext.firstName,
          lastName: patientContext.lastName,
          gender: patientContext.gender,
          dateOfBirth: patientContext.dob
        },
        version: 1,
        dataOrigin: 0
      },
      draftAutoCreated: true
    });
    
    // Start ambient recording
    await DragonCopilotSDK.dragon.recording.startRecording({ 
      recordingMode: "ambient" 
    });
    
  } catch (error) {
    console.error("Failed to initialize ambient session:", error);
  }
}

For more details on implementing dictation and ambient recording modes, see the following articles:

  • Last updated on