Dragon Copilot Embedded for Mobile - overview

Introduction

Dragon Copilot Embedded for Mobile is an SDK that you can integrate with your iOS apps. When initialized, the SDK launches the Dragon Copilot web UI and gives your app access to the full suite of Dragon Copilot features and capabilities.

This document outlines the steps to embed Dragon Copilot Embedded for Mobile seamlessly into your iOS mobile apps.

This enables your users to:

• Perform ambient recording and stream audio to Dragon Copilot for generative AI processing in a reliable and secure manner.
• Review AI generated notes and transcripts in the web UI.
• Use hands-free dictation of speech to text.
• Ask patient-related questions and receive answers from the AI engine.

Supported platforms

• iOS 17 and 18
• iPhone and iPad devices

Supported development environments

• Xcode 16.2
• Swift, Objective-C, and React Native.

Swift and Objective-C

Xcode setup: Open your project in Xcode and add the SDK as a package dependency. Make sure the necessary frameworks and libraries are linked to your project.

React Native

Middleware: Since Dragon Copilot Embedded for Mobile is built with native iOS technology, you need to write a middleware as a bridge between React Native and the SDK.

Dependencies: Add the SDK as a package dependency in your React Native project. Make sure the necessary frameworks and libraries are linked to your project.

Application logs

Dragon Copilot Embedded for Mobile generates application logs, which are stored locally on the device. Additionally, these logs are sent to Azure Application Insights for centralized monitoring and analysis.

These logs don't contain any PHI (Protected Health Information) or PII (Personally Identifiable Information). The logs include Microsoft identifiers to provide context, such as the device ID, partner ID, customer ID, product ID, and correlation ID.

Error handling and codes

Dragon Copilot Embedded for Mobile performs various HTTP requests to retrieve information on the app and the user. It includes a robust error handling mechanism to ensure reliable and consistent behavior during these requests. When making HTTP requests, our system automatically retries the request if it encounters any retryable HTTP status codes. This helps to handle transient errors and improve the reliability of the system.

Retry logic:

• Retryable status codes: If the HTTP request fails with a retryable status code (e.g., 500, 502, 503, 504), the system will retry the request.
• Retry attempts: The request will be retried up to 3 times.
• Delay between retries: There is a consistent delay of 2 seconds between each retry attempt.

Common Issues

Authentication issues: If the partner-issued authentication token doesn't meet Dragon Copilot's requirements, the SDK will fail to authenticate. Make sure the token is valid and meets all necessary criteria.

Missing required fields: If any required fields are missing, the app might not start, or the UI will display a generic error page with an error code.

A valid server environment ID is needed to start, and a correlation ID is required to show session details.

Resiliency

Streaming is the primary method of delivering audio to Dragon Copilot. However, Dragon Copilot Embedded for Mobile also stores the audio locally as a backup in case of network interruptions. This enables users to record continuously even when network interruptions occur; the upload is completed when the connection is restored. Your mobile app doesn't have to implement audio backup functionality because this is fully managed by the SDK.

Dragon Copilot Embedded for Mobile saves backup audio files locally to iOS Core Data and encrypts them using AES 256. The SDK uses encryption keys that are stored in the iOS keychain. These keys are unique per recording and never leave the device, so the audio can only be decrypted on the respective device.

The following diagram illustrates the backup and encryption strategy employed by Dragon Copilot Embedded for Mobile.

Sample code

For Swift sample code, see our sample app.

For instructions and code sample to integrate Dragon Copilot Embedded for Mobile with a React Native app, see: Embed Dragon Copilot in your React Native mobile app.