Edit

QuickStart: Add closed captions to your calling app

  • Article

Important

Functionality described on this section is currently in private preview. Private preview includes access to SDKs and documentation for testing purposes that are not yet available publicly. Apply to become an early adopter by filling out the form for preview access to Azure Communication Services.

Prerequisites

Refer to the Voice Calling Quickstart to set up a sample app with voice calling.

Models

Name Description
CaptionsCallFeature API for call captions.
StartCaptionsOptions Used for representing options to start closed captions
CaptionsHandler Callback definition for handling the CaptionsReceivedEventType event.
CaptionsInfo Data structure received for each CaptionsReceivedEventType event.

Methods

Start captions

  1. Get the ongoing call object established during the prerequisite steps.
  2. Get the captions feature object.
  3. Set the captionsReceived event handler via the on API.
  4. Call startCaptions on the feature object with the desired options.
const captionsHandler = (data: CaptionsInfo) => { /* USER CODE HERE - E.G. RENDER TO DOM */ };

try {
    const callCaptionsApi = call.feature(Features.Captions);
    callCaptionsApi.on('captionsReceived', captionsHandler);
    if (!callCaptionsApi.isCaptionsActive) {
        await callCaptionsApi.startCaptions({ spokenLanguage: 'en-us' });
    }
} catch (e) {
    console.log('Internal error occurred when Starting Captions');
}

Stopping captions

  1. Get the captions feature object.
  2. Call off with the previous specified handler.

Note

Captions will still be processed, but this client will stop handling them.

const callCaptionsApi = call.feature(Features.Captions);
callCaptionsApi.off('captionsReceived', captionsHandler);

Get available languages

Access the availableLanguages property on the call.feature(Features.Captions) API.

const callCaptionsApi = call.feature(Features.Captions);
const availableLanguages = callCaptionsApi.availableLanguages;

Update language

Pass a value in from the available languages array to ensure that the requested language is supported.

await callCaptionsApi.selectLanguage(availableLanguages[0]);

Clean up

If you want to clean up and remove a Communication Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it. Learn more about cleaning up resources here.

Prerequisites

Refer to the Voice Calling Quickstart to set up a sample app with voice calling.

Models

Name Description
CaptionsCallFeature Used to start and manage closed captions.
StartCaptionsOptions Used for representing options to start closed captions.
CaptionsListener Used to handle captions events.
CaptionsInfo Used to representing captions data.

Methods

Start captions

  1. Get the ongoing call object established during the prerequisite steps.
  2. Get the Captions Feature object
CaptionsCallFeature captionsCallFeature = call.api(Features.CAPTIONS);
  1. Define the CaptionsListener
CaptionsListener captionsListener = new CaptionsListener() {
    @Override
    public void onCaptions(CaptionsInfo captionsInfo) {
        String captionsText = captionsInfo.getText(); // get transcribed text
        String speakerName = captionsInfo.getSpeaker().getDisplayName; // get display name of current speaker
        Date timeStamp = captionsInfo.getTimestamp(); // get timestamp corresponding to caption

        // display captionsText and information as needed...
    }
};
  1. Register the captionsListener
  2. Invoke startCaptions with the desired options.
public void startCallCaptions() {
    captionsCallFeature.addOnCaptionsReceivedListener(captionsListener);
    if (!captionsCallFeature.isCaptionsActive) {
        StartCaptionsOptions startCaptionsOptions = new StartCaptionsOptions();
        startCaptionsOptions.setLanguage("en-us");

        try {
            captionsCallFeature.startCaptions(startCaptionsOptions);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Stopping captions

Remove the previously registered captionsListener.

public void stopCaptions() {
    captionsCallFeature.removeOnCaptionsReceivedListener(captionsListener);
}

Get available languages

Call the getAvailableLanguages method on the CaptionsCallFeature API.

String[] captionsAvailableLanguages = captionsCallFeature.getAvailableLanguages();

Update language

Pass a value in from the available languages array to ensure that the requested language is supported.

public void switchCaptionsLanguage() {
    if (!captionsCallFeature.isCaptionsActive) {
        return;
    }
    try {
        captionsCallFeature.select(captionsAvailableLanguages[0]);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

Clean up

Learn more about cleaning up resources here.

Prerequisites

Refer to the Voice Calling Quickstart to set up a sample app with voice calling.

Models

Name Description
CaptionsCallFeature Used to start and manage closed captions.
StartCaptionsOptions Used for representing options to start closed captions.
CaptionsCallFeatureDelegate Used to handle captions events.
CaptionsInfo Used to representing captions data.

Methods

Start captions

  1. Get the ongoing call object established during the prerequisite steps.
  2. Get the Captions Feature object
var captionsFeature = self.call!.feature(Features.captions)
  1. Define the CaptionsCallFeatureDelegate
@State var callObserver:CallObserver?
extension CallObserver: CaptionsCallFeatureDelegate {
    init(view:<nameOfView>) {
        owner = view
        super.init()
    }
    public func captionsCallFeature(_ captionsFeature: CaptionsCallFeature, didChangeCaptionsState args: PropertyChangedEventArgs) {
        os_log("Call captions state changed to %d", log:log, captionsFeature.isCaptionsActive)
    }
    
    public func captionsCallFeature(_ captionsFeature: CaptionsCallFeature, didReceiveCaptions: CaptionsInfo) {
        let formatter = DateFormatter()
        formatter.dateStyle = .short
        formatter.timeStyle = .short
        let displayedTime = formatter.string(from: didReceiveCaptions.timestamp)
        let captionsText = "\(displayedTime) \(didReceiveCaptions.speaker.displayName): \(didReceiveCaptions.text)"
        // next display captionsText
    }
}
self.callObserver = CallObserver(view:self)
  1. Register the captions feature delegate.
  2. Invoke startCaptions with the desired options.
func startCaptions() {
    captionsFeature!.delegate = self.callObserver
    if(!captionsFeature!.isCaptionsActive) {
        let startCaptionsOptions = StartCaptionsOptions()
        startCaptionsOptions.language = "en-us"
        captionsFeature!.startCaptions(startCaptionsOptions: startCaptionsOptions, completionHandler: { (error) in
            if (error != nil) {
                print ("Call captions Failed to start caption %@", error! as Error)
            }
        })
    }
}

Stopping captions

Remove the previously registered delegate.

func stopCaptions() {
    captionsFeature?.delegate = nil
}

Get available languages

Access the availableLanguages property on the CaptionsCallFeature API.

var captionsAvailableLanguages = captionsFeature!.availableLanguages

Update language

Pass a value in from the available languages array to ensure that the requested language is supported.

func switchCaptionsLanguage() {
    if (!captionsFeature!.isCaptionsActive) {
        return
    }
    captionsFeature!.select(language: captionsAvailableLanguages[0], completionHandler: { (error) in
        if (error != nil) {
            print ("Call captions Failed to switch language %@", error! as Error)
        }
    })
}

Clean up

Learn more about cleaning up resources here.

Clean up resources

If you want to clean up and remove a Communication Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it. Learn more about cleaning up resources.

Next steps

For more information, see the following articles: