Gather user input with Recognize action

Article
11/10/2022
8 minutes to read

Important

Functionality described in this document is currently in public preview. This preview version is provided without a service-level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.

This guide will help you get started with recognizing DTMF input provided by participants through Azure Communication Services Call Automation SDK.

Prerequisites

Azure account with an active subscription, for details see Create an account for free.
Azure Communication Services resource. See Create an Azure Communication Services resource. Note the connection string for this resource.
Create a new web service application using the Call Automation SDK.
The latest .NET library for your operating system.
Obtain the NuGet package from the Azure SDK Dev Feed

Technical specifications

The following parameters are available to customize the Recognize function:

Parameter	Type	Default (if not specified)	Description	Required or Optional
Prompt (for details on Play action, refer to this how-to guide)	FileSource	Not set	This will be the message you wish to play before recognizing input.	Optional
InterToneTimeout	TimeSpan	2 seconds Min: 1 second Max: 60 seconds	Limit in seconds that ACS will wait for the caller to press another digit (inter-digit timeout).	Optional
InitialSilenceTimeout	TimeSpan	5 seconds Min: 0 seconds Max: 300 seconds	How long recognize action will wait for input before considering it a timeout.	Optional
MaxTonesToCollect	Integer	No default Min: 1	Number of digits a developer expects as input from the participant.	Required
StopTones	IEnumeration<DtmfTone>	Not set	The digit participants can press to escape out of a batch DTMF event.	Optional
InterruptPrompt	Bool	True	If the participant has the ability to interrupt the playMessage by pressing a digit.	Optional
InterruptCallMediaOperation	Bool	True	If this flag is set it will interrupt the current call media operation. For example if any audio is being played it will interrupt that operation and initiate recognize.	Optional
OperationContext	String	Not set	String that developers can pass mid action, useful for allowing developers to store context about the events they receive.	Optional

Create a new C# application

In the console window of your operating system, use the dotnet command to create a new web application.

dotnet new web -n MyApplication

Install the NuGet package

During the preview phase, the NuGet package can be obtained by configuring your package manager to use the Azure SDK Dev Feed from here

Establish a call

By this point you should be familiar with starting calls, if you need to learn more about how to start a call view our quickstart. In this instance, we'll answer an incoming call.

Call the recognize action

When your application answers the call, you can provide information about recognizing participant input and playing a prompt.

var targetParticipant = new PhoneNumberIdentifier("+1XXXXXXXXXXX");
                var recognizeOptions = new CallMediaRecognizeDtmfOptions(targetParticipant, maxTonesToCollect)
                {
                    InterruptCallMediaOperation = true,
                    InitialSilenceTimeout = TimeSpan.FromSeconds(30),
                    Prompt = new FileSource(new System.Uri("file://path/to/file")),
                    InterToneTimeout = TimeSpan.FromSeconds(5),
                    InterruptPrompt = true,
                    StopTones = new DtmfTone[] { DtmfTone.Pound },
                };
                await _callConnection.GetCallMedia().StartRecognizingAsync(recognizeOptions).ConfigureAwait(false);

Note: If parameters aren't set, the defaults will be applied where possible.

Receiving recognize event updates

Developers can subscribe to the RecognizeCompleted and RecognizeFailed events on the webhook callback they registered for the call to create business logic in their application for determining next steps when one of the previously mentioned events occurs.

Example of RecognizeCompleted event:

[
    {
        "id": "e9cf1c71-f119-48db-86ca-4f2530a2004d",
        "source": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
        "type": "Microsoft.Communication.RecognizeCompleted",
        "data": {
            "eventSource": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
            "operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
            "resultInformation": {
                "code": 200,
                "subCode": 8531,
                "message": "Action completed, max digits received."
            },
            "recognitionType": "dtmf",
            "collectTonesResult": {
                "tones": [
                    "nine",
                    "eight",
                    "zero",
                    "five",
                    "two"
                ]
            },
            "callConnectionId": "411f0b00-d97f-49ad-a6ff-3f8c05dc64d7",
            "serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
            "correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
        },
        "time": "2022-09-13T00:55:08.2240104+00:00",
        "specversion": "1.0",
        "datacontenttype": "application/json",
        "subject": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted"
    }
]

Example of how you can deserialize the RecognizeCompleted event:

app.MapPost("<WEB_HOOK_ENDPOINT>", async (
    [FromBody] CloudEvent[] cloudEvents,
    [FromRoute] string contextId) =>{
    foreach (var cloudEvent in cloudEvents)
    {
        CallAutomationEventBase @event = CallAutomationEventParser.Parse(cloudEvent);
        If (@event is RecognizeCompleted recognizeCompleted)
        {
            // Access to the Collected Tones
            foreach(DtmfTone tone in recognizeCompleted.CollectTonesResult.Tones) {
                   // work on each of the Dtmf tones.
        }
    }

Example of RecognizeFailed event:

[
    {
        "id": "47d9cb04-7039-427b-af50-aebdd94db054",
        "source": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
        "type": "Microsoft.Communication.RecognizeFailed",
        "data": {
            "eventSource": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
            "operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
            "resultInformation": {
                "code": 500,
                "subCode": 8511,
                "message": "Action failed, encountered failure while trying to play the prompt."
            },
            "callConnectionId": "411f0b00-bb72-4d5b-9524-ae1c29713335",
            "serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
            "correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
        },
        "time": "2022-09-13T00:55:37.0660233+00:00",
        "specversion": "1.0",
        "datacontenttype": "application/json",
        "subject": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed"
    }
]

Example of how you can deserialize the RecognizeFailed event:

app.MapPost("<WEB_HOOK_ENDPOINT>", async (
    [FromBody] CloudEvent[] cloudEvents,
    [FromRoute] string contextId) =>{
    foreach (var cloudEvent in cloudEvents)
    {
        CallAutomationEventBase @event = CallAutomationEventParser.Parse(cloudEvent);
        If (@event is RecognizeFailed recognizeFailed)
        {
            Log.error($”Recognize failed due to: {recognizeFailed.ResultInformation.Message}”);

        }
    }

Prerequisites

Azure account with an active subscription, for details see Create an account for free.
Azure Communication Services resource. See Create an Azure Communication Services resource
Create a new web service application using the Call Automation SDK.
Java Development Kit version 8 or above.
Apache Maven.

Technical specifications

The following parameters are available to customize the Recognize function:

Parameter	Type	Default (if not specified)	Description	Required or Optional
Prompt (for details on Play action, refer to this quickstart)	FileSource	Not set	This will be the message you wish to play before recognizing input.	Optional
InterToneTimeout	TimeSpan	2 seconds Min: 1 second Max: 60 seconds	Limit in seconds that ACS will wait for the caller to press another digit (inter-digit timeout).	Optional
InitialSilenceTimeout	TimeSpan	5seconds Min: 0 seconds Max: 300 seconds	How long recognize action will wait for input before considering it a timeout.	Optional
MaxTonesToCollect	Integer	No default Min: 1	Number of digits a developer expects as input from the participant.	Required
StopTones	IEnumeration<DtmfTone>	Not set	The digit participants can press to escape out of a batch DTMF event.	Optional
InterruptPrompt	Bool	True	If the participant has the ability to interrupt the playMessage by pressing a digit.	Optional
InterruptCallMediaOperation	Bool	True	If this flag is set it will interrupt the current call media operation if any (for example if play is going on) and initiate recognize.	Optional
OperationContext	String	Not set	String that developers can pass mid action, useful for allowing developers to store context about the events they receive.	Optional

Create a new Java application

In your terminal or command window, navigate to the directory where you would like to create your Java application. Run the command below to generate the Java project from the maven-archetype-quickstart template.

mvn archetype:generate -DgroupId=com.communication.quickstart -DartifactId=communication-quickstart -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false

The command above creates a directory with the same name as artifactId argument. Under this directory, src/main/java directory contains the project source code, src/test/java directory contains the test source.

You'll notice that the 'generate' step created a directory with the same name as the artifactId. Under this directory, src/main/java directory contains source code, src/test/java directory contains tests, and pom.xml file is the project's Project Object Model, or POM.

Update your applications POM file to use Java 8 or higher.

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
</properties>

Configure azure SDK dev feed

Since the Call Automation SDK version used in this quickstart isn't yet available in Maven Central Repository, we need to add an Azure Artifacts development feed, which contains the latest version of Call Automation SDK.

Add the azure-sdk-for-java feed to your pom.xml. Follow the instructions after clicking the "Connect to Feed" button.

Add package references

In your POM file, add the following reference for the project

azure-communication-callautomation

<dependency>
  <groupId>com.azure</groupId>
  <artifactId>azure-communication-callautomation</artifactId>
  <version>1.0.0-alpha.20220915.1</version>
</dependency>

Establish a call

By this point you should be familiar with starting calls, if you need to learn more about how to start a call view our quickstart. In this instance we'll answer an incoming call.

Call the recognize action

When your application answers the call, you can provide information about recognizing participant input and playing a prompt.

CallMediaRecognizeOptions callMediaRecognizeOptions = new CallMediaRecognizeDtmfOptions(targetParticipant, maxTonesToCollect)
        .setInterToneTimeout(Duration.ofSeconds(5))
        .setInterruptCallMediaOperation(true)
        .setInitialSilenceTimeout(Duration.ofSeconds(30))
        .setPlayPrompt(new FileSource().setUri("file://path/to/file"))
        .setInterruptPrompt(true);
callMedia.startRecognizing(callMediaRecognizeOptions).block();

Note: If parameters aren't set, the defaults will be applied where possible.

Receiving recognize event updates

Developers can subscribe to RecognizeCompleted and RecognizeFailed events on the webhook callback they registered for the call to create business logic in their application for determining next steps when one of the aforementioned events occurs.

Example of RecognizeCompleted event:

[
    {
        "id": "e9cf1c71-f119-48db-86ca-4f2530a2004d",
        "source": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
        "type": "Microsoft.Communication.RecognizeCompleted",
        "data": {
            "eventSource": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted",
            "operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
            "resultInformation": {
                "code": 200,
                "subCode": 8531,
                "message": "Action completed, max digits received."
            },
            "recognitionType": "dtmf",
            "collectTonesResult": {
                "tones": [
                    "nine",
                    "eight",
                    "zero",
                    "five",
                    "two"
                ]
            },
            "callConnectionId": "411f0b00-d97f-49ad-a6ff-3f8c05dc64d7",
            "serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
            "correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
        },
        "time": "2022-09-13T00:55:08.2240104+00:00",
        "specversion": "1.0",
        "datacontenttype": "application/json",
        "subject": "calling/callConnections/411f0b00-d97f-49ad-a6ff-3f8c05dc64d7/RecognizeCompleted"
    }
]

Example of RecognizeFailed event:

[
    {
        "id": "47d9cb04-7039-427b-af50-aebdd94db054",
        "source": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
        "type": "Microsoft.Communication.RecognizeFailed",
        "data": {
            "eventSource": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed",
            "operationContext": "267e33a9-c28e-4ecf-a33e-b3abd9526e32",
            "resultInformation": {
                "code": 500,
                "subCode": 8511,
                "message": "Action failed, encountered failure while trying to play the prompt."
            },
            "callConnectionId": "411f0b00-bb72-4d5b-9524-ae1c29713335",
            "serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEyLTAxLmNvbnYuc2t5cGUuY29tL2NvbnYvQzNuT3lkY3E0VTZCV0gtcG1GNmc1Zz9pPTQmZT02Mzc5ODYwMDMzNDQ2MTA5MzM=",
            "correlationId": "53be6977-d832-4c42-8527-fb2aa4a78b74"
        },
        "time": "2022-09-13T00:55:37.0660233+00:00",
        "specversion": "1.0",
        "datacontenttype": "application/json",
        "subject": "calling/callConnections/411f0b00-bb72-4d5b-9524-ae1c29713335/RecognizeFailed"
    }
]

Example of how you can deserialize the RecognizeCompleted and RecognizeFailed event:

post("/api/callback", (request, response) -> {
    
   List<CallAutomationEventBase> acsEvents = EventHandler.parseEventList(request.body());

    for (CallAutomationEventBase acsEvent : acsEvents) {
        
       if (acsEvent.getClass() == RecognizeCompleted.class) {

            RecognizeCompleted recognizeCompleted = (RecognizeCompleted) acsEvent;

            for (Tone tone : recognizeCompleted.getCollectTonesResult().getTones()) {
                // work on each of the Dtmf tones
            }

        } else if (acsEvent.getClass() == RecognizeFailed.class) {
            
           RecognizeFailed recognizeFailed = (RecognizeFailed) acsEvent;
            
           Logger.getAnonymousLogger().info("Recognize failed due to: " + recognizeFailed.getResultInformation().getMessage());
            
       }

    }

});

Event codes

Status	Code	Subcode	Message
RecognizeCompleted	200	8531	Action completed, max digits received.
RecognizeCompleted	200	8514	Action completed as stop tone was detected.
RecognizeCompleted	400	8508	Action failed, the operation was canceled.
RecognizeFailed	400	8510	Action failed, initial silence timeout reached
RecognizeFailed	400	8532	Action failed, inter-digit silence timeout reached.
RecognizeFailed	500	8511	Action failed, encountered failure while trying to play the prompt.
RecognizeFailed	500	8512	Unknown internal server error.

Clean up resources

If you want to clean up and remove a Communication Services subscription, you can delete the resource or resource group. Deleting the resource group also deletes any other resources associated with it. Learn more about cleaning up resources.

Next Steps

Learn more about Gathering user input
Learn more about Playing audio in call
Learn more about Call Automation

Gather user input with Recognize action

Prerequisites

Technical specifications

Create a new C# application

Install the NuGet package

Establish a call

Call the recognize action

Receiving recognize event updates

Prerequisites

Technical specifications

Create a new Java application

Configure azure SDK dev feed

Add package references

Establish a call

Call the recognize action

Receiving recognize event updates

Event codes

Clean up resources

Next Steps

Feedback

Additional resources

Additional resources