How to recognize intents with simple language pattern matching

The Azure AI services Speech SDK has a built-in feature to provide intent recognition with simple language pattern matching. An intent is something the user wants to do: close a window, mark a checkbox, insert some text, etc.

In this guide, you use the Speech SDK to develop a C++ console application that derives intents from user utterances through your device's microphone. You learn how to:

• Create a Visual Studio project referencing the Speech SDK NuGet package
• Create a speech configuration and get an intent recognizer
• Add intents and patterns via the Speech SDK API
• Recognize speech from a microphone
• Use asynchronous, event-driven continuous recognition

When to use pattern matching

Use pattern matching if:

• You're only interested in matching strictly what the user said. These patterns match more aggressively than conversational language understanding (CLU).
• You don't have access to a CLU model, but still want intents.

For more information, see the pattern matching overview.

Prerequisites

Be sure you have the following items before you begin this guide:

• An Azure AI services resource or a Unified Speech resource
• Visual Studio 2019 (any edition).

Speech and simple patterns

The simple patterns are a feature of the Speech SDK and need an Azure AI services resource or a Unified Speech resource.

A pattern is a phrase that includes an Entity somewhere within it. An Entity is defined by wrapping a word in curly brackets. This example defines an Entity with the ID "floorName", which is case-sensitive:

    Take me to the {floorName}

All other special characters and punctuation are ignored.

Intents are added using calls to the IntentRecognizer->AddIntent() API.

Create a project

Create a new C# console application project in Visual Studio 2019 and install the Speech SDK.

Start with some boilerplate code

Let's open Program.cs and add some code that works as a skeleton for our project.

using System;
using System.Threading.Tasks;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Intent;

namespace helloworld
{
    class Program
    {
        static void Main(string[] args)
        {
            IntentPatternMatchingWithMicrophoneAsync().Wait();
        }

        private static async Task IntentPatternMatchingWithMicrophoneAsync()
        {
            var config = SpeechConfig.FromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");
        }
    }
}

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your Azure AI services prediction resource.

• Replace "YOUR_SUBSCRIPTION_KEY" with your Azure AI services prediction key.
• Replace "YOUR_SUBSCRIPTION_REGION" with your Azure AI services resource region.

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

Initialize an IntentRecognizer

Now create an IntentRecognizer. Insert this code right below your Speech configuration.

using (var intentRecognizer = new IntentRecognizer(config))
{
    
}

Add some intents

You need to associate some patterns with the IntentRecognizer by calling AddIntent(). We will add 2 intents with the same ID for changing floors, and another intent with a separate ID for opening and closing doors. Insert this code inside the using block:

intentRecognizer.AddIntent("Take me to floor {floorName}.", "ChangeFloors");
intentRecognizer.AddIntent("Go to floor {floorName}.", "ChangeFloors");
intentRecognizer.AddIntent("{action} the door.", "OpenCloseDoor");

Note

There is no limit to the number of entities you can declare, but they will be loosely matched. If you add a phrase like "{action} door" it will match any time there is text before the word "door". Intents are evaluated based on their number of entities. If two patterns would match, the one with more defined entities is returned.

Recognize an intent

From the IntentRecognizer object, you're going to call the RecognizeOnceAsync() method. This method asks the Speech service to recognize speech in a single phrase, and stop recognizing speech once the phrase is identified. For simplicity we'll wait on the future returned to complete.

Insert this code below your intents:

Console.WriteLine("Say something...");

var result = await intentRecognizer.RecognizeOnceAsync();

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, we will print the result.

Insert this code below var result = await recognizer.RecognizeOnceAsync();:

string floorName;
switch (result.Reason)
{
    case ResultReason.RecognizedSpeech:
        Console.WriteLine($"RECOGNIZED: Text= {result.Text}");
        Console.WriteLine($"    Intent not recognized.");
        break;
    case ResultReason.RecognizedIntent:
        Console.WriteLine($"RECOGNIZED: Text= {result.Text}");
        Console.WriteLine($"       Intent Id= {result.IntentId}.");
        var entities = result.Entities;
        if (entities.TryGetValue("floorName", out floorName))
        {
            Console.WriteLine($"       FloorName= {floorName}");
        }
    
        if (entities.TryGetValue("action", out floorName))
        {
            Console.WriteLine($"       Action= {floorName}");
        }
    
        break;
    case ResultReason.NoMatch:
    {
        Console.WriteLine($"NOMATCH: Speech could not be recognized.");
        var noMatch = NoMatchDetails.FromResult(result);
        switch (noMatch.Reason)
        {
            case NoMatchReason.NotRecognized:
                Console.WriteLine($"NOMATCH: Speech was detected, but not recognized.");
                break;
            case NoMatchReason.InitialSilenceTimeout:
                Console.WriteLine($"NOMATCH: The start of the audio stream contains only silence, and the service timed out waiting for speech.");
                break;
            case NoMatchReason.InitialBabbleTimeout:
                Console.WriteLine($"NOMATCH: The start of the audio stream contains only noise, and the service timed out waiting for speech.");
                break;
            case NoMatchReason.KeywordNotRecognized:
                Console.WriteLine($"NOMATCH: Keyword not recognized");
                break;
        }
        break;
    }
    case ResultReason.Canceled:
    {
        var cancellation = CancellationDetails.FromResult(result);
        Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");
    
        if (cancellation.Reason == CancellationReason.Error)
        {
            Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
            Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
            Console.WriteLine($"CANCELED: Did you set the speech resource key and region values?");
        }
        break;
    }
    default:
        break;
}

Check your code

At this point, your code should look like this:

using System;
using System.Threading.Tasks;
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Intent;

namespace helloworld
{
    class Program
    {
        static void Main(string[] args)
        {
            IntentPatternMatchingWithMicrophoneAsync().Wait();
        }

        private static async Task IntentPatternMatchingWithMicrophoneAsync()
        {
            var config = SpeechConfig.FromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");
            using (var intentRecognizer = new IntentRecognizer(config))
            {
                intentRecognizer.AddIntent("Take me to floor {floorName}.", "ChangeFloors");
                intentRecognizer.AddIntent("Go to floor {floorName}.", "ChangeFloors");
                intentRecognizer.AddIntent("{action} the door.", "OpenCloseDoor");

                Console.WriteLine("Say something...");

                var result = await intentRecognizer.RecognizeOnceAsync();

                string floorName;
                switch (result.Reason)
                {
                    case ResultReason.RecognizedSpeech:
                        Console.WriteLine($"RECOGNIZED: Text= {result.Text}");
                        Console.WriteLine($"    Intent not recognized.");
                        break;
                    case ResultReason.RecognizedIntent:
                        Console.WriteLine($"RECOGNIZED: Text= {result.Text}");
                        Console.WriteLine($"       Intent Id= {result.IntentId}.");
                        var entities = result.Entities;
                        if (entities.TryGetValue("floorName", out floorName))
                        {
                            Console.WriteLine($"       FloorName= {floorName}");
                        }

                        if (entities.TryGetValue("action", out floorName))
                        {
                            Console.WriteLine($"       Action= {floorName}");
                        }

                        break;
                    case ResultReason.NoMatch:
                    {
                        Console.WriteLine($"NOMATCH: Speech could not be recognized.");
                        var noMatch = NoMatchDetails.FromResult(result);
                        switch (noMatch.Reason)
                        {
                            case NoMatchReason.NotRecognized:
                                Console.WriteLine($"NOMATCH: Speech was detected, but not recognized.");
                                break;
                            case NoMatchReason.InitialSilenceTimeout:
                                Console.WriteLine($"NOMATCH: The start of the audio stream contains only silence, and the service timed out waiting for speech.");
                                break;
                            case NoMatchReason.InitialBabbleTimeout:
                                Console.WriteLine($"NOMATCH: The start of the audio stream contains only noise, and the service timed out waiting for speech.");
                                break;
                            case NoMatchReason.KeywordNotRecognized:
                                Console.WriteLine($"NOMATCH: Keyword not recognized");
                                break;
                        }
                        break;
                    }
                    case ResultReason.Canceled:
                    {
                        var cancellation = CancellationDetails.FromResult(result);
                        Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");

                        if (cancellation.Reason == CancellationReason.Error)
                        {
                            Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
                            Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
                            Console.WriteLine($"CANCELED: Did you set the speech resource key and region values?");
                        }
                        break;
                    }
                    default:
                        break;
                }
            }
        }
    }
}

Build and run your app

Now you're ready to build your app and test our speech recognition using the Speech service.

1. Compile the code - From the menu bar of Visual Studio, choose Build > Build Solution.
2. Start your app - From the menu bar, choose Debug > Start Debugging or press F5.
3. Start recognition - It will prompt you to say something. The default language is English. Your speech is sent to the Speech service, transcribed as text, and rendered in the console.

For example if you say "Take me to floor 7", this should be the output:

Say something ...
RECOGNIZED: Text= Take me to floor 7.
  Intent Id= ChangeFloors
  FloorName= 7

Create a project

Create a new C++ console application project in Visual Studio 2019 and install the Speech SDK.

Start with some boilerplate code

Let's open helloworld.cpp and add some code that works as a skeleton for our project.

    #include <iostream>
    #include <speechapi_cxx.h>

    using namespace Microsoft::CognitiveServices::Speech;
    using namespace Microsoft::CognitiveServices::Speech::Intent;

    int main()
    {
        std::cout << "Hello World!\n";

        auto config = SpeechConfig::FromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");
    }

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your Azure AI services prediction resource.

• Replace "YOUR_SUBSCRIPTION_KEY" with your Azure AI services prediction key.
• Replace "YOUR_SUBSCRIPTION_REGION" with your Azure AI services resource region.

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

Initialize an IntentRecognizer

Now create an IntentRecognizer. Insert this code right below your Speech configuration.

    auto intentRecognizer = IntentRecognizer::FromConfig(config);

Add some intents

You need to associate some patterns with the IntentRecognizer by calling AddIntent(). We will add 2 intents with the same ID for changing floors, and another intent with a separate ID for opening and closing doors.

    intentRecognizer->AddIntent("Take me to floor {floorName}.", "ChangeFloors");
    intentRecognizer->AddIntent("Go to floor {floorName}.", "ChangeFloors");
    intentRecognizer->AddIntent("{action} the door.", "OpenCloseDoor");

Note

There is no limit to the number of entities you can declare, but they will be loosely matched. If you add a phrase like "{action} door" it will match any time there is text before the word "door". Intents are evaluated based on their number of entities. If two patterns would match, the one with more defined entities is returned.

Recognize an intent

From the IntentRecognizer object, you're going to call the RecognizeOnceAsync() method. This method asks the Speech service to recognize speech in a single phrase, and stop recognizing speech once the phrase is identified. For simplicity we'll wait on the future returned to complete.

Insert this code below your intents:

    std::cout << "Say something ..." << std::endl;
    auto result = intentRecognizer->RecognizeOnceAsync().get();

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, we will print the result.

Insert this code below auto result = intentRecognizer->RecognizeOnceAsync().get();:

switch (result->Reason)
{
case ResultReason::RecognizedSpeech:
        std::cout << "RECOGNIZED: Text = " << result->Text.c_str() << std::endl;
        std::cout << "NO INTENT RECOGNIZED!" << std::endl;
        break;
case ResultReason::RecognizedIntent:
    std::cout << "RECOGNIZED: Text = " << result->Text.c_str() << std::endl;
    std::cout << "  Intent Id = " << result->IntentId.c_str() << std::endl;
    auto entities = result->GetEntities();
    if (entities.find("floorName") != entities.end())
    {
        std::cout << "  Floor name: = " << entities["floorName"].c_str() << std::endl;
    }

    if (entities.find("action") != entities.end())
    {
        std::cout << "  Action: = " << entities["action"].c_str() << std::endl;
    }

    break;
case ResultReason::NoMatch:
{
    auto noMatch = NoMatchDetails::FromResult(result);
    switch (noMatch->Reason)
    {
    case NoMatchReason::NotRecognized:
        std::cout << "NOMATCH: Speech was detected, but not recognized." << std::endl;
        break;
    case NoMatchReason::InitialSilenceTimeout:
        std::cout << "NOMATCH: The start of the audio stream contains only silence, and the service timed out waiting for speech." << std::endl;
        break;
    case NoMatchReason::InitialBabbleTimeout:
        std::cout << "NOMATCH: The start of the audio stream contains only noise, and the service timed out waiting for speech." << std::endl;
        break;
    case NoMatchReason::KeywordNotRecognized:
        std::cout << "NOMATCH: Keyword not recognized" << std::endl;
        break;
    }
    break;
}
case ResultReason::Canceled:
{
    auto cancellation = CancellationDetails::FromResult(result);

    if (!cancellation->ErrorDetails.empty())
    {
        std::cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails.c_str() << std::endl;
        std::cout << "CANCELED: Did you set the speech resource key and region values?" << std::endl;
    }
}
default:
    break;
}

Check your code

At this point, your code should look like this:

#include <iostream>
#include <speechapi_cxx.h>

using namespace Microsoft::CognitiveServices::Speech;
using namespace Microsoft::CognitiveServices::Speech::Intent;

int main()
{
    auto config = SpeechConfig::FromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");
    auto intentRecognizer = IntentRecognizer::FromConfig(config);

    intentRecognizer->AddIntent("Take me to floor {floorName}.", "ChangeFloors");
    intentRecognizer->AddIntent("Go to floor {floorName}.", "ChangeFloors");
    intentRecognizer->AddIntent("{action} the door.", "OpenCloseDoor");

    std::cout << "Say something ..." << std::endl;

    auto result = intentRecognizer->RecognizeOnceAsync().get();

    switch (result->Reason)
    {
    case ResultReason::RecognizedSpeech:
        std::cout << "RECOGNIZED: Text = " << result->Text.c_str() << std::endl;
        std::cout << "NO INTENT RECOGNIZED!" << std::endl;
        break;
    case ResultReason::RecognizedIntent:
        std::cout << "RECOGNIZED: Text = " << result->Text.c_str() << std::endl;
        std::cout << "  Intent Id = " << result->IntentId.c_str() << std::endl;
        auto entities = result->GetEntities();
        if (entities.find("floorName") != entities.end())
        {
            std::cout << "  Floor name: = " << entities["floorName"].c_str() << std::endl;
        }

        if (entities.find("action") != entities.end())
        {
            std::cout << "  Action: = " << entities["action"].c_str() << std::endl;
        }

        break;
    case ResultReason::NoMatch:
    {
        auto noMatch = NoMatchDetails::FromResult(result);
        switch (noMatch->Reason)
        {
        case NoMatchReason::NotRecognized:
            std::cout << "NOMATCH: Speech was detected, but not recognized." << std::endl;
            break;
        case NoMatchReason::InitialSilenceTimeout:
            std::cout << "NOMATCH: The start of the audio stream contains only silence, and the service timed out waiting for speech." << std::endl;
            break;
        case NoMatchReason::InitialBabbleTimeout:
            std::cout << "NOMATCH: The start of the audio stream contains only noise, and the service timed out waiting for speech." << std::endl;
            break;
        case NoMatchReason::KeywordNotRecognized:
            std::cout << "NOMATCH: Keyword not recognized." << std::endl;
            break;
        }
        break;
    }
    case ResultReason::Canceled:
    {
        auto cancellation = CancellationDetails::FromResult(result);

        if (!cancellation->ErrorDetails.empty())
        {
            std::cout << "CANCELED: ErrorDetails=" << cancellation->ErrorDetails.c_str() << std::endl;
            std::cout << "CANCELED: Did you set the speech resource key and region values?" << std::endl;
        }
    }
    default:
        break;
    }
}

Build and run your app

Now you're ready to build your app and test our speech recognition using the Speech service.

1. Compile the code - From the menu bar of Visual Studio, choose Build > Build Solution.
2. Start your app - From the menu bar, choose Debug > Start Debugging or press F5.
3. Start recognition - It will prompt you to say something. The default language is English. Your speech is sent to the Speech service, transcribed as text, and rendered in the console.

For example if you say "Take me to floor 7", this should be the output:

Say something ...
RECOGNIZED: Text = Take me to floor 7.
  Intent Id = ChangeFloors
  Floor name: = 7

Reference documentation | Additional Samples on GitHub

In this quickstart, you install the Speech SDK for Java.

Platform requirements

Choose your target environment:

Java Runtime

The Speech SDK for Java is compatible with Windows, Linux, and macOS.

Windows

On Windows, you must use the 64-bit target architecture. Windows 10 or later is required.

Install the Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017, 2019, and 2022 for your platform. Installing this package for the first time might require a restart.

The Speech SDK for Java doesn't support Windows on ARM64.

Linux

Caution

This article references CentOS, a Linux distribution that is nearing End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the CentOS End Of Life guidance.

The Speech SDK for Java supports the following distributions on the x64, ARM32 (Debian/Ubuntu), and ARM64 (Debian/Ubuntu) architectures:

• Ubuntu 18.04/20.04
• Debian 10/11
• Red Hat Enterprise Linux (RHEL) 7/8
• CentOS 7

Important

Use the most recent LTS release of the Linux distribution. For example, if you are using Ubuntu 20.04 LTS, use the latest release of Ubuntu 20.04.X.

The Speech SDK depends on the following Linux system libraries:

• The shared libraries of the GNU C library, including the POSIX Threads Programming library, libpthreads.
• The OpenSSL library (libssl) version 1.x and certificates (ca-certificates).
• The shared library for ALSA applications (libasound).

You should also install ca-certificates to establish a secure websocket and avoid the WS_OPEN_ERROR_UNDERLYING_IO_OPEN_FAILED error.

Important

The Speech SDK does not yet support OpenSSL 3.0, which is the default in Ubuntu 22.04 and Debian 12.

Ubuntu 18.04/20.04

Run these commands:

sudo apt-get update
sudo apt-get install build-essential libssl-dev ca-certificates libasound2 wget

Debian 10/11

To use the Speech SDK in Alpine Linux, create a Debian chroot environment as documented in the Alpine Linux Wiki on running glibc programs. Then follow the Debian instructions here.

sudo apt-get update
sudo apt-get install build-essential libssl-dev ca-certificates libasound2 wget

RHEL 7/8 and CentOS 7

Caution

This article references CentOS, a Linux distribution that is nearing End Of Life (EOL) status. Please consider your use and planning accordingly. For more information, see the CentOS End Of Life guidance.

Install the development tools and libraries:

sudo yum update
sudo yum groupinstall "Development tools"
sudo yum install alsa-lib openssl wget

Important

• On RHEL/CentOS 7, follow the instructions on how to configure RHEL/CentOS 7 for Speech SDK.
• On RHEL, follow the instructions on how to configure OpenSSL for Linux.

macOS

A macOS version 10.14 or later is required.

Android

The Speech SDK is compatible with Android devices that have 32/64-bit ARM processor and Intel x86/x64 compatible processors.

Install a Java Development Kit such as Azul Zulu OpenJDK. The Microsoft Build of OpenJDK or your preferred JDK should also work.

Install the Speech SDK for Java

Some of the instructions use a specific SDK version such as 1.24.2. To check the latest version, search our GitHub repository.

Choose your target environment:

Java Runtime

This guide shows how to install the Speech SDK for Java on the Java Runtime.

Supported operating systems

The Speech SDK for Java package is available for these operating systems:

• Windows: 64-bit only.
• Mac: macOS X version 10.14 or later.
• Linux: See the supported Linux distributions and target architectures.

Maven

Follow these steps to install the Speech SDK for Java using Apache Maven:

1. Install Apache Maven.

2. Open a command prompt where you want the new project, and create a new pom.xml file.

3. Copy the following XML content into pom.xml:

   <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
       <modelVersion>4.0.0</modelVersion>
       <groupId>com.microsoft.cognitiveservices.speech.samples</groupId>
       <artifactId>quickstart-eclipse</artifactId>
       <version>1.0.0-SNAPSHOT</version>
       <build>
           <sourceDirectory>src</sourceDirectory>
           <plugins>
           <plugin>
               <artifactId>maven-compiler-plugin</artifactId>
               <version>3.7.0</version>
               <configuration>
               <source>1.8</source>
               <target>1.8</target>
               </configuration>
           </plugin>
           </plugins>
       </build>
       <dependencies>
           <dependency>
           <groupId>com.microsoft.cognitiveservices.speech</groupId>
           <artifactId>client-sdk</artifactId>
           <version>1.37.0</version>
           </dependency>
       </dependencies>
   </project>
   

4. Run the following Maven command to install the Speech SDK and dependencies.

   mvn clean dependency:copy-dependencies
   

Eclipse

Create an Eclipse project and install the Speech SDK

1. Install the Eclipse Java IDE. This IDE requires Java to already be installed.

2. Start Eclipse.

3. In Eclipse Launcher, in the Workspace box, enter the name of a new workspace directory. Then select Launch.

   

4. In a moment, the main window of the Eclipse IDE appears. Close the Welcome screen if one is present.

5. From the Eclipse menu, select File > New > Project.

6. The New Project dialog box appears. Select Java Project, and then select Next.

   

7. The New Java Project wizard starts. In the Project name field, enter quickstart. Choose JavaSE-1.8 as the execution environment. Select Finish.

   

8. If the Open Associated Perspective? window appears, select Open Perspective.

9. In Package Explorer, right-click the quickstart project. Select Configure > Convert to Maven Project from the context menu.

   

10. The Create new POM window appears. In the Group Id field, enter com.microsoft.cognitiveservices.speech.samples. In the Artifact Id field, enter quickstart. Then select Finish.

    

11. Open the pom.xml file and edit it:

    1. Add a dependencies element at the end of the file, before the closing tag </project>, with the Speech SDK as a dependency:

    <dependencies>
      <dependency>
        <groupId>com.microsoft.cognitiveservices.speech</groupId>
        <artifactId>client-sdk</artifactId>
        <version>1.37.0</version>
      </dependency>
    </dependencies>
    

    1. Save the changes.

Gradle

Gradle configurations

Gradle configurations require an explicit reference to the .jar dependency extension:

// build.gradle

dependencies {
    implementation group: 'com.microsoft.cognitiveservices.speech', name: 'client-sdk', version: "1.37.0", ext: "jar"
}

Android

This guide shows how to install the Speech SDK for Java on Android.

The Speech SDK for Android is packaged as an Android Archive (AAR) file, which includes the necessary libraries and required Android permissions.

Install the Speech SDK by using Android Studio

Create a new project in Android Studio and add the Speech SDK for Java as a library dependency. The setup is based on the Speech SDK Maven Package and Android Studio Chipmunk 2021.2.1.

Create an empty project

1. Open Android Studio, and select New project.

   

2. In the New project window that appears, select Phone and Tablet > Empty Activity, and then select Next.

   

3. Enter SpeechQuickstart in the Name text box.

   

4. Enter samples.speech.cognitiveservices.microsoft.com in the Package name text box.

5. Select a project directory in the Save location selection box.

6. Select Java in the Language selection box.

7. Select API 23: Android 6.0 (Marshmallow) in the Minimum API level selection box.

8. Select Finish.

Android Studio takes some time to prepare your new project. For your first time using Android Studio, it might take a few minutes to set preferences, accept licenses, and complete the wizard.

Install the Speech SDK for Java on Android

Add the Speech SDK as a dependency in your project.

1. Select File > Project structure > Dependencies > app.

2. Select the plus symbol (+) to add a dependency under Declared Dependencies. Then select Library dependency from the dropdown menu.

   

3. In the Add Library Dependency window that appears, enter the name and version of the Speech SDK for Java: com.microsoft.cognitiveservices.speech:client-sdk:1.37.0. Then select Search.

4. Make sure that the selected Group ID is com.microsoft.cognitiveservices.speech, and then select OK.

5. Select OK to close the Project Structure window and apply your changes to the project.

Start with some boilerplate code

1. Open Main.java from the src dir.

2. Replace the contents of the file with following:

package quickstart;
import java.util.Dictionary;
import java.util.concurrent.ExecutionException;

import com.microsoft.cognitiveservices.speech.*;
import com.microsoft.cognitiveservices.speech.intent.*;

public class Program {
    public static void main(String[] args) throws InterruptedException, ExecutionException {
        IntentPatternMatchingWithMicrophone();
    }

    public static void IntentPatternMatchingWithMicrophone() throws InterruptedException, ExecutionException {
        SpeechConfig config = SpeechConfig.fromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");
    }
}

Create a Speech configuration

Before you can initialize an IntentRecognizer object, you need to create a configuration that uses the key and location for your Azure AI services prediction resource.

• Replace "YOUR_SUBSCRIPTION_KEY" with your Azure AI services prediction key.
• Replace "YOUR_SUBSCRIPTION_REGION" with your Azure AI services resource region.

This sample uses the FromSubscription() method to build the SpeechConfig. For a full list of available methods, see SpeechConfig Class.

Initialize an IntentRecognizer

Now create an IntentRecognizer. Insert this code right below your Speech configuration.

try (IntentRecognizer intentRecognizer = new IntentRecognizer(config)) {
    
}

Add some intents

You need to associate some patterns with the IntentRecognizer by calling addIntent(). We will add 2 intents with the same ID for changing floors, and another intent with a separate ID for opening and closing doors. Insert this code inside the try block:

intentRecognizer.addIntent("Take me to floor {floorName}.", "ChangeFloors");
intentRecognizer.addIntent("Go to floor {floorName}.", "ChangeFloors");
intentRecognizer.addIntent("{action} the door.", "OpenCloseDoor");

Note

There is no limit to the number of entities you can declare, but they will be loosely matched. If you add a phrase like "{action} door" it will match any time there is text before the word "door". Intents are evaluated based on their number of entities. If two patterns would match, the one with more defined entities is returned.

Recognize an intent

From the IntentRecognizer object, you're going to call the recognizeOnceAsync() method. This method asks the Speech service to recognize speech in a single phrase, and stop recognizing speech once the phrase is identified. For simplicity we'll wait on the future returned to complete.

Insert this code below your intents:

System.out.println("Say something...");

IntentRecognitionResult result = intentRecognizer.recognizeOnceAsync().get();

Display the recognition results (or errors)

When the recognition result is returned by the Speech service, we will print the result.

Insert this code below IntentRecognitionResult result = recognizer.recognizeOnceAsync().get();:

if (result.getReason() == ResultReason.RecognizedSpeech) {
    System.out.println("RECOGNIZED: Text= " + result.getText());
    System.out.println(String.format("%17s", "Intent not recognized."));
}
else if (result.getReason() == ResultReason.RecognizedIntent) {
    System.out.println("RECOGNIZED: Text= " + result.getText());
    System.out.println(String.format("%17s %s", "Intent Id=", result.getIntentId() + "."));
    Dictionary<String, String> entities = result.getEntities();

    if (entities.get("floorName") != null) {
        System.out.println(String.format("%17s %s", "FloorName=", entities.get("floorName")));
    }
    if (entities.get("action") != null) {
        System.out.println(String.format("%17s %s", "Action=", entities.get("action")));
    }
}
else if (result.getReason() == ResultReason.NoMatch) {
    System.out.println("NOMATCH: Speech could not be recognized.");
}
else if (result.getReason() == ResultReason.Canceled) {
    CancellationDetails cancellation = CancellationDetails.fromResult(result);
    System.out.println("CANCELED: Reason=" + cancellation.getReason());

    if (cancellation.getReason() == CancellationReason.Error)
    {
        System.out.println("CANCELED: ErrorCode=" + cancellation.getErrorCode());
        System.out.println("CANCELED: ErrorDetails=" + cancellation.getErrorDetails());
        System.out.println("CANCELED: Did you update the subscription info?");
    }
}

Check your code

At this point, your code should look like this:

package quickstart;
import java.util.Dictionary;
import java.util.concurrent.ExecutionException;

import com.microsoft.cognitiveservices.speech.*;
import com.microsoft.cognitiveservices.speech.intent.*;

public class Main {
    public static void main(String[] args) throws InterruptedException, ExecutionException {
        IntentPatternMatchingWithMicrophone();
    }

    public static void IntentPatternMatchingWithMicrophone() throws InterruptedException, ExecutionException {
        SpeechConfig config = SpeechConfig.fromSubscription("YOUR_SUBSCRIPTION_KEY", "YOUR_SUBSCRIPTION_REGION");

        try (IntentRecognizer intentRecognizer = new IntentRecognizer(config)) {
            intentRecognizer.addIntent("Take me to floor {floorName}.", "ChangeFloors");
            intentRecognizer.addIntent("Go to floor {floorName}.", "ChangeFloors");
            intentRecognizer.addIntent("{action} the door.", "OpenCloseDoor");

            System.out.println("Say something...");

            IntentRecognitionResult result = intentRecognizer.recognizeOnceAsync().get();
            if (result.getReason() == ResultReason.RecognizedSpeech) {
            System.out.println("RECOGNIZED: Text= " + result.getText());
            System.out.println(String.format("%17s", "Intent not recognized."));
            }
            else if (result.getReason() == ResultReason.RecognizedIntent) {
                System.out.println("RECOGNIZED: Text= " + result.getText());
                System.out.println(String.format("%17s %s", "Intent Id=", result.getIntentId() + "."));
                Dictionary<String, String> entities = result.getEntities();

                if (entities.get("floorName") != null) {
                    System.out.println(String.format("%17s %s", "FloorName=", entities.get("floorName")));
                }
                if (entities.get("action") != null) {
                    System.out.println(String.format("%17s %s", "Action=", entities.get("action")));
                }
            }
            else if (result.getReason() == ResultReason.NoMatch) {
                System.out.println("NOMATCH: Speech could not be recognized.");
            }
            else if (result.getReason() == ResultReason.Canceled) {
                CancellationDetails cancellation = CancellationDetails.fromResult(result);
                System.out.println("CANCELED: Reason=" + cancellation.getReason());

                if (cancellation.getReason() == CancellationReason.Error)
                {
                    System.out.println("CANCELED: ErrorCode=" + cancellation.getErrorCode());
                    System.out.println("CANCELED: ErrorDetails=" + cancellation.getErrorDetails());
                    System.out.println("CANCELED: Did you update the subscription info?");
                }
            }
        }
    }
}

Build and run your app

Now you're ready to build your app and test our intent recognition using the speech service and the embedded pattern matcher.

Select the run button in Eclipse or press ctrl+F11, then watch the output for the "Say something..." prompt. Once it appears speak your utterance and watch the output.

For example if you say "Take me to floor 7", this should be the output:

Say something ...
RECOGNIZED: Text= Take me to floor 7.
  Intent Id= ChangeFloors
  FloorName= 7

Next steps

Improve your pattern matching by using custom entities.