Voice commands for Windows Phone 8

Article
05/20/2016

[ This article is for Windows Phone 8 developers. If you’re developing for Windows 10, see the latest documentation. ]

Users can use voice commands to launch your app and to execute an action. For example, a user using the Contoso Widgets app could tap the Start button and say "Contoso Widgets, show best sellers" to both launch the Contoso Widgets app and to navigate to a “best sellers” page, or some other action that the developer specifies. You can add voice command functionality to your app by completing these three steps:

Create a Voice Command Definition (VCD) file. This is an XML document that defines all the spoken commands that users can say to initiate actions when launching your app.
Add code to initialize the VCD file with the phone's speech feature.
Add code to handle navigation and to execute commands.

Note

To use voice commands, you must set the ID_CAP_SPEECH_RECOGNITION, ID_CAP_MICROPHONE, and ID_CAP_NETWORKING capabilities in the app manifest file. If you don’t set these capabilities, your app might not work correctly. For more info, see App capabilities and hardware requirements for Windows Phone 8.

This topic contains the following sections.

Creating a VCD file

The following sections describe how to add a VCD file to your project, and how to create the contents of the file.

Adding a VCD file to your project

The following steps describe how to add a VCD file to your project.

In Visual Studio, right-click the project name, select Add->New Item, and then select Voice Command Definition.
Type a name for the VCD file, and then select Add.
In Solution Explorer, select the VCD file.
In the Properties window, set Build action to Content, and then set Copy to output directory to Copy if newer.

Creating the contents of the VCD file

Each voice command contains:

An example phrase of how a user should typically invoke the command.
The words or phrases that your app will recognize to initiate the command.
The text that your app will display and speak to the user when the command is recognized.
The screen that your app will navigate to when the command is recognized.

The following code example shows a sample VCD file:

<?xml version="1.0" encoding="utf-8"?>
<VoiceCommands xmlns="https://schemas.microsoft.com/voicecommands/1.0">
  <CommandSet xml:lang="en-us">
    <CommandPrefix> Contoso Widgets </CommandPrefix>
    <Example> Show today's specials </Example>
    <Command Name="showWidgets">
      <Example> Show today's specials </Example>
      <ListenFor> [Show] {widgetViews} </ListenFor>
      <Feedback> Showing {widgetViews} </Feedback>
      <Navigate Target="/favorites.xaml"/>
    </Command>

    <PhraseList Label="widgetViews">
      <Item> today's specials </Item>
      <Item> best sellers </Item>
    </PhraseList>
  
  </CommandSet>
  
  <!-- Other CommandSets for other languages -->

</VoiceCommands>

Choosing a command prefix

Setting a value for the CommandPrefix element is optional, and is useful to set in two main scenarios:

Your app name is not pronounceable. For example, if your app's name is Contoso Widg3ts, you could set the CommandPrefix element to "Contoso Widgets".
You're localizing your voice commands. If you choose to have a command set for each supported language, you can set the CommandPrefix element differently according to each language. For example, if your app is named "Contoso Table", you could set the CommandPrefix element to "Contoso Mesa" for any Spanish language command set.

Warning

It is not possible to subset match text in the CommandPrefix element. For example, if your CommandPrefix element is set to "Contoso Weather", "Contoso Weather [phrase]" would be recognized, but "Contoso [phrase]" or "Weather [phrase]" would not be recognized.

For more info about the CommandPrefix element, see Voice command element and attribute reference for Windows Phone 8.

Specifying words or phrases in the VCD file

Each Command element must contain at least one ListenFor element. Each ListenFor element contains the word or words that will initiate the action specified by the Command element.

ListenFor elements cannot be programmatically modified. However, PhraseList elements that are associated with ListenFor elements can be programmatically modified. For example, let's say you have a movie viewer app and want to allow users to launch a movie simply by saying the app name followed by "Play MovieName". It would be an overwhelming task to have to create a separate ListenFor element for each possible movie. Instead, you could dynamically populate the PhraseList at run time with all possible movie options. In the ListenFor element itself, you could say something like: <ListenFor>Play {movies}</ListenFor>, where “movies” is the label name for the PhraseList. You would need to create a blank, labeled PhraseList element at minimum to be able to programmatically modify it at run time. See the Programmatically modifying phrase lists section later on for more info about how to do this. Words or phrases in PhraseList elements can be referenced from any Command element in the containing CommandSet.

Initializing the VCD file

You use a single method call to initialize the VCD file during your app's first run. Initializing the VCD file registers the commands to listen for with the speech system on the user's phone.

The following code example shows how you can initialize a VCD file:

await VoiceCommandService.InstallCommandSetsFromFileAsync(
    new Uri("ms-appx:///ContosoWidgets.xml"));

Extracting URI parameters and executing voice commands

At this point, your app can load a VCD file that defines the voice commands your app will listen for. Now you need to tell your app how to respond to specific voice commands. Depending on the purpose of your app, this could mean that you display a map, read back a reservation confirmation, display flight status, or navigate to another screen.

First, you need to figure out if your app was launched by a voice command, and then determine what the name and parameters of the voice command are. You accomplish this by looking at the QueryString property of the NavigationContext class. When you've determined what voice command was used, you can take appropriate action in your app.

Going back to the sample in the Creating a VCD File section, this means that if a user speaks "Contoso Widgets, show best sellers", the NavigationContext.QueryString dictionary will contain entries for the keys showWidgets, widgetViews, and reco.

Note

A ListenFor element can include an asterisk character inside a pair of curly braces to implement wildcard functionality. For more information, see Voice command element and attribute reference for Windows Phone 8. If the voice command for a wildcard-enabled ListenFor element is matched, the reco key will contain the string "…".

The following example shows what the launch URI looks like when your VCD file navigates to the favorites.xaml page using the showWidgets voice command.

"/favorites.xaml?voiceCommandName=showWidgets&widgetViews=best%20sellers&reco=Contoso%20Widgets%Show%20best%20sellers"

Important Note:
The VCD file allows you to map different voice commands to different screens; you don't need to use a URI mapper for voice commands. But if you use a URI mapper for other features, such as Search extensibility, be sure to pass-through voice command launches and preserve the full URI scheme so that the VCD information is not lost.

When your screen launches, it can use the QueryString property of the NavigationContext class to extract the URI parameter values and get info about the voice command. The following code example shows how to use QueryString to get info about the voice command, and how to initiate an appropriate action afterwards:

protected override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
  base.OnNavigatedTo(e);

  // Is this a new activation or a resurrection from tombstone?
  if (e.NavigationMode == System.Windows.Navigation.NavigationMode.New)
  {

    // Was the app launched using a voice command?
    if (NavigationContext.QueryString.ContainsKey("voiceCommandName"))
    {

      // If so, get the name of the voice command.
      string voiceCommandName 
        = NavigationContext.QueryString["voiceCommandName"];

      // Define app actions for each voice command name.
      switch (voiceCommandName)
      {
        case "showWidgets":
          string viewWidget = NavigationContext.QueryString["widgetViews"];
                   
          // Add code to display specified widgets.
          break;
 
        // Add cases for other voice commands. 
        default:

          // There is no match for the voice command name.
          break;
      }
    }
  }
}

Localizing voice commands

A VCD file allows you to specify multiple language versions for the commands used to launch your app and execute a command. You can create multiple CommandSets, each with a different xml:lang attribute to allow your app to be used in different markets. For example, an app for the United States might have a CommandSet for English and a CommandSet for Spanish.

Warning

To launch an app and initiate an action using a voice command, the app must register a VCD file that contains a CommandSet with a language that matches the speech language that the user selected on the phone. This language is set by the user on the phone's Settings > System > Speech > Speech Language screen.

Programmatically modifying phrase lists

The API for Voice Commands includes two methods that allow you to dynamically update phrase lists in your VCD file. Updating the PhraseList programmatically is useful for cases in which the voice command is specific to a task involving updated data such as a user's favorites list or updated app data. To update a phrase list in the VCD file, you first get the CommandSet that contains the phrase list you want to update. You then store the retrieved CommandSet in a VoiceCommandSet object. The call specifies the CommandSet by the value of its Name attribute, which must be unique in the VCD file.

In the following code example, the command set is retrieved by using the string "en-US", which must be set as the CommandSet element's Name attribute.

VoiceCommandSet widgetVcs =
    VoiceCommandService.InstalledCommandSets["en-US"];

Now you can update the items in the phrase list by specifying its label attribute and an array of strings, which will comprise the new content of the phrase list.

await widgetVcs.UpdatePhraseListAsync("saved", 
    new string[] {"today’s special", "best sellers", "hammer", "plane", "scroll saw"});

Note that this call replaces the entire contents of the phrase list. If you want to add only new items to existing items in the phrase list, you need to specify both the existing items and the new items in this call. In the preceding example, the phrase list already contained the values "today’s special" and "best sellers", but they must be specified in the call or they will be overwritten.

Handling phone backups

If a phone backup occurs and your app reinstalls automatically, any voice command data is not preserved. To avoid this scenario and to make sure your app's voice command data stays intact, consider initializing your VCD file each time your app launches, or store a setting that indicates if the VCD is currently installed and check the setting each time your app launches.

GUI screens for voice commands

The voice commands feature includes GUI screens that help users discover which apps on the phone support voice commands, and which voice commands can be used to launch an app.

A user selects an app from the phone's App list.
A user speaks a voice command, from which the app, but not the requested action, is recognized. The disambiguation screen that appears contains a question mark button that links to the Did you know? screen.

The What can I say screen

The What can I say screen lists names of apps that support voice commands and an example of a command that can be used to launch each app and initiate an action. The sample command is the contents of an example element that is the child of the CommandSet element in the VCD file. This screen appears when a user presses the Start button and speaks "what can I say?", or presses the question mark button on the global Listening screen.

The Did you know? screen

The Did you know? screen lists available voice commands that can be used to launch an app. The screen is accessed when a user taps the Did you know? link on the What can I say screen, and includes the app name (or the value of the CommandPrefix element, if the developer specified it in the VCD file), and the voice command examples.

A link to the Did you know? screen also appears if a user speaks an app name (or CommandPrefix element value), followed by an unrecognized voice command.

Voice command errors

You might encounter voice command errors when working with the feature. For more info about these errors, see Handling errors in speech apps for Windows Phone 8.

Share via