Presenting prompts, confirmations, and disambiguation choices for Windows Phone 8

Article
05/20/2016

[ This article is for Windows Phone 8 developers. If you’re developing for Windows 10, see the latest documentation. ]

Helping users know what to say to your app, and know what was recognized, can improve their recognition experience. Windows Phone 8 provides built-in GUI screens for speech recognition, which you can use to help users provide input that your app expects, and to confirm a user's input.

Speech recognition GUI flow

When your app attempts speech recognition using a SpeechRecognizerUI object, several GUI screens display in the following order.

If the speech recognizer uses a predefined dictation or web search grammar:

The Listening screen.
The Thinking screen (undocumented in this topic).
The Heard you say or error screen.

If the speech recognizer uses a custom grammar:

The Listening screen.
The Did you say screen, if what the user said could be interpreted as more than one potential result.
The Heard you say or error screen.

The following image shows an example of the flow between GUI screens for a speech recognizer that uses a custom grammar. In this example, speech recognition was successful.

Listening screen

The Listening screen displays text to prompt a user's input, and optionally provides examples of words or phrases that the app can recognize. The Listening screen displays when an app calls the RecognizeWithUIAsync()()() method.

Modifying the Listening screen

You can modify content on the Listening screen by using the following properties of the SpeechRecognizerUISettings class.

Property	Description
SpeechRecognizerUISettingsListenText()()()	Specify custom text on the Listening screen to let users know what kind of info your app is expecting, for example "Favorite color?", "What kind of list do you want to create?", or "Select a trivia category". Keep the string as short as possible, as there is a two-line limit, If you specify custom text for this property, it’s the app's responsibility to provide a localized string. If you don’t specify text for this property, the text "Listening…" displays by default and will be automatically localized. If the speech recognizer has an associated language, the text will display in that language. Otherwise, the text will display in the default system Speech language.
SpeechRecognizerUISettingsExampleText()()()	Text that you specify that provides one or more examples of what your app is listening for, for example " 'blue', 'orange' ", " 'plain list', 'checklist', 'reminder' ", or " 'geography', 'movies', 'food' ". Alternately, you could use this property to display descriptive text about how to use the speech feature, or any other content. If no text is specified, this section of the Listening screen is blank.

SpeechRecognizerUISettingsListenText()()()

Specify custom text on the Listening screen to let users know what kind of info your app is expecting, for example "Favorite color?", "What kind of list do you want to create?", or "Select a trivia category". Keep the string as short as possible, as there is a two-line limit, If you specify custom text for this property, it’s the app's responsibility to provide a localized string.

If you don’t specify text for this property, the text "Listening…" displays by default and will be automatically localized. If the speech recognizer has an associated language, the text will display in that language. Otherwise, the text will display in the default system Speech language.

SpeechRecognizerUISettingsExampleText()()()

Text that you specify that provides one or more examples of what your app is listening for, for example " 'blue', 'orange' ", " 'plain list', 'checklist', 'reminder' ", or " 'geography', 'movies', 'food' ". Alternately, you could use this property to display descriptive text about how to use the speech feature, or any other content.

If no text is specified, this section of the Listening screen is blank.

Did you say screen

If a user says a word or phrase that almost exactly matches more than one potential result, the Did you say screen displays up to twenty phrases from the enabled grammars that are possible matches. If five or fewer phrases are displayed, the phrases are spoken to the user as long as the ReadoutEnabled()()() property is enabled, and "Play audio confirmations" is enabled in the phone's speech settings. Note that the Did you say screen doesn’t display when you use dictation or web search grammars to match speech input.

Modifying the Did you say screen

You can modify the functionality of the Did you say screen by using the following properties of the SpeechRecognizerUISettings class.

Property	Description
SpeechRecognizerUISettingsReadoutEnabled()()()	This property controls whether the phone speaks successfully recognized text back to the user from the Heard you say screen, and whether it speaks options from the Did you say screen. The default setting is true, and the setting applies to both screens. Note that if text-to-speech (TTS) readout is disabled in the settings for speech on the phone, readout will effectively be disabled even if SpeechRecognizerUISettingsReadoutEnabled()()() is set to true.

SpeechRecognizerUISettingsReadoutEnabled()()()

This property controls whether the phone speaks successfully recognized text back to the user from the Heard you say screen, and whether it speaks options from the Did you say screen. The default setting is true, and the setting applies to both screens. Note that if text-to-speech (TTS) readout is disabled in the settings for speech on the phone, readout will effectively be disabled even if SpeechRecognizerUISettingsReadoutEnabled()()() is set to true.

Heard you say screen

The Heard you say screen displays when a call to the RecognizeWithUIAsync()()() method produces a successful recognition, and shows the text of the recognized speech. You can control whether this screen displays and whether to speak recognized text back to the user.

Modifying the Heard you say screen

You can modify the functionality of the Heard you say screen by using the following properties of the SpeechRecognizerUISettings class.

Property	Description
SpeechRecognizerUISettingsReadoutEnabled()()()	This property controls whether the phone speaks successfully recognized text back to the user from the Heard you say screen, and whether it speaks options from the Did you say screen. The default setting is true, and the setting applies to both screens. Note that if text-to-speech (TTS) readout is disabled in the settings for speech on the phone, readout will effectively be disabled even if SpeechRecognizerUISettingsReadoutEnabled()()() is set to true.
SpeechRecognizerUISettingsShowConfirmation()()()	Set this property to false to prevent the Heard you say screen from displaying.

SpeechRecognizerUISettingsReadoutEnabled()()()

SpeechRecognizerUISettingsShowConfirmation()()()

Set this property to false to prevent the Heard you say screen from displaying.

Error screen

The error screen displays when speech recognition was unsuccessful. The screen's title differs depending on the type of error that occurred.

The following image shows one possible error screen.

Speech recognition GUI customization example

The following code example creates a simple grammar, and sets ReadoutEnabled to false to deactivate the readout of recognized text to the user. Note that the Heard you say screen will appear because the screen displays by default.

// Declare the SpeechRecognizerUI object.
SpeechRecognizerUI recoWithUI;

private async void TriviaCategoryPicker_Click(object sender, RoutedEventArgs e)
{
  recoWithUI = new SpeechRecognizerUI();

  // Build a string array, create a grammar from it, and add it to the speech recognizer's grammar set.
  string[] triviaCategories = { "geography", "movies", "food" };
  recoWithUI.Recognizer.Grammars.AddGrammarFromList("categories", triviaCategories);

  // Display text to prompt the user's input.
  recoWithUI.Settings.ListenText = "Select a trivia category";

  // Display an example of ideal expected input.
  recoWithUI.Settings.ExampleText = @"Ex. 'geography', 'movies', 'food'";

  // Deactivate the readout of recognized text to the user.
  recoWithUI.Settings.ReadoutEnabled = false;

  // Load the grammar set and start recognition.
  SpeechRecognitionUIResult result = await recoWithUI.RecognizeWithUIAsync(); 
}

Partager via