Share via

Handling Data

  • Article
  Microsoft Speech Technologies Homepage

The speech recognition engine sends recognized text results to both voice-only and multimodal speech applications using Semantic Markup Language (SML). Use the following Microsoft ASP.NET Speech Controls to validate the SML results and bind them to other controls or script variables.

Speech Control Description
Speech CompareValidator Use the Speech CompareValidator (CompareValidator) control to validate the data in a SemanticItem by comparing its data to the data in another control, or to a constant value.
Speech CustomValidator Use the Speech CustomValidator (CustomValidator) control to validate the data in a SemanticItem using a custom client-side script.
Speech Listen (Multimodal applications only) Use the Speech Listen (Listen) control to bind data to specific graphical elements on a Web page.
Speech Prompt (Multimodal applications only) Use the Speech Prompt (Prompt) control to specify any prompts spoken to the user.
Speech RecordSound (Voice-only applications only) Use the Speech RecordSound (RecordSound) control to record input spoken by a user.
Speech SemanticMap Use the Speech SemanticMap (SemanticMap) control to define semantic properties that a QA control sends to an application using SML output.

SemanticMap controls are comprised of a set of SemanticItem controls, which contain information about a QA control's semantic state and its binding and autopostback characteristics.
  • Validating Data
  • Binding Data to Controls or Script Variables
  • Using Normalization Functions to Control the Display of Data
  • Recording Input Spoken by a User

Validating Data

Use the CompareValidator and CustomValidator controls to validate data. Although both controls are similar in purpose, the CompareValidator control supports validation with a fixed set of operators, and the CustomValidator control supports validation by means of application-specific client-side scripts.

Validating Data by Comparing Values

The SemanticItemToValidate property of the CompareValidator control references the SemanticItem that contains the data to be validated. The CompareValidator is activated when this data is changed or confirmed, depending on the setting of the ValidationEvent property. The data to be validated is compared either with data in the control referenced by the SemanticItemToCompare property, or with constant data in the ValueToCompare property.

The Operator property specifies the type of comparison, such as equal to or less than. If the result of the comparison is False, CompareValidator invalidates the item specified by SemanticItemToValidate. If the result is False and the InvalidateBoth property is True, the SemanticItemToValidate and SemanticItemToCompare properties are both invalidated.

When the result of the comparison is False, the CompareValidator plays an associated prompt to inform the user that the data is not valid.

CompareValidator General panel

To validate data by comparing data

  1. From the Speech Toolbox, drag a CompareValidator control onto the design canvas of an ASP.NET (.aspx) page.

  2. Right-click the CompareValidator control, and on the shortcut menu, click Property Builder to open the General panel of the CompareValidator Property Builder.

  3. In the ID text box, type a unique ID for this CompareValidator control.

    Note  The ID of a Speech Control, Application Speech Control, or Basic Speech Control must consist entirely of ASCII characters in the ranges A-Z, a-z and 0-9. IDs containing characters outside this range may fail to display properly, and references to these IDs may fail.

  4. If applicable, in the Speech Control Settings box, type or select the ID of the SpeechControlSettingsItem to apply to this CompareValidator control.

  5. In the ValidationEvent list, do one of the following.

    • Click onconfirmed to specify that the CompareValidator control runs each time the State property of the associated SemanticItem control changes to Confirmed.
    • Select onchanged to specify that the CompareValidator control runs each time the Text property of the associated SemanticItem control changes.

  6. In the SemanticItemToValidate list, type or select the ID of the SemanticItem containing the data to be validated, and then do one of the following.

    • If comparing data to a SemanticItem, type or select the ID of the SemanticItem containing comparison data in the SemanticItemToCompare box.
    • If comparing data to a constant value, enter the constant value in the ValueToCompare field.

  7. In the Type list, click the data type of the data being compared, and then in the Operator list, click the comparison operator.

To inform the user of invalid data

  1. In the CompareValidator Property Builder, under the Voice Output heading, click Prompt.
  2. Associate either an inline prompt or a prompt function with the CompareValidator to play a prompt when the compared data is not valid.

Note  The process of associating prompts with a CompareValidator control is identical to the process of associating prompts with a QA control.

Validating Data using Client-side Scripts

The SemanticItemToValidate property of the CustomValidator control references the SemanticItem containing the data to be validated. The CustomValidator is activated when this data is changed or confirmed, depending on the setting of the ValidationEvent property. The data to be validated is evaluated by the client-side script procedure specified in the ClientValidationFunction property. If the script returns False, the data in the control is invalidated.

When the result of the comparison is False, the CustomValidator plays an associated prompt to inform the user that the data is not valid.

CustomValidator General panel

To validate data using client-side scripts

  1. From the Speech Toolbox, drag a CustomValidator control onto the design canvas of an .aspx page.
  2. Right-click the CompareCustomValidator control, and on the shortcut menu, click Property Builder to open the General panel of the CustomValidator Property Builder.
  3. In the ID text box, type a unique ID for this CustomValidator control.
  4. In the ValidationEvent list, do one of the following.
    • Click onconfirmed to specify that the CustomValidator control runs each time the State property of the associated SemanticItem control changes to Confirmed.
    • Click onchanged to specify that the CompareValidator control runs each time the Text property of the associated SemanticItem control changes.
  5. In the SemanticItemToValidate box, type or select the ID of the SemanticItem containing the data to be validated.
  6. In the ClientValidationFunction text box, type the name of the client-side script that will evaluate the data.

To inform the user of invalid data

  1. In the CustomValidator Property Builder, under the Voice Output heading, click Prompt.
  2. Associate either an inline prompt or a prompt function with the CustomValidator to play a prompt when the compared data is not valid.

Note  The process of associating prompts with a CustomValidator control is identical to the process of associating prompts with a QA control.

Binding Data to Controls or Script Variables

Whether a speech application is multimodal or voice-only, bind recognized data to controls on a Web page or to JScript variables.

Binding Data in Multimodal Applications

Use Listen controls to bind data to target elements on a Web page, such as text boxes. Use Prompt controls to create prompts from the data in the target elements of the Listen controls.

Listen controls contain Grammar and Bind objects that enable developers to associate grammars with the Listen control and bind data to target elements.

Listen control

To bind data to Web page elements

  1. From the Speech Toolbox, drag a Listen control onto the design canvas of an .aspx page.
  2. Right-click the control, and on the shortcut menu, click Property Builder to open the General panel of the Listen Property Builder.
  3. In the ID text box, type a unique ID for this Listen control.
  4. In the tree view pane on the left, under the Input heading, click General.
  5. Associate grammars with this control. This process is identical to the process for associating grammars with other Speech Controls.
  6. In the Listen Mode list, click the recognition mode of the speech recognition engine, and then do the following.
    • In the Target Element column, type or select the ID of the Web page element to which recognized results are bound.
    • In the Target Attribute column, type or select whether the data consists of the recognized semantic text value or the selectedIndex.
    • If applicable, in the Target Method column, type the name of a method called when data binds to the target element.
    • If applicable, in the Test column, enter a method execution condition.
    • In the Value column, type or select the XPath expression that maps to the resulting SML node.

Bind data to a JScript variable using the value of a SemanticItem, or the value of a Web page element. The following code example illustrates setting the value of a SemanticItem with an ID of siCity to the JScript variable theCity. Follow the same logic to set the value of a Web page element to a JScript variable, replacing siCity with the ID of the Web page element.

  siCity.value = theCity

Also use the previous data binding conventions when creating prompt functions.

Use the data in Web page elements or SemanticItems to create prompts that contain the data.

Prompt control

To create prompts from Web page element data

  1. From the Speech Toolbox, drag a Prompt control onto the design canvas of an .aspx page.
  2. Right-click the control, and on the shortcut menu, click Property Builder to open the General panel of the Prompt Property Builder.
  3. In the ID text box, type a unique ID for this Prompt control.
  4. In the tree view pane on the left, under the Output heading, click General.
  5. In the text box The inline content of the prompt, type any supporting inline prompt text to be prepended or appended to bound data.
  6. Click Insert Value element, and then do the following.
    • In the Target Element box, type or select the ID of the Web page element that contains the applicable data.
    • In the Target Attribute box, type or select a value that specifies whether the data consists of the text value or the selectedIndex.
  7. If applicable, click Insert Content element, and then do the following.
    • In the Href box, click Browse to browse to a file that contains relevant content.
    • In the Type text box, type the MIME type corresponding to the speech output format.

The following code example illustrates how to enter an inline prompt that will say "The event is scheduled for Saturday at 7:00PM," where the text "The event is scheduled for" and "at" is typed text, and "Saturday" and "7:00PM" are retrieved from Web page elements with IDs of theDay and theTime, respectively.

  The event is scheduled for 
<Value TargetElement="theDay" TargetAttribute="value"/>
at 
<Value TargetElement="theTime" TargetAttribute="value"/>

For an example of using Listen controls in a multimodal application, see the Using Tap-and-talk sample application in the SASDK.

Binding Data in Voice-only Applications

For voice-only applications, bind data to Web page elements and JScript variables using the value or selectedIndex of SemanticItems and Web page elements as source data, just as with multimodal applications.

Multimodal and voice-only applications retrieve data from different locations. Multimodal applications use Listen and Prompt controls to retrieve data from Web page elements. Voice-only applications retrieve data from SemanticItems.

Using Normalization Functions to Control the Display of Data

Normalization functions are client-side functions that perform normalization of the recognition results in a matched SML node. These functions enable developers to control the display of data on a Web page for multimodal applications. If a user speaks a phone number to a multimodal application, a normalization function can format the phone number so it appears on the Web page as a typical phone number, such as (000) 555-5555, instead of 0005555555.

To create a client normalization function

  1. In Solution Explorer, right-click a project name, point to Add, and then on the sub-shortcut menu, click Add New Item.
  2. In the Categories pane, expand the Web Project Items heading, click Utility, and then on the Templates pane, click JScript File.
  3. In the Name text box, optionally type a new name for the file (with a .js extension), and then click Open to open a blank JScript file in Speech Control Editor labeled JScript1.js.
  4. In the blank file, type a normalization function, and then save the file with the project.

When the XPathTrigger property of the Answer object is matched, RunSpeech looks for a ClientNormalizationFunction property. If ClientNormalizationFunction is specified, RunSpeech calls it, and passes the entire SML node to it. ClientNormalizationFunction returns the results as a string of normalized text.

  
    string FunctionName (smlNode, SemanticItem)
{
    // Client-side code referenced by ClientNormalizationFunction
}

The following sample illustrates the client normalization functions used to normalize a spoken phone number in the Phone Number sample application in the SASDK. The resulting output would look like (000) 555 5555.

  //formatting the area code part to look like: (NNN)
function FormatAreaCodeString(theValue){ 
    if (theValue == null) 
        return; 
    else { 
        var sAreaCodeString = "(" + theValue.value + ")"; 
        document.all.PhoneNumberTB.value = sAreaCodeString +
          document.all.PhoneNumberTB.value;        
    } 
} 

//formatting the local number part to look like: NNN NNNN 
function FormatLocalNumberString(theValue){ 
    var sLocalNumberString = theValue.value; 
    if (sLocalNumberString == null || sLocalNumberString.length != 7) 
        return;                 
    else {            
        var re = /(\d{3})(\d{4})/;                   
        var aMatchCollection = sLocalNumberString.match(re);                     
        sLocalNumberString = aMatchCollection[1] + " " + aMatchCollection[2]; 
        document.all.PhoneNumberTB.value = document.all.PhoneNumberTB.value + " " +
          sLocalNumberString;                   
    } 
}

To associate a client normalization function with an Answer or Confirm object

  1. Right-click a QA control on an .aspx page, and on the shortcut menu, click Property Builder to open the General panel of the QA Property Builder.
  2. In the tree view pane on the left, under the Input heading, click General.
  3. On the Answers tab, in the ClientNormalizationFunction column, type the name of a normalization script for each matched SML node that should be normalized.
  4. If applicable, on the Extra Answers tab, in the ClientNormalizationFunction column, type the name of a normalization script for each matched SML node that should be normalized.
  5. If applicable, on the Confirms tab, in the ClientNormalizationFunction column, type the name of a normalization script for each matched SML node that should be normalized.

Note  Although this sample mainly illustrates server-side functionality, it also uses a client normalization script that transforms recognized SML results into a proper format so that the results can be processed by a C# script on the server.

Recording Input Spoken by a User

Use the RecordSound control to record input spoken by a user, and copy the recorded file to the Web server from Microsoft Speech Server 2004 (MSS) for future playback.

For example, use the RecordSound control to record the user's response to the application prompt, "Please say your name and address after the beep."

To record input spoken by a user

  1. From the Speech Toolbox, drag a RecordSound control onto the design canvas of an .aspx page.
  2. Right-click the control, and on the shortcut menu, click Properties to open the Properties window.
  3. In the SavePath box of the Properties window, type the full local path on the Web server where the file will be saved. For example, C:\RecordedSounds\Sound001.wav.
  4. In the PlaybackUrlBase box of the Properties window, type the base URL used to construct a playback path for replaying the sound from the Web server. For example, https://localhost/CompanySounds.
  5. Enter other property values as appropriate.

When the saved file is accessed for playback, the base URL is concatenated with the saved file. For example:

  PlaybackUrlBase = https://localhost/CompanySounds
Saved file = Sound001.wav
Downloaded file = https://localhost/CompanySounds/Sound001.wav

See Also

Creating Speech-Enabled Web Pages