Stay On The Line...

  • Article

POSTED BY: RENAUD LECOEUCHE, MSS Software Dev Engineer

Some voice applications consider the user being silent as a valid answer. For example, an application could say: “Please choose one of the following products or stay on the line to speak to a sales representative” followed by a list of products that are available for phone order. If the user stays silent for a second or so, the call continues and a transfer is made.

As you probably know, the way to ask a question and get an answer in MSS 2007 Beta is to use a QuestionAnswerActivity. However the QuestionAnswerActivity will keep repeating until it gets a valid recognition. The question then is: How to model this kind of dialog with MSS?

The answer is easy. Here is how to proceed:

  1. Create a QuestionAnswerActivity that recognizes the product names.
  2. Create a SpeechSequenceActivity.
  3. Add the QuestionAnswerActivity to the SpeechSequenceActivity.
  4. Go to the SpeechEvents view of the SpeechSequenceActivity.
  5. Add a ConsecutiveSilencesSpeechEventActivity and set its MaximumSilences property to 1.
  6. Add a GotoActivity to the ConsecutiveSilencesSpeechEventActivity that redirects to the part of the dialog you want to run when the user remains silent.

You are done! Adding a ConsecutiveSilencesSpeechEventActivity allows you to take arbitrary actions when the user stays silent. Because SpeechEvents are associated with SpeechSequenceActivity activities, you can localize their actions to a single QuestionAnswerActivity or a group of related activities.

How speech events work
Speech events provide a powerful way of controlling your application flow. Let me clarify how they work. Speech events are scoped to a SpeechSequenceActivity activity. When the speech sequence is executing, its associated speech events track the changes in the workflow’s History property. This is how they know what is happening in the dialog and when they can execute.

For example, a ConsecutiveSilencesSpeechEventActivity activity increments its count when a SilenceHistoryItem is assign to the History property. SpeechEvents count the number of events globally. The ConsecutiveSilencesSpeechEventActivity activity increments its count when a SilenceHistoryItem from any activity is assigned to the History property. It does not matter whether the first silence and second silence come from the same or different activities. Similarly, it resets its count when a recognition from any activity is detected. The same is true for other speech events.

Activating speech events
Now that we know how a speech event tracks what happens in the dialog, we can look at how the speech events are activated. The workflow decides whether to run a speech event when a QuestionAnswerActivity activity completes a turn. This is the only point where a speech event can be started. If the speech event is active because its count matches the maximum allowed number of events, the workflow suspends the QuestionAnswerActivity activity and runs the speech event. Once the speech event completes, the QuestionAnswerActivity activity restarts, unless it was cancelled. I’d like to emphasize the point: you cannot force a speech event to execute only by changing the History property of the workflow. Changing the History property will change the speech events counts, but it won’t force activation. Activation is decided by the workflow.

Bail-out logic
Let’s go back to the solution presented above. There is a caveat worth mentioning. SpeechEvents are often used to trigger a bail-out: if the user stays silent three times in a row, the dialog is transferred or terminated.

In our example above, the silence that is used to move to another part of the dialog will count towards any bail-out limit you may have. If you have a SpeechEvent that triggers after three silences, the user can only stay silent twice now. This is due to the global nature of the speech event count: recall that it does not matter where silence comes from.

In this case, one comes from the initial question answer activity and the next ones come from the following question answer activity. In order to reset the count, you need to assign a RecognitionHistoryItem to the History property of the workflow. This makes sense: the first silence that was detected is not really a silence but a valid way for the user to interact with the system. By setting the history, you let the speech events know that. Setting the History is easy:

Workflow.History = new RecognitionHistoryItem(QAThatTriggeredTheSilence, false /*no bargein*/);

does the job. Changing the History property in this way will ensure that your bail-out logic works as expected.