Handle events (Android)

2025-03-07

Your app can be notified about various events that happen during speech recognition.

Session events

Your app can be notified whenever recording is started or stopped during a session.

In your app, include a class that implements the SessionEventListener interface; generally, your speech-enabled activity class. Call the addSessionEventListener() method to register an instance of this class as a listener for session events in the session instance. Make sure that you unregister the listener object before the end of its life cycle; call the removeSessionEventListener() method.

Example session event listener

public class MySpeechEnabledActivity extends Activity implements SessionEventListener {
    ...
    // Activity lifecycle callbacks to register and unregister for session event callbacks properly
    @Override
    protected void onStart() {
        // When the Activity starts (it is already created), register for session event callbacks
        Session.getSharedSession().addSessionEventListener(this);
        super.onStart();
    }
    @Override
    protected void onStop() {
        // When the Activity stops (before being destroyed), unregister for session event callbacks
        Session.getSharedSession().removeSessionEventListener(this);
        super.onStop();
    }
    ...
    // SessionEventListener callback methods
    @Override
    public void onRecordingStarted() {
    ...
    }
    @Override
    public void onRecordingStopped() {
    ...
    }
}

Session event listener interface

The SessionEventListener interface contains the following methods:

onRecordingStarted() - called when recording is started.
onRecordingStopped() - called when recording is stopped, either on request or because of an error.

If your app is notified about session events, you can (among other tasks) implement a record button on your GUI. The appearance of the button reflects the current recording state (for example, a caption/image to indicate recording is started/stopped). The onRecordingStarted() and onRecordingStopped() callbacks notify your app of changes in the recording state, which can be triggered by the user, an error, an idle timeout or any other cause, and enable you to update the appearance of your record button and your internal representation of the recording state.

VuiController events

Your app can be notified about events related to individual VuiForms.

To receive VuiController events, your speech-enabled activity must implement the VuiControllerEventListener interface. The VuiController instance automatically detects if this is the case and calls the appropriate callbacks.

Example: public class MyActivity extends Activity implements VuiControllerEventListener { ...

The VuiControllerEventListener interface contains the following methods:

onProcessingStarted() - called when the user starts their first utterance in any speech-enabled control and the speech recognition process starts.
onProcessingFinished() - called when the last utterance spoken is recognized and the GUI is updated with its results; the speech recognition process is finished, there are no more utterances to be processed.
onProcessingStarted(View view) - called when the user finishes their first utterance in a speech-enabled control and the speech recognition process starts. A reference to the control is supplied as the callback parameter.
onProcessingFinished(View view) - called when the last utterance for a speech-enabled control is recognized and the GUI is updated with the recognition results. A reference to the control is supplied as the callback parameter.
onCommandRecognized(String id, String spokenPhrase, String content, HashMap<String, String> placeholderValues) – called when an application command has been recognized. The following information is delivered by the event about the recognized voice command:

id - the identifier of the recognized command.

spokenPhrase - the actual phrase that was recognized.

content – deprecated, this parameter is always empty.

placeholderValues - the placeholder identifier and value pairs that are present in the recognized voice command.

For more information, see the application commands concept and the application commands use case.

Important information

The processing started/stopped events are always fired in pairs; for every onProcessingStarted() and onProcessingStarted(View view) call, there's a corresponding onProcessingFinished() or onProcessingFinished(View view) call.
If your app is notified about VuiController events, you can (among other tasks) change the GUI based on whether the speech recognition process is ongoing or has finished, either globally or for a specific control.

Globally: For example, you want to stop the user navigating away from the current speech-enabled form while speech recognition processing is ongoing (to make sure that no recognition results are lost). In this case, the onProcessingStarted() and the onProcessingStopped() callbacks can be used to disable and enable navigation between forms.

Specific control: For example, the GUI layout of your app changes based on whether the sound is recorded into a specific control. In this case, wait to change the GUI until all the utterances recorded into that control are completely recognized. By performing the GUI change from the onProcessingFinished(View view) callback, you can make sure that all the relevant recognition results are displayed.
The onProcessingStarted and onProcessingStopped events (global and text control-specific) are reliable only if recording has already been stopped; the user might start speaking again after the onProcessingStopped event is fired.
The onProcessingStopped(View view) event is always fired for the control that has the speech focus when the user starts to dictate (for which the onProcessingStarted(View view) event was fired earlier). This guarantees that onProcessingStarted(View view) and onProcessingStopped(View view) events are always fired in pairs for all controls.
The onProcessingStarted(View view) and onProcessingStopped(View view) callbacks can't track which controls have received recognized results.

onProcessingStarted example

Let's look at the following scenario:

There are two controls in the form: Field 1 and Field 2. Field 1 has the speech focus when recording is started.
The user records three utterances without waiting for the results: 'this is a test', 'next field' and 'this is another test'. Then recording is stopped.
When all speech recognition processing is finished, Field 1 contains "This is a test", Field 2 contains "This is another test" and Field 2 has the speech focus.

How the onProcessingStarted(View view) and onProcessingStopped(View view) events work in this scenario:

The user says 'this is a test' when Field 1 has the speech focus: An onProcessingStarted(View view) event is fired for Field 1.
The user says 'next field' and 'this is another test' when Field 1 has the speech focus. Speech recognition processing is ongoing, therefore no onProcessingStarted(View view) events are fired for these utterances.
When the recognition results for 'this is another test' are displayed, speech recognition processing is finished, therefore an onProcessingFinished(View view) event is fired for the control which had the speech focus when the user started to say it, in this case Field 1.
The user didn't start saying anything when the speech focus was in Field 2. No onProcessingStarted(View view) or onProcessingStopped(View view) events are fired for this control even though Field 2 contains the recognition results.

Conclusion: You can use the onProcessingStarted(View view) and onProcessingStopped(View view) events for this scenario but they aren't reliable in the case of voice navigation or while recording is on.

Share via