Continue Speech Recogonization with Custom keyword

Question

Hi ,

I am going to develop voice recolonization in this i have two custom keyword and more than 20 custom Command.

so i have created custom keyword on my studio login and downloaded .table file and i am checking user voice with .table file locally that is working fine.

Now for custom command i am checking first .table file is the keyword is available ? in given user voice because custom command also have keyword .

for example - i have these two custom keyword -

1. john
2. hey john

and i have below custom command

1.john go to route for changing the my monitor display color to red.
2.hey john go to route for changing the my monitor display color to red.

her is my code implementation -

        List<Task<KeywordRecognitionResult>> lstTasks = new List<Task<KeywordRecognitionResult>>();
                lstTasks.Add(ProcessHeyWakeUp("keywords/john.table"));
                lstTasks.Add(ProcessHeyWakeUp("keywords/Heyjohn.table"));
                var kailash = await Task.WhenAny(lstTasks.ToArray());

                if (kailash.Result.Text == "john \n")
                {

                    NotifyUser("Got John Command" + kailash.Result.Text);

                }
                else if (kailash.Result.Text == "Hey John")
                {
                    NotifyUser("Got Hey John Command" + kailash.Result.Text);

                }

               var stream = AudioDataStream.FromResult(kailash.Result);
                stream.DetachInput();
                var savePicker = new FileSavePicker();
                savePicker.SuggestedStartLocation = PickerLocationId.DocumentsLibrary;
                savePicker.FileTypeChoices.Add("WAV File", new List<string>() { ".wav" });
                savePicker.SuggestedFileName = "audio";
                var tempFilePath = $"{ApplicationData.Current.TemporaryFolder.Path}\\audio.wav";
                await stream.SaveToWaveFileAsync(tempFilePath);
                var config1 = SpeechConfig.FromSubscription(@"mykey", @"eastus");
                config1.EndpointId = "myendpoint";

                using (var audioInput = AudioConfig.FromWavFileOutput(tempFilePath))
                {
                    using (var recognizer = new SpeechRecognizer(config1, audioInput))
                    {
                        var fullcommand = await recognizer.RecognizeOnceAsync();
                        if (fullcommand.Text != "Hey john." || fullcommand.Text != "john.")
                        {

                            await Dispatcher.RunAsync(
                            Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
                            {
                                await VcpSpeechMessage(fullcommand.Text);
                            });

                        }



                    }
                }

Task Method -

public async Task<KeywordRecognitionResult> ProcessHeyWakeUp(string str)
{

    var WakeUpModel = KeywordRecognitionModel.FromFile(str);
    var audioConfig = AudioConfig.FromDefaultMicrophoneInput();
    var keywordRecognizer = new KeywordRecognizer(audioConfig);
    return await Task<KeywordRecognitionResult>.Run(async () =>
    {
        var matchResult = await keywordRecognizer.RecognizeOnceAsync(WakeUpModel).ConfigureAwait(true);

        return matchResult;
    });

}

When i run this code it's recognizing custom keyword that is working fine , but when i speak custom command it's
not recognizing full command some time it's recognizing like -

.john go to route for changing
hey john go to route for changing the my monitor

my problem it's not recognizing full command text , i don't want to put any Delay or Thread Sleep. because i don't know the length of custom command it can be change. So any Delay is not option for me.

can you help to solve this problem ?

Answer 1

@kailash solanki Your matchResult is local to the method ProcessHeyWakeUp() so it seems like the keywordRecognizer.RecognizeOnceAsync() is returning the result as soon as your custom keyword is recognized. This is the normal behavior of this method. You can lookup more details here.

Starts a keyword recognition session. This session will last until the first keyword is recognized. When this happens, a Recognized event will be raised and the session will end. To rearm the keyword, the method needs to be called again after the event is emitted.

You should instead try to achieve something like this where you declare matchResult as global and read this result in your audiostream. Returning after keyword recognition might not always give you the complete sentence. You can also try to use a visual option or a button click as mentioned in the example and then detach your input stream instead. I hope this helps to recognize the full sentence and your keyword.