I want to use the created AudioClip with DictationHandler.

Question

Development Environment
HoloLens2
Unity 2020.3.11f
Visual Studio 2019
Microsoft Mixed Reality Toolkit v2.7.3

I am creating an application to speech-to-text conversion using DcitationHandler(dictation) on HoloLens2.
But there is a problem, I can get the sound of my own voice well with the HoloLens2 built-in microphone, but I cannot get the sound of the surroundings well with the HoloLens2 built-in microphone.
I used MicStream.StreamCategory.ROOM_CAPTURE in the HoloToolkit.Unity.InputModule.MicStream Class(MicStream.cs) to get the sound of the surroundings with the HoloLens2 built-in microphone and create an AudioClip.
Is it possible to use this AudioClip with DictationHandler? I would like to know how to do that.
Also, please let me know if there is a better way other than Dictationhandler or something.

Answer 1

Hello, Welcome to Microsoft Q&A,

MRTK uses DictationRecognizer in Unity to implement speech recognition. Referring to its documentation, it seems to be a black box and does not provide an independent method for speech-to-text conversion. You can consult Unity further. Also, you can try Azure Speech service/Speech-to-text, which can recognize speech from files and memory streams, please refer to How to recognize speech - Speech service - Azure Cognitive Services | Microsoft Learn.

----
If the response is helpful, please click "Accept Answer" and upvote it.
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.