SpeechSDK pronunciation assessment recognizes completely wrong words

Question

I use SpeechSDK for pronunciation assessment. When there is no text set in configuration (ie user is not reading, but just speaking) the accuracy of speech-to-text relies heavily on correct pronunciation.

There are cases when user mispronounces some word, not too bad mispronunciation, but the speech-to-text somehow hallucinates and renders 3 different words and says that maybe one was mispronounced and two were perfect pronunciation.

Specific example

User was trying to say: "From the Republican party Donald Trump and from the Democratic party Kamala Harris"

Speech to text recognized: "from the triple bacon martin donald trump and from the democratic party come on hers"

I use continuous recognition with SpeechSDK JS library. Log attached.

When you say those words, it sounds similar to what the user was trying to say, but obviously it's out of context. So I understand where is it coming from. But it defeats the purpose..

My question is - is there some setting I could use to speech-to-text be more context aware?

Answer 1

@Martin With the SDK I think you can add a configuration method EnableContentAssessmentWithTopic for unscripted assessment with SDK version 1.35.0 or later.

This enables content assessment. A content assessment is part of the unscripted assessment for the speaking language learning scenario. By providing a description, you can enhance the assessment's understanding of the specific topic being spoken about.

pronunciationAssessmentConfig.enableContentAssessmentWithTopic("your_topic");  

I tried the same with the speech studio which has all the topics enabled and the accuracy was good.

Also, for issues with JS SDK we have a public github repo where you could add your issue details with logs for SDK team to check for any issues. I hope this helps!!

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.