How to get phonemes from azure speech sdk

Question

How to get phonemes from azure speech sdk

Doug Bergman 1

Hi, I am following the Microsoft Azure Speech-to-Text Python sdk tutorial here. I would like to know if there is a way to return the phonemes, an intermediate step in generating the interpreted text. Is that possible? If so, can you please refer me to the documentation and hopefully some sample code that does this. I searched and could not find anything that already answered my question.

Thanks!
Doug

YutongTie-MSFT 27,426 Reputation points

2020-12-02T01:06:02.223+00:00

Hello Doug,

Thanks for reaching out to us. I don't think there is a official way to do that, but I can communicate with the product group as a feature request. Could you please tell me the specific reason or scenario you want the intermediate result?

Regards,
Yutong
Doug Bergman 1 Reputation point

2020-12-02T01:37:19.037+00:00

Hi Yutong, thanks for the reply. I can't go into great detail but basically I would like to identify the boundaries of phonemes and syllables within wav files. I hope that helps to clarify.

YutongTie-MSFT 27,426 Reputation points

2020-12-02T01:06:02.223+00:00

Hello Doug,

Thanks for reaching out to us. I don't think there is a official way to do that, but I can communicate with the product group as a feature request. Could you please tell me the specific reason or scenario you want the intermediate result?

Regards,
Yutong
Doug Bergman 1 Reputation point

2020-12-02T01:37:19.037+00:00

Hi Yutong, thanks for the reply. I can't go into great detail but basically I would like to identify the boundaries of phonemes and syllables within wav files. I hope that helps to clarify.

Answer 1

Hi @Doug Bergman

Is this not what you'r looking for? Sample response for the Phonemes of the word "Thank":

{
"Duration": 4700000,
"Offset": 11500000,
"Phonemes": [
{
"Duration": 2100000,
"Offset": 11500000,
"Phoneme": "th",
"PronunciationAssessment": {
"AccuracyScore": 100.0
}
},
{
"Duration": 900000,
"Offset": 13700000,
"Phoneme": "ae",
"PronunciationAssessment": {
"AccuracyScore": 100.0
}
},
{
"Duration": 700000,
"Offset": 14700000,
"Phoneme": "ng",
"PronunciationAssessment": {
"AccuracyScore": 100.0
}
},
{
"Duration": 700000,
"Offset": 15500000,
"Phoneme": "k",
"PronunciationAssessment": {
"AccuracyScore": 100.0
}
}
],
"PronunciationAssessment": {
"AccuracyScore": 100.0,
"ErrorType": "None"
},
"Word": "Thank"
}

Sarthak Agarwal 11 Reputation points

2021-01-17T11:06:51.893+00:00

URL: https://learn.microsoft.com/en-us/azure/cognitive-services/speech-service/how-to-pronunciation-assessment?pivots=programming-language-python
Doug Bergman 1 Reputation point

2021-01-18T15:54:15.493+00:00

Hi @Sarthak Agarwal , yes that is what I had in mind. Can you please post the script that returned this json?
Gino Cuevas 1 Reputation point

2021-06-18T13:36:18.883+00:00

Hi @Doug Bergman
Were you able to figure out how @Sarthak Agarwal was able to get the response with Phoneme results?

I can't figure out how to achieve such results.

Answer 2

Hello Doug,

This is

Pronunciation assessment evaluates speech pronunciation and gives speakers feedback on the accuracy and fluency of spoken audio. With pronunciation assessment, language learners can practice, get instant feedback, and improve their pronunciation so that they can speak and present with confidence. Educators can use the capability to evaluate pronunciation of multiple speakers in real-time.

But the pronunciation assessment feature is currently only available in regions westus, eastasia and centralindia, and only supports language en-US.

Please refer to following sample code for how to set up and retrieve.

https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/cpp/windows/console/samples/speech_recognition_samples.cpp#L633

Regards,
Yutong

How to get phonemes from azure speech sdk

2 answers

Activity