How are syllable and word accuracy scores computed from phoneme-level accuracy scores in pronunciation assessments?

Question

How are syllable and word accuracy scores computed from phoneme-level accuracy scores in pronunciation assessments?

Danica Shi 0

I have received the pronunciation assessment result for the word "time" and I have two questions about the results:

How are syllable and word accuracy scores computed from phoneme-level accuracy scores in pronunciation assessments?
As the word "time" only has one syllable, why is the word-level accuracy score different from the syllable-level accuracy score?

Shown below is the pronunciation assessment result for the word "time". Thank you in advance.

{
                            "Word": "time",
                            "Offset": 14900000,
                            "Duration": 4800000,
                            "PronunciationAssessment": {
                                "AccuracyScore": 92,
                                "ErrorType": "None"
                            },
                            "Syllables": [
                                {
                                    "Syllable": "taɪm",
                                    "PronunciationAssessment": {
                                        "AccuracyScore": 72
                                    },
                                    "Offset": 14900000,
                                    "Duration": 4800000
                                }
                            ],
                            "Phonemes": [
                                {
                                    "Phoneme": "t",
                                    "PronunciationAssessment": {
                                        "AccuracyScore": 92,
                                        "NBestPhonemes": [
                                            {
                                                "Phoneme": "t",
                                                "Score": 100
                                            },
                                            {
                                                "Phoneme": "ə",
                                                "Score": 78
                                            },
                                            {
                                                "Phoneme": "ɝ",
                                                "Score": 50
                                            },
                                            {
                                                "Phoneme": "h",
                                                "Score": 34
                                            },
                                            {
                                                "Phoneme": "d",
                                                "Score": 19
                                            }
                                        ]
                                    },
                                    "Offset": 14900000,
                                    "Duration": 1000000
                                },
                                {
                                    "Phoneme": "aɪ",
                                    "PronunciationAssessment": {
                                        "AccuracyScore": 81,
                                        "NBestPhonemes": [
                                            {
                                                "Phoneme": "m",
                                                "Score": 84
                                            },
                                            {
                                                "Phoneme": "aɪ",
                                                "Score": 71
                                            },
                                            {
                                                "Phoneme": "r",
                                                "Score": 68
                                            },
                                            {
                                                "Phoneme": "ə",
                                                "Score": 60
                                            },
                                            {
                                                "Phoneme": "n",
                                                "Score": 49
                                            }
                                        ]
                                    },
                                    "Offset": 16000000,
                                    "Duration": 700000
                                },
                                {
                                    "Phoneme": "m",
                                    "PronunciationAssessment": {
                                        "AccuracyScore": 62,
                                        "NBestPhonemes": [
                                            {
                                                "Phoneme": "m",
                                                "Score": 81
                                            },
                                            {
                                                "Phoneme": "n",
                                                "Score": 72
                                            },
                                            {
                                                "Phoneme": "ə",
                                                "Score": 61
                                            },
                                            {
                                                "Phoneme": "oʊ",
                                                "Score": 50
                                            },
                                            {
                                                "Phoneme": "ŋ",
                                                "Score": 37
                                            }
                                        ]
                                    },
                                    "Offset": 17000000,
                                    "Duration": 3000000
                                }
                            ]
                        }

VasaviLankipalle-MSFT 18,656 Reputation points

2023-08-25T23:50:44.6833333+00:00

Hello @Danica Shi , Thanks for using Microsoft Q&A Platform.

The syllable and word accuracy scores are computed from the phoneme-level accuracy scores by aggregating the accuracy scores of the phonemes that make up each syllable and word, respectively. The accuracy scores are then weighted based on the importance of each level of accuracy in the overall assessment.

Regarding your second question, the word-level accuracy scores are different from the syllable-level accuracy score even for monosyllabic words like "time" because the word-level accuracy score takes into account factors such as fluency and completeness, in addition to accuracy.

As we know the fluency measures how closely the speech matches a native speaker's use of silent breaks between words, while completeness measures how many words are pronounced in the speech to the reference text input. These factors can affect the overall word-level accuracy score, even for monosyllabic words. https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-pronunciation-assessment?pivots=programming-language-python#result-parameters
Danica Shi 0 Reputation points

2023-08-26T13:55:45.3366667+00:00
Hi @VasaviLankipalle-MSFT

Thank you very much for the quick reply.

I have two further questions:

How are the phoneme-level accuracy scores aggregated into the word-level score? By the duration of phonemes? Or are there any other factors?

In the result, the fluency and completeness scores are not found at the word level but at the sentence level. Do you mean the fluency and completeness scores at the sentence level will affect the word-level accuracy score?

Your answer

VasaviLankipalle-MSFT 18,656 Reputation points

2023-08-25T23:50:44.6833333+00:00

Hello @Danica Shi , Thanks for using Microsoft Q&A Platform.

The syllable and word accuracy scores are computed from the phoneme-level accuracy scores by aggregating the accuracy scores of the phonemes that make up each syllable and word, respectively. The accuracy scores are then weighted based on the importance of each level of accuracy in the overall assessment.

Regarding your second question, the word-level accuracy scores are different from the syllable-level accuracy score even for monosyllabic words like "time" because the word-level accuracy score takes into account factors such as fluency and completeness, in addition to accuracy.

As we know the fluency measures how closely the speech matches a native speaker's use of silent breaks between words, while completeness measures how many words are pronounced in the speech to the reference text input. These factors can affect the overall word-level accuracy score, even for monosyllabic words. https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-pronunciation-assessment?pivots=programming-language-python#result-parameters
Danica Shi 0 Reputation points

2023-08-26T13:55:45.3366667+00:00

Hi @VasaviLankipalle-MSFT

Thank you very much for the quick reply.

I have two further questions:

How are the phoneme-level accuracy scores aggregated into the word-level score? By the duration of phonemes? Or are there any other factors?

In the result, the fluency and completeness scores are not found at the word level but at the sentence level. Do you mean the fluency and completeness scores at the sentence level will affect the word-level accuracy score?

Share via

How are syllable and word accuracy scores computed from phoneme-level accuracy scores in pronunciation assessments?

Your answer