Phonemes are not available for pronunciation recognition in french

Question

Phonemes are not available for pronunciation recognition in french

GOMES-ALVES-DOS-SANTOS Bruna 0

On the result of the pronunciation recognition, if we set to "en-US", we have all the results for the phonemes spoken/matches. As below.


"Phonemes": [
                            {
                                "Phoneme": "k",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100,
                                    "NBestPhonemes": [
                                        {
                                            "Phoneme": "k",
                                            "Score": 100
                                        },
                                        {
                                            "Phoneme": "oʊ",
                                            "Score": 4
                                        },
                                        {
                                            "Phoneme": "ɡ",
                                            "Score": 3
                                        },
                                        {
                                            "Phoneme": "w",
                                            "Score": 3
                                        },
                                        {
                                            "Phoneme": "p",
                                            "Score": 2
                                        }
                                    ]
                                },
                                "Offset": 11300000,
                                "Duration": 1200000
                            },
                            {
                                "Phoneme": "w",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100,
                                    "NBestPhonemes": [
                                        {
                                            "Phoneme": "w",
                                            "Score": 100
                                        },
                                        {
                                            "Phoneme": "k",
                                            "Score": 75
                                        },
                                        {
                                            "Phoneme": "r",
                                            "Score": 3
                                        },
                                        {
                                            "Phoneme": "ɛ",
                                            "Score": 1
                                        },
                                        {
                                            "Phoneme": "ɡ",
                                            "Score": 0
                                        }
                                    ]

But for "fr-FR", I just receive an empty string for the phoneme and cannot make a match with the IPA dictionaries.

"Phonemes": [
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 52
                                },
                                "Offset": 10100000,
                                "Duration": 1100000
                            },
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 55
                                },
                                "Offset": 11300000,
                                "Duration": 700000
                            },
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 64
                                },
                                "Offset": 12100000,
                                "Duration": 1100000
                            },
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 49
                                },
                                "Offset": 13300000,
                                "Duration": 500000
                            }
                        ]
                    },
                    {

1 answer

Your answer

Answer 1

navba-MSFT 27,615 Microsoft Employee Moderator

@GOMES-ALVES-DOS-SANTOS Bruna Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.

User's image

This is by design since by design since fr-FR doesn't expose phoneme name yet. Please refer this docs.

.

Also note that if phoneme name is not available, both syllable and spoken phoneme are not available either. Pronunciation assessment supports syllable groups only in en-US with IPA and with SAPI. More info here.

.

Regarding the ETA in the roadmap for the Pronunciation Assessment support for (french locale) fr-FR, We can prioritize our work if your usage on Pronunciation Assessment is large than 1k/month, we can priority our work and support to expose the phoneme name or reach the minimum of Commitment Tiers

User's image

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

**

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

GOMES-ALVES-DOS-SANTOS Bruna 0

Thank you for your answer @navba-MSFT . But could you confirm if it is using correctly the IPA alphabet? Because, for example, for pronouncing "va", I receive this output

{
                        "Word": "va",
                        "Offset": 5500000,
                        "Duration": 2700000,
                        "PronunciationAssessment": {
                            "AccuracyScore": 100,
                            "ErrorType": "None"
                        },
                        "Syllables": [
                            {
                                "Syllable": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100
                                },
                                "Offset": 5500000,
                                "Duration": 1200000
                            },
                            {
                                "Syllable": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100
                                },
                                "Offset": 6800000,
                                "Duration": 1400000
                            }
                        ],
                        "Phonemes": [
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100
                                },
                                "Offset": 5500000,
                                "Duration": 500000
                            },
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100
                                },
                                "Offset": 6100000,
                                "Duration": 600000
                            },
                            {
                                "Phoneme": "",
                                "PronunciationAssessment": {
                                    "AccuracyScore": 100
                                },
                                "Offset": 6800000,
                                "Duration": 1400000
                            }
                        ]
                    }

With a 3 items array for phonemes, but the IPA and SAPI alphabet say to me that "va" on the french language is /va/, only two phonemes and only 1 syllabe. What makes no sense then. Could you help me with this?

GOMES-ALVES-DOS-SANTOS Bruna 0

Or for this case of "temps" that gives me 3 elements phoneme
When the IPA for it is /tɑ̃/ and SAPI is t-an

Printing description of ((Swift.Optional<MicrosoftCognitiveServicesSpeech.SPXWordLevelTimingResult>)0x000060000174ccc0):
(SPXWordLevelTimingResult?) [1] = 0x000060000174ccc0 {
  baseSPXTimingResult@0 = {
    baseNSObject@0 = {
      isa = SPXWordLevelTimingResult
    }
    _offset = 1.1899999999999999
    _duration = 0.14999999999999999
  }
  _word = @"temps"
  _accuracyScore = 100
  _errorType = @"None"
  _phonemes = 0x0000600000c64240 3 elements
  _syllables = 0x0000600000c64870 1 element
}

navba-MSFT 27,615 Reputation points Microsoft Employee Moderator

2024-05-22T04:33:55.92+00:00

@GOMES-ALVES-DOS-SANTOS Bruna Thanks for getting back. I will check this at my end and get back to you with an update.
GOMES-ALVES-DOS-SANTOS Bruna 0 Reputation points

2024-06-14T15:34:39.71+00:00

Hi @navba-MSFT , did you have any feedback on it? I have the same situation in Korean. I'm trying the word "저는" for example, and speech API returns me 5 elements pheneme when the real word has all of these 6 [tɕʌnɯn]
GOMES-ALVES-DOS-SANTOS Bruna 0 Reputation points

2024-06-14T15:35:34.75+00:00

Hi @navba-MSFT , you have any update on this?
navba-MSFT 27,615 Reputation points Microsoft Employee Moderator

2024-06-18T02:52:14.06+00:00

@GOMES-ALVES-DOS-SANTOS Bruna I didn't hear back from the Product Owners. I have followed up again. I will get back once I get an update from them.

Share via

Phonemes are not available for pronunciation recognition in french

1 answer

Your answer