AvaMultilingualNeural voice has abnormal speedup

Question

AvaMultilingualNeural voice has abnormal speedup

Qa 5

https://drive.google.com/file/d/1JzX1Nvxj6NjA7SbnPtZvx1Xt0vrHtYL_/view?usp=drive_link I used this voice when generating in English(US). But it seems like an error occurred. Hope you help me in this case.

VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2024-01-17T21:00:49.77+00:00

Hello @Qa , Thanks for using Microsoft Q&A Platform.

Sorry for the inconveniences caused. We are unable to access the audio file you have shared here. Is it possible to share more details regarding this Multilingual Neural voice issue?

It would be helpful if you can share the speech/audio file and the related text to reproduce the same on our end. Please email the speech files to azcommunity@microsoft.com with the below details,

Thread URL: Link to this thread.

Your Azure Subscription ID, Sample text and audio files response:

Email Subject : Attn Vasavi

Nir Sancho 0

Hi,

This issue reproduces also in english - your solution does not fix it.

Try the following text (voice = en-US-AvaMultilingualNeural) while asking for "wordBoundaries".

"This is a sample text"

If you put it exactly like that the synthesis is returned in a very high rate.

If you add a dot at the end or remove the capital T it returns in normal rate.

If you don't ask for wordBoundaries it returns in normal rate.

Please update if there is any resolution.

Thanks

Nir

See reproduction code below:


(function () {
    'use strict';

    const sdk = require('microsoft-cognitiveservices-speech-sdk');

    const audioFile = 'YourAudioFile.wav';
    // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION"
    const speechConfig = sdk.SpeechConfig.fromSubscription(process.env.SPEECH_KEY, process.env.SPEECH_REGION);
    const audioConfig = sdk.AudioConfig.fromAudioFileOutput(audioFile);

    // The language of the voice that speaks.
    speechConfig.speechSynthesisVoiceName = 'en-US-AvaMultilingualNeural';
    
    speechConfig.speechRecognitionLanguage = 'en-US'
    speechConfig.speechSynthesisLanguage = 'en-US'

    // Create the speech synthesizer.
    let synthesizer = new sdk.SpeechSynthesizer(speechConfig, audioConfig);

    synthesizer.wordBoundary = (s, e) => {
        console.log('tts: wordBoundary', e);
    };

    const text = 'This is a sample text'
    synthesizer.speakTextAsync(
        text,
        function (result) {
            if (result.reason === sdk.ResultReason.SynthesizingAudioCompleted) {
                console.log('synthesis finished.');
            } else {
                console.error('Speech synthesis canceled, ' + result.errorDetails + '\nDid you set the speech resource key and region values?');
            }
            synthesizer.close();
            synthesizer = null;
        },
        function (err) {
            console.trace('err - ' + err);
            synthesizer.close();
            synthesizer = null;
        }
    );
    console.log('Now synthesizing to: ' + audioFile);
})();

Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

2 answers

Your answer

VasaviLankipalle-MSFT 18,676 Reputation points Moderator

2024-01-17T21:00:49.77+00:00

Hello @Qa , Thanks for using Microsoft Q&A Platform.

Sorry for the inconveniences caused. We are unable to access the audio file you have shared here. Is it possible to share more details regarding this Multilingual Neural voice issue?

It would be helpful if you can share the speech/audio file and the related text to reproduce the same on our end. Please email the speech files to azcommunity@microsoft.com with the below details,

Thread URL: Link to this thread.

Your Azure Subscription ID, Sample text and audio files response:

Email Subject : Attn Vasavi
Nir Sancho 0 Reputation points

2024-01-22T07:09:51.0766667+00:00

Hi,

This issue reproduces also in english - your solution does not fix it.

Try the following text (voice = en-US-AvaMultilingualNeural) while asking for "wordBoundaries".

"This is a sample text"

If you put it exactly like that the synthesis is returned in a very high rate.

If you add a dot at the end or remove the capital T it returns in normal rate.

If you don't ask for wordBoundaries it returns in normal rate.

Please update if there is any resolution.

Thanks

Nir

See reproduction code below:

(function () { 'use strict'; const sdk = require('microsoft-cognitiveservices-speech-sdk'); const audioFile = 'YourAudioFile.wav'; // This example requires environment variables named "SPEECH_KEY" and "SPEECH_REGION" const speechConfig = sdk.SpeechConfig.fromSubscription(process.env.SPEECH_KEY, process.env.SPEECH_REGION); const audioConfig = sdk.AudioConfig.fromAudioFileOutput(audioFile); // The language of the voice that speaks. speechConfig.speechSynthesisVoiceName = 'en-US-AvaMultilingualNeural'; speechConfig.speechRecognitionLanguage = 'en-US' speechConfig.speechSynthesisLanguage = 'en-US' // Create the speech synthesizer. let synthesizer = new sdk.SpeechSynthesizer(speechConfig, audioConfig); synthesizer.wordBoundary = (s, e) => { console.log('tts: wordBoundary', e); }; const text = 'This is a sample text' synthesizer.speakTextAsync( text, function (result) { if (result.reason === sdk.ResultReason.SynthesizingAudioCompleted) { console.log('synthesis finished.'); } else { console.error('Speech synthesis canceled, ' + result.errorDetails + '\nDid you set the speech resource key and region values?'); } synthesizer.close(); synthesizer = null; }, function (err) { console.trace('err - ' + err); synthesizer.close(); synthesizer = null; } ); console.log('Now synthesizing to: ' + audioFile); })();
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

Answer 1

Hello @Qa ! Welcome to Microsoft QnA! Please have a look here to set the correct parameters : https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/introducing-more-multilingual-ai-voices-optimized-for/ba-p/4012832 https://learn.microsoft.com/en-us/azure/ai-services/speech-service/speech-synthesis-markup Kindly provide more feedback on how you are using the Voice , in your code or app!

It seems like a Speed issue more likely an encoder issue

Unless you can provide more details , kindly see the Documentation

I hope this helps!

Kindly mark the answer as Accepted and Upvote in case it helped!

Regards

Answer 2

VasaviLankipalle-MSFT 18,676 Moderator

Hello @Qa , thank you for your patience.

Looks like there was a similar issue and the product team has been fixed the bug last week. In this issue there was an abnormal speed rate on multilingual voices when output format is not 24khz. You can try 24k output (mp3 or wav) or 48k.

Are you still facing this issue?

Regards,
Vasavi

-Please kindly accept the answer and vote 'yes' if you feel helpful to support the community, thanks.

Share via

AvaMultilingualNeural voice has abnormal speedup

2 answers

Your answer