Azure AI Speech
An Azure service that integrates speech processing into apps and services.
2,078 questions
Visemes don't match speech for Cantonese Azure text to speech
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 91%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
muxic muxic
0
Reputation points
When trying to use a Cantonese voice e.g. "zh-HK-WanLungNeural" to generate azure speech from text input, e.g. "在美國哪裡可以買到自行車？還有哪裡可以看到蒙娜麗莎？", the length of the visemes generated doesn't match up with the audio. e.g. for 6 seconds of audio, viseme ends at 5 seconds.
Identical code changing the speaker to English or Mandarin with their respective language inputs provides valid visemes.
Visemes are generated in accordance with the docs here: https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-speech-synthesis-viseme?tabs=visemeid&pivots=programming-language-python
Azure AI Speech
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 1%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYM%3C/text%3E%3C/svg%3E)
YutongTie-MSFT 53,976 Reputation points Moderator2024-08-16T05:11:04.52+00:00 Hello @muxic muxic Thanks for reaching out to us, do you have a chance to share the output to us for investigation?
-
muxic muxic 0 Reputation points
2024-08-16T08:48:16.99+00:00 Audio File: https://jmp.sh/s/SWc8BcHq95l8EK52e7hC
Console Output:
在美國哪裡可以買到自行車？還有哪裡可以看到蒙娜麗莎？Viseme event received: audio offset: 0.05s, viseme id: 0.
Viseme event received: audio offset: 0.1s, viseme id: 19.
Viseme event received: audio offset: 0.16875s, viseme id: 15.
Viseme event received: audio offset: 0.2375s, viseme id: 3.
Viseme event received: audio offset: 0.30625s, viseme id: 6.
Viseme event received: audio offset: 0.375s, viseme id: 21.
Viseme event received: audio offset: 0.3875s, viseme id: 4.
Viseme event received: audio offset: 0.43125s, viseme id: 6.
Viseme event received: audio offset: 0.475s, viseme id: 20.
Viseme event received: audio offset: 0.6125s, viseme id: 3.
Viseme event received: audio offset: 0.6125s, viseme id: 20.
Viseme event received: audio offset: 0.7125s, viseme id: 19.
Viseme event received: audio offset: 0.8s, viseme id: 2.
Viseme event received: audio offset: 0.8s, viseme id: 14.
Viseme event received: audio offset: 0.8625s, viseme id: 1.
Viseme event received: audio offset: 0.9s, viseme id: 4.
Viseme event received: audio offset: 0.9375s, viseme id: 12.
Viseme event received: audio offset: 1.1s, viseme id: 3.
Viseme event received: audio offset: 1.1s, viseme id: 6.
Viseme event received: audio offset: 1.175s, viseme id: 6.
Viseme event received: audio offset: 1.4125s, viseme id: 21.
Viseme event received: audio offset: 1.425s, viseme id: 2.
Viseme event received: audio offset: 1.4875s, viseme id: 6.
Viseme event received: audio offset: 1.55s, viseme id: 19.
Viseme event received: audio offset: 1.6375s, viseme id: 8.
Viseme event received: audio offset: 1.6375s, viseme id: 7.
Viseme event received: audio offset: 1.6375s, viseme id: 19.
Viseme event received: audio offset: 1.675s, viseme id: 15.
Viseme event received: audio offset: 1.7125s, viseme id: 6.
Viseme event received: audio offset: 1.7875s, viseme id: 12.
Viseme event received: audio offset: 1.8s, viseme id: 4.
Viseme event received: audio offset: 1.8875s, viseme id: 20.
Viseme event received: audio offset: 2.0375s, viseme id: 19.
Viseme event received: audio offset: 2.0375s, viseme id: 15.
Viseme event received: audio offset: 2.0375s, viseme id: 4.
Viseme event received: audio offset: 2.1125s, viseme id: 0.
Viseme event received: audio offset: 3.5875s, viseme id: 0.
Viseme event received: audio offset: 3.6375s, viseme id: 7.
Viseme event received: audio offset: 3.7625s, viseme id: 2.
Viseme event received: audio offset: 3.8625s, viseme id: 19.
Viseme event received: audio offset: 3.8625s, viseme id: 6.
Viseme event received: audio offset: 3.925s, viseme id: 4.
Viseme event received: audio offset: 3.95625s, viseme id: 7.
Viseme event received: audio offset: 3.9875s, viseme id: 19.
Viseme event received: audio offset: 4.1s, viseme id: 2.
Viseme event received: audio offset: 4.1125s, viseme id: 14.
Viseme event received: audio offset: 4.1875s, viseme id: 1.
Viseme event received: audio offset: 4.2625s, viseme id: 4.
Viseme event received: audio offset: 4.3375s, viseme id: 12.
Viseme event received: audio offset: 4.3375s, viseme id: 3.
Viseme event received: audio offset: 4.4125s, viseme id: 6.
Viseme event received: audio offset: 4.65s, viseme id: 6.
Viseme event received: audio offset: 4.6625s, viseme id: 12.
Viseme event received: audio offset: 4.8s, viseme id: 3.
Viseme event received: audio offset: 4.8875s, viseme id: 19.
Viseme event received: audio offset: 4.8875s, viseme id: 19.
Viseme event received: audio offset: 4.9625s, viseme id: 8.
Viseme event received: audio offset: 5.0s, viseme id: 7.
Viseme event received: audio offset: 5.0375s, viseme id: 21.
Viseme event received: audio offset: 5.05s, viseme id: 7.
Viseme event received: audio offset: 5.15s, viseme id: 20.
Viseme event received: audio offset: 5.2375s, viseme id: 19.
Viseme event received: audio offset: 5.2375s, viseme id: 2.
Viseme event received: audio offset: 5.325s, viseme id: 14.
Viseme event received: audio offset: 5.3875s, viseme id: 4.
Viseme event received: audio offset: 5.44375s, viseme id: 6.
Viseme event received: audio offset: 5.5s, viseme id: 15.
Viseme event received: audio offset: 5.5125s, viseme id: 2.
Viseme event received: audio offset: 5.6125s, viseme id: 0.
The Audio File is 7 seconds long, but visemes end at 5.6 seconds.
-
muxic muxic 0 Reputation points
2024-08-21T03:40:58.2133333+00:00 @YutongTie-MSFT was wondering if you've seen the output, and whether the Microsoft team has begun investigation? Thanks.
Sign in to comment
Sign in to answer