Error rate increasing after training model

Shradha Saini (HOLMES) 1

I have trained the Pronunciation model on Cognitive Service Speech Studio, but after training the Error Rate of trained model has increased from
Base Model - error rate (27.59% , model = (20201019))
Trained Model- error rate (31.47% )

Dataset used for training is English statements read by person in Indian Accent. This dataset is free of errors.

I want to know the reason behind this error increase.

1 answer

Ramr-msft 17,616 Reputation points

2021-03-09T03:57:14.653+00:00

@Shradha Saini (HOLMES) Thanks for the question. Can you please add more details about the use case that you are trying. The WER is built into speech.microsoft.com/customspeech. We have forwarded to the product team to confirm on the error rate.

Here is the doc to improve accuracy.
Please sign in to rate this answer.
Shradha Saini (HOLMES) 1 Reputation point

2021-03-09T05:25:34.717+00:00

Use case : I want to improve the speech to text transcription for specific pronunciations (like Indian English) as I wasn't getting satisfactory result from Base model transcription.

Problem : For this use case I am training the model with Indian English Pronunciation Data to decrease error rate. But for trained model rather than decreasing the error rate it has increased by around 4%. Now the question is what is letting this error to increase.

Conclusion : I want this error rate to go down. I would like to raise support request if it can't be resolved here.

I have gone through this`document link and it suggests to check training data. As I have mentioned there are no errors/noise in the audio data. The audio is very loud and clear, also the transcript is accurate too.

Shradha Saini (HOLMES) 1 Reputation point

2021-03-10T04:42:55.997+00:00

Can I please have an update on this.

Ramr-msft 17,616 Reputation points

2021-03-10T15:39:43.053+00:00

@Shradha Saini (HOLMES) Thanks for the details. We would recommend to raise a Azure support desk ticket from Help+Support blade from Azure portal for your resource if you have a support plan for your subscription. This will help you to share the details securely and work with an engineer who can provide more insights about the issue that if it can be replicated.
Sign in to comment