Internal Server Error when running evaluation on Custom Speech

Question

Internal Server Error when running evaluation on Custom Speech

MavRedSea 0

Trained a Speech to Text Model on Azure, tried running an evaluation on a test set and I'm getting

"Token error rate results are not applicable for some old tests." on Speech Studio. A few weeks ago the same test wasn't giving any issues.

User's image

dupammi 8,615 Reputation points Microsoft External Staff

2024-07-19T04:29:23.07+00:00

Hi @MavRedSea

Thank you for using the Microsoft Q&A forum.

It seems like you're encountering an issue with running evaluations on your Speech to Text model in Azure. The error message "Token error rate results are not applicable for some old tests", suggests that there might be compatibility issues or changes in the test set or the model itself that are causing this problem. To troubleshoot and resolve the issue, check for any recent updates or changes to your model configuration, as updates can sometimes introduce changes in behavior. Double-check the test set to ensure it is correctly formatted and compatible with the model version you are using, as changes in data format or preprocessing can lead to errors. Try re-running the evaluation after ensuring all configurations are correct, as transient issues can sometimes resolve themselves upon retry, and if possible, update your model or test set to the latest version to see if the issue persists, as this can sometimes resolve compatibility issues. If these steps don't resolve the problem, check the quotas and limits for your Speech service. Ensure that you have not exceeded your subscription's limits. Examine your token usage to see if there are any spikes or if you have reached the maximum number of allowed tokens for your evaluation.

Please reach out to Azure support if the issue persists.
MavRedSea 0 Reputation points

2024-07-19T05:41:07.84+00:00

Hi thanks for replying, even the models trained before and tested successfully on the same test dataset are throwing the same errors (including the baseline STT model).

I had a previous model around 6/24 and it was successfully tested on test_data . Rerunning those same model and test data now for evaluation will give me the "Token Error Rate" error as mentioned above
dupammi 8,615 Reputation points Microsoft External Staff

2024-07-22T07:05:02.11+00:00

Hi @MavRedSea

Sorry for the trouble you are facing. I request you to raise a support case through Azure portal.

I hope you understand. Thank you.