Azure Cognitive Services Speech: Unable to get Custom Translator model results from speech translation code

Question

In test C# code that I created based on the speech translation code in the following sample (“Using custom translation in speech translation”), I’m having trouble getting Custom Translator model translation results. The code just returns a cancellation result. (Please see the end of this post for an excerpt of the code.) I’d appreciate any advice on how to correct this bug.

 

“Using custom translation in speech translation”

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/how-to-translate-speech?source=recommendations&tabs=terminal&pivots=programming-language-csharp#using-custom-translation-in-speech-translation

 

I’m reasonably sure the values in the following environment variables are correct. This is because I tested these values in separate test C# code based on the YouTube video (“Building Custom Language Translation Model using Azure Translator Services” by Shweta Lodha) (modified to translate from Japanese to English), and confirmed I was able to get the correct Custom Translator model translation results for non-speech translation (text translation only from Japanese to English).

 

MS_AZURE_SUBSCRIPTION_KEY

MS_AZURE_SPEECH_CATEGORY_ID

 

Test results:

1.      Ran the code, and waited for “Say something...” to be output in the console.

2.      The following was output immediately after the above (with no chance to speak any Japanese to translate into English).

“CANCELED

Translation finished. Press any key to exit.”

 

Console output from the code below:

---

Say something...

CANCELED

Translation finished. Press any key to exit.

---

 

NOTE: To reduce latency, I need to do both the speech-to-text and machine translation in one call to the Azure server (i.e., NOT do speech-to-text and machine translation in two separate calls).

 

Environment I’m using:

·       Windows 10 (Version 22H2 (OS Build 19045.4529))

·       Microsoft .NET Framework (Version 4.8.04084)

·       Microsoft Visual Studio Professional 2019 (Version 16.11.35)

---

Excerpt of the code:

public static void Main(string[] args)
{
    Task.Run(async () => { await TranslateSpeechAsync(); }).Wait();
    Console.WriteLine("Translation finished. Press any key to exit.");
    Console.ReadKey();
}
private static async Task TranslateSpeechAsync()
{
    string msAzureSpeechKey = Environment.GetEnvironmentVariable("MS_AZURE_SPEECH_SERVICE_KEY");
    string msAzureSubscriptionKey = Environment.GetEnvironmentVariable("MS_AZURE_SUBSCRIPTION_KEY");
    string msAzureSpeechRegion = Environment.GetEnvironmentVariable("MS_AZURE_SPEECH_SERVICE_REGION");
    string msAzureSpeechCategoryID = Environment.GetEnvironmentVariable("MS_AZURE_SPEECH_CATEGORY_ID");
    string endpointUrl = "wss://westcentralus.stt.speech.microsoft.com/speech/universal/v2";
    Uri endpoint_uri = new Uri(endpointUrl);
        
    var translationConfig = SpeechTranslationConfig.FromEndpoint(endpoint_uri, msAzureSubscriptionKey);
    translationConfig.SpeechRecognitionLanguage = "ja-JP"; // Language of the speech input
        translationConfig.AddTargetLanguage("en"); // Target translation language
    translationConfig.SetCustomModelCategoryId(msAzureSpeechCategoryID);
    using (var recognizer = new TranslationRecognizer(translationConfig))
    {
        Console.WriteLine("Say something...");
        // Perform speech translation
        var result = await recognizer.RecognizeOnceAsync();
        if (result.Reason == ResultReason.TranslatedSpeech)
        {
            Console.WriteLine($"Recognized: {result.Text}");
            foreach (var element in result.Translations)
            {
                Console.WriteLine($"Translated into '{element.Key}': {element.Value}");
            }
        }
        else if (result.Reason == ResultReason.NoMatch)
        {
            Console.WriteLine("No speech could be recognized.");
        }
        else if (result.Reason == ResultReason.Canceled)
        {
            Console.WriteLine($"CANCELED");  //This is output to the console.
        }
    }
}

Answer 1

Hello Hirai, Tetu,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

Also, thank you for sharing details about your challenges.

Problem

I understand that you are unable to get Custom Translator model results from speech translation code despite you're sure it worked.

Solution

With this kind of issue there a need for certainty, I will advise to add more code as I have written below to get more detail about the incident.

 else if (result.Reason == ResultReason.Canceled)
        {
            var cancellation = CancellationDetails.FromResult(result);
            Console.WriteLine($"CANCELED: Reason={cancellation.Reason}");
            Console.WriteLine($"CANCELED: ErrorDetails={cancellation.ErrorDetails}");
            Console.WriteLine($"CANCELED: ErrorCode={cancellation.ErrorCode}");
        }

Put the above code, below the last else statement in your code. This should give you more insight into why the translation is being canceled. If the issue persists, you can share the specific error details provided by CancellationDetails.

With this you can resolve the issues and if you can't, we are to read from you to help.

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam