Hi team, I am following this link for setting ITN, punctuation: https://learn.microsoft.com/en-us/azure/ai-services/speech-service/display-text-format?pivots=programming-language-python However I couldn't find any related code snippet or samples in Python for those configs? Could someone point me to the right documentation? Thank you so much!

Hi @LeetGPT Thank you for using the Microsoft Q&A forum. To use ITN as part of speech, you can pass the "format" query parameter with the value "detailed" to the Azure Speech to Text API. This will ensure that the API response contains all available text formats, including "Display", "ITN", "Lexical", and "MaskedITN". The inverse-text-normalized (ITN) or canonical form of the recognized text is printed with phone numbers, numbers, abbreviations (such as "doctor smith" to "dr smith"), and other transformations applied when using the "ITN" format. By default, the API response is in "Simple" format, which includes only the recognized text. However, you can use the " Detailed " format to obtain N-Best results in JSON format. By specifying the "Detailed" format query parameter, the API response will include all the available text formats, such as "Display", "ITN", "Lexical", and "MaskedITN". The "ITN" format will display numbers as numerical values instead of text. You can retrieve this response from the API and integrate it into your application. You can find a comprehensive list of all the supported query parameters for the Speech-to-text Rest API on this page and try to use these in your SDK as well. Thank you! I hope this helps. If an answer is helpful, please click on or upvote which might help other community members reading this thread.

How to use Azure Speech to text display text format features in Python?

Accepted answer

dupammi 6,810 Reputation points Microsoft Vendor

2024-04-10T06:55:20.1633333+00:00

Hi @LeetGPT

Thank you for using the Microsoft Q&A forum.

To use ITN as part of speech, you can pass the "format" query parameter with the value "detailed" to the Azure Speech to Text API. This will ensure that the API response contains all available text formats, including "Display", "ITN", "Lexical", and "MaskedITN".

The inverse-text-normalized (ITN) or canonical form of the recognized text is printed with phone numbers, numbers, abbreviations (such as "doctor smith" to "dr smith"), and other transformations applied when using the "ITN" format.

By default, the API response is in "Simple" format, which includes only the recognized text. However, you can use the "Detailed" format to obtain N-Best results in JSON format.

By specifying the "Detailed" format query parameter, the API response will include all the available text formats, such as "Display", "ITN", "Lexical", and "MaskedITN". The "ITN" format will display numbers as numerical values instead of text. You can retrieve this response from the API and integrate it into your application.

You can find a comprehensive list of all the supported query parameters for the Speech-to-text Rest API on this page and try to use these in your SDK as well.

Thank you!

I hope this helps.

If an answer is helpful, please click on or upvote which might help other community members reading this thread.
Please sign in to rate this answer.
LeetGPT 60 Reputation points

2024-04-10T18:04:08.0466667+00:00

Thank you for the answer @dupammi However I am having trouble actually getting the detailed result back from recognized callback:

self.speech_config = speechsdk.SpeechConfig(subscription=selected_key, region=selected_region) self.speech_config.set_property(speechsdk.PropertyId.Speech_LogFilename, "AzureSTTLog") self.speech_config.output_format = speechsdk.OutputFormat.Detailed print(f"output format: {self.speech_config.output_format}") self.audio_config = speechsdk.audio.AudioConfig(stream=self.stream) self.speech_recognizer = speechsdk.SpeechRecognizer(speech_config=self.speech_config, audio_config=self.audio_config) self.__connect_callbacks() def __connect_callbacks(self): self.speech_recognizer.recognized.connect(self.__recognized_callback) self.speech_recognizer.session_stopped.connect(self.__session_stopped_callback) self.speech_recognizer.recognizing.connect(self.__recognizing_callback) def __recognized_callback(self, evt): print(f"result: {evt}") self.transcription += evt.result.text

As you can see I print the result event from __recognized_callback function but the printed result is as follows:

output format: OutputFormat.Detailed result: SpeechRecognitionEventArgs(session_id=b54e3948fa234e0b8251c0e75676ecc2, result=SpeechRecognitionResult(result_id=fdbe35aea05246039a4de3b070fa4c5e, text="So defining sub arrays with ecosystem.", reason=ResultReason.RecognizedSpeech))

Could you let me know what I am missing? Thanks!

LeetGPT 60 Reputation points

2024-04-10T18:04:46.83+00:00

Also printed the output format and double confirmed it is set to be "Detailed".

dupammi 6,810 Reputation points Microsoft Vendor

2024-04-11T01:39:49.77+00:00

Hi @LeetGPT

Thank you for your follow-up on this.

It seems that you were not getting the detailed result back because you are not using the json property of the SpeechRecognitionResult object. The json property contains the detailed recognition result in JSON format, which can be parsed to extract the desired information.

Here's a repro I tried at my side of how to extract the detailed result from the SpeechRecognitionResult object:

import azure.cognitiveservices.speech as speechsdk import json # Replace with your own subscription key, service region, and the path to the audio file you want to recognize. subscription_key = "YOUR_KEY" service_region = "YOUR_REGION" filename = "YOUR_FULL_FILE_PATH" # Create the speech configuration speech_config = speechsdk.SpeechConfig(subscription=subscription_key, region=service_region) # Set the recognition language to the appropriate language speech_config.speech_recognition_language = "en-US" # Ask for detailed recognition result speech_config.output_format = speechsdk.OutputFormat.Detailed # Create an audio configuration specifying the audio file path audio_config = speechsdk.audio.AudioConfig(filename=filename) # Create a speech recognizer with the given settings speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config) # Start the recognizer and wait for a result. result = speech_recognizer.recognize_once() # Check the result if result.reason == speechsdk.ResultReason.RecognizedSpeech: # Print the recognized text print("Recognized Text:", result.text) # Convert the JSON result to a Python dictionary json_result = json.loads(result.json) print(json_result) # Extract ITN and Masked ITN from the JSON result if available if "NBest" in json_result and len(json_result["NBest"]) > 0: itn = json_result["NBest"][0].get("ITN", "") masked_itn = json_result["NBest"][0].get("MaskedITN", "") # Print ITN and Masked ITN print("ITN:", itn) print("Masked ITN:", masked_itn) else: print("ITN and Masked ITN not found in the JSON result.") elif result.reason == speechsdk.ResultReason.NoMatch: print("No speech could be recognized.") elif result.reason == speechsdk.ResultReason.Canceled: cancellation_details = result.cancellation_details print("Speech recognition canceled:", cancellation_details.reason) if cancellation_details.reason == speechsdk.CancellationReason.Error: print("Error details:", cancellation_details.error_details)

What I can suggest you is that modify your code to extract the detailed result from the json property of the SpeechRecognitionResult object.

Output of above code:

Thank you!

I hope this helps.

If an answer is helpful, please click on or upvote which might help other community members reading this thread.

LeetGPT 60 Reputation points

2024-04-11T02:31:41.54+00:00

Appreciate the example! I did find the json result from final recognized result! Just to double confirm, continuous recognition's recognizing callback doesn't have this detailed result right?

def __recognizing_callback(self, evt): print(f"recognizing evt json: {evt.result.json}") if self.on_text is not None: self.on_text(evt.result.text)

This gives me:

recognizing evt json: {"Id":"2675ac5fe2574b7c99aab40947fc4933","Text":"it's easy so i just use","Offset":72000000,"Duration":17600000,"PrimaryLanguage":{"Language":"en-us"},"Channel":0}

dupammi 6,810 Reputation points Microsoft Vendor

2024-04-11T10:08:26.0466667+00:00

Hi @LeetGPT

Glad to hear that you found the JSON result from the final recognized result. Regarding your question, the continuous recognition's recognizing callback doesn't have this detailed result. The recognizing callback only provides the intermediate recognition results, which are not as detailed as the final recognition result.

If you want to get the detailed result, you need to wait for the final recognition result. You can use the recognized event to get the final recognition result.

I hope you understand. Thank you.
Sign in to comment

How to use Azure Speech to text display text format features in Python?

0 additional answers