Azure AI Speech
An Azure service that integrates speech processing into apps and services.
1,768 questions
How can I use Whisper on Azure AI Speech
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 3%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Julian
0
Reputation points
Hi,
I recently switched from using the whisper model via Azure OpenAI to using Azure AI Speech. However, I noticed that the quality of some transcriptions is worse on Azure AI Speech. On the below page it says that it is possible to use the whisper model via Azure AI Speech. Until now I could not find the documentation on how to specify a model when accessing the service with python or javascript. Is there a parameter I can set to get Azure AI Speech to work with whisper and if yes, can I use whisper via Azure AI Speech for real time SST and diarization? The documentation is a big unclear on this point.
Best,
Julian
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 78%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
dupammi 8,535 Reputation points Microsoft Vendor2024-07-05T01:35:20.07+00:00 Hi @Julian
Thank you for using the Microsoft Q&A forum.
You can use OpenAI's Whisper model in Azure AI Speech for batch transcription, but it currently doesn't support real-time speech-to-text (SST) and diarization. To integrate the Whisper model for batch processing, you need to specify it through the REST API. First, list available models in your region using an HTTP GET request to identify the Whisper model. Then, set up a batch transcription request, including the model ID of the Whisper model you found earlier, as shown in this documentation. If transcription quality is subpar, ensure your audio files are high-quality and adhere to supported formats (mp3, wav, ogg). Correctly set your environment variables (SPEECH_KEY, SPEECH_REGION), as described in this guide. For real-time SST and diarization, use Azure AI Speech's native models, as Whisper does not currently support these features in real time. Consider leveraging Whisper for post-processing recorded audio while relying on native models for initial real-time recognition.
Please also refer Quickstart: Speech to text with the Azure OpenAI Whisper model
-
Sign in to comment
Sign in to answer