How to use private endpoint for Azure speech resource in realtime

Ulhas Hulyal, Nilesh 0

We have used below documenation which provide details of realtime transcription using speech SDK.

https://learn.microsoft.com/en-us/azure/ai-services/speech-service/get-started-stt-diarization?tabs=linux&pivots=programming-language-python

We trying to find a solution to use private endpoint for speech resource in realtime speech SDK, we should have transcription and speaker diarization as output.

santoshkc 9,565 Reputation points Microsoft Vendor

2024-10-09T13:11:19.3266667+00:00

Hi @Ulhas Hulyal, Nilesh,

Thank you for reaching out to Microsoft Q&A forum!

To use Azure's Speech SDK with private endpoints for real-time transcription and speaker diarization, you need to configure a private endpoint and custom domain for your Speech resource. This ensures traffic stays within your Virtual Network, using a private IP address, and doesn't traverse the public internet. The Speech SDK will interact with this private endpoint by connecting to the custom domain you've set up, such as https://my-private-link-speech.cognitiveservices.azure.com. When using private endpoints, you'll need to update your DNS configuration to ensure proper resolution within your virtual network. This setup allows secure, real-time speech-to-text and speaker diarization functionalities while maintaining private network communication.

For more info: Use Speech service through a private endpoint.

I hope this helps. Thank you.
Ulhas Hulyal, Nilesh 0 Reputation points

2024-10-09T15:30:46.96+00:00

Hi @santoshkc

We will proceed with implementing the necessary changes to configure a private endpoint and custom domain for the Azure Speech Service.

Once these changes are in place, will the Azure Speech SDK automatically route requests through the private endpoint without requiring any additional modifications to our code?
santoshkc 9,565 Reputation points Microsoft Vendor

2024-10-10T08:54:37.3966667+00:00

Hi @Ulhas Hulyal, Nilesh,

Thank you for your follow up query.

To use the Azure Speech SDK with a private endpoint and custom domain, you will need to adjust your application code. The default regional endpoint URLs (e.g., wss://westeurope.stt.speech.microsoft.com) must be replaced with URLs that include your custom domain (e.g., wss://my-private-link-speech.cognitiveservices.azure.com).

Additionally, you'll need to modify how you instantiate the SpeechConfig in your code. Instead of using the region-based approach, you should create the SpeechConfig using the full private endpoint URL, such as SpeechConfig.FromEndpoint(endPoint, speechKey). This ensures your application routes requests through the private endpoint.

Please see: Speech resource with a custom domain name and a private endpoint.

I hope this helps. Thank you.

Ulhas Hulyal, Nilesh 0

Hi @santoshkc

Thank you for your input.

I have instantiated the SpeechConfig in the code as shown below.

For a speech resource without a private endpoint, it works as expected when "speech_config.FromEndpoint" is removed. However, for a speech resource with a private endpoint, when "speech_config.FromEndpoint" is added, it does not generate transcription results with speaker diarization.

Is there something wrong with the speech configuration setup for the private endpoint?

# Set up speech configuration with subscription key and region`
speech_config = speechsdk.SpeechConfig(subscription="speechkey",region="eastus2")
# Set the private endpoint URL
speech_config.FromEndpoint = "wss://my-private-link-speech.cognitiveservices.azure.com"
# Specify the language (optional, defaults to en-US)
speech_config.speech_recognition_language = "en-US"
# Set up audio configuration with the audio file
audio_config = speechsdk.audio.AudioConfig(filename=path)
# Create ConversationTranscriber
conversation_transcriber = speechsdk.transcription.ConversationTranscriber(speech_config=speech_config, audio_config=audio_config)
transcribing_stop = False

santoshkc 9,565 Reputation points Microsoft Vendor

2024-10-14T06:32:22.4933333+00:00

Hi @Ulhas Hulyal, Nilesh,

It looks like the issue is with how the private endpoint is being configured. Instead of directly assigning the FromEndpoint property, you should use the method SpeechConfig.from_endpoint() to properly set the custom domain for your private endpoint.

Make sure that DNS resolution for the private endpoint is correctly set up in your Virtual Network and that speaker diarization is enabled in your transcription configuration. This should resolve the issue with using the private endpoint for real-time transcription. Thank you.
Ulhas Hulyal, Nilesh 0 Reputation points

2024-10-14T13:14:31.57+00:00

Hi @santoshkc

Thank you for your reply.
We tried by using method SpeechConfig.from_endpoint() in the script but got attached error.
Ulhas Hulyal, Nilesh 0 Reputation points

2024-10-14T13:15:56.4466667+00:00

Hi @santoshkc

Thank you for reply.
We tried by using method SpeechConfig.from_endpoint() in the script but got below error.
santoshkc 9,565 Reputation points Microsoft Vendor

2024-10-14T13:22:29.3733333+00:00
Hi @Ulhas Hulyal, Nilesh,

Sorry for the trouble you are facing. I recommend reporting this issue to the Azure Speech Services support team. They will be able to investigate the issue further and provide a more targeted solution. You can report the issue by following these steps:

Go to the Azure portal and navigate to your Speech Services resource.

Click on the "Support + troubleshooting" tab.

Fill out the required information, including a detailed description of the issue and any steps you have taken to troubleshoot it.

Submit the support request.

The Azure Speech Services support team will review your request and provide assistance as soon as possible.

Share via

How to use private endpoint for Azure speech resource in realtime

Your answer