Streaming Audio from Azure Communication Service Outbound Call to Cognitive Service

Question

Streaming Audio from Azure Communication Service Outbound Call to Cognitive Service

Ashley 60

The objective is to stream audio from an outbound call made with Azure Communication Service to a speech recognizer via WebSocket. The process is designed based on this document and this code.The outbound call has been successfully connected, but an error occurs when attempting to start transcription. The error details are as follows:

[
    {
        'id': 'b83ab652-0ab6-4383-94ad-2c91819d7b1d',
        'source': 'calling/callConnections/23005880-74a4-400f-a7c6-a64021b0eb75',
        'type': 'Microsoft.Communication.TranscriptionFailed',
        'data': {
            'transcriptionUpdate': {
                'transcriptionStatus': 'transcriptionFailed',
                'transcriptionStatusDetails': 'streamUrlMissing'
            },
            'version': '2025-05-15',
            'resultInformation': {
                'code': 400,
                'subCode': 8581,
                'message': 'Action failed, Transport url is not valid or web socket server is not operational.'
            },
            'callConnectionId': '23005880-74a4-400f-a7c6-a64021b0eb75',
            'serverCallId': 'aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29tL2FwaS92Mi9jcC9jb252LXVzZWEtMDEtcHJvZC1ha3MuY29udi5za3lwZS5jb20vY29udi8wXzloRlM4a3NVU0dNemtHY3ZmNzdRP2k9MTAtMTI4LTEyMy0yNDkmZT02Mzg4NTI4MDE4MDU5MDM4Njc=',
            'correlationId': '41926020-8e42-4182-9dcd-bff739751ca1',
            'publicEventType': 'Microsoft.Communication.TranscriptionFailed'
        },
        'time': '2025-06-12T13: 04: 35.4013105+00: 00',
        'specversion': '1.0',
        'datacontenttype': 'application/json',
        'subject': 'calling/callConnections/23005880-74a4-400f-a7c6-a64021b0eb75'
    }
]

Transcription is initiated during the RecognizeCompleted event, and verification shows that the WebSocket is open. Both wss:// and ws:// protocols are yielding the same error. The transcription option set during call_automation_client.create_call is as follows:

transcription_options = TranscriptionOptions(
    transport_url=WEBSOCKET_URI_HOST,
    transport_type=StreamingTransportType.WEBSOCKET,
    locale="en-US",
    start_transcription=True,
    )

Prabhavathi Manchala 2,465 Reputation points Microsoft External Staff Moderator

2025-06-12T14:23:07+00:00
Hi Ashley,

You're getting this error when starting transcription

"transcriptionStatusDetails": "streamUrlMissing", "message": "Action failed, Transport url is not valid or web socket server is not operational."

This error occurs even though the call is connected, start_transcription=True is set, the WebSocket is open, and you're using ws:// or wss://.

This error indicates that ACS is unable to validate or connect to the transport_url specified in TranscriptionOptions. The transcription service requires a fully functional WebSocket endpoint that follows Microsoft’s expected format and behavior.

Common causes include:

The WebSocket server URL (WEBSOCKET_URI_HOST) is missing, incorrect, or inaccessible to ACS.

The server fails to upgrade the connection to WebSocket (missing expected 101 Switching Protocols response).

The server lacks logic to consume audio or doesn't support the required ACS audio format (16-bit PCM, 8kHz mono).

1). Ensure WebSocket server is publicly accessible

The URL you pass in transport_url must be publicly reachable from Azure’s backend services.

Use a public IP or DNS over HTTPS (wss://<your-server>) and ensure there are no firewall rules blocking Azure.

Real-time-transcription

Add real-time transcription into application

2). Your server must handle the WebSocket upgrade handshake

When ACS connects to your server, your backend must respond with 101 Switching Protocols for the upgrade.

It must follow standard WebSocket protocol (RFC 6455) and be compatible with binary audio stream ingestion.

Example: Node.js WebSocket server using ws package

const WebSocket = require('ws'); const wss = new WebSocket.Server({ port: 8080 }); wss.on('connection', function connection(ws) { ws.on('message', function incoming(message) { // Process incoming audio here (e.g., forward to Speech SDK) console.log('received:', message); }); });

3). Confirm your server consumes ACS audio format

Format: 16-bit Linear PCM (Little Endian), 8kHz, mono

You must stream this audio to the Speech SDK or Cognitive Services endpoint from within your WebSocket server.

How to recognize speech(Speech SDK)

4). Validate transport_url format

Ensure you're providing the actual WebSocket URL, not encoded or base64.

transcription_options = TranscriptionOptions( transport_url="wss://yourdomain.com/audio-stream", transport_type=StreamingTransportType.WEBSOCKET, locale="en-US", start_transcription=True, )

Avoid anything like:

"serverCallId": "aHR0cHM6Ly9hcGkuZmxpZ2h0cHJveHkuc2t5cGUuY29t..."

That indicates the URL is base64-encoded, which is not allowed for transport_url.

TranscriptionOptions – Azure Python SDK

Before integrating with ACS, test your WebSocket endpoint using a client like wscat or Postman to ensure it connects and receives binary messages.

Sampath 3,750 Microsoft External Staff Moderator

Hello @Ashley,
Make sure you're using something like this for an outbound call (JavaScript):


const callOptions = { mediaStreamingOptions: { transportUrl: "wss://YOUR_VALID_WS_ENDPOINT", transportType: "websocket", contentType: "audio", audioChannelType: "mixed", startMediaStreaming: true, enableDtmfTones: true, enableBidirectional: false, audioFormat: "Pcm24KMono" }

}; await callAutomationClient.createCall({ source: { communicationUserId: "<ACS_USER_ID>" }, targets: [{ phoneNumber: "<TO_PHONE_NUMBER>" }], callbackUri: "<EVENT_GRID_CALLBACK_URI>", mediaStreamingOptions: callOptions.mediaStreamingOptions });

Or if you're using Python and TranscriptionOptions, validate this part carefully:


transcription_options = TranscriptionOptions(

    transport_url="wss://<your_websocket_host>:<port>",

    transport_type=StreamingTransportType.WEBSOCKET,

    locale="en-US",

    start_transcription=True,

)

Refer this gitfor JavaScript Media Streaming.

Ashley 60 Reputation points

2025-06-16T14:00:38.44+00:00
HI @Sampath , websocket issue is resolved. But I am seeing the below issue now.

I created TranscriptionOptions object , set this object in CreateCallOptions under setTranscriptionOptions and then passed it to client.createCallWithResponse when an outbound call is made with azure communication service.

TranscriptionOptions transcriptionOptions = new TranscriptionOptions(appConfig.getWebSocketUrl(),TranscriptionTransport.WEBSOCKET, "en-US", true); createCallOptions.setTranscriptionOptions(transcriptionOptions); Response<CreateCallResult> result = client.createCallWithResponse(createCallOptions, Context.

But the conversation is being streamed as text instead of binary file in the websocket handler. In the websocket handler class, the method handleTextMessage is being invoked instead of handleBinaryMessage. I have separately tested the websocket by passing audio file and even live streaming from laptop microphone and both the times handleBinaryMessage was invoked. Is this behaviour expected? How do I know when the streaming is complete?

Kindly check. Thanks.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-06-17T11:21:15.3866667+00:00
Hello @Ashley,

It sounds like ACS is sending transcribed text instead of raw binary audio, which is causing your WebSocket handler to trigger handleTextMessage instead of handleBinaryMessage. Here’s how you can troubleshoot and fix it:

If ACS is configured for speech-to-text, it may be streaming transcribed text instead of raw audio.

Check if start_transcription=True is forcing ACS to send text instead of PCM audio.

Incorrect audioFormat in TranscriptionOptions

Ensure that the audioFormat is explicitly set to PCM (e.g., Pcm16KMono or Pcm24KMono).
Example:

TranscriptionOptions transcriptionOptions = new TranscriptionOptions( appConfig.getWebSocketUrl(), TranscriptionTransport.WEBSOCKET, "en-US", true ); transcriptionOptions.setAudioFormat("Pcm16KMono");

Some WebSocket implementations default to text unless explicitly set to handle binary.

Try forcing binary mode:

ws.setBinaryType(BinaryType.BLOB);

Check if the first few messages contain metadata (e.g., JSON headers) before the actual binary audio data.

To Fix It Force ACS to send binary audio

Ensure that audioFormat is correctly set in TranscriptionOptions.
Validate that ACS is configured to stream raw audio instead of transcribed text.

If ACS is sending text first, modify your WebSocket handler to switch to binary mode after receiving the first message.

Some ACS configurations default to text-based transcription instead of raw audio. You may need to explicitly request audio streaming.

References

WebSockets message types: text vs binary

GitHub discussion on WebSocket stream handling
Ashley 60 Reputation points

2025-06-17T13:47:15.8966667+00:00

Hi @Sampath ,

I am using Maven version 1.4.0 for azure-communication-callautomation, which I believe is the latest. I do not see the option for transcriptionOptions.setAudioFormat("Pcm16KMono"); with this version. Can you please let me know which version I should use to set the audio format?
Thanks.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-06-18T13:16:55.5066667+00:00

Thank you for your reply. I will check with the back-end team and get back to you.

1 answer

Your answer

Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-06-13T11:01:29.05+00:00

Hello @Ashley,
Make sure you're using something like this for an outbound call (JavaScript):

const callOptions = { mediaStreamingOptions: { transportUrl: "wss://YOUR_VALID_WS_ENDPOINT", transportType: "websocket", contentType: "audio", audioChannelType: "mixed", startMediaStreaming: true, enableDtmfTones: true, enableBidirectional: false, audioFormat: "Pcm24KMono" } }; await callAutomationClient.createCall({ source: { communicationUserId: "<ACS_USER_ID>" }, targets: [{ phoneNumber: "<TO_PHONE_NUMBER>" }], callbackUri: "<EVENT_GRID_CALLBACK_URI>", mediaStreamingOptions: callOptions.mediaStreamingOptions });

Or if you're using Python and TranscriptionOptions, validate this part carefully:

transcription_options = TranscriptionOptions( transport_url="wss://<your_websocket_host>:<port>", transport_type=StreamingTransportType.WEBSOCKET, locale="en-US", start_transcription=True, )

Refer this gitfor JavaScript Media Streaming.
Ashley 60 Reputation points

2025-06-16T14:00:38.44+00:00

HI @Sampath , websocket issue is resolved. But I am seeing the below issue now.

I created TranscriptionOptions object , set this object in CreateCallOptions under setTranscriptionOptions and then passed it to client.createCallWithResponse when an outbound call is made with azure communication service.

TranscriptionOptions transcriptionOptions = new TranscriptionOptions(appConfig.getWebSocketUrl(),TranscriptionTransport.WEBSOCKET, "en-US", true); createCallOptions.setTranscriptionOptions(transcriptionOptions); Response<CreateCallResult> result = client.createCallWithResponse(createCallOptions, Context.

But the conversation is being streamed as text instead of binary file in the websocket handler. In the websocket handler class, the method handleTextMessage is being invoked instead of handleBinaryMessage. I have separately tested the websocket by passing audio file and even live streaming from laptop microphone and both the times handleBinaryMessage was invoked. Is this behaviour expected? How do I know when the streaming is complete?

Kindly check. Thanks.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-06-17T11:21:15.3866667+00:00

Hello @Ashley,

It sounds like ACS is sending transcribed text instead of raw binary audio, which is causing your WebSocket handler to trigger handleTextMessage instead of handleBinaryMessage. Here’s how you can troubleshoot and fix it:

If ACS is configured for speech-to-text, it may be streaming transcribed text instead of raw audio.

Check if start_transcription=True is forcing ACS to send text instead of PCM audio.

Incorrect audioFormat in TranscriptionOptions

Ensure that the audioFormat is explicitly set to PCM (e.g., Pcm16KMono or Pcm24KMono).
Example:

TranscriptionOptions transcriptionOptions = new TranscriptionOptions( appConfig.getWebSocketUrl(), TranscriptionTransport.WEBSOCKET, "en-US", true ); transcriptionOptions.setAudioFormat("Pcm16KMono");

Some WebSocket implementations default to text unless explicitly set to handle binary.

Try forcing binary mode:

ws.setBinaryType(BinaryType.BLOB);

Check if the first few messages contain metadata (e.g., JSON headers) before the actual binary audio data.

To Fix It Force ACS to send binary audio

Ensure that audioFormat is correctly set in TranscriptionOptions.
Validate that ACS is configured to stream raw audio instead of transcribed text.

If ACS is sending text first, modify your WebSocket handler to switch to binary mode after receiving the first message.

Some ACS configurations default to text-based transcription instead of raw audio. You may need to explicitly request audio streaming.

References

WebSockets message types: text vs binary

GitHub discussion on WebSocket stream handling
Ashley 60 Reputation points

2025-06-17T13:47:15.8966667+00:00

Hi @Sampath ,

I am using Maven version 1.4.0 for azure-communication-callautomation, which I believe is the latest. I do not see the option for transcriptionOptions.setAudioFormat("Pcm16KMono"); with this version. Can you please let me know which version I should use to set the audio format?
Thanks.
Sampath 3,750 Reputation points Microsoft External Staff Moderator

2025-06-18T13:16:55.5066667+00:00

Thank you for your reply. I will check with the back-end team and get back to you.

Answer 1

Grmacjon-MSFT 19,301 Moderator

Hello @Ashley , thanks for your patience. This issue seems to be on your websocket server. Did you validate if that is reachable? Also, it is important to note that ACS only support wss.

Regarding audio format, it is an option only in audio streaming and not transcription. Version 1.4.0 should have latest changes.

Share via

Streaming Audio from Azure Communication Service Outbound Call to Cognitive Service

1 answer

Your answer