Our app uses the Microsoft Speech SDK and STT module, but due to being in a private network, it cannot connect directly to the MS speech server. We are considering building a relay server that can receive and send streams to the MS server. Can we customize the MS Speech SDK to send streams to our server? Are there any restrictions on using the MS SDK or reaching the MS server in specific countries or regions, such as China? Thank you in advance.

Building a Relay Server for Azure Speech Client SDK

ywmo 41

Our app uses the Microsoft Speech SDK and STT module, but due to being in a private network, it cannot connect directly to the MS speech server. We are considering building a relay server that can receive and send streams to the MS server.

Can we customize the MS Speech SDK to send streams to our server?
Are there any restrictions on using the MS SDK or reaching the MS server in specific countries or regions, such as China?

Thank you in advance.

romungi-MSFT 44,761 Reputation points Microsoft Employee

2023-08-28T11:00:33.4166667+00:00

@ywmo I think for the first question, are you planning to capture the audio stream and then send it to speech SDK in public network for processing? If Yes, then that should be possible, you can use push or pull stream to pass audio to the SDK. See samples from github repo.

With respect to usage from china, see the soverign clouds documentation as the endpoints change for regions in China. I do not think there are any documented restrictions, as long as you use the required endpoint and have REST API access(port 443) open.

I would suggest to checkout the speech container setup, which can completely be isolated in your network and you only need access to public cloud to send billing information back to azure.
ywmo 41 Reputation points

2023-08-30T01:15:54.6833333+00:00
Actually, out client app has already implemented stt module with MS Azure, and many related features related to it. So what we are thinking is to use our additional server to just bypass stream (created from client app using MS speech SDK ) and send it to MS speech server. Is this kind of customization possible?

initial : (client app using speech sdk) --> audio streams from speech sdk --> (MS Azure)

TOBE: (client app using speech sdk) --> audio streams from speech sdk --> ( Additional Server) --> audio streams from speech sdk--> (MS Azure)

your suggestion : (client app) --> captured audio stream --> (Additional Server with speech sdk) --> audio stream from speech sdk --> (MS Azure)

is #2 possible with speech sdk?
romungi-MSFT 44,761 Reputation points Microsoft Employee

2023-09-01T08:08:54.32+00:00

@ywmo For 2, if you are capturing the audio stream and passing a request from the SDK before the relay server, you will have to configure your networking to use the additional server. If this is something like a proxy server, you can set the same through SDK. Otherwise too, if the request details are not stripped by the additional server your request should be processed successfully by the endpoint.
ywmo 41 Reputation points

2023-09-04T04:55:30.3766667+00:00

@romungi-MSFT Thanks for your reply. I should read the document. Is there any example code for setting proxy both in client and server?

+) the Current client app is built for Android (with kotlin) and IOS (with Objective-C). Is the proxy method adjustable for these two platform?

There is some function that seems to be related to setting Proxy in objective C, however the comment saids the mac os are not supported :
https://learn.microsoft.com/en-us/objectivec/cognitive-services/speech/spxspeechconfiguration#setproxyusinghostportusernamepassworderror

Share via

Building a Relay Server for Azure Speech Client SDK

Your answer