Speech service with custom endpoints.

Question

When we were using Public Endpoints previously, we were able to start up to 80 concurrent connections per subscription key, and have not experienced any issues. However, when we start using Custom DNS Public Endpoints with whitelisted IP addresses, we are experiencing TooManyConnections, AuthenticationFailure and ServerTimeout error codes. A few hours after running the Speech Service connections with continuous real-time speech transcription, occasionally, there will be “AuthenticationFailure” errors or “ServerTimeout” errors in a few of the connections;

When we initiated the first 32 concurrent connections within 1 minute and then subsequent 32 concurrent connections after 15 minutes, and then another 32 concurrent connections after another 15 minutes. Sometimes we are getting '429 errors' in the 2nd and 3rd batch and sometimes getting “429 errors”for the previous batch of concurrent connections that were running fine for the 15 minutes, not the subsequent batch of concurrent .

Could you please let me know is there a maximum duration (e.g. 12 hours) for a long running real-time speech transcription connection?

Thank you in advance.

Answer 1

@Mallu Swetha (MINDTREE LIMITED) Thanks for reaching out to us, which API you are calling in your case please? To answer the question generally, Speech Service does have certain limitations on the number of concurrent connections as well as the duration for long-running transcription connections.

For standard, non-custom endpoint usage, Microsoft allows up to 20 concurrent connections per subscription. However, when using custom endpoints, this number might be different, and it seems you are experiencing this issue. You might be hitting a limit on the total number of concurrent connections or the rate of establishing new connections.

As for the maximum duration for a long running real-time speech transcription connection, the current limit is approximately 14 hours. After that, you would need to establish a new connection.

The '429 errors' you're experiencing are likely due to the rate limiting imposed by the service. This is often caused by making too many requests in a short period of time.

It's also important to note that "AuthenticationFailure" errors can occur if there's an issue with your subscription key or if it's not being included correctly in your requests.

"ServerTimeout" errors usually indicate that the service is taking longer than expected to respond, which could also be due to the high number of connections.

To resolve these issues, you might want to consider batching your requests or increasing the time delay between each batch of connections. If these issues persist, please let us know, we are happy to process this issue with a support ticket to check on the backend.

Regards,

Yutong

-Please kindly accept the answer if you feel helpful to support the community, thanks a lot.