Long delay on TTS first response

Question

Hi,

I have been using Azure TTS in my project on an AWS server. Sometimes when running my project, TTS will take somewhere around 30 seconds to respond to the first request on the server (every request following is quite fast) yet this issue has never occurred when using TTS on my local machine although is quite infrequent on the ronin, some days it occurs more than others. It more typically occurs after not having used tts for a while. I am currently using a free subscription for this and am requesting using speak_text_async(text).get().

Has anyone experienced anything similar to this?

Cheers

Answer 1

@Flynn Lauridsen The free and standard tiers are governed by limits on the number of transactions per second depending on the API being called. These tiers are also limited by the quotas set for the same. If you are experiencing throttling issues during certain times, then it is most likely that the speech resource is trying to scale the compute that it depends on to support the higher rate of transactions. You can look at the quotas and limits page on how these values could affect your API calls.

From General best practices section of the document, the following explains the behavior you are seeing:

For example, let's say your application is using text-to-speech, and your current workload is 5 TPS. The next second, you increase the load to 20 TPS (that is, four times more). Speech service immediately starts scaling up to fulfill the new load, but is unable to scale as needed within one second. Some of the requests will get response code 429 (too many requests)

If an answer is helpful, please click on or upvote which might help other community members reading this thread.