Configuring Azure Media Service for Scale with On Demand Content

Question

We are building an App Powered by Azure Media services that's streaming content via HLS/DASH to mobile devices. We have 15 to 20 second Video clips that are we want to deliver up to at most 480p.

I know we need to enable CDN, but I'm not sure if we need a standard of premium endpoint. Our goal is to scale up to millions of concurrent streams at scale. How do I think about this?

With CDN enabled, is there an upper bound on concurrent streams with a standard endpoint, assuming the OnDemand on content is highly mostly cached?

If you we have to use a premium endpoint, what happens when you get past max number of 10 nodes?

Finally, if we encode all of our content at one bitrate and resolution, will that help with caching performance? Since we are not really looking to have very HD video, we figured having everyone stream at the the same resolution should give us okay performance.

Please Advise.

@John Deutscher (MSFT) , @David Bristol

Answer 1

The pricing difference between standard and Premium accounts for the fact that the standard uses "shared" VM resources. It is not a dedicated machine for just your workload. The Standard endpoint makes no guarantee that you won't have a "noisy neighbor". You could end up on a deployment that has nobody streaming on it, or you may end up on a very busy one where you could see some occasional impacts. That is the primary difference in the costs.
That and the ability to scale up a lot higher. The Premium endpoints can actually be scaled out to about 40 endpoint units. That's a lot of egress data.

If you configure the CDN properly, using an Origin Shield solution (both Akamai and Verizon offer origin shielding), then you should see very high cache hit ratios on highly popular, viral content. Hopefully you would be able to estimate that 99.5% of the content would be served from the CDN cache, and very few requests would come all the way back to the origin (Streaming Endpoint).

That said, you will likely also have a lot of long-tail content in your service offering, where the content ends up out of cache, and you get a small amount of traffic coming back to the origins.

It's nearly impossible to guess up front what traffic pattern you will see - until you get up in operations and monitor closely. We would highly recommend having the flexibility of scaling up and down with the Premium Endpoints and do a lot of monitoring of the Azure Monitoring telemetry that we provide for egress data and CPU load on the Streaming Endpoints.

Answer 2

Thanks! To elaborate further on this query As always, your configuration is going to depend on the specific traffic pattern that your users are generating against the service, CDN, and StreamingEndpoint(s). One simple example to think about is whether you expect to have 10 videos each generating 1 million simultaneous views or 1 million videos each generating 10 simultaneous views. Both generate 10 million simultaneous views but the caching efficiency is very different for the two cases.

Your scenario is very likely somewhere in the middle but the more unique videos being viewed you have, the more load you will have on the StreamingEndpoints even if we assume perfect caching (which isn’t realistic). The better idea you have of how much load your users are putting on the CDN and in turn how much load the CDN is putting on the StreamingEndpoint, the better you will be able to do capacity planning.

The Streaming Endpoints provide Azure Monitor Metrics which you can use to get an idea of the load these resources are handling:

Monitoring Media Services | Microsoft Learn
Monitoring Media Services data reference | Microsoft Learn

Speaking in broad terms, handling millions of concurrent streams at scale we would generally recommend using one or more premium endpoints with CDN enabled (and CDN protections in place). The default limit is 10 units per Premium StreamingEndpoint and 2 StreamingEndpoints per account but both of these limits can be raised via a support request.

Streaming Endpoints (Origin) - Azure Media Services v3 | Microsoft Learn explains the differences in capabilities of the Premium and Standard StreamingEndpoints.

Encoding all of your content with a single bitrate will reduce the number of urls that the CDN needs to cache (and the StreamingEndpoint needs to serve) so it will improve your cache performance vs having multiple bitrates. That said, having only one bitrate reduces the effectiveness of using adaptive bitrate streaming like HLS and DASH. The client player has only one layer to choose from and thus cannot adapt the content to the network conditions (high bitrate content when the network is good, lower bitrate content when the network is congested). You will have to decide if the single bitrate encode fits the user experience you want to deliver to your customers. That said, if your content is very short (15 to 20 seconds as you said) the client isn’t going to have a lot of time to adapt anyway.

Also I would highly recommend you consider is making your solution highly available. Regional outages happen. Internet routing issues happen. A truly redundant solution would have the ability to process, serve, and cache the videos in a redundant fashion. Redundancy comes with a cost so you will have to decide the business impact of an outage in each of your various dependencies and then evaluate the costs of a mitigation.

The High Availability with Media Services Video on Demand - Azure Media Services v3 | Microsoft Learn sample shows an example of using multiple Azure Media Services accounts in multiple regions to process and serve content.

The sample would need some tweaks to work with multiple CDNs or to have a single CDN pull from multiple StreamingEndpoints but it should give you a starting point on what to consider when thinking through what a highly available solution looks like.

Hope this helps. Let us know.

Answer 3

Thanks for your reply. In our use case, we are building out a social media network. So we could have in theory millions of videos driving content. But the trending/viral videos would be ones that would get the most streams.

I'm still not clear so so how to calculate how much i can stretch the standard endpoint, with CDN enabled. How do I think about this?

If I go with a premium endpoint, the only difference I saw from my use case was that it had less throughput. So in theory, i would need to have 3 premium endpoint ($417/month to match the load of one standard unit ($64.50/month). So if I never scale past the three units, then it's not justifiable, correct? What am I missing here. Given this, I need a way to calculate if I need premium in the 1st place based on my modeling. Are there any other differences between standard and premium outside of the ability to scale up?

As for encoding, I have a minimal requirement on streaming. i can't go below say 360p. If it's below that, then my QOS would be terrible. If I go too much higher, i don't want to bear the cost of streaming for my service. This is why I'm considering doing one bitrate at my minimal threshold, which also lowers my encoding and storage costs.

Given more clarity on these topics, I can best determine how to proceed.