Azure functions and ASP scaling

Question

Hi, something was confusing me when I tried to scale out my Azure function/ASP.

how does the load balancer work in Azure functions?

I thought, intuitively, jobs would be processed at the service plan's best capacity, however, it didn't happen.

I scaled out the service plan to have 100 CONSTANT instances, and I submitted 100 jobs, each job takes 10 seconds to finish. Supposedly, each instance would process 1 job immediately at the same time, and after 10sec all jobs should be done, however, jobs were processed in waves/batches, and only 30-50ish jobs were processed right away in the first wave/batch, Another 20-30 jobs in the next batch after most of the jobs in first wave finished and another more minor wave or batch after another. So, the whole 100 jobs were done in 4 or 5 waves/batches(a total of 40 to 50 seconds). and when I checked the log, only around 50 instances(unique hostname printed) kicked in for these 100 jobs.

Is there anything I did wrong in any configuration that brought other network limits or calculation throttlehold in this example? Is there a way I can fully use all instances? In advance, Is there a way I can easily have 1000 or even more instances for an Azure function application?

Thanks

Answer 1

@Mu Xu Thanks for reaching out.

When you scale out your Azure Functions, the load balancer distributes incoming requests across all available instances. However, the load balancer doesn't guarantee that each instance will receive an equal number of requests. Instead, it tries to balance the load based on the current capacity of each instance.

In your example, it's possible that the load balancer determined that some instances were already at capacity and couldn't handle any more requests, so it directed new requests to other instances. This could explain why your jobs were processed in waves/batches instead of all at once.

Regarding your question about having 1000 or more instances for an Azure Function application, it's possible to achieve this by using the Consumption plan. The Consumption plan automatically scales out to meet demand, so you don't have to worry about manually scaling up or down. However, keep in mind that there are limits to how many instances can be created in the Consumption plan, and you may need to consider other hosting options if you need to scale beyond those limits.

You can refer to this for Event-driven scaling in Azure Functions.

Let me know if you have any queries or concerns.

Please "Accept Answer" if the answer is helpful so that it can help others in the community.