Azure Functions - terms and concept clarifications from documentation and QandA requested

Question

In the documentation and Q and A certain terms are used for Azure functions some of which I beleive functions should be abstracting us from. I come from a VM world and dipping my toe into the function world and want to make sure I am understanding what I am reading.

https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale
Function app is the scaling unit. It can have one or more functions. On the consumption plan each Instance of a function app will get 100 ACU (1CPU) and 1.5GB memory.

Terms I want to sure I understand consistently how they are being used:

1. Instance - I have taken this to mean a single instance of Function app with its own allocated Compute and Memory. Please confirm or explain if I have misunderstood.

Example: https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale
"Event driven. Scale out automatically, even during periods of high load. Azure Functions infrastructure scales CPU and memory resources by adding additional instances of the Functions host, based on the number of incoming trigger events."

2A. VM/HOST - I have seen VM and HOST throw around quite a lot I was expecting only see Instance of function app. We are not in control VM's nor do we want to be if we are using functions. We can limit the instances through SCALE_OUT. I have taken VM and HOST to mean the underlying VM the function app instance or instances are spun up. Please confirm or explain if I have misunderstood.

2B. I believed multiple functions app instances each with their own compute and memory can reside on a single underlying VM. Please confirm or explain if I have misunderstood. If it is one to one does that mean that VM/HOST/Function APP instance are equivalent?

Example: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue-trigger?tabs=csharp#concurrency
"When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. When the number being processed gets down to 8, the runtime gets another batch and starts processing those messages. So the maximum number of concurrent messages being processed per function on one virtual machine (VM) is 24. This limit applies separately to each queue-triggered function on each VM. If your function app scales out to multiple VMs, each VM will wait for triggers and attempt to run functions. For example, if a function app scales out to 3 VMs, the default maximum number of concurrent instances of one queue-triggered function is 72."

3. For queues Batchsize + newBatchThreshold = the max number of messages that the function can process at one time. From testing I see a scale out instance limited function app set to 1 (https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_max_dynamic_application_scale_out) still processing multiple messages concurrently when BatchSice + newBatchThreshold > 1. So some sort of under the hood resource sharing is going on. But at the same time the FUNCTIONS_WORKER_PROCESS_COUNT is defaulted to 1 (https://learn.microsoft.com/en-us/azure/azure-functions/functions-best-practices?tabs=csharp). I would have thought FUNCTIONS_WORKER_PROCESS_COUNT would have been needed to be set to greater than 1 to have more than one message processing at once within a single instance.

Thank you for time,
Nick

Answer 1

@Nicholas Jalonack ,

As you have mentioned in the comments, Host/Instance refer to a Functions host with its own compute and memory corresponding to one instance of the function app.

To answer the question 2B, one to one means one function host is running on one underlying VM. Please note that, the compute resources such as CPU and memory is confined to the underlying VM and the function host running on that VM will utilize the resources. W3wp.exe that runs the function host can be increased using the FUNCTIONS_WORKER_PROCESS_COUNT app setting.

Answering question 3, when WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT is set to 1, the function app will not scale out more than 1, it will always remain running on 1 VM even when there is high load. When FUNCTIONS_WORKER_PROCESS_COUNT is set to 1, only one worker process (w3wp.exe) will run in the underlying VM. So when both the app settings are set to 1, it means one worker process running in one VM.

I hope this helps!

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

Answer 2

We now know how OS processes (W3wp.exe) map to VMs. The final missing piece is how threads map to the process. I'm assuming that, behind the scenes, the function app running in the W32p.exe process is effectively a regular IIS app. So is there a thread pool assigned to this process so that the process can concurrently run functions within the same function app? If so, what is the size of the thread pool? Does the thread pool vary, based on type of function? Unfortunately, despite this capability being described as "serverless", I need to understand in detail how functions scale - from single process to multiple process to multiple VMs. According to the 12-factor guidance, cloud-native apps scale at the process-level (like old CGI web apps), but windows has always scaled within the process initially, via threads. I need to know at what point a process will max out it's thread pool and move to a new VM? Like the author of this question, I find the use of language that describes the scaling and pricing somewhat vague. So, say I have both FUNCTIONS_WORKER_PROCESS_COUNT and WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT set to 2. Both processes will be spawned on VM1 - spun up every 10 secs until reach FUNCTIONS_WORKER_PROCESS_COUNT. Requests are sent evenly to both processes. At what point do those two processes max out (ie number of active threads), triggering VM2? Is it the case that it might not exhaust threads at all and it is simply the planned CPU and or memory limits on a single VM that triggers scaling to an additional VM?