I would like to explicitly control the amount of concurrency for each function app to manage down stream resource limits. I am confused at how WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and batch size would work together. To reach a desired concurrency on a function app would the following be correct: Example queues: Option 1: desired concurrency WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT FUNCTIONS_WORKER_PROCESS_COUNT batch size orchestrator app 1 set 1 max instances default 1 set 1 no concurrency process1 app 4 set 4 max instances default 1 set 1 no concurrency process2 app 2 set 2 max instances default 1 set 1 no concurrency Option 2: desired concurrency WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT FUNCTIONS_WORKER_PROCESS_COUNT batch size orchestrator app 1 set 1 max instances default 1 set 1 no concurrency process1 app 4 set 4 max instances default 1 4 process2 app 2 set 2 max instances default 1 2 Something else...

@Nicholas Jalonack , Thanks for reaching out to Q&A. In order to control the concurrency, you can make use of the batchsize to process the number of messages at time for a single VM. Only when you would like to limit the scale out instances of the backend VM ,you can utilize the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT app setting. Please note that, for consumption/Elastic premium functions , increasing the VM instances is not in your control. You can only limit the VM instances, increasing the VM instances is in control of scale controller . When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. So you can set the batchsize value in the host.json and it should allow you to control the concuurency. Concurrency in queue triggers: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue-trigger?tabs=csharp#concurrency I hope this helps! Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.

Azure Functions - WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and batchSize - how can I get the desired concurrency

Accepted answer

MughundhanRaveendran-MSFT 12,436 Reputation points

2022-02-03T05:18:54.037+00:00

@Nicholas Jalonack ,

Thanks for reaching out to Q&A.

In order to control the concurrency, you can make use of the batchsize to process the number of messages at time for a single VM. Only when you would like to limit the scale out instances of the backend VM ,you can utilize the WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT app setting. Please note that, for consumption/Elastic premium functions , increasing the VM instances is not in your control. You can only limit the VM instances, increasing the VM instances is in control of scale controller.

When there are multiple queue messages waiting, the queue trigger retrieves a batch of messages and invokes function instances concurrently to process them. By default, the batch size is 16. So you can set the batchsize value in the host.json and it should allow you to control the concuurency.

Concurrency in queue triggers: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-queue-trigger?tabs=csharp#concurrency

I hope this helps!

Please 'Accept as answer' and ‘Upvote’ if it helped so that it can help others in the community looking for help on similar topics.
Please sign in to rate this answer.
Nicholas Jalonack 61 Reputation points

2022-02-03T14:11:09.227+00:00

Thank you for your excellent response . Two follow up to ensure I understand please confirm the following:

Question 1:
* WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT limit to the number of VMs
* Batch size equates to the number of instances of a function app allowed on VMS
* FUNCTIONS_WORKERS_PROCESS_COUNT the number of process that can run in an instance of a function app

Over all max concurrency would be: VMs * BATCH * FUNCTIONS_WORKERS_PROCESS_COUNT

Question 2:
Note I want each underlying function invocation to have the full CPU and memory allocation so I would leave FUNCTIONS_WORKERS_PROCESS_COUNT left at default 1
So to specifically control max concurrency if I wanted no more than say 4 instances of function app running at one time resulting in no more than 4 function invocations processing at a time. I could achieve this two ways with the function app settings:

WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT set to 1, Batch size set to 4, = 4 concurrent instances running

WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT set to 4, Batch size set to 1 = 4 concurrent instances running

And if so is option 1 or 2 considered preferred over the other (e.g. connection pooling).

Thanks again,
Nick

Nicholas Jalonack 61 Reputation points

2022-02-03T14:14:53.147+00:00

Formatting issue above appeared after submit:
Over all max concurrency would be: VMs (e.g. 3) x BATCH (e.g. 4) x FUNCTIONS_WORKERS_PROCESS_COUNT (e.g. 1) = 3 x 4 x 1 = 12

MughundhanRaveendran-MSFT 12,436 Reputation points

2022-02-04T12:02:52.267+00:00

@Nicholas Jalonack ,

Question-1:

"Batch size equates to the number of instances of a function app allowed on VMS" - No. Batchsize specifies the number of messages that the function needs to process at a time. When we set to 4 and there are 8 messages in the queue, then 4 messages gets processed at a time and then the next 4. Batchsize doesnt have anything to do with VM.

This would make some difference to Q-2 as well

Nicholas Jalonack 61 Reputation points

2022-02-04T13:35:46.517+00:00

Thank you for your response.

Putting it all togeher:

The functions app is the scale unit.
On the consumption plan you get 1 CPU or 100ACU and 1.5GB memory per instance (https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale)

I am planning to group functions in function apps and which to explicitly control the function invocations in order not to exhaust down stream recourses.

WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT * (batchSize + newBatchThreshold) or
I found in documentation that SCALE_OUT is now preferred to WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT.
https://learn.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_max_dynamic_application_scale_out

I have tested the following two scenarios:

I want only 4 function invocations of a function within a function app running at a time in total period.

ONE: 4 instances - scale out set to 4 , batchSize set to 1, and newBatchThreshold set to 0
This worked, although took some time for 4 hosts to be invoked watching the log streaming. But eventually 4 messages were being processed at a time once the 4th host was spun up. *** THIS would mean each function invocation would have the whole 1 CPU and 1.5 GB of memory allocated to the function app instance.

TWO: 1 instance - scale out set to 1 , batchSize set to 4, and newBatchThreshold set to 0
This work, with 4 msgs at a time processing immediately. *** THIS would mean each function invocation would be sharing the 1 CPU and 1.5 GB of memory allocated to the function app instance.

MughundhanRaveendran-MSFT 12,436 Reputation points

2022-02-04T14:27:07.78+00:00

@Nicholas Jalonack , Thanks for summarizing. Your facts and understanding is correct.
Sign in to comment

Share via

Azure Functions - WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT and batchSize - how can I get the desired concurrency

0 additional answers