About Azure Functions: quotas & limits

Jona 475

Hi,

According to the docs::

https://learn.microsoft.com/en-us/azure/azure-functions/functions-scale#service-limits

Azure Functions instance have up to 1,5GB RAM.

I've been using a function deployed on Azure that read a 100MB pandas dataframe and partition it in almost 40 smaller dataframes. The function sometimes finishes suddenly and others do its job, even when it takes long

I've test locally reading the same dataframe and perform the same task, and my laptop memory spin up to 21GB-25GB while splitting, using thins function.

I could now understand why the function finishes suddenly, let's say it works half and half.

¿Why doesn't the Function finishe all the execution, given that I know now that the operations on that dataframe goes beyong the 1,5GB RAM on Consumption Plan? I can't understand while the Function executes fine (even when it takes very long) given that I surpass its RAM limit.

How can I monitor the memory usage of a Function? is that posible?

Can somebody give me a hand to understand this?

Regards

Vinodh247 23,761 Reputation points MVP

2024-07-11T09:11:44.8866667+00:00
The Consumption plan scales instances dynamically based on the number of incoming events. This means that if your function is triggered frequently, more instances will be added to handle the load, which can help in completing the execution. However, if the function is triggered less frequently, the instances might scale down, leading to incomplete executions. The dynamic scaling and cold start behavior of the Consumption plan can influence the execution outcomes, leading to inconsistent performance.

Switch to Premium Plan or Dedicated (App Service) Plan: These plans offer more memory (up to 14GB in the Premium plan) and might better suit your workload.

Dedicated Plan: If you need consistent performance and higher memory limits, consider using the Dedicated (App Service) Plan.

How can I monitor the memory usage of a Function? is that posible?

Yes. using application insights or monitor metrics.
Jona 475 Reputation points

2024-07-11T16:28:37.1633333+00:00

Hi @Vinodh247

The Consumption plan scales instances dynamically based on the number of incoming events. This means that if your function is triggered frequently, more instances will be added to handle the load, which can help in completing the execution

So if my function is blob-based, if I upload 100 files, there will be 100 functions executing? I ask this, since Consumption Plan has limit on linux machine of 100 instances.

Regards
Vinodh247 23,761 Reputation points MVP

2024-07-12T14:43:46.0133333+00:00

Jona: I have converted the comment as answer below. Please go ahead and 'Upvote'(Thumbs-up) and 'Accept' as an answer if the answer was helpful. This will also help us close this thread and acknowledge the time spent by community volunteers like us.
Jona 475 Reputation points

2024-07-12T15:50:48.02+00:00

Hi @Vinodh247

Yes, that's correct. The Consumption plan has a limit of 100 instances per Linux Function App, as you mentioned. If you upload 100 files to the container, the consumption plan will scale out to create up to 100 concurrent instances of your Function to handle the 100 events. This means that if you upload more than 100 files in a short period, the remaining triggers will be queued and processed once the instances become available. So, if consumption plan is not able to scale out further, some of the events may not be processed immediately.

This is the correct answer for me. Pls, turn it into a asnwer to make it acceptable.

Regards

Accepted answer

Vinodh247 23,761 Reputation points MVP

2024-07-12T14:42:42.5633333+00:00

Yes, that's correct. The Consumption plan has a limit of 100 instances per Linux Function App, as you mentioned. If you upload 100 files to the container, the consumption plan will scale out to create up to 100 concurrent instances of your Function to handle the 100 events. This means that if you upload more than 100 files in a short period, the remaining triggers will be queued and processed once the instances become available. So, if consumption plan is not able to scale out further, some of the events may not be processed immediately.
Please sign in to rate this answer.
Jona 475 Reputation points

2024-07-12T18:38:16.2266667+00:00

Hi @Vinodh247

I forgot a couple of question, before marking the answer as acceptable:

I have 4 Azure Functions Apps running on a Linux machine and each has 5 functions. ¿Each Function Apps can scale to 100 instance, considering all its functions?, or ¿Every function inside of a Azure Functions Apps can scale to 100 independently? If the second question is right, then each of my Azure Functions Apps can have up to 500 running functions.
In the other hand, if the first question is correct, the limit of 100 running functions takes into account all the functions inside an Azure Function Apps

¿There is another limit, like the quantity of Azure Function per suscription or something like that?

Is there a metric to monitor que quantity of running functions?

I've read the docs, but I could't find the proper info.

Regards

Vinodh247 23,761 Reputation points MVP

2024-07-14T01:31:51.63+00:00

Jona:

I have 4 Azure Functions Apps running on a Linux machine and each has 5 functions. ¿Each Function Apps can scale to 100 instance, considering all its functions?, or ¿Every function inside of a Azure Functions Apps can scale to 100 independently? If the second question is right, then each of my Azure Functions Apps can have up to 500 running functions. In the other hand, if the first question is correct, the limit of 100 running functions takes into account all the functions inside an Azure Function Apps

Each Function App can scale up to 100 instances. This scaling applies to the entire Function App, meaning all the functions within a single Function App share these 100 instances. This means the total maximum number of running functions across your 4 apps is 4 * 100 = 400, not 4 * 5 * 100 = 2000.

There is another limit, like the quantity of Azure Function per subscription or something like that, is there a metric to monitor que quantity of running functions

This can vary depending on the subscription type and the plan (Consumption Plan, Premium Plan, Dedicated (App Service) Plan). But across all plans, there’s a hard limit of 5000 Function Apps per subscription. This means that regardless of the plan, you can’t exceed this total number of Function Apps in your subscription.

Is there a metric to monitor que quantity of running functions?

Azure Monitor: For monitoring various metrics related to Azure Functions like execution count, execution time, and failure count.

Application Insights: Integrated with Azure Functions and provides detailed telemetry data. You can set up Application Insights to monitor performance, failures, and usage.

Azure Metrics: For monitoring execution counts, errors, exec units and duration etc.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

About Azure Functions: quotas & limits

0 additional answers

Your answer