write operations to excel template is consuming tremendous compute using Azure functions

Question

write operations to excel template is consuming tremendous compute using Azure functions

Bala Reddy 0

We have an application that takes input as a data payload(schema in json format) from API to convert the data received to an templated excel file in the backend.

we figured out that the excel generation process is consuming huge(comparatively high amounts of system's CPU and RAM)

This whole execution has been hosted on the Azure functions.

Looking for assistance on how this can be optimized to overcome the high utilization of the compute power.

Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-03-31T12:01:26.5833333+00:00
Hello @Bala Reddy,

Could you please share more details to understand the issue better.

Is your function app hosted on consumption plan, elastic or dedicated plan ?

Which nugget packages and which methods you are trying to insert the data to the excel file?

As you mentioned you have observed high CPU while performing some operation could you share more insights on that what exactly you are doing?

Are you seeing high CPU for every invocation or only for specific invocations and what is the size of you Json payload that you are getting another system?

Also, are you following any documentation steps? if yes, could you please share any.
Bala Reddy 0 Reputation points

2025-03-31T17:22:36.11+00:00
Hi @Ranashekar Guda Thanks for responding, please find my responses inline.

Is your function app hosted on consumption plan, elastic or dedicated plan ?

dedicated plan as we already knew that consumption plan will not suffice the compute needs of this specific execution

Which nugget packages and which methods you are trying to insert the data to the excel file?

post method is used and we tried to make the payloads(json/simple text) as simple as possible with 200 records chunks of data. Our goal is to send huge amount of data via api calls and json payloads

As you mentioned you have observed high CPU while performing some operation could you share more insights on that what exactly you are doing?

every time we exceed certain amounts of data payloads received via api operation on the application end and while writing the data to excel template(unfortunately, our excel template is with heavy validations)

Are you seeing high CPU for every invocation or only for specific invocations and what is the size of you Json payload that you are getting another system?

it's straightforward and as the payload with too much data or too many payloads received at the excel generation end makes it overloading on the compute part of the ASP attached.

we already have isolated the excel generation piece from the data receiving part of the application

I hope below diagram helps

Also, are you following any documentation steps? if yes, could you please share any.

I am not sure what are you referring to, it's fresh application designed to serve the mentioned flow as explained in above diagram, have following the necessary best practices recommended by Azure while using Azure services.

issue is exhausting the ASP compute(we tested with p3v3 plan, ASP's cpu and memory spikes really quick and consumes maximum number of instances and autoscaling also not helping as spike is really quick and application crashes) allocated to the function app in the excel generation part when receiving 1k number of chunks of 200 records of json payload each.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2025-04-01T12:57:33.1133333+00:00
@Bala Reddy ,

Based on what you've shared, it seems that the heavy Excel template and the 1,000 chunks of 200 records are pushing your P3V3 plan to its limit. Here are some suggestions to lighten the load:

Stream the data into the Excel file using something like NPOI and save it directly to Blob Storage to avoid holding everything in memory.

Reduce the number of validations in the template or handle them elsewhere to save CPU resources.

Process smaller batches, such as 10 chunks at a time, in parallel to improve performance.

Consider using Durable Functions to split the work into steps and avoid spikes.

If issues persist, consider upgrading to a higher tier or adjusting the number of instances running simultaneously.

Try these suggestions with a small test to see if they help reduce crashes.
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-02T12:02:38.5833333+00:00

Hi @Bala Reddy,
Just checking in to see if the below answer provided by @Khadeer Ali helped.
Bala Reddy 0 Reputation points

2025-04-02T14:00:52.7766667+00:00

Hi @Ranashekar Guda & @Khadeer Ali I appreciate your help and recommendations on this topic, we have some of the above Ali mentioned recommendations already on our list to research, develop and test, however, it would be helpful if you could point us to any articles and/or public git repositories for efficient handling of the excel operations with combination of Azure services like eventhubs/eventgrids/functions development to deal with the complexity of the goal we are trying to achieve
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-03T11:11:40.66+00:00

Hello @Bala Reddy,

Thank you for your detailed response. I understand you're looking for more resources to help efficiently handle the Excel operations while integrating with Azure services like Event Hubs, Event Grids, and Azure Functions. Below are some helpful links and GitHub repositories that might assist you in optimizing the solution.

https://github.com/Azure/azure-quickstart-templates

https://github.com/EPPlusSoftware/EPPlus

https://github.com/Azure/azure-functions-durable-extension

https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.eventhub
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-04T10:33:03.2833333+00:00

Hi @Bala Reddy,

Just checking in to see if above information was helpful. If you have any further updates on this issue, please feel free to post back.
Bala Reddy 0 Reputation points

2025-04-10T16:12:39.9766667+00:00

Thanks @Ranashekar Guda & @Khadeer Ali for chiming in and assisting, we are currently evaluating different options to optimize the overall application performance and including few of the recommendations you did above. Have a wonderful time ahead...!
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-10T19:30:36.41+00:00

Hello @Bala Reddy,

Thank you for the update! We're glad to hear that the suggestions were helpful and feel free to reach out if you need any further assistance.

Your answer

Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-03-31T12:01:26.5833333+00:00

Hello @Bala Reddy,

Could you please share more details to understand the issue better.

Is your function app hosted on consumption plan, elastic or dedicated plan ?

Which nugget packages and which methods you are trying to insert the data to the excel file?

As you mentioned you have observed high CPU while performing some operation could you share more insights on that what exactly you are doing?

Are you seeing high CPU for every invocation or only for specific invocations and what is the size of you Json payload that you are getting another system?

Also, are you following any documentation steps? if yes, could you please share any.
Khadeer Ali 5,990 Reputation points Microsoft External Staff Moderator

2025-04-01T12:57:33.1133333+00:00

@Bala Reddy ,

Based on what you've shared, it seems that the heavy Excel template and the 1,000 chunks of 200 records are pushing your P3V3 plan to its limit. Here are some suggestions to lighten the load:

Stream the data into the Excel file using something like NPOI and save it directly to Blob Storage to avoid holding everything in memory.

Reduce the number of validations in the template or handle them elsewhere to save CPU resources.

Process smaller batches, such as 10 chunks at a time, in parallel to improve performance.

Consider using Durable Functions to split the work into steps and avoid spikes.

If issues persist, consider upgrading to a higher tier or adjusting the number of instances running simultaneously.

Try these suggestions with a small test to see if they help reduce crashes.
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-02T12:02:38.5833333+00:00

Hi @Bala Reddy,
Just checking in to see if the below answer provided by @Khadeer Ali helped.
Bala Reddy 0 Reputation points

2025-04-02T14:00:52.7766667+00:00

Hi @Ranashekar Guda & @Khadeer Ali I appreciate your help and recommendations on this topic, we have some of the above Ali mentioned recommendations already on our list to research, develop and test, however, it would be helpful if you could point us to any articles and/or public git repositories for efficient handling of the excel operations with combination of Azure services like eventhubs/eventgrids/functions development to deal with the complexity of the goal we are trying to achieve
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-03T11:11:40.66+00:00

Hello @Bala Reddy,

Thank you for your detailed response. I understand you're looking for more resources to help efficiently handle the Excel operations while integrating with Azure services like Event Hubs, Event Grids, and Azure Functions. Below are some helpful links and GitHub repositories that might assist you in optimizing the solution.

https://github.com/Azure/azure-quickstart-templates

https://github.com/EPPlusSoftware/EPPlus

https://github.com/Azure/azure-functions-durable-extension

https://github.com/Azure/azure-quickstart-templates/tree/master/quickstarts/microsoft.eventhub
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-04T10:33:03.2833333+00:00

Hi @Bala Reddy,

Just checking in to see if above information was helpful. If you have any further updates on this issue, please feel free to post back.
Bala Reddy 0 Reputation points

2025-04-10T16:12:39.9766667+00:00

Thanks @Ranashekar Guda & @Khadeer Ali for chiming in and assisting, we are currently evaluating different options to optimize the overall application performance and including few of the recommendations you did above. Have a wonderful time ahead...!
Ranashekar Guda 2,820 Reputation points Microsoft External Staff Moderator

2025-04-10T19:30:36.41+00:00

Hello @Bala Reddy,

Thank you for the update! We're glad to hear that the suggestions were helpful and feel free to reach out if you need any further assistance.

Share via

write operations to excel template is consuming tremendous compute using Azure functions

Your answer