ARM Template limitation of maximum 4MB size

Question

Hi Team ,

We have implemented CI/CD process for Azure data factory using the recommended approach(https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-improvements)

However in few of the project , due to large number of data factory assets , we get ARM templates in adf_publish branch which is more than 4MB of size and unable to deploy the changes in higher environment due to these size limitation with ARM template

Currently we see some new approach defined in the above link for CI/CD approach to use cetain npm utilities to generate the ARM template

So my question is whether any fix applied using new utilities so it don't go beyond 4MB in size

Appreciate your quick help

Answer 1

Hi @MB, Sasekumar ,

After doing and additional research and having conversation with internal team, looks like ADF Product team has published a new feature to overcome this ARM template limitation. To accommodate large factories while generating the full Resource Manager template for a factory, Data Factory now generates linked Resource Manager templates. With this feature, the entire factory payload is broken down into several files so that you aren't constrained by the limits.

To use linked templates instead of the full Resource Manager template, update your CI/CD task to point to ArmTemplate_master.json instead of ArmTemplateForFactory.json (the full Resource Manager template). Resource Manager also requires that you upload the linked templates into a storage account so Azure can access them during deployment

For more info please refer to this doc - Linked Resource Manager templates with CI/CD

Hope this info helps.

----------

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators