Azure Data Factory ARM Template parameter limit issue when using CI/CD

chandrasekhar munagala 21 Reputation points

I have created an Data Factory in Azure with a about 30+ pipelines in it and they all work fine and I can debug and run them without issue on the Data Factory.I have also setup Azure DevOps integration with the Data Factory with a CI/CD pipeline.

Initially this pipeline worked fine as well and I was able to use the generated ARM template to recreate the Pipelines/Triggers/etc on a production Data Factory instance.

Now I have added some pipelines to the Data Factory and when it generates the ARM template now it creates a template that with 293 parameters, which when it is run through the CI/CD pipeline fails because only 256 parameters are allowed with an ARM template.

So my question is, is there a way to handle this? The things that I can think of doing are creating multiple Data Factories and only having a couple of pipelines in each one. I don't really like this idea because it would become very large and cumbersome over time.

How to get around Azure Data Factory ARM Template parameter limits?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,726 questions
No comments
{count} votes

Accepted answer
  1. Saurabh Sharma 17,331 Reputation points Microsoft Employee

    @chandrasekhar munagala Thanks for using Microsoft Q&A !!

    There is no direct way of going beyond parameters limit of 256, however, you can reduce the number of parameters by using objects that contains multiple properties. You can refer to these documentations for reference -

    You can also try using Custom Parameters with the Resource Manager template as described here which helps you to override the default parameterization template. Please let me know if this helps.

0 additional answers

Sort by: Most helpful