Azure Data Factory ARM Template parameter limit issue when using CI/CD

Question

I have created an Data Factory in Azure with a about 30+ pipelines in it and they all work fine and I can debug and run them without issue on the Data Factory.I have also setup Azure DevOps integration with the Data Factory with a CI/CD pipeline.

Initially this pipeline worked fine as well and I was able to use the generated ARM template to recreate the Pipelines/Triggers/etc on a production Data Factory instance.

Now I have added some pipelines to the Data Factory and when it generates the ARM template now it creates a template that with 293 parameters, which when it is run through the CI/CD pipeline fails because only 256 parameters are allowed with an ARM template.

So my question is, is there a way to handle this? The things that I can think of doing are creating multiple Data Factories and only having a couple of pipelines in each one. I don't really like this idea because it would become very large and cumbersome over time.

How to get around Azure Data Factory ARM Template parameter limits?

Answer 1

@chandrasekhar munagala Thanks for using Microsoft Q&A !!

There is no direct way of going beyond parameters limit of 256, however, you can reduce the number of parameters by using objects that contains multiple properties. You can refer to these documentations for reference -

• Template Parameters
• ARM template best practices

You can also try using Custom Parameters with the Resource Manager template as described here which helps you to override the default parameterization template. Please let me know if this helps.