Azure Devops - ARM deployment of Azure data factory

Question

Azure Devops - ARM deployment of Azure data factory

J, Ezhilarason (Cognizant) 0

I have a requirement to move few of the pipelines and its dependency to another ADF using ARM template. There are around 80 pipelines in the ADF i need to move 25 from it . I exported the ARM template & picked the required pipelines and created the template json , the json seems to be having issues when running the Devops pipline (ARM deployment).As it has large json files i'm not able to figure out the errors. is there a better approach to move the pipelines or is there a way to find the issues in ARM json files. Devops pipeline doesn't recognize the ARM file as a valid template file

RevelinoB 3,675 Reputation points

2023-04-25T07:20:09.1833333+00:00

Hi J, Ezhilarason. I understand that you are trying to move 25 out of 80 pipelines from one ADF to another ADF using ARM template, but you are facing some issues while running the DevOps pipeline for ARM deployment. Since the JSON file is quite large, you are finding it difficult to identify the errors. To tackle this issue, you could try breaking down the large JSON file into smaller ones and then running the DevOps pipeline. You could also use ARM template validator or other JSON validation tools to validate the JSON file and fix the errors. Additionally, you could update the JSON file to comply with the ARM template schema and try running the DevOps pipeline again. Another option could be to manually copy and paste the required pipelines and their dependencies from one ADF to another using Azure Data Factory's native copy and paste functionality. This approach may be more time-consuming but could be a simpler solution, especially if the JSON file is proving difficult to work with. I hope this helps, and if you have any further queries, please feel free to ask?
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-05-03T22:56:20.34+00:00

Hello J, Ezhilarason (Cognizant),

I am checking to see if you got a chance to look into my earlier response. Please let me know if you have any further questions.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-05-04T23:24:09.0766667+00:00

Hello J, Ezhilarason (Cognizant),

I am checking to see if you have any further questions here.

1 answer

Your answer

RevelinoB 3,675 Reputation points

2023-04-25T07:20:09.1833333+00:00

Hi J, Ezhilarason. I understand that you are trying to move 25 out of 80 pipelines from one ADF to another ADF using ARM template, but you are facing some issues while running the DevOps pipeline for ARM deployment. Since the JSON file is quite large, you are finding it difficult to identify the errors. To tackle this issue, you could try breaking down the large JSON file into smaller ones and then running the DevOps pipeline. You could also use ARM template validator or other JSON validation tools to validate the JSON file and fix the errors. Additionally, you could update the JSON file to comply with the ARM template schema and try running the DevOps pipeline again. Another option could be to manually copy and paste the required pipelines and their dependencies from one ADF to another using Azure Data Factory's native copy and paste functionality. This approach may be more time-consuming but could be a simpler solution, especially if the JSON file is proving difficult to work with. I hope this helps, and if you have any further queries, please feel free to ask?
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-05-03T22:56:20.34+00:00

Hello J, Ezhilarason (Cognizant),

I am checking to see if you got a chance to look into my earlier response. Please let me know if you have any further questions.
Bhargava-MSFT 31,261 Reputation points Microsoft Employee Moderator

2023-05-04T23:24:09.0766667+00:00

Hello J, Ezhilarason (Cognizant),

I am checking to see if you have any further questions here.

Answer 1

Hello J, Ezhilarason (Cognizant),

To move pipelines and their dependencies to another ADF using ARM templates, you can use the Export ARM template feature in Azure Data Factory to export the ARM template for the entire ADF

Then, you can modify the ARM template to include only the pipelines and their dependencies that you want to move<sup>.</sup>

If you are having issues with the ARM template, you can use the Azure Resource Manager Template Toolkit to validate and troubleshoot the template<sup>.</sup>

https://learn.microsoft.com/en-us/azure/azure-resource-manager/templates/test-toolkit

Alternatively, you can validate the ARM template using Azure CLI

Ex:

az deployment group validate --resource-group testrg --template-file {template-file}

https://learn.microsoft.com/en-us/cli/azure/deployment/group?view=azure-cli-latest#az-deployment-group-validate

This command will help you identify any issues in the ARM template before deploying it.

Once you have a valid ARM template, you can deploy it using your DevOps pipeline.

I hope this helps. Please let us know if you have any further questions.

Share via

Azure Devops - ARM deployment of Azure data factory

1 answer

Your answer