Git repository - Azure Data Factory

Question

Hi,

Currently in our azure data factory, we are having three branches, one for dev, one for production working branch and final collaboration branch is production branch. Now we had requirement to add additional pipelines. So after developing in my dev branch, can I push my newly created pipelines and its resources to prod working branch via pull request or whether it will create any merge conflicts

Answer 1

Hi @Anonymous ,
Thanks for using Microsoft Q&A platform.
When a Pull request is raised from any working branch (also known as feature branch) , once it is approved and merged, the changes will only be reflected in the main branch (also known as collaboration branch) .

• If you want to push the newly created pipeline from one feature branch(dev working) to another feature branch(prod working), you can download the support files of the pipeline in your local system.
• Go to Git repository , select the Prod working branch and upload the unzipped json file . Make sure your pipeline json file should be uploaded in pipeline folder.

In your case, main branch is working as Production environment. The issue with this approach is that if you want to test end to end flow using triggers in dev then it won't be possible as every time you raise a PR, pipelines will be merged with Production environment.

The best practice is to create different ADF for different environment and push your fully tested code from Dev to higher environments using Release pipelines.

Here are few documents you can refer to :
https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery
https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-automate-azure-pipelines

Refer to the below video for the end to end process of how to implement CICD for ADF pipelines using Azure DevOps.
https://www.youtube.com/watch?v=jJcikWOUqOk