Configuring Azure Data Factory to deploy to dev, ppe, prod environments

Michelle 1 Reputation point
2021-03-22T22:00:27.927+00:00

I've got a data factory that I've set up, but I need to be able to recreate the same data factory with the same pipelines and linked services in different environments. I've connected my Data Factory to an azure devops account, but it's not clear to me how to change things like linked service values (keyvault url, sql server url, etc) depending on which environment I'm trying to deploy to.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,600 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Vaibhav Chaudhari 38,606 Reputation points
    2021-03-23T05:20:15.767+00:00

    Setting up CICD for ADF is little lengthy process.

    You have to link Dev ADF to GIT as you already have done. And create empty UAT & PROD ADF in Data factory mode.

    Do the test changes in Dev ADF in local branch, merge the changes from local to master branch. Open pipeline, select master branch and publish the changes so that these are available in separate adf_publish branch.

    Create a release pipeline and deploy changes from adf_publish branch to UAT & Prod ADF. In release pipeline, you can configure the values in such way that while deploying it will use UAT & PROD specific variables such UAT & PROD SQL server name, or key vault URLs etc.

    Refer these and give it a try -

    https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-deployment
    https://www.youtube.com/watch?v=cLf3nAiGG3Q

    ----------

    Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

    2 people found this answer helpful.