Seperate repositories for type of ADF pipelines - Recommendation ?

Question

Seperate repositories for type of ADF pipelines - Recommendation ?

sam nick 366

to-date, all pipelines developed have been using ADF without a devops github repository format. Now i need to do that. This is a new concept for me, so i was thinking of having multiple repositories for types of pipelines

ex: All pipelines that load from source A to db is one repository
Another pipeline that has a completely different purpose into a different repository.

Is that approach recommended or just have all pipelines into a single repository ?

AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-17T15:10:33.787+00:00

Hi @sam nick ,
Just checking in to see if the below answers helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-18T05:03:17.813+00:00

Hi @sam nick ,
We haven't heard back from you on the above query. Just checking if you got a chance to see the suggested responses. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-28T14:11:43.617+00:00

Hi @sam nick ,
Just following up to see if the answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

2 answers

Your answer

AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-17T15:10:33.787+00:00

Hi @sam nick ,
Just checking in to see if the below answers helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-18T05:03:17.813+00:00

Hi @sam nick ,
We haven't heard back from you on the above query. Just checking if you got a chance to see the suggested responses. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2022-02-28T14:11:43.617+00:00

Hi @sam nick ,
Just following up to see if the answer helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 1

That's not a recommended approach.

In a data factory, you have 5 pipelines. You can't link these 5 individual pipelines to 5 individual repositories as far as I know.

Ideal way is to create a development data factory, link it to a repository and all the 5 pipelines, related datasets and linked services code gets saved to that single linked repository.

Official document here: https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery#cicd-lifecycle

----------

Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

Answer 2

Hi @sam nick ,

Yes, I think it's good approach to have separate ADFs for different projects/repositories for different set of pipelines. You can also setup a CI/CD Pipeline for your ADF projects. This will help you to manage separate ADF environments (development, test, production). Please refer to this documentation for more details.

Please "Accept the answer" if this helps or let me know more details about your scenario.

More references:
best-practices-to-implement-an-azure-data-factory
azure-data-factory-3-ways-to-integrate-adf-pipeline-with-azure-devops-git

Share via

Seperate repositories for type of ADF pipelines - Recommendation ?

2 answers

Your answer