Exporting Selected Azure Data Factory Pipelines

Zouraiz Butt 0 Reputation points


I have a question, I have Data Factory which has almost 850 pipelines.

Out of 850 pipelines. 250 of them are created using as Back up or created for versioning of pipelines.

now i want to take backup of those pipelines and want to store their ARM template in backup folder and delete them for ADF
one solution I have in mind is that i create a pipeline and put all 250 pipelines in a newly created Pipeline and then export it. but that is a very tedious work.

how can i export them by Selecting those pipelines and creating their arm template and then by selecting them to delete them in one click

can anyone help me out on this ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,948 questions
{count} votes

1 answer

Sort by: Most helpful
  1. BhargavaGunnam-MSFT 28,271 Reputation points Microsoft Employee

    Hello Zouraiz Butt,

    As Samy mentioned, you can use the PowerShell script to export pipelines as required by you programmatically.

    Please follow the steps below.

    • Use PowerShell to get a list of all the pipelines in your Data Factory.
    • Filter the list to include only the pipelines you want to export and delete.
    • For each pipeline in the filtered list, use PowerShell to export the ARM template to a backup folder.
    • Once all the ARM templates have been exported, use PowerShell to delete the pipelines from your Data Factory.
    # Connect to your Azure account
    $factoryName = "MyDataFactory"
    # Get a list of all the pipelines in your Data Factory
    $pipelines = Get-AzDataFactoryV2Pipeline -DataFactoryName $factoryName
    # Filter the list to include only the pipelines you want to export and delete
    $backupPipelines = $pipelines | Where-Object { $_.Name -like "*" }
    # Export the ARM templates for each pipeline in the filtered list
    foreach ($pipeline in $backupPipelines) {
        $template = Get-AzDataFactoryV2Pipeline -DataFactoryName $factoryName -ResourceGroupName $pipeline.ResourceGroupName -Name $pipeline.Name | ConvertTo-Json -Depth 100
        $template | Out-File "C:\Backup\$($pipeline.Name).json"
    foreach ($pipeline in $backupPipelines) {
        Remove-AzDataFactoryV2Pipeline -DataFactoryName $factoryName -ResourceGroupName $pipeline.ResourceGroupName -Name $pipeline.Name

    This script gets a list of all the pipelines in your Data Factory

    You can filter the list of pipelines Where-Object { $_.Name -like "pipeline1" } section

    So this will export pipeline1 to C:\backup folder '

    Remove-AzDataFactoryV2Pipeline removes the pipeline1 from the datafactory.

    I have executed the script and successfully delete the pipeline1 from my data factory. Please test the script and let me know if you run into any issues.

    User's image

    I hope this helps.

    If this answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.