How to pass parameters from DevOps to Azure Data Factory Pipeline

LoganIN 61 Reputation points
2022-01-01T13:55:11.807+00:00

Hello Everyone,

We have the ADF pipeline where people key in the following parameters values in the Global Parameter section and trigger the pipeline and It's working fine without any issues. But we would like to use DevOps to pass those parameters values instead of directly changing the Global Parameter section.

It would be really helpful if any examples provided on this requirement. Thanks in advance!

Parameters:
AzureStorageAccount
AzureSourceContainerName
AzureSourceFoldername
AzureTargettier
DestinationFolder

FYI, the above global parameters values used in multiple activities like Web, Custom(PowerShell) & Copy data.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,521 questions
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
35,944 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 37,896 Reputation points Microsoft Employee
    2022-01-03T17:12:21.693+00:00

    Hi @LoganIN ,

    Thank you for posting query in Microsoft Q&A Platform.

    Global parameters is ADF are static. We cant supply values to them dynamically.

    If you want to supply values to your parameters dynamically while running pipelines then consider having them as pipeline parameters.

    From DevOps you can deploy your ADF with the help ARM Templates deployment. While deploying itself you can set values to global parameters as well.

    If your ask is not related to deploying ADF from DevOps and looking for something more. Then, please feel free to share more details about what you mean when you say "passing values from DevOps"? How are planning to run pipelines in your case ?

    0 comments
  2. LoganIN 61 Reputation points
    2022-01-04T15:14:08.033+00:00

    Hi @ShaikMaheer-MSFT ,

    Thank you for your response.

    My requirement is to run the ADF pipeline from the Azure DevOps. I did further research and found below blog and was able to trigger the ADF pipeline. However i'm getting an error when the ADF pipeline triggers from DevOps.

    Error details are:
    Operation on target Set-AzBlobRehydration failed: Can not access user batch account, please check batch account settings.

    The Set-AzBlobRehydration is the custom activity which runs the powershell script stored in the Azure Storage account. Looks like the storage account connection string values are getting modified whenever i release the pipeline.

    I tried passing the storage account key by using the override parameter as well but no luck. Any idea how to resolve this issue?

    Thanks in advance!

    https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery