Feature Request: prevent global parameters in git collaboration branch from being published to live mode

Question

Our global parameters between differents ADF environments are managed using Terraform. As such, we'd like to prevent (using a checkbox) that the global parameters present in the git collaboration branch would make it to the live mode when the Publish button is pressed, hence preventing global parameters (managed by Terraform) from getting overwritten.

Managing Global Parameters with Terraform is very convenient, because it is a perfect tool to handle environment specific settings (address of the key vault, Databricks workspace URL, Databricks cluster ID, ...). But at the moment, when a new global parameter is set with Terraform (directly in live mode), Data Factory detects a discrepancy between the git collaboration branch and the live mode when publishing, and overwrites the parameters managed by Terraform.

The alternative is to handle all the global parameters in all environments manually, but this process is error-prone and lacks automation.

Also note that the current "Include in ARM template" checkbox has nothing to do with this feature request (as already clarified with Azure support), as it aims to prevent deploying the global parameters to the next environment when using a CICD pipeline to promote ADF content, but not to prevent global parameters in the git collaboration branch from getting published to live mode.

Answer 1

Hi @Michael Van de Borne ,
Welcome to Microsoft Q&A platform and thanks for posting your query.
As I understand your ask here, you are trying to find a workaround to deploy global parameters in ADF from dev environment to higher environments.

You can leverage Powershell script to achieve this requirement. If you keep the 'Include in ARM template' box unchecked and publish the ADF changes, or export an ARM template with global parameters, a folder called globalParameters is created with a file called your-factory-name_GlobalParameters.json.

You need to download this .json file in the local and provide the same while running the below script . Provide details like subscription id, and adf name of the higher environment and you should see the Global parameters reflecting in higher env.

Kindly check this doc for more details: Deploying using PowerShell

param  
(  
    [parameter(Mandatory = $true)] [String] $globalParametersFilePath,  
    [parameter(Mandatory = $true)] [String] $resourceGroupName,  
    [parameter(Mandatory = $true)] [String] $dataFactoryName  
)  
  
Import-Module Az.DataFactory  
  
$newGlobalParameters = New-Object 'system.collections.generic.dictionary[string,Microsoft.Azure.Management.DataFactory.Models.GlobalParameterSpecification]'  
  
Write-Host "Getting global parameters JSON from: " $globalParametersFilePath  
$globalParametersJson = Get-Content $globalParametersFilePath  
  
Write-Host "Parsing JSON..."  
$globalParametersObject = [Newtonsoft.Json.Linq.JObject]::Parse($globalParametersJson)  
  
# $gp in $factoryFileObject.properties.globalParameters.GetEnumerator())   
# may  be used in case you use non-standard location for global parameters. It is not recommended.   
foreach ($gp in $globalParametersObject.GetEnumerator()) {  
    Write-Host "Adding global parameter:" $gp.Key  
    $globalParameterValue = $gp.Value.ToObject([Microsoft.Azure.Management.DataFactory.Models.GlobalParameterSpecification])  
    $newGlobalParameters.Add($gp.Key, $globalParameterValue)  
}  
  
$dataFactory = Get-AzDataFactoryV2 -ResourceGroupName $resourceGroupName -Name $dataFactoryName  
$dataFactory.GlobalParameters = $newGlobalParameters  
  
Write-Host "Updating" $newGlobalParameters.Count "global parameters."  
  
Set-AzDataFactoryV2 -InputObject $dataFactory -Force  

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators