How to Override Linked Service Parameters (e.g., Databricks domain) in Azure Synapse CI/CD YAML Pipeline?

Question

How to Override Linked Service Parameters (e.g., Databricks domain) in Azure Synapse CI/CD YAML Pipeline?

Napsty 65

Hello,

I’m currently working on a CI/CD pipeline for Azure Synapse, integrated with Azure DevOps as the source repository. I’ve successfully built and tested the entire deployment pipeline between the Development and QA Synapse workspaces. Everything works as expected when deploying artifacts across environments.

Here is a visual overview of my CI/CD system:

User's image

Since the official documentation for YAML-based Synapse deployments is limited (most resources reference the classic Release Pipeline mechanism, which is being deprecated), I decided to implement a YAML-based deployment using the following configuration:

Current CD pipeline:

pool:
  vmImage: 'windows-latest'

stages:
- stage: Deploy
  displayName: 'Deploy Synapse Workspace'
  jobs:
  - deployment: DeployWorkspace
    environment: 'QA'
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self
          - task: Synapse workspace deployment@2
            inputs:
                operation: 'deploy'
                TemplateFile: 'jk-napsty-synapse-dev/TemplateForWorkspace.json'
                ParametersFile: 'jk-napsty-synapse-dev/TemplateParametersForWorkspace.json'
                azureSubscription: 'Azure subscription 1(mySuscriptionID)'
                ResourceGroupName: 'synapse-rg-qa'
                TargetWorkspaceName: 'jk-napsty-synapse-qa'
                DeleteArtifactsNotInTemplate: false
                DeployManagedPrivateEndpoints: false
                OverrideArmParameters: '-workspaceName jk-napsty-synapse-qa -AzureDatabricks1_accessToken abc'
                FailOnMissingOverrides: false
                Environment: 'prod'
                npmpackage: 'prod'

My Issue:

I’d like to override specific parameters in a Linked Service, especially for environment-specific values such as the Databricks domain URL. Here’s an example of how a Databricks linked service is defined:

{
    "name": "AzureDatabricks1",
    "type": "Microsoft.Synapse/workspaces/linkedservices",
    "properties": {
        "annotations": [],
        "type": "AzureDatabricks",
        "typeProperties": {
            "domain": "https://adb-4075628429852041.1.azuredatabricks.net",
            "newClusterNodeType": "Standard_DS3_v2",
            "newClusterNumOfWorker": "1",
            "newClusterVersion": "10.4.x-scala2.12",
            "clusterOption": "Fixed",
            "newClusterInitScripts": [],
            "encryptedCredential": "ew0KICAiVmVyc2lvbiI6ICIyMDE3LTExLTMwIiwNCiAgIlByb3RlY3Rpb25Nb2RlIjogIktleSIsDQogICJTZWNyZXRDb250ZW50VHlwZSI6ICJQbGFpbnRleHQiLA0KICAiQ3JlZGVudGlhbElkIjogIlNZTkFQU0VANzEwNzg5QjQtRjA0Qi00OTk5LUEwNDYtRjgwQzYzNDY4RTREXzYwNTI1YTg5LTY3MmItNDM4Ni1hNTY4LWFlZjhjZDM4YWY5YSINCn0="
        },
        "connectVia": {
            "referenceName": "AutoResolveIntegrationRuntime",
            "type": "IntegrationRuntimeReference"
        }
    }
}

However, in the exported TemplateParametersForWorkspace.json, I do not see the domain or other properties listed, only the accessToken:

"parameters": {
  "AzureDatabricks1_accessToken": {
    "value": ""
  }
}

Even though I use OverrideArmParameters to pass a new access token (as shown in the YAML), the value is not getting deployed as expected.

Questions:

How can I override additional linked service properties like the Databricks domain URL via the YAML pipeline?
Is there a way to expose more parameters (such as domain, newClusterVersion, etc.) from the Linked Service into the TemplateParametersForWorkspace.json so I can override them at deployment time?
What is the recommended approach to parameterize environment-specific values in Linked Services when using YAML CI/CD pipelines for Synapse?

Thank you in advance for your help! Any guidance or examples would be greatly appreciated.

Venkat Reddy Navari 5,840 Reputation points Microsoft External Staff Moderator

2025-07-04T12:05:27.6466667+00:00

Napsty We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

1 answer

Your answer

Venkat Reddy Navari 5,840 Reputation points Microsoft External Staff Moderator

2025-07-04T12:05:27.6466667+00:00

Napsty We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Answer 1

Hello !

Thank you for posting on Microsoft Learn.

When Synapse exports the ARM templates using the publish process, it includes only certain linked service parameters, like accessToken, by default in TemplateParametersForWorkspace.json. Unfortunately, other properties (e.g., domain, newClusterVersion) are hardcoded in the template and not parameterized, unless you manually adjust the template.

You need to manually edit TemplateForWorkspace.json and TemplateParametersForWorkspace.json to add the additional parameters you want to override (like domain).

Find the Linked Service definition (AzureDatabricks1) and change this section:

"domain": "https://adb-4075628429852041.1.azuredatabricks.net",

To:

"domain": "[parameters('AzureDatabricks1_domain')]",

Add a new parameter definition like:

"AzureDatabricks1_domain": {
  "type": "string",
  "defaultValue": "https://adb-4075628429852041.1.azuredatabricks.net"
}

Now you can override it using OverrideArmParameters in your YAML.

Then you need to update your YAML deployment step like this:

OverrideArmParameters: >
  -workspaceName jk-napsty-synapse-qa
  -AzureDatabricks1_accessToken $(databricksToken)
  -AzureDatabricks1_domain https://adb-qa.azuredatabricks.net

Replace the domain value per environment.

If you want to automate this for multiple environments and linked services, consider writing a small PowerShell or Python script to:

inject parameters into the template files
replace static values with parameter references.
maintain environment-specific override JSON or YAML fragments

Share via

How to Override Linked Service Parameters (e.g., Databricks domain) in Azure Synapse CI/CD YAML Pipeline?

1 answer

Your answer