How to set ADF linked services and triggers parameterized for respective environment when we create pull request

Question

How to Set ADF linked services and triggers parameterized for respective environment when we create pull request like dev-parameter.json

Answer 1

Hi Madhu Ram,

Thanks for reaching out to Microsoft Q&A.

Implement a configuration that is dynamically adjusted based on the target environment when creating a pull request. This approach typically involves using parameter files for each environment (such as dev-parameter.json, test-parameter.json, and prod-parameter.json) and Azure DevOps pipelines or similar CI/CD tools to deploy to the right environment.

Create Parameter Files for Each Environment: parameter.json

{
  "linkedService:AzureBlobStorage": {
    "properties": {
      "typeProperties": {
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=devstorage;AccountKey=XXXXXX;"
      }
    }
  },
  "trigger:DailyTrigger": {
    "properties": {
      "typeProperties": {
        "recurrence": {
          "frequency": "Day",
          "interval": 1,
          "startTime": "2024-10-01T00:00:00Z"
        }
      }
    }
  }
}

Add Parameters to Linked Services and Triggers

When defining linked services and triggers in ADF, include parameters that will reference environment-specific values.

• In Linked Services, you can parameterize fields like connection strings, database names, etc.
• In Triggers, you can parameterize properties such as recurrence intervals, start times, etc.

Example Linked Service with Parameters::

{
  "name": "AzureBlobStorage",
  "properties": {
    "type": "AzureBlobStorage",
    "typeProperties": {
      "connectionString": {
        "value": "@{linkedService().connectionString}",
        "type": "SecureString"
      }
    }
  }
}

Use Parameter Files in CI/CD Pipeline

When creating a pull request or triggering a deployment to a specific environment, use your CI/CD pipeline (Azure DevOps, GitHub Actions, etc.) to inject the parameter file (parameters.json) based on the target environment.

Here’s how to implement this in Azure DevOps Pipeline:

• Define the environment variable in the pipeline:

    parameters:
      - name: environment
        displayName: Select Environment
        type: string
        default: dev
        values:
          - dev
          - test
          - prod
  

• Use the environment parameter to select the correct parameter file:

    steps:
    - task: AzureDataFactoryV2@1
      inputs:
        azureSubscription: 'Azure_Subscription_Name'
        ResourceGroupName: 'Your_Resource_Group'
        DataFactoryName: 'Your_Data_Factory_Name'
        Stage: 'armTemplateDeployment'
        TemplateLocation: 'LinkedArtifact'
        csmFile: 'ARMTemplate/ADF_Template.json'
        csmParametersFile: '$(Pipeline.Workspace)/arm-templates/$(parameters.environment)-parameters.json'
        deploymentMode: 'Incremental'
    
    
  

  This pipeline will dynamically pick the appropriate parameter file (dev-parameters.json, prod-parameters.json, etc.) based on the selected environment.
• Automate the Parameter Injection in the Deployment Process To fully automate this process, your pull request workflow should integrate the CI/CD process. You can configure it to automatically deploy to different environments when a PR is merged, using the correct parameter file for that environment.

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.