Override parameters for deployment using Synapse deployment task for workspace

Question

Hello,

I am trying to deploy my Synapse workspace to the Test environment using the Synapse deployment task.
Since this task has the option to deploy the code from a non-publish branch, I have used the 'Validate and Deploy' operation type for deployment.

Since I do not have a dedicated parameter file for this mode of deployment, how can I make use of the 'Override Parameters' section to point to the Test instances? For example, I need to change the 'abc_dev' storage account name to 'abc_test'.

@PRADEEPCHEEKATLA-MSFT , could you please advise on this one?

Answer 1

Hello @Alok Thampi ,
Thanks for the reply.
My understanding is that when you use the 'Validate and Deploy’ option, by default it points to workspace_publish for the template parameters(same as the deploy option).

TemplateParametersForWorkspace.json file containing a global parameter for your workspace name and a parameter for each workspace
ex: default linked service: the default SQL Server and the default Storage account.

But if you check the TemplateForWorkspace.json in your publish branch, you will find several other properties(the artifact whose properties are not exposed by the default workspace parameters template.)

If you need to override the artifact properties that are not parameterized by the default parameters template(TemplateParametersforworkspace.json), then you need to use "custom parameters template".

To override the default parameter template, create a custom parameter template named template-parameters-definition.json in the root folder of your Git branch. You must use this exact file name. When Azure Synapse workspace publishes from the collaboration branch or the deployment task validates the artifacts in other branches, it reads this file and uses its configuration to generate the parameters. If Azure Synapse workspace doesn't find that file, is uses the default parameter template.

Ex: The below code in the custom template-parameters-defintion.json will parameterize all parameters at the pipeline level in the Synapse Deployment task.

"Microsoft.Synapse/workspaces/pipelines": {  
 "properties": {  
            "parameters":{  
                    "*": {"*":"="}  
                 
                }  

Reference documents:

https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/cicd-automation-in-synapse-analytics-taking-advantage-of-custom/ba-p/3513546

https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/continuous-integration-delivery#create-custom-parameters-in-the-workspace-template

I hope this helps. Please let me know if you have any further questions.

Answer 2

Hi @Alok Thampi ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

As I understand your query, you are trying to override parameters while deploying synapse code from dev to higher environment. Please let me know if my understanding is incorrect.

Firstly, go to the variables tab and create new variables corresponding to the parameters. We will also be creating a system.debug value to give us extra information in our release pipeline, it's value is True.

Now we will go to the OverrideArmParameters text area. We will use the following syntax : -variableNameFromTheJsonFile $(devOpsPipelineVarriable)

For example:

        -bballasw-WorkspaceDefaultSqlServer_connectionString $(WorkspaceDefault) -Lahman_connectionString $(Lahman)  

Yours may vary based your number of secureStrings and names. Now let's click Save on our pipeline.

For more details, kindly visit :

1. CI CD in Azure Synapse Analytics Part 4 - The Release Pipeline
2. Update pipeline parameters in the deployment to another environment through Azure DevOps

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 3

Hi @BhargavaGunnam-MSFT and @AnnuKumari-MSFT

Thank you both for the inputs.
I assume that the solutions that you had provided is based on the deployment using the workspace_publish branch (where we have both the TemplateForWorkspace.json and TemplateParametersForWorkspace.json files generated).

To give more context - I have 2 situations that I am trying to solve, and they are as below.

• Situation 1
  I am trying to deploy from a non-publish branch which is the reason why I used ‘Validate and Deploy’ option in the deployment task. Since this is a non-publish branch, it does not contain the TemplateParametersForWorkspace.json file from where I can directly pick the parameter names to put in the override parameters section. As this branch contains multiple folders, how do I find the parameter name to be overridden? In the screenshot below, what would be the parameter name that I need to put in the override parameters section to change the value of abc_dev to abc_test for the linked service ‘LS_SYN_SERVERLESSPOOL’?

I understand that this parameter is directly available in the ‘TemplateParametersForWorkspace.json‘ file if I had deployed from the workspace publish branch.
Or is it that we should always refer to the workspace publish branch for these parameter names even though we are doing the deployment from a non-publish branch?

• Situation 2
  How do I override the values that are specific to Synapse data pipelines? For example, below are some pipeline parameters that I have defined in my data pipelines. These contain ‘dev’ in the values which should be renamed as ‘tst’ when I move the code to the Test environment. How can I override these values during deployment? (By still using the Synapse deployment task)

I tried to check for these values in the TemplateParametersForWorkspace.json file of the publish branch (though I would not be deploying from the publish branch) but it doesn’t appear there either.

Answer 4

Hi @BhargavaGunnam-MSFT

My question is - I want to overwrite my linked services using collaboration branch.

Is there any way to do this?

Thanks,

Nikhil

Answer 5

Hello @Alok Thampi ,

Welcome to the MS Q&A platform.

Synapse deployment task supports two types of operations, validate and validate and deploy.

Deploy: The inputs of the operation deploy include the Synapse workspace template and parameter template, which can be created after publishing in the workspace publish branch or after the validation. It is same as the version 1.x.

Validate and deploy can be used to directly deploy the workspace from non-publish branch with the artifact root folder.

You can choose the operation types based on the use case

The goal of the Validate and Deploy operation is to deploy the artifacts to a target from any user branch (a non-publish branch). It does the job of the Validate operation (generating the ARM templates based on the specified branch) and adds this extra deploy step to deploy only the artifacts from that branch..

The OverrideParameters template works in the same way for both operations.

We have very good documentation explaining the "validate and deploy" option.

Reference document: https://techcommunity.microsoft.com/t5/azure-synapse-analytics-blog/automating-the-publishing-of-workspace-artifacts-in-synapse-cicd/ba-p/3603042

For your question: you need to change the 'abc_dev' storage account name to 'abc_test' on the connectionstring value in the release pipeline.

You can find the name of the connection string parameter from the "TemplateParametersForWorkspace.Json" file

I hope this helps. Please let me know if you have any further questions.

------------------------------

• Please don't forget to click on and upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators