synapse workspace publish templates, remove hardcoded resources

Question

For deployment of our production synapse workspace, we have a dev workspace synced to a Git repo with a publish branch and a release pipeline to publish the template to our production workspace.

In this template (TemplatesForWorkspace.json) some resources seem to be hard coded. In particular the spark pool compute resource is specified to be the nonprod resources.

Obviously we want the prod resources to be different (because there is no option to use the nonprod resources from the prod workspace). What are the steps needed to generate the template with the resources to be parameterized or defined in the template instead of linked to the non-prod resources?

Answer 1

@De Bondt, Hannes (Mechelen)

Thanks for using Microsoft Q&A forum and posting your query.

To modify your Synapse workspace template (TemplatesForWorkspace.json) and remove hardcoded resources, particularly for the Spark pool,

Steps to Parameterize Resources in Synapse Workspace Template

1. Identify Hardcoded Resources:
   • Open your TemplatesForWorkspace.json file and locate the sections where resources are hardcoded, such as the Spark pool configuration.
2. Define Parameters:
   • At the beginning of your JSON template, define parameters for the resources you want to make configurable. For example:JSON

          "parameters": {
            "sparkPoolName": {
              "type": "string",
              "defaultValue": "your-default-spark-pool-name",
              "metadata": {
                "description": "Name of the Spark pool to use."
              }
            },
            "sparkPoolSize": {
              "type": "string",
              "defaultValue": "Small",
              "metadata": {
                "description": "Size of the Spark pool."
              }
            }
          }
     

3. Update Resource Definitions:
   • Replace the hardcoded values in the resource definitions with the parameters you just created. For example:JSON

          {
            "type": "Microsoft.Synapse/workspaces/sparkPools",
            "name": "[parameters('sparkPoolName')]",
            "properties": {
              "nodeCount": "[parameters('sparkPoolSize')]"
            }
          }
     

4. Use Parameter Values in Deployment:
   • When deploying the template, ensure that you pass the appropriate values for these parameters based on the environment (production vs. non-production). This can be done in your CI/CD pipeline or deployment scripts.
5. Test the Template:
   • Deploy the modified template to a test environment to ensure that the parameterization works as expected. Verify that the correct resources are created based on the parameters provided.
6. Update CI/CD Pipeline:
   • If you have a release pipeline set up, ensure that it is configured to pass the correct parameter values for production deployments.

refer:https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/source-control

Continuous integration & delivery in Azure Synapse Analytics - Azure Synapse Analytics | Microsoft Learn

https://techcommunity.microsoft.com/blog/azuresynapseanalyticsblog/automating-the-publishing-of-workspace-artifacts-in-synapse-cicd/3603042

Hope this helps. Do let us know if you any further queries.