Azure Synapse Analytics: How to quickly move/replicate work (Integration Datasets, Pipelines, etc.) from one Dev workspace to another Dev workspace (when connected to DevOps)

Question

I have a workspace in Azure Synapse Analytics that is connected to a DevOps repo. I've created a "template" of work that basically is just some pipelines, dataflows, and integration datasets. I would like to move all of this work to another Dev workspace. I know I could recreate all of this work from scratch but would prefer a method that doesn't require so much manual labor.

I don't want to manually review all, or even any, of the work after it has been migrated. I don't mind setting up a process for the actual migration, I just don't want to have to re-test everything after it has been moved to another workspace.

I have briefly tried the following methods: (but have had some issues)

1. Using pipeline templates
   1. items are duplicated if multiple pipelines are tied to it or reference it
      1. ex: creates a duplicate integration dataset instead of connecting to the one that already exists
2. Downloading/editing JSON files from DevOps
   1. some things don't seem to work even when the JSON files reflect all of the correct information (I haven't looked too heavily into why. I could have overseen a mistake)

This is by no means of a complete list of:

• methods I'm willing to use for migrating work
• issues I'm having with those methods

I would appreciate any guidance on this problem to help me better understand my options for migrating work between Azure Synapse Workspaces with as little review/testing needed.

Answer 1

First, ensure all your current work is committed and pushed to the Azure DevOps repository from the source workspace.

Then you can use the built-in export function to export your entire Synapse workspace artifacts (pipelines, dataflows, datasets, linked services...).

Now you need to clone Repository for Target Workspace:

• Clone the repository linked to the source workspace.
• Create a new branch (or use an existing one) where you will push the changes intended for the target workspace.

If you encounter issues with direct JSON transfer, ensure that the JSON files are correctly configured for the target workspace, especially:

• Linked services: Update any connection strings or credentials that might differ between workspaces.
• Integration runtimes: Ensure any self-hosted integration runtimes are correctly referenced.

Import to Target Workspace:

• Push the adjusted branch to your Azure DevOps repository.
• In the target workspace, configure the workspace to connect to the correct branch of the Azure DevOps repository.
• Once the repository is connected, use the 'Publish All' feature to deploy all artifacts to the target workspace.