Unable to deploy code into another synapse workspace where Linked Service and Integration runtime are different

Question

Hello

I have azure synapse workspace and ETL code on that workspace.

I need to deploy the same code on new workspace and new subscription/tenant.

I am using Synapse deployment (template and params file).

I created yaml file where I can run the deployment.

From source we have this:

Linked service from source is different than target workspace

Integration runtime is different than the target (both SH but linked service created are different)

I made changes but it runs it still looks to old IR even though I replaced the IR in the Template.

I run out of ideas.

Please someone help!

Thanks

Xhev

Answer 1

Hi @Xhevahir Mehalla
Welcome to Microsoft Q&A platform and thanks for posting your question here.

It seems that you are facing an issue where the deployment of your ETL code in Azure Synapse is still referencing the old Integration Runtime despite making changes to the template. This can happen due to a few reasons.

One possible reason is caching. Sometimes, the deployment process might cache certain configurations. Ensure that you have cleared any caches or state files that might be holding onto the old IR settings.

Another possible reason is the parameter file. If you are using a parameters file along with your ARM template, double-check that the parameters file is updated with the new IR details and is being correctly referenced during the deployment.

You should also review the ARM template to ensure that all instances of the old IR are replaced with the new IR. It’s possible that there might be multiple references within the template.

Additionally, ensure that your YAML file is correctly configured to use the updated ARM template and parameters file. The YAML file orchestrates the deployment, so any discrepancies here can lead to the old IR being used.

If the linked services are still pointing to the old IR, they need to be updated. Linked services are the connectors that define how Azure Synapse connects to external resources, and they must be consistent with the new environment.

If you are using a self-hosted IR, there might be additional steps required to disconnect the old IR and connect the new one. This includes updating the IR on the Azure portal and within any configuration files.

Finally, make sure that the scope of your deployment is correct. If you’re deploying at a subscription level, it might not update the resources at the resource group or workspace level where the IR is defined.

To resolve this issue, you can follow these steps:

• Update Linked Services: Make sure that all linked services in your ARM template are updated to reference the new IR.
• Check ARM Template: Verify that the ARM template is correctly updated with the new IR settings.
• Use Azure DevOps or GitHub: Consider using Azure DevOps or GitHub for deploying your ETL code, as they provide tools like the Synapse Workspace Deployment tool which can help manage deployments across different environments.

Reference
https://learn.microsoft.com/en-in/answers/questions/1668992/deploy-the-etl-code-from-dev-synapse-analytics-wor

I hope this information helps you. Let me know if you have any further questions or concerns.