Converting azure synapse pipelines to azure data factory pipelines

Question

Our application was developed in Synapse workspace. We want to move to Azure data factory. Please let me know the process to move from Synapse to data factory.

Answer 1

Migrating from Azure Synapse Analytics to Azure Data Factory (ADF) involves transferring Synapse Pipelines to Data Factory Pipelines. While both services offer data integration capabilities, they have distinct functionalities and design philosophies.

I am assuming the following (this is a step-by-step guide ) :

1. Assessment:
   • Identify your Synapse workspace's pipelines, datasets, linked services, and triggers.
   • List all external dependencies, such as storage accounts, databases, and external systems that are connected to your Synapse workspace.
2. Environment Setup:
   • Create an Azure Data Factory instance in the Azure portal if you haven't already.
   • Set up necessary authentication mechanisms. For example, set up Managed Identity or Service Principal to provide ADF with access to Azure resources.
3. Migrate Linked Services:
   • In Synapse, Linked Services define the connection parameters to the external systems. In ADF, the concept remains the same.
   • Recreate these linked services in ADF manually or by exporting the ARM templates from Synapse and modifying them for ADF.
4. Migrate Datasets:
   • Datasets in Synapse represent the data structures, such as tables or files.
   • You can manually recreate these in ADF or, similar to linked services, use ARM templates for migration.
5. Migrate Activities:
   • Activities in Synapse might have different configurations than those in ADF.
   • Review each activity in Synapse and recreate them in ADF. Some might be straightforward, while others might need adjustments or reconfiguration.
   • Pay close attention to mapping data flows and other transformation activities, as there might be differences between Synapse and ADF.
6. Migrate Triggers:
   • If you have any scheduled or event-driven triggers in Synapse, replicate them in ADF.