Migration Data Factory Pipeline to Synapse Pipeline or Fabric

Question

We are currently working on a architecture for building a data warehouse. Our current state has Data Factory pulling data from various sources. As future state soon we are planning to migrate it to either synapse or Microsoft fabric. Looking for possible suggestions / solutions for the future migration. Due to costing we are unable to implement Modern Data Platform as of now.

Answer 1

Hello Sundar Garimella,

Both synapse and fabric are great choices for building a modern data platform, but they have slightly different use cases.

Microsoft Fabric is a unified SaaS offering expanded with current Microsoft analytical tools like Azure Synapse Analytics, Azure Data Factory and Power BI. Fabric facilitates seamless data exchange, easy data discovery, and reduced integration time and effort. Fabric brings all of these services in one place.

Please see the difference between these two platforms.

• Fabric is a SaaS offering, and Synapse is a PaaS offering.
• Unlike Synapse, Fabric does not have a dedicated SQL pool or relational storage. In Fabric, warehouse data is persisted in delta lake format.
• Fabric warehouse is powered by the Polaris engine, which also drives the Serverless SQL pool in Synapse. This MPP (massively parallel processing) engine scales AUTOMATICALLY to accommodate different data workloads.
•  Dataflows Gen2 (Power Query) can be used instead of Mapping Data Flows since Fabric does not support them. Synapse Mapping Data Flows is the “no code / low code” graphical user interface that can transform data in Synapse.
• Fabric does not allow OPENROWSET(). You may still query the data from the lake(house) in Fabric using T-SQL, but you must modify all queries that utilize OPENROWSET syntax.
• The Synapse Link feature is not available in Fabric. Synapse Link allows querying the data from Cosmos DB and/or Microsoft Dataverse.
• The Data Activator module, new in Fabric, is designed for real-time data detection and monitoring. It will be able to send notifications and perform actions when it detects specific patterns in data.
• The Spark engine in Fabric spins up significantly faster than the Synapse engine.

If you want to build a data warehouse and perform analytics on large-scale data, then Azure Synapse Analytics is a great choice. On the other hand, if you are looking to build and deploy microservices-based applications, then Azure Fabric is a great choice.

Below are a few documents URs and great blog posts to help you get started.

https://learn.microsoft.com/en-us/fabric/

https://blog.fabric.microsoft.com/en-us/blog/introducing-synapse-data-warehouse-in-microsoft-fabric/

https://endjin.com/blog/2023/05/azure-synapse-analytics-versus-microsoft-fabric-a-side-by-side-comparison

https://www.linkedin.com/pulse/what-microsoft-fabric-how-differs-from-synapse-hemavathi-thiruppathi/

https://learn.microsoft.com/en-us/fabric/onelake/onelake-azure-synapse-analytics

https://radacad.com/what-is-microsoft-fabric-and-why-it-is-a-big-deal

I hope this helps. Please let me know if you have any specific questions.

If this answers your question, please consider accepting the answer by hitting the Accept answer and up-vote as it helps the community look for answers to similar questions.