Upgrade best practices for Azure Data Factory to Fabric Data Factory

This article provides implementation-level best practices for each component of an Azure Data Factory (ADF) to Fabric Data Factory upgrade: connections, integration runtimes, activities, dataflows, global parameters, and tooling. For high-level upgrade planning, including choosing an upgrade path and prioritizing pipelines, see Plan your upgrade from Azure Data Factory to Fabric.

Considerations before upgrading

Before upgrading, evaluate what to reuse, translate, or redesign. Follow these steps to ensure a smooth transition:

  1. Identify authentication patterns, such as managed identity or key-based authentication.
  2. Review network requirements, including private endpoints and gateways.
  3. Map scheduling and trigger semantics, and align monitoring and alerting setups.
  4. Compare ADF features with their Fabric counterparts. For pipelines containing Mapping Data Flows, assess whether they can be upgraded to MDF transforms in Dataflow Gen2 and identify any unsupported connectors, transformations, authentication methods, networking configurations, or other preview limitations. Also note gaps for capabilities such as SSIS.
  5. Define nonfunctional targets, such as SLAs, throughput, cost limits, and observability.
  6. Build a test scenario with sample datasets and expected outputs to objectively compare ADF and Fabric runs.
  7. Plan for secrets rotation, naming conventions, and workspace taxonomy so your migration improves - not just reproduces - your current data integration strategy.

A phased approach with side-by-side validation and rollback plans minimizes risk while enabling faster execution, centralized monitoring, and deeper integration with Microsoft Fabric.

For large migrations, consider working with certified Microsoft partners or your Microsoft account team for guidance.

Best practices for migrating ADF linked services and datasets to Fabric connections

In Azure Data Factory (ADF), linked services and datasets define connections and data structures. In Fabric, these map to connections and activity settings, with a stronger focus on workspace-level reuse and managed identity. Here’s how to adapt your ADF assets:

  1. Review Connector continuity between Azure Data Factory and Fabric to confirm support for your data sources and sinks.
  2. Consolidate redundant connections to streamline management.
  3. Adopt managed identity for secure and consistent authentication.
  4. Standardize folder and table parameterization using clear naming conventions, for example: conn-sql-warehouse-sales or ds-lh-raw-orders.

To ensure consistency and scalability, fully document each source and destination with:

  • Owners
  • Sensitivity levels
  • Retry settings

This documentation helps templatize operations across pipelines and improves governance.

Integration runtimes and OPDG, virtual network gateway

Azure Data Factory (ADF) uses Integration Runtimes (IRs) to define compute resources for data processing. These include:

  • Cloud IRs for Azure-hosted compute.
  • Self-hosted IRs (SHIRs) for on-premises or privately networked sources.
  • SSIS IRs for SQL Server Integration Services.
  • VNet-enabled IRs for secure network connectivity.

In Fabric, these map to cloud execution, On-premises Data Gateway (OPDG), and Virtual Network Data Gateway options. Here’s how to plan your migration:

  1. Identify pipelines that rely on SHIRs and plan their gateway mapping and throughput sizing.
  2. Validate DNS, egress, firewall rules, and authentication for each connector.
  3. Rehearse failover scenarios to ensure reliability.
  4. When possible, migrate to private endpoints or virtual network data gateways to simplify security reviews and reduce operational overhead.

Fabric simplifies compute management by using cloud-based resources within your Fabric capacities. SSIS IRs aren’t available in Fabric. For on-premises connectivity, use the On-premises Data Gateway (OPDG). For secure network connectivity, use the Virtual Network Data Gateway.

When migrating:

  • Public network Azure IRs don’t need to be moved.
  • Recreate SHIRs as OPDGs.
  • Replace VNet-enabled Azure IRs with Virtual Network Data Gateways.

Best practices for migrating ADF pipeline activities to Fabric

All core activities in Azure Data Factory (ADF), such as Copy, Lookup, Stored Procedure/SQL Script, Web, and Control flow, have direct equivalents in Fabric. However, there are some differences in properties, expression syntax, and limits. When migrating, review the following:

  • Retry policies and timeouts.
  • Pagination settings for REST sources.
  • Binary versus tabular copy configurations.
  • Foreach and filter patterns.
  • System variables used in dynamic content.

Fabric often provides more native options for certain tasks. For example, use SQL Script in a Warehouse instead of a generic Stored Procedure call for better lineage and monitoring. To streamline migration, centralize common expressions like paths, dates, and tenant-specific URIs into pipeline parameters. This reduces drift and speeds up testing.

For more information, see Activity continuity between Azure Data Factory and Fabric.

Best practices for upgrading Mapping Data Flows

The built-in upgrade experience supports eligible Azure Data Factory and Azure Synapse Analytics pipelines that contain Mapping Data Flows. Supported Mapping Data Flows are upgraded to Mapping Data Flow (MDF) transforms in Dataflow Gen2, preserving their transformation logic and Spark-based execution model. This capability is currently in preview.

Before upgrading, review the supported connectors, transformations, authentication methods, networking requirements, and current limitations. Current preview limitations include flowlets, Data Flow Library, user-defined functions, Managed Virtual Network connectivity, and direct Dataflow Gen2 execution.

For Mapping Data Flows that use unsupported capabilities, consider redesigning the transformation by using:

  • Power Query in Dataflow Gen2 for governed, low-code data preparation and rowset transformations.
  • Fabric Warehouse SQL for set-based ELT operations.
  • Spark notebooks for custom Spark code and advanced transformations.
  • Fabric pipeline activities or other Fabric items when they provide a more appropriate implementation.

After upgrading, validate connections, parameters, transformation results, schema, row counts, rerun behavior, monitoring, and performance before production cutover. For requirements and step-by-step instructions, see Upgrade Azure Data Factory Mapping Data Flow pipelines to Fabric.

Global parameters in Azure Data Factory

Fabric uses workspace-level Variable Libraries to define constants across Fabric items. When migrating to Microsoft Fabric Data Factory, you'll need to convert your Azure Data Factory global parameters to variable libraries.

For full conversion guidance, see Convert ADF Global Parameters to Fabric Variable Libraries.

Azure Marketplace partner offerings

Trusted migration partners, like Bitwise Global, provide tools to help with your migration. These tools can:

  • Scan your Azure Data Factory (ADF) environment.
  • Generate target Fabric artifacts.
  • Perform impact analysis and lineage tracking.
  • Create automated test plans.

These solutions are especially helpful if you have:

  • Hundreds of pipelines.
  • Diverse connectors.
  • Strict downtime requirements.

Partner tools standardize mapping rules, generate conversion reports, and run parallel validation tests. This allows you to compare row counts, checksums, and performance between your old and new environments. Even if you don’t use a partner for the entire migration, their discovery and assessment modules can help you start your internal planning and reduce uncertainties.

Use Copilot and AI tools to accelerate ADF-to-Fabric migration

Large language models (LLMs) like Microsoft Copilot, ChatGPT, and Claude can speed up migration tasks. These tools are useful for:

  • Refactoring expressions.
  • Converting Azure Data Factory (ADF) JSON to Fabric syntax.
  • Writing MERGE statements.
  • Generating connection templates.
  • Drafting validation scripts.

Use them to create documentation, such as runbooks, data dictionaries, and upgrade checklists, so engineers and operators stay aligned. However, keep these tools in the loop, not in charge:

  • Avoid pasting sensitive information into AI tools.
  • Validate all items in a development environment.
  • Use automated tests like row counts, schema comparisons, and business-rule checks to catch subtle issues, such as type mismatches or locale-specific date parsing.

For more information, see Use Copilot in Data Factory and AI in Microsoft Fabric.

Upgrade paths

For detailed guidance on upgrade path options (including mounting ADF items for continuity, using the built-in upgrade experience, PowerShell conversion, and replatforming strategies), see Upgrade paths in the upgrade planning guide.

Related content

  • Last updated on