A guide to Fabric Dataflows for Azure Data Factory Mapping Data Flow users
Microsoft Fabric’s Data Factory experience provides an intuitive and user-friendly interface using Power Query Online that can help you streamline your data transformation workflows when authoring Dataflow Gen2. If you’re a developer with a background in Azure Data Factory's Mapping Data Flows, you’ll find this guide helpful in mapping your existing Mapping Data Flow transformations to the Dataflow Gen2 Power Query user interface.
Global search box
When first starting to author Dataflows, you can also leverage the Global search box (shortcut: Alt + Q) found in the middle of the editor to search for connectors, transformations (actions), queries, and more while you learn your way around the Power Query Online interface.
Note
Learn more about Global search box.
Transformations
When authoring Dataflow Gen2 using the Power Query Online interface, you can use the following table to understand how the current Mapping Data Flows capabilities translate and their equivalent toolbar navigation location in Dataflow Gen2 where applicable.
Multiple inputs/outputs
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| New branch | Reference |
|
Important: Right click a query and select Reference. |
| Join | Merge queries |
|
Navigation: Home > Merge queries Important: Merge queries – Merge this query into another query in this project. Merge queries as new – Merge this query into another query in this project to create a new query. |
| Conditional Split | Reference |
|
Important: Right click a query and select Reference, you can then apply additional transformation logic to limit specific rows manually. |
| Union | Append queries |
|
Navigation: Home > Append queries Important: Append queries – Append this query into another query in this project. Append queries as new – Append this query into another query in this project to create a new query. |
| Lookup | Merge queries |
|
Navigation: Home > Merge queries Important: Select Left outer from the Join kind options. |
Schema modifier
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| Derived Column | Custom column |
|
Navigation: Add column > Custom column Important: Column from examples - Use examples to create a new column in this table. (Ctrl + E) Custom column - Create a new column based on other columns, or a custom expression. Invoke custom function - Invoke a custom function for each row of this table. Conditional column - Create a new column that conditionally adds the values in the currently selected column. |
| Replace values | |
Navigation: Transform > Replace values |
|
| Select | Choose columns |
Note: Includes the ability to select, drop and rename (Name as) columns and whether to drop or pass on duplicate columns |
Navigation: Home > Choose columns |
| (Remove Mapping) | Remove columns |
|
|
| Navigation: Home > Remove columns |
|
| (Name as) | Rename |
|
|
| Navigation: Transform > Rename |
|
| Aggregate | Group by |
|
|
| Navigation: Transform > Group by |
|
| Surrogate Key | Index column |
|
|
| Navigation: Add column > Index column Important: From 0 - Create a new column with an index starting at 0. From 1 - Create a new column with an index starting at 1. Custom... - Create a new column with an index starting at a specified value and a specified increment. |
|
| Pivot | Pivot column |
|
|
| Navigation: Transform > Pivot column |
|
| Unpivot | Unpivot columns |
|
|
| Navigation: Transform > Unpivot columns Important: Unpivot columns - Translate all but the currently unselected columns into attribute-value pairs. Unpivot other columns - Select all but the currently selected columns into attribute-value pairs. Unpivot only selected columns - Translate only the currently selected columns into attribute-value pairs. |
|
| Rank | Rank column |
|
|
| Navigation: Add column > Rank column |
|
| External Call | Custom column |
|
Navigation: Add column > Custom column Expression: Web.Contents |
| Cast | Data type |
|
Navigation: Transform > Data type |
Formatters
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| Flatten | (Expand column) |
|
|
| Important: The expand option is available when structured data types exist in your column. |
|
| Parse | Parse |
|
Navigation: Transform > Text column > Parse |
Row modifier
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| Filter | Filter rows |
|
Navigation: Home > Filter rows |
| Sort | Sort |
|
Navigation: Home > Sort |
Flowlets
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| Flowlet | (Custom functions) |
|
Learn more about how to Create a custom function |
Destination
| Mapping Data Flow | Dataflow Gen2 |
|---|---|
| Sink | Add data destination |
|
Navigation: Home > Add data destination |
Considerations and limitations
The following mapping data flow transformations are not supported in Dataflow Gen2. To vote for support of these operations, please visit Fabric ideas.
- Assert
- Alter Row
- Stringify
- Window