Mapping data flow transformation overview
APPLIES TO: Azure Data Factory Azure Synapse Analytics
Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!
Data flows are available both in Azure Data Factory and Azure Synapse Pipelines. This article applies to mapping data flows. If you are new to transformations, please refer to the introductory article Transform data using a mapping data flow.
Below is a list of the transformations currently supported in mapping data flow. Click on each transformations to learn its configuration details.
|Define different types of aggregations such as SUM, MIN, MAX, and COUNT grouped by existing or computed columns.
|Set insert, delete, update, and upsert policies on rows.
|Set assert rules for each row.
|Change column data types with type checking.
|Route rows of data to different streams based on matching conditions.
|Generate new columns or modify existing fields using the data flow expression language.
|Call external endpoints inline row-by-row.
|Check whether your data exists in another source or stream.
|Filter a row based upon a condition.
|Take array values inside hierarchical structures such as JSON and unroll them into individual rows.
|Build and include custom re-usable transformation logic.
|Combine data from two sources or streams.
|Reference data from another source.
|Apply multiple sets of operations and transformations against the same data stream.
|Parse text columns in your data stream that are strings of JSON, delimited text, or XML formatted text.
|An aggregation where one or more grouping columns has its distinct row values transformed into individual columns.
|Generate an ordered ranking based upon sort conditions
|Alias columns and stream names, and drop or reorder columns
|A final destination for your data
|Sort incoming rows on the current data stream
|A data source for the data flow
|Turn complex types into plain strings
|Add an incrementing non-business arbitrary key value
|Combine multiple data streams vertically
|Pivot columns into row values
|Define window-based aggregations of columns in your data streams.