Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
APPLIES TO: 
Azure Data Factory
Azure Synapse Analytics
Tip
Data Factory in Microsoft Fabric is the next generation of Azure Data Factory, with a simpler architecture, built-in AI, and new features. If you're new to data integration, start with Fabric Data Factory. Existing ADF workloads can upgrade to Fabric to access new capabilities across data science, real-time analytics, and reporting.
Data flows are available in both Azure Data Factory pipelines and Azure Synapse Analytics pipelines. This article applies to mapping data flows. If you're new to transformations, refer to the introductory article Transform data using mapping data flows.
Tip
For the equivalent transformation (Append queries) in Dataflow Gen2, see A guide to Dataflow Gen2 for mapping data flow users.
Union combines multiple data streams into one, with the SQL Union of those streams as the new output from the Union transformation. All of the schema from each input stream will be combined inside of your data flow, without needing to have a join key.
You can combine n-number of streams in the settings table by selecting the "+" icon next to each configured row, including both source data and streams from existing transformations in your data flow.
Here's a short video walk through of the union transformation in the mapping data flow:
In this case, you can combine disparate metadata from multiple sources (in this example, three different source files) and combine them into a single stream:
To achieve this, add more rows in the Union Settings by including all source you wish to add. There's no need for a common lookup or join key:
If you set a Select transformation after your Union, you'll be able to rename overlapping fields or fields that weren't named from headerless sources. Select "Inspect" to see the combined metadata with 132 total columns in this example from three different sources:
Name and position
When you choose "union by name", each column value drops into the corresponding column from each source, with a new concatenated metadata schema.
If you choose "union by position", each column value drops into the original position from each corresponding source, resulting in a new combined stream of data where the data from each source is added to the same stream:
