Azure Data flow: Flowlets for any input

Jan 1 Reputation point
2022-11-25T17:33:50.983+00:00

I successfully created a flowlet for any input that trims down the amount of columns to a representative set. However when using the flowlet, the original projection of the input columns gets lost which complicates the implementation of the data flow, since there is no access to the mapping anymore.

For any kind of input, for example structered like this:

264361-flowlet-input.png

The output schema always looks like this (empty list):

264250-flowlet-output.png

The flowlet used does not remove or alter any of the existing columns and the returned output shown in the data preview also confirms this, so the original projection of the input columns should in theory still work. Is there any way to preserve the mapping without creating a flowlet for every specific case (registering all the inputs of the data flow)? Or is this something that might be resolved in a future update?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,653 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,078 questions
{count} votes

1 answer

Sort by: Most helpful
  1. ShaikMaheer-MSFT 38,326 Reputation points Microsoft Employee
    2022-12-06T14:10:05.65+00:00

    Hi @Jan ,

    Thank you for sharing details.

    yes, it's expected behavior, let me explain in simple words, Flowlet is like a function which take some columns as input and returns some columns back.

    In your case, your flowlet actually outputing back only one column which is RandSortCol. Hence, the moment you add that flowlet in to dataflow using flowlet transformation it is returning back only one column which is RandSortCol.

    Kindly consider checking below video if mine, where i explained about flowlets.
    Flowlets in Mapping data flow in Azure Data Factory

    Hope this helps. Please let me know if any queries.

    -----------

    Please consider hitting Accept Answer button. Accepted answers help community as well.