Azure Data flow: Flowlets for any input

Question

I successfully created a flowlet for any input that trims down the amount of columns to a representative set. However when using the flowlet, the original projection of the input columns gets lost which complicates the implementation of the data flow, since there is no access to the mapping anymore.

For any kind of input, for example structered like this:

The output schema always looks like this (empty list):

The flowlet used does not remove or alter any of the existing columns and the returned output shown in the data preview also confirms this, so the original projection of the input columns should in theory still work. Is there any way to preserve the mapping without creating a flowlet for every specific case (registering all the inputs of the data flow)? Or is this something that might be resolved in a future update?

Answer 1

Hi @Jan ,

Thank you for sharing details.

yes, it's expected behavior, let me explain in simple words, Flowlet is like a function which take some columns as input and returns some columns back.

In your case, your flowlet actually outputing back only one column which is RandSortCol. Hence, the moment you add that flowlet in to dataflow using flowlet transformation it is returning back only one column which is RandSortCol.

Kindly consider checking below video if mine, where i explained about flowlets.
Flowlets in Mapping data flow in Azure Data Factory

Hope this helps. Please let me know if any queries.

-----------

Please consider hitting Accept Answer button. Accepted answers help community as well.