question

Cedersved-0221 avatar image
0 Votes"
Cedersved-0221 asked MartinJaffer-MSFT commented

Mapping data flow does not load any rows to 1 out of 2 branches

I have this mapping data flow, with two branches from 'RemoveHdlColumns'.
When I run this the rows from MergingInput is only loaded through the bottom branch.
All the rows are loaded for the 'Select2' transformation, but NO rows hits the 'Select1' transformation (or HashAllColumns or ChangedRows).
Select1 and Select2 are identical, without any expression - just set to auto mapping.

Why is this?



8968-adfmdf.png


azure-data-factory
adfmdf.png (47.7 KiB)
· 4
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Here's a revised version where i duplicated the 'MerginInput' source. I also added a new, unnecessary sink to the top branch - which makes the output with the row counts a bit more clear. But I don't understand the monitor view of data flows. In the attached picture, I get no row count for the 'EstablishRowHashMetadata' transformation. Not in the bottom view of the flows to the sink, and not in the right pane if I click the transformation. It makes it seem that no rows were passed to this transformation. But appearantly it processed rows, as the stream for EstablishRowHashMetadata was used to identify the 2 rows in the ChangedRows exist transformation.

9000-adfmdf2.png


0 Votes 0 ·
adfmdf2.png (98.5 KiB)

@Cedersved-0221 Thank you for bringing this issue to our attention. I have a question. Is the Data Flow writing the output your expect, and this is just a problem with the reporting. Or, is the Data Flow not working properly and giving you bad output?

0 Votes 0 ·

@majaffer It is not giving the output I expected. I did not get any rows in the sink with the currentUTC set in the CreateValidFrom transformation. So the reporting seemed to true from the perspective of no rows being passed beyond the Join1 transformation. Join1 inner joins the source (RemoveHdlColumns or Select2) with the output of ChangedRows using the same expression to generate a hash column as in the HashAllColumns transformation (generating a md5 hash using byPosition for the first 100 columns).


0 Votes 0 ·
Show more comments

0 Answers