It looks like you're passing the p_extract_end_dt
parameter from the pipeline to the data flow, but the value might not be properly applied to the sys_update_dt
field during the upsert process in the sink.
Understanding the Upsert Behavior in Data Flow
When using upsert
in an Azure Data Factory (ADF) Data Flow, the update behavior depends on whether the column is explicitly mapped and modified during the transformation.
Here are few possible causes & solutions:
Ensure Parameter Mapping in Data Flow - Inside the data flow, you need to explicitly assign sys_update_dt
to p_extract_end_dt
in a Derived Column transformation before the sink. If you didn’t do this, sys_update_dt
might retain its original value from the source instead of being updated.
Check Sink Mapping - Open the Sink transformation and verify that sys_update_dt
is mapped correctly. If it’s missing, manually map it to the correct column.
Upsert Behavior - In an upsert operation, only explicitly modified columns will be updated in the target table. If sys_update_dt
is not treated as an update, it may retain its existing value.
Check for "Alter Row" Transformation (if any) - If you're using an Alter Row transformation to control updates, ensure it allows the sys_update_dt
column to be modified.
Expected Behavior
If mapped correctly, sys_update_dt
should be updated to 2025-02-12 16:09:09.000. If not mapped, sys_update_dt
may retain its previous value or not be updated.
I hope this information helps.
Thank you.