delete if - data flow - issue

arkiboys 9,686 Reputation points
2021-10-04T14:31:53.137+00:00

Hello,
I am using parquet file for both source and sink.
The screen-shots below show the settings for update if and delete if settings in the sink activity of the data flow which is set to Delta dataset...
Question,
do you see why the Delete if does not work (If I delete a row at source, then the destination still keeps the row)
note that the upsert if seems to be working fine

137462-image.png
137369-image.png
137463-image.png

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,130 questions
{count} vote

Accepted answer
  1. ShaikMaheer-MSFT 38,326 Reputation points Microsoft Employee
    2021-10-05T07:57:27.39+00:00

    Hi @arkiboys ,

    Thank you for posting query in Microsoft Q&A Platform.

    You are using true() as condition for both "Upsert If" and "Delete If" in Alter row transformation. That means, all rows from data will match both Upsert If & Delete If conditions. But Order takes priority here. Since rows are already marked as Upsert If(as Upsert If defined first), rows will not be marked as delete if.

    Use the Alter Row transformation to set insert, delete, update, and upsert policies on rows. You can add one-to-many conditions as expressions. These conditions should be specified in order of priority, as each row will be marked with the policy corresponding to the first-matching expression. Click here to know more about Alter row transformation.

    Please check below screenshots with sample example to explain same.

    137682-image.png

    137674-image.png

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful