Alter Row transformation for implementing upsert

Lavanya Bhajanthri 200 Reputation points
2023-12-20T20:48:37.5233333+00:00

I intend to utilize the Alter Row transformation for implementing upsert functionality. In my target table, there are two columns, InsertedTime and UpdatedTime. For a new record insertion, I want to update both columns with the current datetime, and for an update, I only want to update the UpdatedTime column. I'm currently using Alter Row for both Insert and Update operations. How can I achieve the following:

If it's an Insert operation, set values for both InsertedTime and UpdatedTime columns.

If it's an Update operation, set values only for the UpdatedTime column.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,623 questions
{count} votes

Accepted answer
  1. Smaran Thoomu 24,110 Reputation points Microsoft External Staff Moderator
    2023-12-21T10:55:55.9333333+00:00

    Hi @Lavanya Bhajanthri

    Welcome to Microsoft Q&A Platform. Thank you for posting your query.

    Did you try using the Alter row for any of the Insert and Update Conditions yet?

    I have added the ADF dataflow with transformations to help with the initial query!

    Step1:
    Use Join for Source and Target Tables using Left Outer Join

    Use Select Transformation to avoid column names confusion!

    Use Derived Column transformation to create 2 new columns:

    InsertOrUpdate = iif(isNull(Trg_ID), 'Insert', 'Update')
    NewValue = Src_Value
    User's image

    Step2:

    Use Split Condition: For Update and Insert

    Update Path:
    User's image

    *
    Use below condition ,It is for 2 paths (Update & Insert)
    *
    User's image

    Add Derived Column:

    User's image

    Add Alter Row:User's image

    SINK: Select "Allow Update" With Key Column as 'ID'

    Mapping: update path Sink
    User's image

    Insert Path:
    User's image

    Add Derived Column to the Insert condition.

    2 new Columns to capture CurrentUTC(), or whatever the time format you want!!!

    User's image

    Add Alter Row:

    User's image

    Add SINK: Select "Allow Insert" method

    Mapping SINK for the Insert path.

    User's image

    SQL Server Tables:

    Before :

    User's image

    After the ADF pipeline run:

    User's image

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.


0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.