multiple sink transaction in Azure Data Factory

44 0 Reputation points
2023-11-22T02:16:56.9366667+00:00

hI, I have one dataflow and one pipeline. And there are two sinks in the dataflow. I want to do that if there are some error happened in the any dataflow, then these two dataflow all rollback.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,179 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Nandan Hegde 31,511 Reputation points MVP
    2023-11-22T03:46:31.06+00:00

    Hey,

    Unfortunately ADF doesnt support transaction or roll back aspect as an in built functionality.

    You would have to write your own custom logics or flows to ensure the rollback aspects.

    Like in case if your sinks are 2 different tables within the same Azure SQL database, you can copy both into 2 staging tables and then use an SP activity which would have transaction to load both into final tables.

    For other sinks, you can initially preserve the initial state of the sinks in some location and in case of any failure, revert the sink back to original state via the copy preserved.