write to delta sink error

Question

hello,
I do not know how to solve this in dataflow of the data factory.
Job failed due to reason: at Sink 'sink1': Files were added to the root of the table by a concurrent update. Please try the operation again.

note that I do run it a multiple times and each times it gives the same message.
any suggestions?
thank you

Answer 1

Thank you @Pratik Somaiya

Hello @arkiboys ,
Please correct me if my understanding is wrong. You see the error when your data flow tries to update the same table concurrently.

This exception occurs when a concurrent operation adds files in the same partition (or **anywhere in an unpartitioned table) that your operation reads. The file additions can be caused by INSERT, DELETE, UPDATE, or MERGE operations.

To mitigate the issue, you can do one of the following:
Partition your table, and try to operate on different partitions of your table concurrently
Schedule the writes one after other to avoid conflicts
Add retries to your code when you face this exception.

Answer 2

Hello @arkiboys

As per the error it seems to be a locking issue

Is there a way you can try running the data flow activities in a sequential manner?