Copy Data Activity Behaviour in ADF

Question

Hi Team,

We are using copy data activity to transfer the data from on- premisis to Azure sql DB.Recently while jobs are running one of the user updated the record at on premisis DB.ADF picked up the record twice because a user was making changes for one of column with new description at the same time while job is running and causes duplicate record entry in our sytem with second record reflecting with updated description and first record appearing in our system as old description.

Basically the entry appearing before and after change is avilable which we are expecting only updated record into sytem.Since user is doing updation at the same time 2 records are entered in to system.

Please suggest what isolatin level need to select considering performance and same time control the same kind of issues in furthur or is there any way to limit this.

We are performaing data load from on-premisis Sql server to Azure SQL Server.

Thanks,
Raj

Answer 1

Hello @Raj0125 ,
Thanks for the question and using MS Q&A platform.
As we understand the ask here is how can we avoid multiple entry on the sink side , please do let us know if its not accurate.
I think you should go ahead with the "Read commited option " , but then when the records where updated on the source side as the UPDATE statement was part of transaction , if yes then it should work fine .

One other thought is may be you can take advantage of the write behaviuor . So its the recors already exists it will just update that .

Please do let me if you have any queries.
Thanks
Himanshu

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators