how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Question

how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Te Liu 1 Microsoft Employee

I want to copy data from blob in account storage to SQL server regularly every day. But I want to skip record if this record is already exist in table. thanks :)

Te Liu 1 Reputation point Microsoft Employee

2022-11-08T09:14:19.847+00:00

Do we have a way to check if the record is already exist and skip this record when needed.
PRADEEPCHEEKATLA 90,631 Reputation points Moderator

2022-11-14T09:23:48.427+00:00

Hello @Te Liu ,

Just checking in to see if the below answer provided by @Nandan Hegde helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

1 answer

Your answer

Te Liu 1 Reputation point Microsoft Employee

2022-11-08T09:14:19.847+00:00

Do we have a way to check if the record is already exist and skip this record when needed.
PRADEEPCHEEKATLA 90,631 Reputation points Moderator

2022-11-14T09:23:48.427+00:00

Hello @Te Liu ,

Just checking in to see if the below answer provided by @Nandan Hegde helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 1

Nandan Hegde 36,066 MVP Volunteer Moderator

Hey, You can copy the entire data in a staging table and then leverage Stored proceudre to do the rest of copying only new data into the final table

Share via

how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

1 answer

Your answer