how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Question

how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Te Liu 1 Microsoft Employee

I want to copy data from blob in account storage to SQL server regularly every day. But I want to skip record if this record is already exist in table. thanks :)

Te Liu 1 Reputation point Microsoft Employee

2022-11-08T09:14:19.847+00:00

Do we have a way to check if the record is already exist and skip this record when needed.
PRADEEPCHEEKATLA-MSFT 59,326 Reputation points Microsoft Employee

2022-11-14T09:23:48.427+00:00

Hello @Te Liu ,

Just checking in to see if the below answer provided by @Nandan Hegde helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Te Liu 1 Reputation point Microsoft Employee

2022-11-08T09:14:19.847+00:00

Do we have a way to check if the record is already exist and skip this record when needed.
PRADEEPCHEEKATLA-MSFT 59,326 Reputation points Microsoft Employee

2022-11-14T09:23:48.427+00:00

Hello @Te Liu ,

Just checking in to see if the below answer provided by @Nandan Hegde helped. If this answers your query, do click Accept Answer and Up-Vote for the same. And, if you have any further query do let us know.

Answer 1

Nandan Hegde 22,721

Hey, You can copy the entire data in a staging table and then leverage Stored proceudre to do the rest of copying only new data into the final table

how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

1 answer

Activity