Hey, You can copy the entire data in a staging table and then leverage Stored proceudre to do the rest of copying only new data into the final table
how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Te Liu
1
Reputation point Microsoft Employee
I want to copy data from blob in account storage to SQL server regularly every day. But I want to skip record if this record is already exist in table. thanks :)