how to avoid duplicated record in sql server when use Synapse copy data from blob to dedicated SQL pool scheduled

Te Liu 1 Reputation point Microsoft Employee
2022-11-08T09:08:44.573+00:00

I want to copy data from blob in account storage to SQL server regularly every day. But I want to skip record if this record is already exist in table. thanks :)

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,687 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,180 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Nandan Hegde 31,511 Reputation points MVP
    2022-11-08T09:37:39.297+00:00

    Hey, You can copy the entire data in a staging table and then leverage Stored proceudre to do the rest of copying only new data into the final table

    0 comments No comments