Copy delta format data from one storage account to another in synapse

- 51 Reputation points Microsoft Employee
2023-04-26T00:26:20.74+00:00

Hi, Is there a way to copy delta format table data from one ADLS account to another? Ask is: We have a partner team who maintains data in delta format in their own storage account. They have it as day-wise partitioned. We need to daily copy only the modified partitions from their storage account and add it to a delta table in our ADLS account. My understanding is that we cannot simply copy parquet files, as they are tied to delta_log files. I see there is a Azure Databricks delta linked service, but that is not applicable in our case as our partner and us both use synapse. Is there any other linked service that can be used for this job?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,549 questions
{count} votes

Accepted answer
  1. KranthiPakala-MSFT 46,432 Reputation points Microsoft Employee
    2023-04-27T14:23:01.1633333+00:00

    Hi @- ,

    Welcome to Microsoft Q&A forum and thanks for reaching out here.

    You can utilize mapping data flows in Synapse Analytics and use Inline Dataset type as Delta in your source and sink transformation to copy the delta format table data from one ADLS to another.

    User's image

    Here is a document where detailed explanation about the functionality is provided: Delta format in Azure Data Factory & Synapse Analytics

    Hope this helps.


    Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful