How to load Delta table with parquet file in ADLS

heta desai 247 Reputation points
2022-08-18T18:08:10.547+00:00

I am trying to pull data from SQL server using copy activity into ADLS in parquet format using synapse pipeline. I need to load parquet file data into delta lake table. I have already created a delta lake table with LOCATION and copy activity will write parquet file to same location where delta table location in ADLS.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,328 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,456 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. KranthiPakala-MSFT 46,422 Reputation points Microsoft Employee
    2022-08-19T23:58:21.937+00:00

    Hello @Anonymous ,

    Thanks for the question and using MS Q&A platform.

    As per my understanding you would like to know how to copy parquet file data from ADLS to a delta table using Synapse/ADF pipelines. Please correct me if I'm wrong.

    Could you please confirm what Delta Lake tables you are referring to? Are you referring to Azure Databricks Delta Lake or Synapse Delta Lake? If you are referring to Azure databricks delta lake you can use a copy activity and copy data from your source Parquet file to your ADB Delta Lake using below connector in your sink Dataset (Azure Databricks Delta lake Dataset).

    233041-image.png

    In case if you are referring to Synapse Delta Lake, then you will have to use Mapping Dataflow and use Inline dataset as your sink configuration and Copy activity cannot be used as Delta format is supported only in Mapping Data flow as inline dataset.

    232979-image.png

    Reference docs

    Hope this info helps.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how