Share via

how ADF connection to Azure Delta Lake ?

shane want 0 Reputation points
2024-07-22T06:14:23.61+00:00

Are there any plans to provide connection between ADF v2/Managing Data Flow and Azure Delta Lake? It would be great new source and sync for ADF pipeline and Managing Data Flows to provide full ETL/ELT CDC capabilities to simplify complex lambda data warehouse architecture requirements.

Azure SQL Database

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. shane want 0 Reputation points
    2024-07-22T06:22:16.0533333+00:00

    In general, the service supports Delta Lake with the following capabilities to meet your various needs.

    • Copy activity supports Azure Databricks Delta Lake connector to copy data from any supported source data store to Azure Databricks delta lake table, and from delta lake table to any supported sink data store. It leverages your Databricks cluster to perform the data movement, see details in Prerequisites section.
    • Mapping Data Flow supports generic Delta format on Azure Storage as source and sink to read and write Delta files for code-free ETL, and runs on managed Azure Integration Runtime.
    • Databricks activities supports orchestrating your code-centric ETL or machine learning workload on top of delta lake.
    0 comments