Delta Lake with Databricks and Synapse

James H. Robinson 161 Reputation points
2021-08-31T02:07:38.07+00:00

What advantages are there to using Databricks and Synapse with Delta Lake? I understand that AAS can connect to Synapse but not Databricks. If that's the case, why not just use Synapse with Delta Lake?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,558 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,019 questions
0 comments No comments
{count} votes

Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 82,671 Reputation points Microsoft Employee
    2021-08-31T10:21:33.993+00:00

    Hello @James H. Robinson ,

    Thanks for the question and using MS Q&A platform.

    What is Delta Lake?

    Delta Lake is an open-source data format that enables you to update your big data sets with guaranteed ACID transaction behavior.

    Delta Lake is a layer placed on top of your existing Azure Data Lake data that can be fully managed using Apache Spark APIs available in both Azure Synapse and Azure Databricks.

    127866-image.png

    Delta Lake is one of the most popular updateable big data formats in big data solutions, and frequently used by many data engineers who need to prepare, clean, or update data stored in data lake, or apply machine learning experiments.

    Yes, you cannot connect to Azure Analysis services is not supported as a source in Azure Databricks.

    Reason: Supported Data Sources in Azure Databricks and Data Sources in Azure Analysis Services.

    If you are using Delta Lake with Azure Synapse Analytics, you can connect with Azure Analysis services.

    Data sharing without copy, load, or transformation of Delta Lake files is the main benefit of serverless SQL pools. The serverless endpoint in Azure Synapse represents a bridge between a reporting/analytics layer where you use Power BI or Azure Analysis Services, and your data stored in Delta Lake format.

    127807-image.png

    For more information, refer to Query Delta Lake files using T-SQL language in Azure Synapse Analytics.

    Hope this helps. Do let us know if you any further queries.

    ---------------------------------------------------------------------------

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful