compute cluster and serverless cluster

Shambhu Rai 1,411 Reputation points
2023-07-26T15:53:28.36+00:00

Hi Expert,

How we can have same configuration and performance in compute cluster and serverless cluster

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,559 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,373 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,514 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
{count} votes

1 answer

Sort by: Most helpful
  1. PRADEEPCHEEKATLA 90,641 Reputation points Moderator
    2023-07-31T06:56:22.95+00:00

    @Shambhu Rai - Thanks for the question and using MS Q&A platform.

    Azure Databricks provides two types of SQL endpoints: SQL Data Warehouse and Serverless SQL Pool**.** SQL Data Warehouse is a dedicated cluster that provides high-performance analytics and data warehousing capabilities. On the other hand, Serverless SQL Pool is a pay-per-query service that allows you to query data stored in Azure Blob Storage, Azure Data Lake Storage, and Azure Synapse Analytics.

    SQL Data Warehouse is a good choice when you need to run complex queries on large datasets and require high-performance analytics. It provides a dedicated cluster that can be scaled up or down based on your needs. However, it requires more management and maintenance compared to Serverless SQL Pool.

    Serverless SQL Warehouse, on the other hand, is a good choice when you need to query data on an ad-hoc basis and don't want to manage a dedicated cluster. It provides a pay-per-query model, which means you only pay for the queries you run. It also provides automatic scaling and can handle large datasets. However, it may not be suitable for running complex queries on very large datasets.

    The following diagram shows important differences between the serverless data plane and classic data plane for both serverless features.

    Compare classic and serverless data plane for Databricks SQL

    For more details, refer to Compare serverless compute to other Azure Databricks architectures.

    Hope this helps. Do let us know if you any further queries.


    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.