Selection of Cluster in Databricks and Synapse

Sukumar Vinnakota 66 Reputation points
2022-07-12T14:48:46.967+00:00

I want to compare the sizing of clusters in synapse and databricks. If I select the small cluster with 4 core 32 GB RAM in synapse, what is the node type should I prefer for a standard type cluster in databricks . Please note that I need the type of node must be in general purpose.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,186 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,331 questions
0 comments No comments
{count} votes

Accepted answer
  1. PRADEEPCHEEKATLA 90,541 Reputation points
    2022-07-13T07:05:39.04+00:00

    Hello @Sukumar Vinnakota ,

    Thanks for the question and using MS Q&A platform.

    Azure Databricks - Node Sizes

    Azure Databricks offers three distinct workloads on several VM Instances tailored for your data analytics workflow—the Jobs Compute and Jobs Light Compute workloads make it easy for data engineers to build and execute jobs and the All-Purpose Compute workload makes it easy for data scientists to explore, visualise, manipulate and share data and insights interactively.

    Note: In Standard cluster mode, you have 33 instances available under General purpose type. You can pick any instance as per your requirement.

    220321-image.png

    Azure Synapse Analytics - Node Sizes

    A Spark pool can be defined with node sizes that range from a Small compute node with 4 vCore and 32 GB of memory up to a XXLarge compute node with 64 vCore and 512 GB of memory per node. Node sizes can be altered after pool creation although the instance may need to be restarted.

    220272-image.png

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.