Azure Databricks pools reduce cluster start and auto-scaling times by maintaining a set of idle, ready-to-use instances. When a cluster is attached to a pool, cluster nodes are created using the pool’s idle instances. If the pool has no idle instances, the pool expands by allocating a new instance from the instance provider in order to accommodate the cluster’s request. When a cluster releases an instance, it returns to the pool and is free for another cluster to use. Only clusters attached to a pool can use that pool’s idle instances.
You can specify a different pool for the driver node and worker nodes, or use the same pool for both.
For an introduction to pools and configuration recommendations, view the Databricks pools video:
Azure Databricks does not charge DBUs while instances are idle in the pool. Instance provider billing does apply. See pricing.
This section describes how to work with pools using the UI:
Submit and view feedback for