Select and Configure Compute in Azure Databricks
Intermediate
Data Engineer
Azure Databricks
Azure Databricks provides multiple compute options optimized for different workloads. This module explores how to choose the right compute type, configure performance settings, manage access permissions, and install libraries. You'll learn when to use serverless versus classic compute, how to optimize clusters for cost and performance, and best practices for securing compute resources.
Learning objectives
By the end of this module, you'll be able to:
- Choose appropriate compute types for different Azure Databricks workloads
- Configure compute performance settings including node types, autoscaling, and termination
- Enable compute features like Photon acceleration and select appropriate Databricks Runtime versions
- Configure compute access permissions and dedicated group access modes
- Install and manage libraries on compute resources using various methods
Prerequisites
The following prerequisites should be completed:
- Basic understanding of Azure Databricks concepts
- Familiarity with Unity Catalog and workspace management
- Knowledge of Spark fundamentals and cluster architecture