Understand Azure Databricks architecture
Intermediate
Data Engineer
Azure Databricks
Azure Databricks architecture separates control and compute planes while organizing resources through a hierarchical structure. This module explores how the account hierarchy works, the differences between serverless and classic compute planes, and the various storage options available including default storage, external storage, and Unity Catalog managed storage for organizing and governing your data.
Learning objectives
By the end of this module, you'll be able to:
- Describe the Azure Databricks account hierarchy and how it organizes resources
- Explain the difference between control plane and compute plane
- Understand the role of workspace storage in Azure Databricks
- Describe default storage capabilities in serverless workspaces
- Explain how external locations connect cloud storage to Unity Catalog
- Understand how Unity Catalog managed storage organizes data across catalogs and schemas
Prerequisites
The following prerequisites should be completed:
- Basic understanding of cloud storage concepts
- Familiarity with Azure Databricks workspaces
- Knowledge of Unity Catalog fundamentals