Classic compute plane networking

This guide introduces features to customize network access between the Azure Databricks control plane and the classic compute plane. Connectivity between the control plane and the serverless compute plane is always over the cloud network backbone and not the public internet.

To learn more about the control plane and the compute plane, see Azure Databricks architecture overview.

The features in this section focus on establishing and securing the connection between the Azure Databricks control plane and classic compute plane. This connection is labeled as 2 the diagram below:

Network connectivity overview diagram

For more information on configuring Azure networking features between Azure Databricks and Azure storage, see Grant your Azure Databricks workspace access to Azure Data Lake Storage Gen2.

Enable secure cluster connectivity

Databricks recommends that you enable secure cluster connectivity on your Azure Databricks workspaces. When secure cluster connectivity is enabled, compute resources in the classic control plane connect to the control plane through a relay. This means compute plane has no open ports and classic control plane resources have no public IP addresses. This simplifies network administration by removing the need to configure ports on security groups or network peering. To learn more about deploying a workspace with secure cluster connectivity, see Secure cluster connectivity (No Public IP / NPIP).

Deploy a workspace in your own virtual network

By default, every Azure Databricks deployment creates a locked virtual network (VNet) in your Azure subscription. Classic compute resources are created in that virtual network. You can choose to create a new workspace in your own customer-managed virtual network (also known as VNet injection) instead, enabling you to:

To deploy a workspace in your own virtual network, see Deploy Azure Databricks in your Azure virtual network (VNet injection). You can also peer the Azure Databricks virtual network with another Azure virtual network, see Peer virtual networks.

Enable private connectivity from the control plane to the classic compute plane

Azure Private Link provides private connectivity from Azure VNets and on-premises networks to Azure services without exposing the traffic to the public network. You can enable private connectivity from the classic compute plane to Azure Databricks workspace’s core services in the control plane by enabling Azure Private Link.

For more information, see Enable Azure Private Link back-end and front-end connections.