Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
The Isolated environment architecture inherits the Hardened connectivity baseline and adds two requirements: private workspace access and a required external firewall. Workspace access is gated by VPN or inbound Private Link, never the public internet. All classic compute egress flows through the firewall for inspection and policy enforcement.
This architecture has:
- Complete network isolation: All traffic flows through private connections.
- Private workspace access: VPN or inbound Private Link only. The workspace is unreachable from the public internet.
- Required egress inspection: Firewall inspection of all classic compute outbound traffic.
- Data exfiltration prevention: Network-layer controls block unauthorized data transfer.
Use this architecture when:
- Workspace access must be private, like over VPN or inbound Private Link.
- Handling data in highly regulated industries, for example financial services, healthcare, government.
- Compliance frameworks require egress controls (for example, SOC 2, HIPAA, PCI DSS, and FedRAMP).
- Implementing enterprise zero-trust security frameworks.
- Data exfiltration prevention is a requirement.
Prerequisites
- Azure Azure Databricks Premium tier with a VNet-injected workspace.
- Existing VPN infrastructure or inbound Private Link connectivity.
- Firewall or network virtual appliance (NVA).
Architecture overview
The Isolated environment architecture routes all traffic through private connections with firewall inspection:
| Traffic type | Path |
|---|---|
| User access | Users → VPN or inbound Private Link → Workspace |
| Classic compute → control | Compute → Classic Private Link → Azure Databricks control plane |
| Classic compute → cloud | Compute → Service endpoints or UDRs → Azure services |
| Serverless → your resources | Serverless compute → NCC private endpoints → Your Azure resources |
| Classic compute → egress | Compute → External firewall (required) → Inspected internet |
Required components
Inbound
Workspace is reachable only over private connections: VPN, inbound Private Link, or both depending on your existing infrastructure. Customers typically select one rather than stacking them.
Private access settings (disable public access)
This is the gating control that actually blocks public ingress. Without it, the workspace still accepts internet traffic even with Private Link configured. Private Link becomes an additional path, not the only path.
Set the workspace's Public Network Access to Disabled in the Azure portal. This blocks public ingress to the workspace UI and APIs.
Workspace ingress controls
Configure workspace ingress through context-based ingress (CBI), the recommended ingress policy framework. CBI rules combine network source (IP ranges), identity, authentication mechanism, and access scope into a single allow/deny model, so the network-source attribute does the same job as the standalone IP access list feature, plus more.
IP access lists remain supported and can be configured alongside CBI. When both are configured, a request must be allowed by both controls.
Configuration levels:
- Account-level CBI policies: Apply to all workspaces in the account. See Manage context-based ingress policies.
- Workspace-level IP access lists: Apply to a single workspace. See Configure IP access lists for workspaces.
- Account-level IP access lists: Apply to the account console. See Configure IP access lists for the account console.
Best practices:
- Start broad, refine based on actual usage.
- Document IP ranges with purpose and expiration dates.
- Maintain administrator access through a known-good IP range.
- Review quarterly and remove obsolete ranges.
Warning
Ingress policies and IP access lists can lock you out of your workspace if misconfigured. Always maintain administrator access through a known-good IP range.
Delta Sharing recipient access control
Delta Sharing uses its own IP access lists configured on recipient objects. This is separate from workspace IP access lists and is not covered by context-based ingress. Applies to open sharing (non-Azure Databricks recipients) only.
See Restrict Delta Sharing recipient access using IP access lists (open sharing).
Inbound connectivity
Establishes private connectivity for user access to the workspace UI and API. Users reach the workspace over VPN or inbound Private Link, never the public internet.
Custom DNS
Configure private DNS to resolve Azure Databricks endpoints to private IP addresses.
Azure automatically creates private DNS zones when creating private endpoints.
Outbound
Serverless egress controls (network policies and NCC private endpoints) are inherited from the Hardened connectivity baseline. This architecture makes the external firewall, optional in Hardened, required for full classic compute egress inspection.
External firewall (required)
Route all egress traffic through a firewall for inspection, logging, and policy enforcement. Options include:
- Azure Firewall or third-party network virtual appliances (NVAs).
Tip
Use service endpoint policies for Azure Databricks artifact storage to bypass the firewall, reducing data transfer costs. Artifact storage alone can account for up to 11 GB downloaded per cluster node.
See IP addresses and domains for Azure Databricks services and assets for the required Azure Databricks endpoints that firewall rules must allow.
Tip
For maximum lockdown, consider hosting a private package repository (such as JFrog Artifactory or Sonatype Nexus) for Python, R, and Maven packages. This eliminates the need for firewall rules allowing access to public package indexes like PyPI.
Warning
Azure Databricks control plane and SCC relay connections use TLS with certificate pinning. Do not enable TLS inspection (decrypt and re-encrypt) on traffic between your clusters and the Azure Databricks control plane. Doing so causes cluster failures. Configure firewall rules to allow these connections by destination FQDN or IP without TLS interception. See IP addresses and domains for Azure Databricks services and assets for required endpoints.
Important
Incorrectly configured firewall rules can break Azure Databricks capabilities. Test thoroughly in a non-production environment.
Data exfiltration protection
Configure network policies and firewall controls to prevent unauthorized data exfiltration:
- Serverless egress control through network policies.
- Classic compute egress through firewall/NVA.
- Private endpoint rules for approved data destinations.
See Data exfiltration protection for implementation guidance.
Classic compute baseline
The classic compute baseline is inherited from Managed security, and cloud service endpoints are inherited from Hardened connectivity. No additional classic compute components are required for this architecture.
The baseline includes VNet injection, Secure Cluster Connectivity (SCC), and classic Private Link. Cloud service endpoints include user-defined routes (UDRs), service endpoints, and private endpoints for customer-managed storage accounts.
Egress approaches for data access
There are two approaches to handling outbound data access from compute resources:
NAT gateway with firewall: Deploy a NAT gateway for outbound connectivity and route traffic through a firewall for inspection. This approach allows controlled access to external package repositories and APIs and maintains visibility into traffic patterns. Use this approach when you must access external resources but require inspection and logging.
No NAT gateway (fully private): Remove the NAT gateway entirely to eliminate all public communication from compute resources. All data access occurs through private endpoints and VPC endpoints only. This approach has the highest level of security by removing the possibility of data exfiltration through public egress paths. Use this approach when your organization prohibits any public internet communication from compute resources.
Implementation
Start from a deployed Hardened connectivity baseline. The following phases add the private workspace access and required external firewall that define this architecture.
Phase 1: Inbound controls
- Configure inbound Private Link so user access to the Azure Azure Databricks UI and APIs is routed privately instead of over public IPs. See Configure Inbound Private Link.
- Set the workspace's Public Network Access to Disabled in the Azure portal. This is what actually blocks public ingress. Without it, the workspace still accepts internet traffic even with inbound Private Link configured.
- Test user access via VPN or Private Link to confirm that authenticated users can reach the workspace only through the private network path, and that public access is blocked.
Phase 2: External firewall (required)
- Deploy Azure Firewall or a third-party network virtual appliance (NVA) in a hub VNet and connect the workspace VNet using VNet peering or a virtual hub.
- Configure user-defined routes (UDRs) in the workspace subnets with a default route to the firewall so outbound traffic from Azure Databricks compute flows through the hub firewall. See User-defined route settings for Azure Databricks.
- Configure Azure Firewall application and network rules to allow only required Azure Databricks endpoints (see IP addresses and domains for Azure Databricks services and assets) without TLS interception on control plane and SCC relay traffic.
Phase 3: Validation
- Verify egress control by reviewing Azure Firewall logs and diagnostics to verify Azure Databricks traffic is inspected and limited according to policy.
- Confirm no public IP addresses are assigned to cluster nodes or other Azure Databricks-managed compute resources in the workspace VNet.
- Validate that all control, data, and incoming traffic flows through the configured Private Link endpoints and hub firewall as designed.
The Azure Databricks Terraform SRA provides Infrastructure-as-Code templates as a starting point for this deployment pattern.
Validation
After you deploy the architecture, run the following checks to confirm that full network isolation, private connectivity, and egress controls work as configured.
| Check | Expected result |
|---|---|
| Workspace accessible via VPN | Yes |
| Workspace accessible without VPN | No |
| Clusters launch with SCC | Yes, no public IPs |
| Data access through private connections | Yes |
| Egress blocked without firewall approval | Yes |
| DNS resolves to private IPs | Yes |
Troubleshooting
If a validation check fails or a workload can't connect to a required endpoint, use the cloud-specific table that follows to diagnose common issues.
| Issue | Cause | Resolution |
|---|---|---|
| Cluster fails to start | Firewall blocking required endpoints or misconfigured private endpoints for SCC, Azure Databricks control plane, or storage accounts (NSG rules, routing) | Review firewall logs and add Azure Databricks infrastructure rules; verify private endpoint NSG rules allow traffic from cluster subnets; check UDRs |
| DNS resolution fails | Private DNS misconfigured | Verify private DNS zones and VNet links |
| Storage access fails | Private endpoint or routing issue | Check private endpoint configuration and route tables |
| Package installation fails | PyPI blocked by firewall | Add PyPI to firewall allow list |
Ongoing maintenance
- Firewall rules: Review and update egress allow-lists regularly.
- DNS management: Update records when you add workspaces.
- Endpoint monitoring: Track private endpoint health and data transfer costs.
- Network policies: Add private endpoints for new approved data sources.
- Remove firewall: If firewall operational overhead is too high or compliance requirements relax, you can remove the firewall component and keep private connectivity and VPN access.
- Downgrade to Hardened connectivity: If private workspace access becomes a productivity barrier.
Next steps
| Resource | Description |
|---|---|
| Data exfiltration protection | Detailed reference architecture for combining network and Unity Catalog controls to prevent data exfiltration. |
| Networking | Networking options and concepts for Azure Databricks. |