Configure enhanced security and compliance settings

2025-05-28

This page describes how to configure enhanced security and compliance settings on your Azure Databricks workspace.

Important

Enabling the compliance security profile or adding compliance standards to a workspace is intended to be a permanent change.
You cannot remove the compliance profile or individual standards from a workspace if it has ever processed regulated data. To revert, you must delete the workspace and create a new one without the profile or with a different standard. For help, contact Azure Databricks support.

Requirements

Your Azure Databricks workspace is on the Premium pricing tier.
Your Databricks account must include the Enhanced Security and Compliance add-on.

Ensure that sensitive information is never entered in customer-defined input fields, such as workspace names, cluster names, tags, and job names.

Compliance security profile requirements

If the workspace is configured to restrict outbound network access, you must configure your network to additionally allow traffic to port 2443. See Deploy Azure Databricks in your Azure virtual network (VNet injection).
Generation 2 VMs and Arm64-based virtual machines are not supported. Azure Databricks does not allow starting compute with those instance types when the compliance security profile is enabled.
You must use the following VM instance types:
- General purpose: Dv5 and Dsv5-series, Ddv5 and Ddsv5-series, Dlsv5 and Dldsv5-series, Dasv5 and Dadsv5-series, Dasv6 and Dadsv6-series, Dalsv6 and Daldsv6-series, Dsv6-series, Dplsv6 and Dpldsv6-series, Dpsv6 and Dpdsv6-series
- Memory intensive workloads: Ev5 and Esv5-series, Edv5 and Edsv5-series, Easv5 and Eadsv5-series, Easv6 and Eadsv6-series, Epsv6 and Epdsv6-series
- Compute optimized: Falsv6-series, Famsv6-series, Fasv6-series

Note

Databricks Assistant is disabled by default on workspaces that have enabled the compliance security profile. Workspace admins can enable it by following the instructions For an account: Disable or enable Azure AI-powered AI features.

Set account-level defaults for all new workspaces

Account admins can configure settings for security profile (with compliance standards) or enhanced security monitoring at an account level to apply to all new workspaces. When you enable the compliance security profile as a default for new workspaces, enhanced security monitoring and automatic cluster update are also enabled for new workspaces.

As an account admin, go to the account console.
In the sidebar, click Settings.
Click the Security and compliance tab.
In the sidebar, click Enhanced Security and Compliance Settings.
To enable the compliance security profile, next to Compliance security profile, click Configure.

In the Compliance security profile for new workspaces dialog, select Enabled, select one or compliance standards or select None and click Save.
To enable enhanced security monitoring, select the checkbox Enhanced security monitoring for new workspaces.

Enable enhanced security and compliance features on a workspace

You can create a workspace with enhanced security and compliance features using the Azure portal, the Azure CLI, Powershell, an ARM template, or Terraform.

Use the Azure portal

In the Azure portal, click the Settings > Security & compliance an existing Azure Databricks workspace or on the Azure Databricks workspace creation page.
To enable the compliance security profile, select the checkbox next to Enable compliance security profile. In the dropdown, select one or more compliance standards or select None. The dropdown lists compliance standards available in your workspace region.

If you enable the compliance security profile or add compliance standards, those selections are permanent for that workspace.
To enable enhanced security monitoring, select the checkbox Enable enhanced security monitoring.
To enable automatic cluster update, select the checkbox Enable automatic cluster update.

To configure the maintenance window and its frequency, see Automatic cluster update

Use the Azure CLI

You can create a workspace with enhanced security and compliance features using the Azure CLI. Possible compliance standards include: HIPAA, PCI_DSS, HITRUST, IRAP_PROTECTED, UK_CYBER_ESSENTIALS_PLUS, CANADA_PROTECTED_B, or NONE. You can select more than one compliance standard. For example:

az databricks workspace create --resource-group MyResourceGroup --name MyWorkspace --location westus --sku premium --enable-compliance-security-profile --compliance-standards='["HIPAA"]' --enable-automatic-cluster-update --enable-enhanced-security-monitoring

Use Powershell

You can create a workspace with enhanced security and compliance features using Powershell. Possible compliance standards include: HIPAA, PCI_DSS, HITRUST, IRAP_PROTECTED, UK_CYBER_ESSENTIALS_PLUS, CANADA_PROTECTED_B, or NONE. You can select more than one compliance standard. For example:

New-AzDatabricksWorkspace -Name MyWorkspace -ResourceGroupName MyResourceGroup -Location westus -Sku "Premium" -EnhancedSecurityMonitoring 'Enabled' -AutomaticClusterUpdate 'Enabled' -EnhancedSecurityCompliance 'Enabled' -ComplianceStandard @("HIPAA","PCI_DSS")

Use an ARM template

You can configure the Enhanced Security and Compliance add-on features with an ARM template that Databricks provides. It contains additional parameters that you can set to Enabled or Disabled. If you want to add them to an existing template to update the workspace, you can do so. You can set features independently except as indicated:

complianceSecurityProfile: Enables the compliance security profile. Once enabled, this feature is permanently enabled on the workspace.
complianceStandards: Configures an array of compliance standards to use with the compliance security profile.
- If complianceSecurityProfile is set to Disabled, pass an empty array.
- If complianceSecurityProfile is set to Enabled, you must pass an array of one or more strings that specify which (if any) compliance standards you want for your workspace. Possible selections are HIPAA, PCI_DSS, HITRUST, IRAP_PROTECTED, UK_CYBER_ESSENTIALS_PLUS, CANADA_PROTECTED_B, or NONE. Add the single array element NONE if you are using the compliance security profile only for its security benefits but not to process any regulated data.
enhancedSecurityMonitoring — Enables enhanced security monitoring. If the compliance security profile is enabled, you must set this feature to Enabled explicitly in the template.
automaticClusterUpdate — Enables automatic cluster update. If the compliance security profile is enabled, you must set this feature to Enabled explicitly in the template. To configure the maintenance window and its frequency, see Automatic cluster update.

To update a workspace with one or more of these features, follow the same instructions for deploying a custom template as you would for creating a new workspace with a template. However, check that you use your original template and then copy the fields from the provided example template into your existing workspace template.

Workspace template with enhanced security and compliance features

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "disablePublicIp": {
      "type": "bool",
      "defaultValue": false,
      "metadata": {
        "description": "Specifies whether to deploy Azure Databricks workspace with secure cluster connectivity (No Public IP) enabled."
      }
    },
    "workspaceName": {
      "type": "string",
      "metadata": {
        "description": "The name of the Azure Databricks workspace to create."
      }
    },
    "pricingTier": {
      "type": "string",
      "defaultValue": "premium",
      "allowedValues": ["standard", "premium"],
      "metadata": {
        "description": "The pricing tier of workspace."
      }
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    },
    "automaticClusterUpdate": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": ["Disabled", "Enabled"],
      "metadata": {
        "description": "Enable/Disable automatic cluster update"
      }
    },
    "enhancedSecurityMonitoring": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": ["Disabled", "Enabled"],
      "metadata": {
        "description": "Enable/Disable enhanced security monitoring"
      }
    },
    "complianceSecurityProfile": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": ["Disabled", "Enabled"],
      "metadata": {
        "description": "Enable/Disable the Compliance Security Profile"
      }
    },
    "complianceStandards": {
      "type": "array",
      "defaultValue": [],
      "allowedValues": [
        [],
        ["NONE"],
        ["HIPAA"],
        ["PCI_DSS"],
        ["HITRUST"],
        ["IRAP_PROTECTED"],
        ["UK_CYBER_ESSENTIALS_PLUS"],
        ["CANADA_PROTECTED_B"]
      ],
      "metadata": {
        "description": "Specify the desired compliance standards for your compliance security profile"
      }
    }
  },
  "variables": {
    "managedResourceGroupName": "[format('databricks-rg-{0}-{1}', parameters('workspaceName'), uniqueString(parameters('workspaceName'), resourceGroup().id))]",
    "trimmedMRGName": "[substring(variables('managedResourceGroupName'), 0, min(length(variables('managedResourceGroupName')), 90))]",
    "managedResourceGroupId": "[format('{0}/resourceGroups/{1}', subscription().id, variables('trimmedMRGName'))]"
  },
  "resources": [
    {
      "type": "Microsoft.Databricks/workspaces",
      "apiVersion": "2023-09-15-preview",
      "name": "[parameters('workspaceName')]",
      "location": "[parameters('location')]",
      "sku": {
        "name": "[parameters('pricingTier')]"
      },
      "properties": {
        "managedResourceGroupId": "[variables('managedResourceGroupId')]",
        "parameters": {
          "enableNoPublicIp": {
            "value": "[parameters('disablePublicIp')]"
          }
        },
        "enhancedSecurityCompliance": {
          "automaticClusterUpdate": {
            "value": "[parameters('automaticClusterUpdate')]"
          },
          "complianceSecurityProfile": {
            "value": "[parameters('complianceSecurityProfile')]",
            "complianceStandards": "[parameters('complianceStandards')]"
          },
          "enhancedSecurityMonitoring": {
            "value": "[parameters('enhancedSecurityMonitoring')]"
          }
        }
      }
    }
  ],
  "outputs": {
    "workspace": {
      "type": "object",
      "value": "[reference(resourceId('Microsoft.Databricks/workspaces', parameters('workspaceName')), '2023-09-15-preview', 'full')]"
    }
  }
}

Use Terraform

Enhanced security and compliance can also be enabled on a Azure Databricks workspace using the azurerm Terraform plugin for Databricks. For more information about the azurerm Terraform plugin, see azurerm_databricks_workspace.

For example, to create an Azure Databricks workspace with compliance controls enabled, use the following:

resource "azurerm_databricks_workspace" "this" {
  name                        = "${local.prefix}-workspace"
  resource_group_name         = azurerm_resource_group.this.name
  location                    = azurerm_resource_group.this.location
  sku                         = "premium"
  managed_resource_group_name = "${local.prefix}-workspace-rg"
  tags                        = local.tags

  enhanced_security_compliance {
  automatic_cluster_update_enabled    = true
  compliance_security_profile_enabled   = true
  compliance_security_profile_standards = ["HIPAA", "PCI_DSS", "HITRUST", "IRAP_PROTECTED", "UK_CYBER_ESSENTIALS_PLUS", "CANADA_PROTECTED_B", "NONE"]
  enhanced_security_monitoring_enabled  = true
  }
}

Confirm that the compliance security profile is enabled for a workspace

You can confirm a workspace is using the compliance security profile in the Security and compliance tab on the workspace page in the account console.

Shield account.

The workspace also has a shield logo displayed in the workspace UI. A shield logo appears in the top-right of the page, to the right of the workspace name. Click the workspace name to see a list of the workspaces that you have access to. The workspaces that enable the compliance security profile have a shield icon.

If the shield icons are missing for a workspace with the compliance security profile enabled, contact your Azure Databricks account team.