Configure HSM customer-managed keys for DBFS using the Azure portal

Article
05/23/2024

Note

This feature is available only in the Premium plan.

You can use the Azure portal to configure your own encryption key to encrypt the workspace storage account. This article describes how to configure your own key from Azure Key Vault Managed HSM. For instructions on using a key from Azure Key Vault vaults, see Configure customer-managed keys for DBFS using the Azure portal.

Important

The Key Vault must be in the same Azure tenant as your Azure Databricks workspace.

For more information about customer-managed keys for DBFS, see Customer-managed keys for DBFS root.

Create an Azure Key Vault Managed HSM and an HSM key

You can use an existing Azure Key Vault Managed HSM or create and activate a new one following Quickstart: Provision and activate a Managed HSM using Azure CLI. The Azure Key Vault Managed HSM must have Purge Protection enabled.

To create an HSM key, follow Create an HSM key.

Prepare the workspace storage account

Go to your Azure Databricks service resource in the Azure portal.
In the left menu, under Automation, select Export template.
Click Deploy.
Click Edit template, search for prepareEncryption, and modify the vault to true type. For example:
```
  "prepareEncryption": {
           "type": "Bool",
           "value": "true"
        }
```
Click Save.
Click Review + Create to deploy the change.
On the right, under Essentials, click JSON View.
Search for storageAccountIdentity, and copy the principalId.

Configure the Managed HSM role assignment

Go to your Managed HSM resource in the Azure portal.
In the left menu, under Settings, select Local RBAC.
Click Add.
In the Role field, select Managed HSM Crypto Service Encryption User.
In the Scope field, select All keys (/).
In the Security principal field, enter the principalId of the workspace storage account in the search bar. Select the result.
Click Create.
In the left menu, under Settings, select Keys and select your key.
In the Key Identifier field, copy the text.

Encrypt the workspace storage account using your HSM key

Go to your Azure Databricks service resource in the Azure portal.
In the left menu, under Settings, select Encryption.
Select Use your own key, enter your Managed HSM key’s Key Identifier, and select the Subscription that contains the key.
Click Save to save your key configuration.

Regenerate (rotate) keys

When you regenerate a key, you must return to the Encryption page in your Azure Databricks service resource, update the Key Identifier field with your new key identifier, and click Save. This applies to new versions of the same key as well as new keys.

Important

If you delete the key that is used for encryption, the data in DBFS root cannot be accessed.

Share via