Encryption for Azure Synapse Analytics workspaces
This article will describe:
- Encryption of data at rest in Synapse Analytics workspaces.
- Configuration of Synapse workspaces to enable encryption with a customer managed key.
- Managing keys used to encrypt data in workspaces.
Encryption of data at rest
A complete Encryption-at-Rest solution ensures the data is never persisted in unencrypted form. Double encryption of data at rest mitigates threats with two, separate layers of encryption to protect against compromises of any single layer. Azure Synapse Analytics offers a second layer of encryption for the data in your workspace with a customer-managed key. This key is safeguarded in your Azure Key Vault, which allows you to take ownership of key management and rotation.
The first layer of encryption for Azure services is enabled with platform-managed keys. By default, Azure Disks, and data in Azure Storage accounts are automatically encrypted at rest. Learn more about how encryption is used in Microsoft Azure in the Azure Encryption Overview.
Note
Some items considered customer content, such as table names, object names, and index names, might be transmitted in log files for support and troubleshooting by Microsoft.
Azure Synapse encryption
This section will help you better understand how customer-managed key encryption is enabled and enforced in Synapse workspaces. This encryption uses existing keys or new keys generated in Azure Key Vault. A single key is used to encrypt all the data in a workspace. Synapse workspaces support RSA 2048 and 3072 byte-sized keys, and RSA-HSM keys.
Note
Synapse workspaces do not support the use of EC, EC-HSM, and oct-HSM keys for encryption.
The data in the following Synapse components is encrypted with the customer-managed key configured at the workspace level:
- SQL pools
- Dedicated SQL pools
- Serverless SQL pools
- Data Explorer pools
- Apache Spark pools
- Azure Data Factory integration runtimes, pipelines, datasets.
Workspace encryption configuration
Workspaces can be configured to enable double encryption with a customer-managed key at the time of workspace creation. Enable double encryption using a customer-managed key on the "Security" tab when creating your new workspace. You can choose to enter a key identifier URI or select from a list of key vaults in the same region as the workspace. The Key Vault itself needs to have purge protection enabled.
Important
The configuration setting for double encryption cannot be changed after the workspace is created.
Key access and workspace activation
The Azure Synapse encryption model with customer-managed keys involves the workspace accessing the keys in Azure Key Vault to encrypt and decrypt as needed. The keys are made accessible to the workspace either through an access policy or Azure Key Vault RBAC. When granting permissions via an Azure Key Vault access policy, choose the "Application-only" option during policy creation (select the workspaces managed identity and do not add it as an authorized application).
The workspace managed identity must be granted the permissions it needs on the key vault before the workspace can be activated. This phased approach to workspace activation ensures that data in the workspace is encrypted with the customer-managed key. Encryption can be enabled or disabled for individual dedicated SQL Pools. Each dedicated pool is not enabled for encryption by default.
Use a User-assigned Managed identity
Workspaces can be configured to use a User-assigned Managed identity to access your customer-managed key stored in Azure Key Vault. Configure a User-assigned Managed identity to avoid phased activation of your Azure Synapse workspace when using double encryption with customer-managed keys. The Managed Identity Contributor built-in role is required to assign a user-assigned managed identity to an Azure Synapse workspace.
Note
A User-assigned Managed Identity cannot be configured to access customer-managed key when Azure Key Vault is behind a firewall.
Permissions
To encrypt or decrypt data at rest, the managed identity must have the following permissions. Similarly, if you are using a Resource Manager template to create a new key, the 'keyOps' parameter of the template must have the following permissions:
- WrapKey (to insert a key into Key Vault when creating a new key).
- UnwrapKey (to get the key for decryption).
- Get (to read the public part of a key)
Workspace activation
If you do not configure a user-assigned managed identity to access customer managed keys during workspace creation, your workspace will remain in a "Pending" state until activation succeeds. The workspace must be activated before you can fully use all functionality. For example, you can only create a new dedicated SQL pool once activation succeeds. Grant the workspace managed identity access to the key vault and select the activation link in the workspace Azure portal banner. Once the activation completes successfully, your workspace is ready to use with the assurance that all data in it's protected with your customer-managed key. As previously noted, the key vault must have purge protection enabled for activation to succeed.
Manage the workspace customer-managed key
You can change the customer-managed key used to encrypt data from the Encryption page in the Azure portal. Here too, you can choose a new key using a key identifier or select from Key Vaults that you have access to in the same region as the workspace. If you choose a key in a different key vault from the ones previously used, grant the workspace-managed identity "Get", "Wrap", and "Unwrap" permissions on the new key vault. The workspace will validate its access to the new key vault and all data in the workspace will be re-encrypted with the new key.
Important
When changing the encryption key of a workspace, retain the old key until you replace it in the workspace with a new key. This allows decryption of data with the old key before it gets re-encrypted with the new key. The state of the SQL pool (Online/Offline) does not affect the workspace customer managed key (CMK) rotation process.
SQL pools that are offline during the CMK rotation will remain encrypted with the old key or key version. If the old key or key version is disabled or expired, the pools will not resume as decryption is not possible. Upon resuming these pools, the old key or key version must 1) be enabled and 2) have an expiration date set in the future to allow decryption and subsequent re-encryption with the new key or key version.
To ensure a smooth CMK rotation, if some SQL pools are offline during the process, the old key or key version should remain enabled and have its expiration date set in the future. This is crucial until the offline pools are successfully resumed and re-encrypted with the new key or key version.
It is highly recommended not to delete old keys or key versions, as they might still be needed to decrypt backups. Instead, after all SQL pools have been re-encrypted with the new key or key version, disable the old key or key version. This ensures the old key or key version remains available for decrypting older backups if necessary.
Azure Key Vaults policies for automatic, periodic rotation of keys, or actions on the keys can result in the creation of new key versions. You can choose to re-encrypt all the data in the workspace with the latest version of the active key. To-re-encrypt, change the key in the Azure portal to a temporary key and then switch back to the key you wish to use for encryption. As an example, to update data encryption using the latest version of active key Key1, change the workspace customer-managed key to temporary key, Key2. Wait for encryption with Key2 to finish. Then switch the workspace customer-managed key back to Key1-data in the workspace will be re-encrypted with the latest version of Key1.
Note
Azure Synapse Analytics does not automatically re-encrypt data when new key versions are created. To ensure consistency in your workspace, force the re-encryption of data using the process detailed above.
SQL Transparent Data Encryption with service-managed keys
SQL Transparent Data Encryption (TDE) is available for dedicated SQL Pools in workspaces not enabled for double encryption. In this type of workspace, a service-managed key is used to provide double encryption for the data in the dedicated SQL pools. TDE with the service-managed key can be enabled or disabled for individual dedicated SQL pools.
Cmdlets for Azure SQL Database and Azure Synapse
To configure TDE through PowerShell, you must be connected as the Azure Owner, Contributor, or SQL Security Manager.
Use the following cmdlets for Azure Synapse workspace.
| Cmdlet | Description |
|---|---|
| Set-AzSynapseSqlPoolTransparentDataEncryption | Enables or disables transparent data encryption for a SQL pool. |
| Get-AzSynapseSqlPoolTransparentDataEncryption | Gets the transparent data encryption state for a SQL pool. |
| New-AzSynapseWorkspaceKey | Adds a Key Vault key to a workspace. |
| Get-AzSynapseWorkspaceKey | Gets the Key Vault keys for a workspace |
| Update-AzSynapseWorkspace | Sets the transparent data encryption protector for a workspace. |
| Get-AzSynapseWorkspace | Gets the transparent data encryption protector |
| Remove-AzSynapseWorkspaceKey | Removes a Key Vault key from a workspace. |