Edit

Share via

Azure Monitor customer-managed keys

  • Article

Data in Azure Monitor is encrypted with Microsoft-managed keys. You can use your own encryption key to protect data in your workspaces. Customer-managed keys in Azure Monitor give you control on the encryption key lifecycle, and access to logs. Once configure, new data ingested to linked workspaces is encrypted with your key in Azure Key Vault, or Azure Key Vault Managed "HSM".

Customer-managed key overview

Data Encryption at Rest is a common privacy and security requirement in organizations. You can let Azure completely manage encryption at rest, or use various options to closely manage encryption and encryption keys.

Azure Monitor ensures that all data and saved queries are encrypted at rest using Microsoft-managed keys (MMK). Azure Monitor's use of encryption is identical to the way Azure Storage encryption operates.

To control the key lifecycle with ability to revoke access data, encrypt data with your own key in Azure Key Vault, or Azure Key Vault Managed "HSM". Customer-managed keys capability is available on dedicated clusters and provide you with higher-level of protection and control.

Data ingested to dedicated clusters is encrypted twice - at the service level using Microsoft-managed keys or customer-managed keys, and at the infrastructure level, using two different encryption algorithms and two different keys. Double encryption protects against a scenario where one of the encryption algorithms or keys is compromised. Dedicated clusters also let you protect data with Lockbox.

Data ingested in the last 14 days, or recently used in queries, is kept in hot-cache (SSD-backed) for query efficiency. SSD data is encrypted with Microsoft-managed keys regardless of whether you configure customer-managed keys, but your control over SSD access adheres to key revocation.

Important

Dedicated clusters use a commitment tier pricing model of at least 100 GB per day.

How customer-managed keys work in Azure Monitor

Azure Monitor uses managed identity to grant access to your key in Azure Key Vault. The identity of the Log Analytics clusters is supported at the cluster level. To provide customer-managed keys on multiple workspaces, a Log Analytics cluster resource serves as an intermediate identity connection between your Key Vault and your Log Analytics workspaces. The cluster's storage uses the managed identity associated with the cluster to authenticate to your Azure Key Vault through Microsoft Entra ID.

Clusters support two managed identity types: System-assigned and User-assigned, while a single identity can be defined in a cluster depending on your scenario.

  • System-assigned managed identity is simpler and generated automatically with cluster when identity type is set to SystemAssigned. This identity is used later to grant storage access to your Key Vault for data encryption and decryption.
  • User-assigned managed identity lets you configure customer-managed keys at cluster creation, when identity type is set to UserAssigned, and granting it permissions in your Key Vault prior to cluster creation.

You can configure customer-managed keys on a new cluster, or an existing dedicated cluster with linked workspaces ingesting data. Similarly, you can unlink workspaces from a cluster at any time. New data ingested to linked workspaces is encrypted with your key, and older data remains encrypted with Microsoft-managed keys. The configuration doesn't interrupt ingestion or queries, where queries are performed across old and new data seamlessly. When you unlink workspaces from a cluster. New data ingested is encrypted with Microsoft-managed keys.

Important

The customer-managed keys capability is regional. Your Azure Key Vault, dedicated cluster, and linked workspaces must be in the same region, but can be in different subscriptions.

Screenshot of customer-managed key overview.

  1. Key Vault
  2. Log Analytics cluster resource that has a managed identity with permissions to Key Vault - the identity is propagated to the underlay dedicated cluster storage
  3. Dedicated cluster
  4. Workspaces linked to dedicated cluster

Encryption keys types

There are three types of keys involved in Storage data encryption:

  • "KEK" - Key Encryption Key (your Customer-managed key)
  • "AEK" - Account Encryption Key
  • "DEK" - Data Encryption Key

The following rules apply:

  • The cluster storage has unique encryption key for every Storage Account, which is known as the "AEK".
  • The "AEK" is used to derive "DEKs, which are the keys that are used to encrypt each block of data written to disk.
  • When configuring customer-managed key "KEK" in your cluster, the cluster storage performs 'wrap' and 'unwrap' requests to your Key Vault for "AEK" encryption and decryption.
  • Your "KEK" never leaves your Key Vault, and, if you store your key in Azure Key Vault Managed "HSM", it never leaves the hardware.
  • Azure Storage uses the managed identity associated with the cluster for authentication. It accesses Azure Key Vault via Microsoft Entra ID.

Customer-Managed key provisioning steps

  1. Creating Azure Key Vault and storing key
  2. Creating a dedicated cluster
  3. Granting permissions to your Key Vault
  4. Updating a dedicated cluster with key identifier details
  5. Linking workspaces

Customer-managed key configuration doesn't support setting up identity and key identifier details currently. These can be be performed via PowerShell, CLI, or REST requests.

Required permissions

To perform cluster-related actions, you need these permissions:

Action Permissions or role needed
Create a dedicated cluster Microsoft.Resources/deployments/*and Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example
Change cluster properties Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example
Link workspaces to a cluster Microsoft.OperationalInsights/clusters/write, Microsoft.OperationalInsights/workspaces/write, and Microsoft.OperationalInsights/workspaces/linkedservices/write permissions, as provided by the Log Analytics Contributor built-in role, for example
Check workspace link status Microsoft.OperationalInsights/workspaces/read permissions to the workspace, as provided by the Log Analytics Reader built-in role, for example
Get clusters or check a cluster's provisioning status Microsoft.OperationalInsights/clusters/read permissions, as provided by the Log Analytics Reader built-in role, for example
Update commitment tier or billingType in a cluster Microsoft.OperationalInsights/clusters/write permissions, as provided by the Log Analytics Contributor built-in role, for example
Grant the required permissions Owner or Contributor role that has */write permissions, or the Log Analytics Contributor built-in role, which has Microsoft.OperationalInsights/* permissions
Unlink a workspace from cluster Microsoft.OperationalInsights/workspaces/linkedServices/delete permissions, as provided by the Log Analytics Contributor built-in role, for example
Delete a dedicated cluster Microsoft.OperationalInsights/clusters/delete permissions, as provided by the Log Analytics Contributor built-in role, for example

Storing encryption key ("KEK")

A portfolio of Azure Key Management products lists the vaults and managed HSMs that can be used.

Create or use an existing Azure Key Vault in the region that the cluster is planed. In your Key vault, generate or import a key to be used for logs encryption. The Azure Key Vault must be configured as recoverable, to protect your key and the access to your data in Azure Monitor. You can verify this configuration under properties in your Key Vault, both Soft delete and Purge protection should be enabled.

Important

It's recommended to set up notification and respond to Azure Key Vault events such as key nearing expiry, received via Azure Event Grid. When key expires, ingestion and queries aren't affected, but you can't perform update on the key and will need to contact support.

Screenshot of soft delete and purge protection settings.

These settings can be updated in Key Vault via CLI and PowerShell:

Create cluster

Clusters use managed identity for data encryption with your Key Vault. Configure identity type property to SystemAssigned or UserAssigned when creating your cluster to allow access to your Key Vault for data encryption and decryption operations.

FOr example, add these properties in request body for System-assigned managed identity

JSON 
{
  "identity": {
    "type": "SystemAssigned"
    }
}

Note

Identity type can be changed after the cluster is created with no interruption to ingestion or queries, with the following considerations

  • Identity and key can't be updated simultaneously in cluster —update in two consecutive operations
  • Updating SystemAssigned to UserAssigned—Grant UserAssign identity in Key Vault, then update identity in cluster
  • Updating UserAssigned to SystemAssigned—Grant SystemAssigned identity in Key Vault, then update identity in cluster

Follow the procedure illustrated in dedicated cluster article.

Grant Key Vault permissions

There are two permission models in Key Vault to grant access to your cluster and underlay storage—Azure role-based access control (Azure RBAC), and Vault access policies (legacy).

  1. Assign Azure RBAC you control (recommended)

    To add role assignments, you must have a role with Microsoft.Authorization/roleAssignments/write and Microsoft.Authorization/roleAssignments/delete permissions, such as User Access Administrator or Owner.

    1. Open your Key Vault in Azure portal and select Settings > Access configuration > Azure role-based access control and Apply
    2. Click Go to access control(IAM) button and add Key Vault Crypto Service Encryption User role assignment.
    3. Select Managed identity in Members tab and select the subscription for identity and the identity as member

    Screenshot of Grant Key Vault RBAC permissions.

  2. Assign vault access policy (legacy)

    Open your Key Vault in Azure portal and select Access Policies > Vault access policy > + Add Access Policy to create a policy with these settings:

    • Key permissions—select Get, Wrap Key and Unwrap Key.
    • Select principal—depending on the identity type used in the cluster (system or user assigned managed identity)
      • System assigned managed identity - enter the cluster name or cluster principal ID
      • User assigned managed identity - enter the identity name

    Screenshot of Grant Key Vault access policy permissions.

    The Get permission is required to verify that your Key Vault is configured as recoverable to protect your key and the access to your Azure Monitor data.

Update cluster with key identifier details

All operations on the cluster require the Microsoft.OperationalInsights/clusters/write action permission. It permission could be granted via the Owner or Contributor that contains the */write action, or via the Log Analytics Contributor role that contains the Microsoft.OperationalInsights/* action.

This step updates dedicated cluster storage with the key and version to use for "AEK" wrap and unwrap.

Important

  • Key rotation can be automatic or per explicit key version, see Key rotation to determine suitable approach before updating the key identifier details in cluster.
  • Cluster update should not include both identity and key identifier details in the same operation. If you need to update both, the update should be in two consecutive operations.

Screenshot of Grant Key Vault permissions.

Update KeyVaultProperties in cluster with key identifier details.

The operation is asynchronous and can take a while to complete.

N/A

Important

This step should be performed only after the cluster provisioning. If you link workspaces and ingest data prior to the provisioning, ingested data will be dropped and won't be recoverable.

Follow the procedure illustrated in Dedicated Clusters article.

Follow the procedure illustrated in Dedicated Clusters article.

Key revocation

Important

  • The recommended way to revoke access to your data is by disabling your key, or deleting Access Policy in your Key Vault.
  • Setting the cluster's identity type to None also revokes access to your data, but this approach isn't recommended since you can't revert it without contacting support.

The cluster's storage always respects changes in key permissions within an hour or sooner, and storage become unavailable. New data ingested to linked workspaces is dropped and unrecoverable. Data is inaccessible on these workspaces and queries fail. Previously ingested data remains in storage as long as your cluster and your workspaces aren't deleted. Inaccessible data is governed by the data-retention policy and purged when retention is reached. Data ingested in the last 14 days and data recently used in queries is also kept in hot-cache (SSD-backed) for query efficiency. The data on SSD gets deleted on key revocation operation and becomes inaccessible. The cluster storage attempts to reach Key Vault for wrap and unwrap periodically, and once key is enabled, unwrap succeeds, SSD data is reloaded from storage, and data ingestion and query are resumed within 30 minutes.

Key rotation

Key rotation has two modes:

  • Autorotation—update "keyVaultProperties" properties in cluster and omit "keyVersion" property, or set it to "". Storage automatically uses the latest key version.
  • Explicit key version update—update "keyVaultProperties" properties and update the key version in "keyVersion" property. Key rotation requires explicit update of "keyVersion" property in cluster. For more information, see Update cluster with Key identifier details. If you generate a new key version in Key Vault but don't update the key in the cluster, the cluster storage keeps using your previous key. If you disable or delete the old key before updating a new one in the cluster, you get into key revocation state.

All your data remains accessible during and after the key rotation operation. Data always encrypted with the Account Encryption Key ("AEK"), which is encrypted with your new Key Encryption Key ("KEK") version in Key Vault.

Customer-managed key for saved queries and log search alerts

The query language used in Log Analytics is expressive and can contain sensitive information in comments, or in the query syntax. Some organizations require that such information is kept protected under Customer-managed key policy and you need save your queries encrypted with your key. Azure Monitor enables you to store saved queries and log search alerts encrypted with your key in your own Storage Account when linked to your workspace.

Customer-managed key for Workbooks

With the considerations mentioned for Customer-managed key for saved queries and log search alerts, Azure Monitor enables you to store Workbook queries encrypted with your key in your own Storage Account, when selecting Save content to an Azure Storage Account in Workbook 'Save' operation.

Screenshot of Workbook save.

Note

Queries remain encrypted with Microsoft key ("MMK") in the following scenarios regardless Customer-managed key configuration: Azure dashboards, Azure Logic App, Azure Notebooks and Automation Runbooks.

When linking your Storage Account for saved queries, the service stores saved queries and log search alert queries in your Storage Account. Having control on your Storage Account encryption-at-rest policy, you can protect saved queries and log search alerts with Customer-managed key. You will, however, be responsible for the costs associated with that Storage Account.

Considerations before setting Customer-managed key for queries

  • You need to have "write" permissions on your workspace and Storage Account.
  • Make sure to create your Storage Account in the same region as your Log Analytics workspace is located, with Customer-managed key encryption. This is important since saved queries are stored in table storage and it can only be encrypted at Storage Account creation.
  • Queries saved in query pack aren't encrypted with Customer-managed key. Select Save as Legacy query when saving queries instead, to protect them with Customer-managed key.
  • Saves queries in storage are considered service artifacts and their format might change.
  • Linking a Storage Account for queries removes existing saves queries from your workspace. Copy saves queries that you need before this configuration. You can view your saved queries using PowerShell.
  • Query 'history' and 'pin to dashboard' aren't supported when linking Storage Account for queries.
  • You can link a single Storage Account to a workspace for both saved queries and log search alert queries.
  • Log search alerts are saved in blob storage and Customer-managed key encryption can be configured at Storage Account creation, or later.
  • Fired log search alerts won't contain search results or alert query. You can use alert dimensions to get context in the fired alerts.

Configure BYOS for saved queries

Link a Storage Account for queries to keep saved queries in your Storage Account.

N/A

After the configuration, any new saved search query will be saved in your storage.

Configure BYOS for log search alert queries

Link a Storage Account for Alerts to keep log search alert queries in your Storage Account.

N/A

After the configuration, any new alert query will be saved in your storage.

Customer Lockbox

Lockbox gives you the control to approve or reject Microsoft engineer request to access your data during a support request.

Lockbox is provided in dedicated cluster in Azure Monitor, where your permission to access data is granted at the subscription level.

Learn more about Customer Lockbox for Microsoft Azure

Customer-Managed key operations

Customer-Managed key is provided on dedicated cluster and these operations are referred in dedicated cluster article

  • Get all clusters in resource group
  • Get all clusters in subscription
  • Update capacity reservation in cluster
  • Update billingType in cluster
  • Unlink a workspace from cluster
  • Delete cluster

Limitations and constraints

  • A maximum of five active clusters can be created in each region and subscription.

  • A maximum number of seven reserved clusters (active or recently deleted) can exist in each region and subscription.

  • A maximum of 1,000 Log Analytics workspaces can be linked to a cluster.

  • A maximum of two workspace link operations on particular workspace is allowed in 30 day period.

  • Moving a cluster to another resource group or subscription isn't currently supported.

  • Cluster update shouldn't include both identity and key identifier details in the same operation. In case you need to update both, the update should be in two consecutive operations.

  • Lockbox isn't available in China currently.

  • Lockbox doesn't apply to tables with the Auxiliary plan.

  • Double encryption is configured automatically for clusters created from October 2020 in supported regions. You can verify if your cluster is configured for double encryption by sending a GET request on the cluster and observing that the isDoubleEncryptionEnabled value is true for clusters with Double encryption enabled.

    • If you create a cluster and get an error—"region-name doesn’t support Double Encryption for clusters", you can still create the cluster without Double encryption, by adding "properties": {"isDoubleEncryptionEnabled": false} in the REST request body.
    • Double encryption settings can't be changed after the cluster is created.

  • Deleting a linked workspace is permitted while linked to cluster. If you decide to recover the workspace during the soft-delete period, it returns to previous state and remains linked to cluster.

  • Customer-managed key encryption applies to newly ingested data after the configuration time. Data that was ingested before the configuration remains encrypted with Microsoft keys. You can query data ingested before and after the customer-managed key configuration seamlessly.

  • The Azure Key Vault must be configured as recoverable. These properties aren't enabled by default and should be configured using CLI or PowerShell:

  • Your Azure Key Vault, cluster and workspaces must be in the same region and in the same Microsoft Entra tenant, but they can be in different subscriptions.

  • Setting the cluster's identity type to None also revokes access to your data, but this approach isn't recommended since you can't revert it without contacting support. The recommended way to revoke access to your data is key revocation.

  • You can't use Customer-managed key with User-assigned managed identity if your Key Vault is in Private-Link (vNet). You can use System-assigned managed identity in this scenario.

  • The Auxiliary table plan doesn't support customer-managed keys. Data in tables with the Auxiliary plan is encrypted with Microsoft-managed keys, even if you protect the data in the rest of your Log Analytics workspace using your own encryption key.

Troubleshooting

  • Behavior per Key Vault availability:

    • Normal operation—storage caches "AEK" for short periods of time and goes back to Key Vault to unwrap periodically.

    • Key Vault connection errors—storage handles transient errors (timeouts, connection failures, "DNS" issues), by allowing keys to stay in cache during the availability issue, and it overcomes blips and availability issues. The query and ingestion capabilities continue without interruption.

  • Key Vault access rate - the frequency with which the cluster storage accesses Key Vault for wrap and unwrap - is between 6 to 60 seconds.

  • If you update your cluster while it's at provisioning state, or updating state, the update fails.

  • If you get conflict—error when creating a cluster, a cluster with the same name may have been deleted in the last 14 days and kept reserved. Deleted cluster name becomes available 14 days after deletion.

  • Linking a workspace to a cluster fails if the workspace is linked to another cluster.

  • If you create a cluster and specify the KeyVaultProperties immediately, the operation might fail until an identity is assigned to the cluster, and granted to Key Vault.

  • If you update existing cluster with KeyVaultProperties and 'Get' key Access Policy is missing in Key Vault, the operation fails.

  • If you fail to deploy your cluster, verify that your Azure Key Vault, cluster and linked workspaces are in the same region. The can be in different subscriptions.

  • If you rotate your key in Key Vault and don't update the new key identifier details in the cluster, cluster keep using previous key and your data will become inaccessible. Update new key identifier details in cluster to resume data ingestion and query. You can update key version with "''" to have storage always use lates key version automatically.

  • Some operations are long running and can take a while to complete, include cluster create, cluster key update and cluster delete. You can check the operation status by sending GET request to cluster or workspace and observe the response. For example, unlinked workspace won't have the clusterResourceId under features.

  • Error messages

    Cluster Update

    • 400 - Cluster is in deleting state. Async operation is in progress. Cluster must complete its operation before any update operation is performed.
    • 400 - KeyVaultProperties isn't empty but has a bad format. See key identifier update.
    • 400 - Failed to validate key in Key Vault. Could be due to lack of permissions or when key doesn’t exist. Verify that you set key and Access Policy in Key Vault.
    • 400 - Key isn't recoverable. Key Vault must be set to Soft-delete and Purge-protection. See Key Vault documentation
    • 400 - Operation can't be executed now. Wait for the Async operation to complete and try again.
    • 400 - Cluster is in deleting state. Wait for the Async operation to complete and try again.

    Cluster Get

    • 404--Cluster not found, the cluster might have been deleted. If you try to create a cluster with that name and get a conflict, the cluster is in the deletion process.

Next steps