Client-side encryption for blobs

Стаття
10/03/2024

The Azure Blob Storage client library for .NET supports encrypting data within client applications before uploading to Azure Storage, and decrypting data while downloading to the client. The library also supports integration with Azure Key Vault for storage account key management.

Important

Blob Storage supports both service-side and client-side encryption. For most scenarios, Microsoft recommends using service-side encryption features for ease of use in protecting your data. To learn more about service-side encryption, see Azure Storage encryption for data at rest.

For a step-by-step tutorial that leads you through the process of encrypting blobs using client-side encryption and Azure Key Vault, see Encrypt and decrypt blobs in Microsoft Azure Storage using Azure Key Vault.

About client-side encryption

The Azure Blob Storage client library uses Advanced Encryption Standard (AES) to encrypt user data. There are two versions of client-side encryption available in the client library:

Version 2 uses Galois/Counter Mode (GCM) mode with AES.
Version 1 uses Cipher Block Chaining (CBC) mode with AES.

Warning

Using version 1 of client-side encryption is no longer recommended due to a security vulnerability in the client library's implementation of CBC mode. For more information about this security vulnerability, see Azure Storage updating client-side encryption in SDK to address security vulnerability. If you are currently using version 1, we recommend that you update your application to use version 2 and migrate your data. See the following section, Mitigate the security vulnerability in your applications, for further guidance.

Mitigate the security vulnerability in your applications

Due to a security vulnerability discovered in the Blob Storage client library's implementation of CBC mode, Microsoft recommends that you take one or more of the following actions immediately:

Consider using service-side encryption features instead of client-side encryption. For more information about service-side encryption features, see Azure Storage encryption for data at rest.
If you need to use client-side encryption, then migrate your applications from client-side encryption v1 to client-side encryption v2.

The following table summarizes the steps to take if you choose to migrate your applications to client-side encryption v2:

Client-side encryption status	Recommended actions
Application is using client-side encryption a version of the client library that supports only client-side encryption v1.	Update your application to use a version of the client library that supports client-side encryption v2. See SDK support matrix for client-side encryption for a list of supported versions. Learn more... Update your code to use client-side encryption v2. Learn more... Download any encrypted data to decrypt it, then re-encrypt it with client-side encryption v2. Learn more...
Application is using client-side encryption with a version of the client library that supports client-side encryption v2.	Update your code to use client-side encryption v2. Learn more... Download any encrypted data to decrypt it, then re-encrypt it with client-side encryption v2. Learn more...

Additionally, Microsoft recommends that you take the following steps to help secure your data:

Configure your storage accounts to use private endpoints to secure all traffic between your virtual network (VNet) and your storage account over a private link. For more information, see Use private endpoints for Azure Storage.
Limit network access to specific networks only.

SDK support matrix for client-side encryption

The following table shows which versions of the client libraries for .NET, Java, and Python support different versions of client-side encryption:

	.NET	Java	Python
Client-side encryption v2 and v1	Versions 12.13.0 and later	Versions 12.18.0 and later	Versions 12.13.0 and later
Client-side encryption v1 only	Versions 12.12.0 and earlier	Versions 12.17.0 and earlier	Versions 12.12.0 and earlier

Note

Client-side encryption v2.1 is available in the Java SDK for versions 12.27.0 and later. This version allows you to configure the region length for authenticated encryption, from 16 bytes to 1 GiB. For more information, see the Java example at Example: Encrypting and decrypting a blob with client-side encryption v2.

If your application is using client-side encryption with an earlier version of the .NET, Java, or Python client library, you must first upgrade your code to a version that supports client-side encryption v2. Next, you must decrypt and re-encrypt your data with client-side encryption v2. If necessary, you can use a version of the client library that supports client-side encryption v2 side-by-side with an earlier version of the client library while you're migrating your code. For code examples, see Example: Encrypting and decrypting a blob with client-side encryption v2.

How client-side encryption works

The Azure Blob Storage client libraries use envelope encryption to encrypt and decrypt your data on the client side. Envelope encryption encrypts a key with one or more additional keys.

The Blob Storage client libraries rely on Azure Key Vault to protect the keys that are used for client-side encryption. For more information about Azure Key Vault, see What is Azure Key Vault?.

Encryption and decryption via the envelope technique

Encryption via the envelope technique works as follows:

The Azure Storage client library generates a content encryption key (CEK), which is a one-time-use symmetric key.
User data is encrypted using the CEK.
The CEK is then wrapped (encrypted) using the key encryption key (KEK). The KEK is identified by a key identifier and can be either an asymmetric key pair or a symmetric key. You can manage the KEK locally or store it in an Azure Key Vault.

The Azure Storage client library itself never has access to KEK. The library invokes the key wrapping algorithm that is provided by Key Vault. Users can choose to use custom providers for key wrapping/unwrapping if desired.
The encrypted data is then uploaded to Azure Blob Storage. The wrapped key together with some additional encryption metadata is stored as metadata on the blob.

Decryption via the envelope technique works as follows:

The Azure Storage client library assumes that the user is managing the KEK either locally or in an Azure Key Vault. The user doesn't need to know the specific key that was used for encryption. Instead, a key resolver that resolves different key identifiers to keys can be set up and used.
The client library downloads the encrypted data along with any encryption material that is stored in Azure Storage.
The wrapped CEK is then unwrapped (decrypted) using the KEK. The client library doesn't have access to the KEK during this process, but only invokes the unwrapping algorithm of the Azure Key Vault or other key store.
The client library uses the CEK to decrypt the encrypted user data.

Encryption/decryption on blob upload/download

The Blob Storage client library supports encryption of whole blobs only on upload. For downloads, both complete and range downloads are supported. Client-side encryption v2 chunks data into 4 MiB buffered authenticated encryption blocks which can only be transformed whole. To adjust the chunk size, make sure you're using the most recent version of the SDK that supports client-side encryption v2.1. The region length is configurable from 16 bytes up to 1 GiB.

During encryption, the client library generates a random initialization vector (IV) of 16 bytes and a random CEK of 32 bytes, and performs envelope encryption of the blob data using this information. The wrapped CEK and some additional encryption metadata are then stored as blob metadata along with the encrypted blob.

When a client downloads an entire blob, the wrapped CEK is unwrapped and used together with the IV to return the decrypted data to the client.

Downloading an arbitrary range in the encrypted blob involves adjusting the range provided by users to get a small amount of additional data that can be used to successfully decrypt the requested range.

All blob types (block blobs, page blobs, and append blobs) can be encrypted/decrypted using this scheme.

Warning

If you are editing or uploading your own metadata for the blob, you must ensure that the encryption metadata is preserved. If you upload new metadata without also preserving the encryption metadata, then the wrapped CEK, IV, and other metadata will be lost and you will not be able to retrieve the contents of the blob. Calling the Set Blob Metadata operation always replaces all blob metadata.

When reading from or writing to an encrypted blob, use whole blob upload commands, such as Put Blob, and range or whole blob download commands, such as Get Blob. Avoid writing to an encrypted blob using protocol operations such as Put Block, Put Block List, Put Page, or Append Block. Calling these operations on an encrypted blob can corrupt it and make it unreadable.

Example: Encrypting and decrypting a blob with client-side encryption v2

The code example in this section shows how to use client-side encryption v2 to encrypt and decrypt a blob.

Important

If you have data that has been previously encrypted with client-side encryption v1, then you'll need to decrypt that data and re-encrypt it with client-side encryption v2. See the guidance and sample for your client library below.

To use client-side encryption from your .NET code, reference the Blob Storage client library. Make sure that you're using version 12.13.0 or later. If you need to migrate from version 11.x to version 12.13.0, see the Migration guide.

Two additional packages are required for Azure Key Vault integration for client-side encryption:

The Azure.Core package provides the IKeyEncryptionKey and IKeyEncryptionKeyResolver interfaces. The Blob Storage client library for .NET already defines this assembly as a dependency.
The Azure.Security.KeyVault.Keys package (version 4.x and later) provides the Key Vault REST client and the cryptographic clients that are used with client-side encryption. Make sure that this package is referenced in your project if you're using Azure Key Vault as your key store.

Azure Key Vault is designed for high-value master keys, and throttling limits per key vault reflect this design. As of version 4.1.0 of Azure.Security.KeyVault.Keys, the IKeyEncryptionKeyResolver interface doesn't support key caching. Should caching be necessary due to throttling, you can use the approach demonstrated in this sample to inject a caching layer into an Azure.Security.KeyVault.Keys.Cryptography.KeyResolver instance.

Developers can provide a key, a key resolver, or both a key and a key resolver. Keys are identified using a key identifier that provides the logic for wrapping and unwrapping the CEK. A key resolver is used to resolve a key during the decryption process. The key resolver defines a resolve method that returns a key given a key identifier. The resolver provides users the ability to choose between multiple keys that are managed in multiple locations.

On encryption, the key is always used and the absence of a key results in an error.

On decryption, if the key is specified and its identifier matches the required key identifier, that key is used for decryption. Otherwise, the client library attempts to call the resolver. If there's no resolver specified, then the client library throws an error. If a resolver is specified, then the key resolver is invoked to get the key. If the resolver is specified but doesn't have a mapping for the key identifier, then the client library throws an error.

To use client-side encryption, create a ClientSideEncryptionOptions object and set it on client creation with SpecializedBlobClientOptions. You can't set encryption options on a per-API basis. Everything else is handled by the client library internally.

// Your key and key resolver instances, either through Azure Key Vault SDK or an external implementation.
IKeyEncryptionKey key;
IKeyEncryptionKeyResolver keyResolver;

// Create the encryption options to be used for upload and download.
ClientSideEncryptionOptions encryptionOptions = new ClientSideEncryptionOptions(ClientSideEncryptionVersion.V2_0)
{
   KeyEncryptionKey = key,
   KeyResolver = keyResolver,
   // String value that the client library will use when calling IKeyEncryptionKey.WrapKey()
   KeyWrapAlgorithm = "some algorithm name"
};

// Set the encryption options on the client options.
BlobClientOptions options = new SpecializedBlobClientOptions() { ClientSideEncryption = encryptionOptions };

// Create blob client with client-side encryption enabled.
// Client-side encryption options are passed from service clients to container clients, 
// and from container clients to blob clients.
// Attempting to construct a BlockBlobClient, PageBlobClient, or AppendBlobClient from a BlobContainerClient
// with client-side encryption options present will throw, as this functionality is only supported with BlobClient.
BlobClient blob = new BlobServiceClient
(new Uri($"https://{accountName}.blob.core.windows.net"), new DefaultAzureCredential(), options).GetBlobContainerClient("my-container").GetBlobClient("myBlob");

// Upload the encrypted contents to the blob.
blob.Upload(stream);

// Download and decrypt the encrypted contents from the blob.
MemoryStream outputStream = new MemoryStream();
blob.DownloadTo(outputStream);

You can apply encryption options to a BlobServiceClient, BlobContainerClient, or BlobClient constructors that accept BlobClientOptions objects.

If a BlobClient object already exists in your code but lacks client-side encryption options, then you can use an extension method to create a copy of that object with the given ClientSideEncryptionOptions. This extension method avoids the overhead of constructing a new BlobClient object from scratch.

using Azure.Storage.Blobs.Specialized;

// An existing BlobClient instance and encryption options.
BlobClient plaintextBlob;
ClientSideEncryptionOptions encryptionOptions;

// Get a copy of the blob that uses client-side encryption.
BlobClient clientSideEncryptionBlob = plaintextBlob.WithClientSideEncryptionOptions(encryptionOptions);

After you update your code to use client-side encryption v2, make sure that you decrypt and re-encrypt any existing encrypted data, as described in Re-encrypt previously encrypted data with client-side encryption v2.

To use client-side encryption from your Java code, reference the Blob Storage client library. Make sure that you're using version 12.18.0 or later. If you need to migrate from an earlier version of the Java client library, see the Blob Storage migration guide for Java.

To use client-side encryption v2.1, include a dependency on azure-storage-blob-cryptography version 12.27.0 or later. Client-side encryption v2 has a fixed chunk size of 4 MiB, while v2.1 includes the ability to configure the region length for authenticated encryption. The region length is configurable from 16 bytes up to 1 GiB.

To use client-side encryption v2.1, create a BlobClientSideEncryptionOptions instance and optionally set the region length using the setAuthenticatedRegionDataLengthInBytes method. Then pass the encryption options to the EncryptedBlobClientBuilder constructor.

Add the following import directives to your code:

import com.azure.core.cryptography.*;
import com.azure.storage.blob.specialized.cryptography.*;

The following code example shows how to use client-side encryption v2.1 to encrypt a blob for upload:

// Your key instance, either through Azure Key Vault SDK or an external implementation
AsyncKeyEncryptionKey keyEncryptionKey;
AsyncKeyEncryptionKeyResolver keyResolver;
String keyWrapAlgorithm = "algorithm name";

// Sets the region length to 4 KiB
BlobClientSideEncryptionOptions encryptionOptions = new BlobClientSideEncryptionOptions()
    .setAuthenticatedRegionDataLengthInBytes(1024 * 4);
 
EncryptedBlobClient ebc = new EncryptedBlobClientBuilder(EncryptionVersion.V2_1)
    .blobClient(client)
    .key(key, keyWrapAlgorithm)
    .keyResolver(keyResolver)
    .credential(new DefaultAzureCredentialBuilder().build())
    .endpoint("https://<storage-account-name>.blob.core.windows.net/")
    .clientSideEncryptionOptions(encryptionOptions)
    .buildEncryptedBlobClient();

ebc.upload(BinaryData.fromString("sample data"));

To learn more about the library used for client-side encryption, see Azure Storage Blobs Cryptography client library for Java.

If you're migrating from client-side encryption v1, make sure that you decrypt and re-encrypt any existing encrypted data, as described in Re-encrypt previously encrypted data with client-side encryption v2.

To use client-side encryption from your Python code, reference the Blob Storage client library. Make sure that you're using version 12.13.0 or later. If you need to migrate from an earlier version of the Python client library, see the Blob Storage migration guide for Python.

The following example shows how to use client-side migration v2 from Python:

blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)
    blob_client.require_encryption = True
    blob_client.key_encryption_key = kek
    blob_client.encryption_version = ‘2.0’  # Use Version 2.0!

    with open("decryptedcontentfile.txt", "rb") as stream:
        blob_client.upload_blob(stream, overwrite=OVERWRITE_EXISTING)

Re-encrypt previously encrypted data with client-side encryption v2

Any data that was previously encrypted with client-side encryption v1 must be decrypted and then re-encrypted with client-side encryption v2 to mitigate the security vulnerability. Decryption requires downloading the data and re-encryption requires reuploading it to Blob Storage.

For a sample project that shows how to migrate data from client-side encryption v1 to v2 and how to encrypt data with client-side encryption v2 in .NET, see the Encryption migration sample project.

Client-side encryption and performance

Keep in mind that encrypting your storage data results in additional performance overhead. When you use client-side encryption in your application, the client library must securely generate the CEK and IV, encrypt the content itself, communicate with your chosen keystore for key-enveloping, and format and upload additional metadata. This overhead varies depending on the quantity of data being encrypted. We recommend that customers always test their applications for performance during development.

Поділитися через