Integrate Azure HPC Cache with Azure Kubernetes Service (AKS)
Article
Azure HPC Cache speeds access to your data for high-performance computing (HPC) tasks. By caching files in Azure, Azure HPC Cache brings the scalability of cloud computing to your existing workflow. This article shows you how to integrate Azure HPC Cache with Azure Kubernetes Service (AKS).
You need Azure CLI version 2.7 or later. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.
Register the hpc-cache extension in your Azure subscription. For more information on using HPC Cache with Azure CLI, see the HPC Cache CLI prerequisites.
Review the HPC Cache prerequisites. You need to satisfy the following before you can run an HPC Cache:
The cache requires a dedicated subnet with at least 64 IP addresses available.
The subnet must not host other VMs or containers.
The subnet must be accessible from the AKS nodes.
If you need to run your application as a user without root access, you may need to disable root squashing by using the change owner (chown) command to change directory ownership to another user. The user without root access needs to own a directory to access the file system. For the user to own a directory, the root user must chown a directory to that user, but if the HPC Cache is squashing root, this operation is denied because the root user (UID 0) is being mapped to the anonymous user. For more information about root squashing and client access policies, see HPC Cache access policies.
Install the hpc-cache Azure CLI extension
Important
AKS preview features are available on a self-service, opt-in basis. Previews are provided "as is" and "as available," and they're excluded from the service-level agreements and limited warranty. AKS previews are partially covered by customer support on a best-effort basis. As such, these features aren't meant for production use. For more information, see the following support articles:
To install the hpc-cache extension, run the following command:
az extension add --name hpc-cache
Run the following command to update to the latest version of the extension released:
az extension update --name hpc-cache
Register the StorageCache feature flag
Register the Microsoft.StorageCache resource provider using the az provider register command.
az provider register --namespace Microsoft.StorageCache --wait
It takes a few minutes for the status to show Registered. Verify the registration status by using the az feature show command:
az feature show --namespace "Microsoft.StorageCache"
Create the Azure HPC Cache
Get the node resource group using the az aks show command with the --query nodeResourceGroup query parameter.
az aks show --resource-group myResourceGroup --name myAKSCluster --query nodeResourceGroup -o tsv
Your output should look similar to the following example output:
MC_myResourceGroup_myAKSCluster_eastus
Create a dedicated HPC Cache subnet using the az network vnet subnet create command. First define the environment variables for RESOURCE_GROUP, VNET_NAME, VNET_ID, and SUBNET_NAME. Copy the output from the previous step for RESOURCE_GROUP, and specify a value for SUBNET_NAME.
Create the HPC Cache using the az hpc-cache create command. The following example creates the HPC Cache in the East US region with a Standard 2G cache type named MyHpcCache. Specify a value for --location, --sku-name, and --name.
Creation of the HPC Cache can take up to 20 minutes.
Create and configure Azure storage
Create a storage account using the az storage account create command. First define the environment variable STORAGE_ACCOUNT_NAME.
Important
You need to select a unique storage account name. Replace uniquestorageaccount with your specified name. Storage account names must be between 3 and 24 characters in length and can contain only numbers and lowercase letters.
STORAGE_ACCOUNT_NAME=uniquestorageaccount
The following example creates a storage account in the East US region with the Standard_LRS SKU. Specify a value for --location and --sku.
Assign the Storage Blob Data Contributor Role on your subscription using the az role assignment create command. First, define the environment variables STORAGE_ACCOUNT_ID and AD_USER.
STORAGE_ACCOUNT_ID=$(az storage account show --name $STORAGE_ACCOUNT_NAME --query "id" -o tsv)
AD_USER=$(az ad signed-in-user show --query objectId -o tsv)
az role assignment create --role "Storage Blob Data Contributor" --assignee $AD_USER --scope $STORAGE_ACCOUNT_ID
Create the Blob container within the storage account using the az storage container create command. First, define the environment variable CONTAINER_NAME and replace the name for the Blob container.
CONTAINER_NAME=mystoragecontainer
az storage container create --name $CONTAINER_NAME --account-name $STORAGE_ACCOUNT_NAME --auth-mode login
Provide permissions to the Azure HPC Cache service account to access your storage account and Blob container using the az role assignment commands. First, define the environment variables HPC_CACHE_USER and HPC_CACHE_ID.
HPC_CACHE_USER="StorageCache Resource Provider"
HPC_CACHE_ID=$(az ad sp list --display-name "${HPC_CACHE_USER}" --query "[].objectId" -o tsv)
az role assignment create --role "Storage Account Contributor" --assignee $HPC_CACHE_ID --scope $STORAGE_ACCOUNT_ID
az role assignment create --role "Storage Blob Data Contributor" --assignee $HPC_CACHE_ID --scope $STORAGE_ACCOUNT_ID
Add the blob container to your HPC Cache as a storage target using the az hpc-cache blob-storage-target add command. The following example creates a blob container named MyStorageTarget to the HPC Cache MyHpcCache. Specify a value for --name, --cache-name, and --virtual-namespace-path.
Create an Azure Private DNS zone for the client-facing IP addresses using the az network private-dns zone create command. First define the environment variable PRIVATE_DNS_ZONE and specify a name for the zone.
PRIVATE_DNS_ZONE="myhpccache.local"
az network private-dns zone create \
--resource-group $RESOURCE_GROUP \
--name $PRIVATE_DNS_ZONE
Create the round-robin DNS name for the client-facing IP addresses using the az network private-dns record-set a create command. First, define the environment variables DNS_NAME, HPC_MOUNTS0, HPC_MOUNTS1, and HPC_MOUNTS2. Replace the value for the property DNS_NAME.
az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS0
az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS1
az network private-dns record-set a add-record -g $RESOURCE_GROUP -z $PRIVATE_DNS_ZONE -n $DNS_NAME -a $HPC_MOUNTS2
Create a persistent volume
Create a file named pv-nfs.yaml to define a persistent volume and then paste in the following manifest. Replace the values for the property server and path.
The source for this content can be found on GitHub, where you can also create and review issues and pull requests. For more information, see our contributor guide.
Azure Kubernetes Service feedback
Azure Kubernetes Service is an open source project. Select a link to provide feedback:
We discuss various factors that influence high-performance computing (HPC) storage selection in Azure. We review NFS, file access, directory services, parallel file systems, and hybrid file-access considerations.