Create and use a volume with Azure Files in Azure Kubernetes Service (AKS)

A persistent volume represents a piece of storage that has been provisioned for use with Kubernetes pods. A persistent volume can be used by one or many pods, and can be dynamically or statically provisioned. If multiple pods need concurrent access to the same storage volume, you can use Azure Files to connect using the Server Message Block (SMB) protocol. This article shows you how to dynamically create an Azure Files share for use by multiple pods in an Azure Kubernetes Service (AKS) cluster.

This article shows you how to:

  • Work with a dynamic persistent volume (PV) by installing the Container Storage Interface (CSI) driver and dynamically creating one or more Azure file shares to attach to a pod.
  • Work with a static PV by creating one or more Azure file shares, or use an existing one and attach it to a pod.

For more information on Kubernetes volumes, see Storage options for applications in AKS.

Before you begin

  • An Azure storage account.

  • The Azure CLI version 2.0.59 or later installed and configured. Run az --version to find the version. If you need to install or upgrade, see Install Azure CLI.

Dynamically provision a volume

This section provides guidance for cluster administrators who want to provision one or more persistent volumes that include details of one or more shares on Azure Files for use by a workload. A persistent volume claim (PVC) uses the storage class object to dynamically provision an Azure Files file share.

Dynamic provisioning parameters

Name Meaning Available Value Mandatory Default value
skuName Azure Files storage account type (alias: storageAccountType) Standard_LRS, Standard_ZRS, Standard_GRS, Standard_RAGRS, Standard_RAGZRS,Premium_LRS, Premium_ZRS No StandardSSD_LRS
Minimum file share size for Premium account type is 100 GB.
ZRS account type is supported in limited regions.
NFS file share only supports Premium account type.
fsType File System Type ext4, ext3, ext2, xfs Yes ext4 for Linux
location Specify Azure region where Azure storage account will be created. For example, eastus. No If empty, driver uses the same location name as current AKS cluster.
resourceGroup Specify the resource group where the Azure Disks will be created Existing resource group name No If empty, driver uses the same resource group name as current AKS cluster.
shareName Specify Azure file share name Existing or new Azure file share name. No If empty, driver generates an Azure file share name.
shareNamePrefix Specify Azure file share name prefix created by driver. Share name can only contain lowercase letters, numbers, hyphens, and length should be fewer than 21 characters. No
folderName Specify folder name in Azure file share. Existing folder name in Azure file share. No If folder name does not exist in file share, mount will fail.
shareAccessTier Access tier for file share General purpose v2 account can choose between TransactionOptimized (default), Hot, and Cool. Premium storage account type for file shares only. No Empty. Use default setting for different storage account types.
accountAccessTier Access tier for storage account Standard account can choose Hot or Cool, and Premium account can only choose Premium. No Empty. Use default setting for different storage account types.
server Specify Azure storage account server address Existing server address, for example accountname.privatelink.file.core.windows.net. No If empty, driver uses default accountname.file.core.windows.net or other sovereign cloud account address.
disableDeleteRetentionPolicy Specify whether disable DeleteRetentionPolicy for storage account created by driver. true or false No false
allowBlobPublicAccess Allow or disallow public access to all blobs or containers for storage account created by driver. true or false No false
requireInfraEncryption Specify whether or not the service applies a secondary layer of encryption with platform managed keys for data at rest for storage account created by driver. true or false No false
storageEndpointSuffix Specify Azure storage endpoint suffix. core.windows.net, core.chinacloudapi.cn, etc. No If empty, driver uses default storage endpoint suffix according to cloud environment. For example, core.windows.net.
tags Tags are created in new storage account. Tag format: 'foo=aaa,bar=bbb' No ""
matchTags Match tags when driver tries to find a suitable storage account. true or false No false
--- Following parameters are only for SMB protocol --- ---
subscriptionID Specify Azure subscription ID where Azure file share is created. Azure subscription ID No If not empty, resourceGroup must be provided.
storeAccountKey Specify whether to store account key to Kubernetes secret. true or false
false means driver leverages kubelet identity to get account key.
No true
secretName Specify secret name to store account key. No
secretNamespace Specify the namespace of secret to store account key.

Note:
If secretNamespace isn't specified, the secret is created in the same namespace as the pod.
default,kube-system, etc No Pvc namespace, for example csi.storage.k8s.io/pvc/namespace
useDataPlaneAPI Specify whether to use data plane API for file share create/delete/resize. This could solve the SRP API throttling issue because the data plane API has almost no limit, while it would fail when there is firewall or Vnet setting on storage account. true or false No false
--- Following parameters are only for NFS protocol --- ---
rootSquashType Specify root squashing behavior on the share. The default is NoRootSquash AllSquash, NoRootSquash, RootSquash No
mountPermissions Mounted folder permissions. The default is 0777. If set to 0, driver doesn't perform chmod after mount 0777 No
--- Following parameters are only for VNet setting. For example, NFS, private end point --- ---
vnetResourceGroup Specify VNet resource group where virtual network is defined. Existing resource group name. No If empty, driver uses the vnetResourceGroup value in Azure cloud config file.
vnetName Virtual network name Existing virtual network name. No If empty, driver uses the vnetName value in Azure cloud config file.
subnetName Subnet name Existing subnet name of the agent node. No If empty, driver uses the subnetName value in Azure cloud config file.
fsGroupChangePolicy Indicates how volume's ownership is changed by the driver. Pod securityContext.fsGroupChangePolicy is ignored. OnRootMismatch (default), Always, None No OnRootMismatch

Create a storage class

A storage class is used to define how an Azure file share is created. A storage account is automatically created in the node resource group for use with the storage class to hold the Azure Files file share. Choose of the following Azure storage redundancy for skuName:

  • Standard_LRS - standard locally redundant storage (LRS)
  • Standard_GRS - standard geo-redundant storage (GRS)
  • Standard_ZRS - standard zone redundant storage (ZRS)
  • Standard_RAGRS - standard read-access geo-redundant storage (RA-GRS)
  • Premium_LRS - premium locally redundant storage (LRS)
  • Premium_ZRS - premium zone redundant storage (ZRS)

Note

Minimum premium file share is 100GB.

For more information on Kubernetes storage classes for Azure Files, see Kubernetes Storage Classes.

Create a file named azure-file-sc.yaml and copy in the following example manifest. For more information on mountOptions, see the Mount options section.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: my-azurefile
provisioner: file.csi.azure.com # replace with "kubernetes.io/azure-file" if aks version is less than 1.21
allowVolumeExpansion: true
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - actimeo=30
parameters:
  skuName: Premium_LRS

Create the storage class with the kubectl apply command:

kubectl apply -f azure-file-sc.yaml

Create a persistent volume claim

A persistent volume claim (PVC) uses the storage class object to dynamically provision an Azure file share. The following YAML can be used to create a persistent volume claim 100 GB in size with ReadWriteMany access. For more information on access modes, see the [Kubernetes persistent volume][access-modes] documentation.

Now create a file named azure-file-pvc.yaml and copy in the following YAML. Make sure that the storageClassName matches the storage class created in the last step:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-azurefile
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: my-azurefile
  resources:
    requests:
      storage: 100Gi

Note

If using the Premium_LRS sku for your storage class, the minimum value for storage must be 100Gi.

Create the persistent volume claim with the kubectl apply command:

kubectl apply -f azure-file-pvc.yaml

Once completed, the file share will be created. A Kubernetes secret is also created that includes connection information and credentials. You can use the kubectl get command to view the status of the PVC:

kubectl get pvc my-azurefile

The output of the command resembles the following example:

NAME           STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE
my-azurefile   Bound     pvc-8436e62e-a0d9-11e5-8521-5a8664dc0477   10Gi       RWX            my-azurefile      5m

Use the persistent volume

The following YAML creates a pod that uses the persistent volume claim my-azurefile to mount the Azure Files file share at the /mnt/azure path. For Windows Server containers, specify a mountPath using the Windows path convention, such as 'D:'.

Create a file named azure-pvc-files.yaml, and copy in the following YAML. Make sure that the claimName matches the PVC created in the last step.

kind: Pod
apiVersion: v1
metadata:
  name: mypod
spec:
  containers:
  - name: mypod
    image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    volumeMounts:
    - mountPath: "/mnt/azure"
      name: volume
  volumes:
    - name: volume
      persistentVolumeClaim:
        claimName: my-azurefile

Create the pod with the kubectl apply command.

kubectl apply -f azure-pvc-files.yaml

You now have a running pod with your Azure Files file share mounted in the /mnt/azure directory. This configuration can be seen when inspecting your pod using the kubectl describe command. The following condensed example output shows the volume mounted in the container:

Containers:
  mypod:
    Container ID:   docker://053bc9c0df72232d755aa040bfba8b533fa696b123876108dec400e364d2523e
    Image:          mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    Image ID:       docker-pullable://nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
    State:          Running
      Started:      Fri, 01 Mar 2019 23:56:16 +0000
    Ready:          True
    Mounts:
      /mnt/azure from volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-8rv4z (ro)
[...]
Volumes:
  volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  my-azurefile
    ReadOnly:   false
[...]

Mount options

The default value for fileMode and dirMode is 0777 for Kubernetes version 1.13.0 and above. If dynamically creating the persistent volume with a storage class, mount options can be specified on the storage class object. The following example sets 0777:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: my-azurefile
provisioner: file.csi.azure.com # replace with "kubernetes.io/azure-file" if aks version is less than 1.21
allowVolumeExpansion: true
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=0
  - gid=0
  - mfsymlinks
  - cache=strict
  - actimeo=30
parameters:
  skuName: Premium_LRS

Using Azure tags

For more details on using Azure tags, see Use Azure tags in Azure Kubernetes Service (AKS).

Statically provision a volume

This section provides guidance for cluster administrators who want to create one or more persistent volumes that include details of an existing Azure Files share to use with a workload.

Static provisioning parameters

Name Meaning Available Value Mandatory Default value
volumeAttributes.resourceGroup Specify an Azure resource group name. myResourceGroup No If empty, driver uses the same resource group name as current cluster.
volumeAttributes.storageAccount Specify an existing Azure storage account name. storageAccountName Yes
volumeAttributes.shareName Specify an Azure file share name. fileShareName Yes
volumeAttributes.folderName Specify a folder name in Azure file share. folderName No If folder name doesn't exist in file share, mount would fail.
volumeAttributes.protocol Specify file share protocol. smb, nfs No smb
volumeAttributes.server Specify Azure storage account server address Existing server address, for example accountname.privatelink.file.core.windows.net. No If empty, driver uses default accountname.file.core.windows.net or other sovereign cloud account address.
--- Following parameters are only for SMB protocol --- --- ---
volumeAttributes.secretName Specify a secret name that stores storage account name and key. No
volumeAttributes.secretNamespace Specify a secret namespace. default,kube-system, etc. No PVC namespace (csi.storage.k8s.io/pvc/namespace)
nodeStageSecretRef.name Specify a secret name that stores storage account name and key. Existing secret name Yes
nodeStageSecretRef.namespace Specify a secret namespace. Kubernetes namespace Yes
--- Following parameters are only for NFS protocol --- --- ---
volumeAttributes.fsGroupChangePolicy Indicates how a volumes ownership is changed by the driver. Pod securityContext.fsGroupChangePolicy is ignored. OnRootMismatch (default), Always, None No OnRootMismatch
volumeAttributes.mountPermissions Specify mounted folder permissions. The default is 0777 No

Create an Azure file share

Before you can use an Azure Files file share as a Kubernetes volume, you must create an Azure Storage account and the file share. In this article, you'll create the storage container in the node resource group.

  1. Get the resource group name with the az aks show command and add the --query nodeResourceGroup query parameter. The following example gets the node resource group for the AKS cluster named myAKSCluster in the resource group named myResourceGroup.

    az aks show --resource-group myResourceGroup --name myAKSCluster --query nodeResourceGroup -o tsv
    

    The output of the command resembles the following example:

    MC_myResourceGroup_myAKSCluster_eastus
    
  2. The following command creates a storage account using the Standard_LRS SKU. Replace the following placeholders:

    • myAKSStorageAccount with the name of the storage account
    • nodeResourceGroupName with the name of the resource group that the AKS cluster nodes are hosted in
    • location with the name of the region to create the resource in. It should be the same region as the AKS cluster nodes.
    az storage account create -n myAKSStorageAccount -g nodeResourceGroupName -l location --sku Standard_LRS
    
  3. Run the following command to export the connection string as an environment variable. This is used when creating the Azure file share in a later step.

    export AZURE_STORAGE_CONNECTION_STRING=$(az storage account show-connection-string -n storageAccountName -g resourceGroupName -o tsv)
    
  4. Create the file share using the Az storage share create command. Replace the placeholder shareName with a name you want to use for the share.

    az storage share create -n shareName --connection-string $AZURE_STORAGE_CONNECTION_STRING
    
  5. Run the following command to export the storage account key as an environment variable.

    STORAGE_KEY=$(az storage account keys list --resource-group $AKS_PERS_RESOURCE_GROUP --account-name $AKS_PERS_STORAGE_ACCOUNT_NAME --query "[0].value" -o tsv)
    
  6. Run the following commands to echo the storage account name and key. Copy this information as these values are needed when you create the Kubernetes volume later in this article.

    echo Storage account name: $AKS_PERS_STORAGE_ACCOUNT_NAME
    echo Storage account key: $STORAGE_KEY
    

Create a Kubernetes secret

Kubernetes needs credentials to access the file share created in the previous step. These credentials are stored in a Kubernetes secret, which is referenced when you create a Kubernetes pod.

Use the kubectl create secret command to create the secret. The following example creates a secret named azure-secret and populates the azurestorageaccountname and azurestorageaccountkey from the previous step. To use an existing Azure storage account, provide the account name and key.

kubectl create secret generic azure-secret --from-literal=azurestorageaccountname=$AKS_PERS_STORAGE_ACCOUNT_NAME --from-literal=azurestorageaccountkey=$STORAGE_KEY

Mount file share as an inline volume

Note

Inline volume can only access secrets in the same namespace as the pod. To specify a different secret namespace, please use the persistent volume example below instead.

To mount the Azure Files file share into your pod, configure the volume in the container spec. Create a new file named azure-files-pod.yaml with the following contents. If you changed the name of the file share or secret name, update the shareName and secretName. If desired, update the mountPath, which is the path where the Files share is mounted in the pod. For Windows Server containers, specify a mountPath using the Windows path convention, such as 'D:'.

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  nodeSelector:
    kubernetes.io/os: linux
  containers:
  - image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    name: mypod
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    volumeMounts:
      - name: azure
        mountPath: /mnt/azure
  volumes:
  - name: azure
    csi:
      driver: file.csi.azure.com
      readOnly: false
      volumeAttributes:
        secretName: azure-secret  # required
        shareName: aksshare  # required
        mountOptions: "dir_mode=0777,file_mode=0777,cache=strict,actimeo=30,nosharesock"  # optional

Use the kubectl apply command to create the pod.

kubectl apply -f azure-files-pod.yaml

You now have a running pod with an Azure Files file share mounted at /mnt/azure. You can verify the share is mounted successfully using the kubectl describe command:

kubectl describe pod mypod

Mount file share as a persistent volume

The following example demonstrates how to mount a file share as a persistent volume.

  1. Create a file named azurefiles-pv.yaml and copy in the following YAML. Under csi, update resourceGroup, volumeHandle, and shareName. For mount options, the default value for fileMode and dirMode is 0777.

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: azurefile
    spec:
      capacity:
        storage: 5Gi
      accessModes:
        - ReadWriteMany
      persistentVolumeReclaimPolicy: Retain
      storageClassName: azurefile-csi
      csi:
        driver: file.csi.azure.com
        readOnly: false
        volumeHandle: unique-volumeid  # make sure this volumeid is unique for every identical share in the cluster
        volumeAttributes:
          resourceGroup: resourceGroupName  # optional, only set this when storage account is not in the same resource group as node
          shareName: aksshare
        nodeStageSecretRef:
          name: azure-secret
          namespace: default
      mountOptions:
        - dir_mode=0777
        - file_mode=0777
        - uid=0
        - gid=0
        - mfsymlinks
        - cache=strict
        - nosharesock
        - nobrl
    
  2. Run the following command to create the persistent volume using the kubectl create command referencing the YAML file created earlier:

    kubectl create -f azurefiles-pv.yaml
    
  3. Create a azurefiles-mount-options-pvc.yaml file with a PersistentVolumeClaim that uses the PersistentVolume and copy the following YAML.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: azurefile
    spec:
      accessModes:
        - ReadWriteMany
      storageClassName: azurefile-csi
      volumeName: azurefile
      resources:
        requests:
          storage: 5Gi
    
  4. Use the kubectl commands to create the PersistentVolumeClaim.

kubectl apply -f azurefiles-mount-options-pvc.yaml
  1. Verify your PersistentVolumeClaim is created and bound to the PersistentVolume by running the following command.

    kubectl get pvc azurefile
    

    The output from the command resembles the following example:

    NAME        STATUS   VOLUME      CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    azurefile   Bound    azurefile   5Gi        RWX            azurefile      5s
    
  2. Update your container spec to reference your PersistentVolumeClaim and update your pod. For example:

    ...
      volumes:
      - name: azure
        persistentVolumeClaim:
          claimName: azurefile
    
  3. Because a pod spec can't be updated in place, use kubectl delete and kubectl apply commands to delete and then re-create the pod:

    kubectl delete pod mypod
    
    kubectl apply -f azure-files-pod.yaml
    

Next steps

For Azure File CSI driver parameters, see CSI driver parameters.

For associated best practices, see Best practices for storage and backups in AKS.