Provision Azure NetApp Files dual-protocol volumes for Azure Kubernetes Service

Article
08/01/2024

After you configure Azure NetApp Files for Azure Kubernetes Service, you can provision Azure NetApp Files volumes for Azure Kubernetes Service.

Azure NetApp Files supports volumes using NFS (NFSv3 or NFSv4.1), SMB, and dual-protocol (NFSv3 and SMB, or NFSv4.1 and SMB).

This article shows you how to statically provisioning volumes for dual-protocol access using NFS or SMB.

Before you begin

Make sure you have already created a dual-protocol volume. See create a dual-protocol volume for Azure NetApp Files.

Provision a dual-protocol volume in Azure Kubernetes Service

This section describes how to expose an Azure NetApp Files dual-protocol volume statically to Kubernetes. Instructions are provided for both SMB and NFS protocols. You can expose the same volume via SMB to Windows worker nodes and via NFS to Linux worker nodes.

Create the persistent volume for NFS

Define variables for later usage. Replace myresourcegroup, myaccountname, mypool1, myvolname with an appropriate value from your dual-protocol volume.
```
RESOURCE_GROUP="myresourcegroup"
ANF_ACCOUNT_NAME="myaccountname"
POOL_NAME="mypool1"
VOLUME_NAME="myvolname"
```

List the details of your volume using the az netappfiles volume show command.

az netappfiles volume show \
    --resource-group $RESOURCE_GROUP \
    --account-name $ANF_ACCOUNT_NAME \
    --pool-name $POOL_NAME \
    --volume-name $VOLUME_NAME -o JSON

The following output is an example of the above command executed with real values.

{
  ...
  "creationToken": "myfilepath2",
  ...
  "mountTargets": [
    {
      ...
      "ipAddress": "10.0.0.4",
      ...
    }
  ],
  ...
}

Create a file named pv-nfs.yaml and copy in the following YAML. Make sure the server matches the output IP address from the previous step, and the path matches the output from creationToken above. The capacity must also match the volume size from Step 2.
```
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-nfs
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  mountOptions:
    - vers=3
  nfs:
    server: 10.0.0.4
    path: /myfilepath2
```
Create the persistent volume using the kubectl apply command:
```
kubectl apply -f pv-nfs.yaml
```
Verify the status of the persistent volume is Available by using the kubectl describe command:
```
kubectl describe pv pv-nfs
```

Create a persistent volume claim for NFS

Create a file named pvc-nfs.yaml and copy in the following YAML. This manifest creates a PVC named pvc-nfs for 100Gi storage and ReadWriteMany access mode, matching the PV you created.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-nfs
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: ""
  resources:
    requests:
      storage: 100Gi

Create the persistent volume claim using the kubectl apply command:
```
kubectl apply -f pvc-nfs.yaml
```
Verify the Status of the persistent volume claim is Bound by using the kubectl describe command:
```
kubectl describe pvc pvc-nfs
```

Mount within a pod using NFS

Create a file named nginx-nfs.yaml and copy in the following YAML. This manifest defines a nginx pod that uses the persistent volume claim.

kind: Pod
apiVersion: v1
metadata:
  name: nginx-nfs
spec:
  containers:
  - image: mcr.microsoft.com/oss/nginx/nginx:1.15.5-alpine
    name: nginx-nfs
    command:
    - "/bin/sh"
    - "-c"
    - while true; do echo $(date) >> /mnt/azure/outfile; sleep 1; done
    volumeMounts:
    - name: disk01
      mountPath: /mnt/azure
  volumes:
  - name: disk01
    persistentVolumeClaim:
      claimName: pvc-nfs

Create the pod using the kubectl applykubectl-apply command:
```
kubectl apply -f nginx-nfs.yaml
```
Verify the pod is Running by using the kubectl apply command:
```
kubectl describe pod nginx-nfs
```

Verify your volume has been mounted on the pod by using kubectl exec to connect to the pod, and then use df -h to check if the volume is mounted.

kubectl exec -it nginx-nfs -- sh

/ # df -h
Filesystem             Size  Used Avail Use% Mounted on
...
10.0.0.4:/myfilepath2  100T  384K  100T   1% /mnt/azure
...

Create a secret with the domain credentials

Create a secret on your AKS cluster to access the AD server using the kubectl create secret command. This secret will be used by the Kubernetes persistent volume to access the Azure NetApp Files SMB volume. Use the following command to create the secret, replacing USERNAME with your username, PASSWORD with your password, and DOMAIN_NAME with your Active Directory domain name.

kubectl create secret generic smbcreds --from-literal=username=USERNAME --from-literal=password="PASSWORD" --from-literal=domain='DOMAIN_NAME'

To verify the secret has been created, run the kubectl get command.

kubectl get secret

NAME       TYPE     DATA   AGE
smbcreds   Opaque   2      20h

Install an SMB CSI driver

You must install a Container Storage Interface (CSI) driver to create a Kubernetes SMB PersistentVolume.

Install the SMB CSI driver on your cluster using helm. Be sure to set the windows.enabled option to true:

helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts   
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system --version v1.10.0 –-set windows.enabled=true

For other methods of installing the SMB CSI Driver, see Install SMB CSI driver master version on a Kubernetes cluster.

Verify the csi-smb controller pod is running and each worker node has a pod running using the kubectl get pods command:

kubectl get pods -n kube-system | grep csi-smb

csi-smb-controller-68df7b4758-xf2m9   3/3     Running   0          3m46s
csi-smb-node-s6clj                    3/3     Running   0          3m47s
csi-smb-node-win-tfxvk                3/3     Running   0          3m47s

Create the persistent volume for SMB

Define variables for later usage. Replace myresourcegroup, myaccountname, mypool1, myvolname with an appropriate value from your dual-protocol volume.
```
RESOURCE_GROUP="myresourcegroup"
ANF_ACCOUNT_NAME="myaccountname"
POOL_NAME="mypool1"
VOLUME_NAME="myvolname"
```

List the details of your volume using az netappfiles volume show command.

az netappfiles volume show \
    --resource-group $RESOURCE_GROUP \
    --account-name $ANF_ACCOUNT_NAME \
    --pool-name $POOL_NAME \
    --volume-name "$VOLUME_NAME -o JSON

The following output is an example of the above command executed with real values.

{
  ...
  "creationToken": "myvolname",
  ...
  "mountTargets": [
    {
      ...
      "
         "smbServerFqdn": "ANF-1be3.contoso.com",
      ...
    }
  ],
  ...
}

Create a file named pv-smb.yaml and copy in the following YAML. If necessary, replace myvolname with the creationToken and replace ANF-1be3.contoso.com\myvolname with the value of smbServerFqdn from the previous step. Be sure to include your AD credentials secret along with the namespace where it's located that you created in a prior step.

apiVersion: v1
kind: PersistentVolume
metadata:
  name: anf-pv-smb
spec:
  storageClassName: ""
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
    - dir_mode=0777
    - file_mode=0777
    - vers=3.0
  csi:
    driver: smb.csi.k8s.io
    readOnly: false
    volumeHandle: myvolname  # make sure it's a unique name in the cluster
    volumeAttributes:
      source: \\ANF-1be3.contoso.com\myvolname
    nodeStageSecretRef:
      name: smbcreds
      namespace: default

Create the persistent volume using the kubectl apply command:
```
kubectl apply -f pv-smb.yaml
```
Verify the status of the persistent volume is Available using the kubectl describe command:
```
kubectl describe pv anf-pv-smb
```

Create a persistent volume claim for SMB

Create a file name pvc-smb.yaml and copy in the following YAML.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: anf-pvc-smb
spec:
  accessModes:
    - ReadWriteMany
  volumeName: anf-pv-smb
  storageClassName: ""
  resources:
    requests:
      storage: 100Gi

Create the persistent volume claim using the kubectl apply command:
```
kubectl apply -f pvc-smb.yaml
```
Verify the status of the persistent volume claim is Bound by using the kubectl describe command:
```
kubectl describe pvc anf-pvc-smb
```

Mount within a pod using SMB

Create a file named iis-smb.yaml and copy in the following YAML. This file will be used to create an Internet Information Services pod to mount the volume to path /inetpub/wwwroot.

apiVersion: v1
kind: Pod 
metadata:
  name: iis-pod
  labels:
     app: web
spec:
  nodeSelector:
    "kubernetes.io/os": windows
  volumes:
  - name: smb
    persistentVolumeClaim:
      claimName: anf-pvc-smb 
  containers:
  - name: web
    image: mcr.microsoft.com/windows/servercore/iis:windowsservercore 
    resources:
      limits:
        cpu: 1
        memory: 800M
    ports:
      - containerPort: 80
    volumeMounts:
    - name: smb
      mountPath: "/inetpub/wwwroot"
      readOnly: false

Create the pod using the kubectl apply command:
```
kubectl apply -f iis-smb.yaml
```

Verify the pod is Running and /inetpub/wwwroot is mounted from SMB by using the kubectl describe command:

kubectl describe pod iis-pod

The output of the command resembles the following example:

Name:         iis-pod
Namespace:    default
Priority:     0
Node:         akswin000001/10.225.5.246
Start Time:   Fri, 05 May 2023 09:34:41 -0400
Labels:       app=web
Annotations:  <none>
Status:       Running
IP:           10.225.5.248
IPs:
  IP:  10.225.5.248
Containers:
  web:
    Container ID:   containerd://39a1659b6a2b6db298df630237b2b7d959d1b1722edc81ce9b1bc7f06237850c
    Image:          mcr.microsoft.com/windows/servercore/iis:windowsservercore
    Image ID:       mcr.microsoft.com/windows/servercore/iis@sha256:0f0114d0f6c6ee569e1494953efdecb76465998df5eba951dc760ac5812c7409
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 05 May 2023 09:34:55 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  800M
    Requests:
      cpu:        1
      memory:     800M
    Environment:  <none>
    Mounts:
      /inetpub/wwwroot from smb (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mbnv8 (ro)
...

Verify your volume has been mounted on the pod by using the kubectl exec command to connect to the pod. Then use the dir command in the correct directory to check if the volume is mounted and the size matches the size of the volume you provisioned.

kubectl exec -it iis-pod –- cmd.exe