nodeAffinity tags "topology.disk.csi.azure.com/zone" for managed-premium storage class are too restrictive

Question

Question: Is there a way to use LRS and set the nodeAffinity tags correctly?

AKS is unable to schedule pods - the pods are in Pending state.
Checking the PVC shows that there are nodeAffinity issues.
Further diagnostics reveal that the PV has the following tags:

$>k get pv pvc-xyz -ojson | jq -r '.spec.nodeAffinity.required.nodeSelectorTerms[].matchExpressions[] | select(.key=="topology.disk.csi.azure.com/zone").values[]'

westeurope-1

This appears to be incorrect.

We are using managed-premium storage class (which uses Premium_LRS i.e. Locally Redundant Storage):

$>k get sc managed-premium -oyaml | grep storageaccounttype

storageaccounttype: Premium_LRS

According to Errors when mounting Azure disk volumes , we should use ZRS (Zone Redundant Storage) or Azure Files. Azure Files is not possible due to latency. ZRS increases our costs.

I believe that Kubernetes should not need to use Zone-Redundant storage.

---

Question: Is there a way to use LRS and set the nodeAffinity tags correctly?

Answer 1

Hi @Jean-Pierre Fouche ,

Thanks for reaching out to us on Microsoft Q&A forum.

Since LRS (Locally Redundant Storage) does not provide redundancy across multiple availability zones, it is crucial to ensure that your AKS cluster has sufficient nodes in the specified zone.

If you have defined an affinity rule that requires pods to run only in a specific zone and there are no available nodes in that zone, the pods will remain in a pending state.

If there are no available nodes that match the node affinity requirements specified in the PV, any pods trying to use that PV will remain in a pending state. This occurs because Kubernetes cannot find suitable nodes to schedule those pods.

Only nodes labeled with topology.disk.csi.azure.com/zone=westeurope-1 can access the PV.

This is crucial for ensuring that pods requiring access to this PV are scheduled on appropriate nodes, preventing scheduling conflicts.

Pod Scheduling:

Ensure that your AKS cluster has sufficient nodes in the specified zone (westeurope-1). If necessary, scale your AKS cluster or add more nodes specifically in that zone.

Use Preferred Scheduling:

Instead of using requiredDuringSchedulingIgnoredDuringExecution, which enforces strict matching, If want more flexibility consider using preferredDuringSchedulingIgnoredDuringExecution . This allows Kubernetes to schedule the pod on any available node if no node matches the preferred criteria.

If the information is helpful, please consider by clicking the "Upvote".

If you have any further queries, please let us know we are glad to help you.