Getting error: "could not get account name from azure-secret" after upgrading to K8s 1.23

Question

After upgrading AKS cluster to kubernetes from 1.22.x to 1.23.8 there started to appear issues when mounting existing azurefile PVC in non default namespaces.

MountVolume.MountDevice failed for volume "xxxxxxxx-pv" : fetching NodeStageSecretRef yyyyyyyy/azure-secret failed: kubernetes.io/csi: failed to find the secret azure-secret in the namespace yyyyyyyy with error: secrets "azure-secret" not found

Since azure-secret actually did exist only in default namespace a quick solution was to apply the same secret in each affected namespace.

But now I keep receiving this error message in my kubelet syslog -

W0829 05:46:43.445593 5011 azure_file.go:123] getStorageAccountName(azure-secret) returned with error: could not get account name from azure-secret, correct format: azure-storage-account-(.+)-secret

What is going on?

Answer 1

Our internal team has rolled out a fix which would be gradually rolled out in all the regions. To solve the current issue, you need to do an image upgrade for agent node to get fixed versions.

Guide to perform upgrade : Upgrade Azure Kubernetes Service (AKS) node images - Azure Kubernetes Service | Microsoft Learn

Below is actual flag, which does only node image upgrade.

Related issue thread :
After cluster upgrade from 1.18.9 to 1.19.7 azureFile volume secrets are searched in wrong namespace · Issue #99061 · kubernetes/kubernetes · GitHub

Reference:
https://github.com/Azure/AKS/issues/2871#issuecomment-1081349817
https://github.com/kubernetes/kubernetes/issues/99061
https://learn.microsoft.com/en-us/azure/aks/node-image-upgrade