Authorization error accessing Blob from AKS

Question

I have created a storage class with with storage account credentials as described here: https://learn.microsoft.com/en-us/azure/aks/azure-csi-blob-storage-provision?tabs=mount-nfs%2Csecret

allowVolumeExpansion: true

apiVersion: storage.k8s.io/v1

kind: StorageClass

metadata:

name: azureblob-fuse-premium-creds

annotations:

storageclass.kubernetes.io/is-default-class: "true"

mountOptions:

• -o allow_other
• --file-cache-timeout-in-seconds=120
• --use-attr-cache=true
• --cancel-list-on-mount-seconds=10
• -o attr_timeout=120
• -o entry_timeout=120
• -o negative_timeout=120
• --log-level=LOG_WARNING
• --cache-size-mb=1000

parameters:

skuName: Premium_LRS

location: eastus

resourceGroup: ***

storageAccount: ***

networkEndpointType: privateEndpoint

protocol: fuse

containerName: ***

server: ***.blob.core.windows.net

subscriptionID:

storeAccountKey: ***

provisioner: blob.csi.azure.com

reclaimPolicy: Retain

volumeBindingMode: Immediate

But when I create a PVC with that storageclass, the following warning is thrown and pvc is in pending state:

Warning ProvisioningFailed 17s (x5 over 32s) blob.csi.azure.com_csi-blob-controller-56b4b745b9-csdzp_e4444625-f2ad-4012-9382-3308a60d3037 failed to provision volume with StorageClass "azureblob-fuse-premium-creds": rpc error: code = Internal desc = failed to create container(minio-test-custom-sc) on account(minioazureblob) type(Premium_LRS) rg(RG-QpiPro) location(eastus) size(1), error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"AuthorizationFailed","message":"The client '' with object id 'b37bf055-eedb-42cc-a5c3-47a21e607363' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/blobServices/containers/write' over scope '/subscriptions/---/resourceGroups/---/providers/Microsoft.Storage/storageAccounts/minioazureblob/blobServices/default/containers/test-custom-sc' or the scope is invalid. If access was recently granted, please refresh your credentials."}}

I'm able to create blobs on Azure console, using same Resource Group, Subscription ID.

Answer 1

Hi Yannam C Chiranjeevi - Thanks for reaching out.

What is the auth mechanism you are using for mounting? From the exception it appears you are probably using SPN or MSI which seem to be missing the permission to write/create the container.
failed to create container(minio-test-custom-sc)

Below are the 2 options to isolate:

Try configuring using Access Key as auth mechanism. This will ideally tend to work fine as Access Key as superior permissions.

Alternatively, try providing the contributor level role to the object ID b37bf055-eedb-42cc-a5c3-47a21e607363 and then test ahead.

Hope that helps!

Please let me know if there are any further queries/concerns, will be glad to assist.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Hello Yannam C Chiranjeevi,

Greetings! Welcome to Microsoft Q&A Platform.

The error message you’re seeing indicates an authorization issue. Specifically, the client with object ID b37bf055-eedb-42cc-a5c3-47a21e607363 does not have the necessary permissions to perform the action microsoft.storage/storageaccounts/blobservices/containers/write on the specified resource you must assign Azure roles to your application to manage azure resources as per this article. Azure AD roles and azure roles are two different roles, to learn more refer this article.

If you’ve recently granted permissions or made changes, try refreshing your credentials by signing out and signing back in to the Azure portal. Sometimes, cached credentials can cause issues.

Please verify if you have allowed access from All the Networks or there is some restriction w.r.t. specific IP, VNET etc? If there are, please flip to allowing access from all the networks and then test it once. This shall help with the isolation.

Ensure that the storage class azureblob-fuse-premium-creds is correctly configured and available in your cluster.

Use kubectl describe pvc <pvc-name> to get more details about the PVC and understand why it is in the pending state.

Check the logs of the storage provisioner for more detailed insights. Use kubectl logs -n <provisioner-namespace> <provisioner-pod-name> to view the logs.

reference threads - https://stackoverflow.com/questions/66003204/pvc-get-stuck-in-pending-waiting-for-a-volume-to-be-created-either-by-external, https://stackoverflow.com/questions/66202397/failed-to-provision-volume-with-storageclass-could-not-get-storage-key-for-sto, https://github.com/ceph/ceph-csi/issues/1062.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

---

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.