Terraform backend unable to access Blob using SP & OIDC Connect. Getting 403 || Client#ListBlobs

Question

Hi

I am using github action for Infra provisioning, Backend is configured as mentioned in code block.
Storage account has Storage account contrinutor role(inherited from Resource group) for Service principle used in backend.

In activity logs i can see "List Storage Account Keys Succeded" using same SP. But in Github action i get the error while performing terraform init.
"Error: Failed to get existing workspaces: containers.Client#ListBlobs: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="AuthorizationFailure" Message="This request is not authorized to perform this operation."

note: sub1 and sub2 are different

Public network access is disabled.

Instead have private endpoint.

 
terraform {
  required_version = "1.7.1"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "=3.89.0"
    }
  }

   backend "azurerm" {
    storage_account_name = "NAME"
    container_name       = "CONTAINER_NAME"
    key                  = "BLOB_KEY"
    resource_group_name  = "RG_NAME"
    use_oidc             = true
    subscription_id      = "MY_SUB_ID_1"	
    tenant_id            = "MY_TENANT_ID"
    client_id            = "MY_CLIENT_ID"
     }
}
provider "azurerm" {
  features {}
  subscription_id = "MY_SUB_ID_2"
  client_id       = "MY_CLIENT_ID"
  tenant_id       = "MY_TENANT_ID"
  use_oidc        = true
}

Answer 1

Hi @Gaurav Kathuria - There can be multiple reasons for the auth failures. It can be due to missing permission w.r.t. data plane or also some kind of network level permission.

A quick test will be to allow access to all the networks and then test the same. If that works, then probably you are getting blocked on the network and in that case ensure that the machine from where you are accessing is part of same VNET associated to PE.

Another option is to verify that SPN also have a data plane role such as Storage Blob Data Reader or Storage Blob Data Contributor etc. to see if that helps out.

Answer 2

Hello Gaurav Kathuria,

Greetings! Welcome to Microsoft Q&A forum.The 403 Forbidden error that you are encountered during terraform init indicates an authorization issue. There are some possible ways to overcome this issue by checking the SP has the necessary permissions to perform the required operations. Ensure that the SP has the correct permissions for the specific storage account and container you’re using as the backend.

Also check if the SP can access the storage account through the private endpoint and ensure that the storage account has appropriate access level by go into your storage account > IAM > Add role assignment, and add the special permissions for this type of request:

• Storage Blob Data Contributor
• Storage Queue Data Contributor

Similar SO thread for reference - https://stackoverflow.com/questions/77388978/failed-to-get-existing-workspaces-containers-clientlistblobs-code-containern, https://learn.microsoft.com/en-us/azure/developer/terraform/authenticate-to-azure?tabs=bash

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.