403 Error When Accessing Private Storage Account in Pipeline

Question

Hi Azure Community,

I'm encountering an issue with our pipeline setup, specifically in the unittest step where we build and execute the devcontainer with a unittest command. Here’s the setup:

• The unittest requires access to a private storage account.
• During the exec phase, I set environment variables (like AZURE_TENANT_ID, AZURE_CLIENT_CERTIFICATE_PATH, AZURE_CLIENT_ID) within the devcontainer.
• The service principal used has the Storage Blob Data Contributor role assigned to the target storage account.

Despite this setup, I receive the following error when the command runs in the pipeline: 403: This request is not authorized to perform this operation using this permission.

Strangely, when I run the same command on my local machine, everything works fine. Here’s what I’ve checked so far:

1. Service Principal Roles: Verified that the service principal has the appropriate roles.
2. Environment Variables: Confirmed that the necessary environment variables are correctly set within the devcontainer.
3. Certificate Mounting: Verified that the certificate is mounted as expected within the devcontainer.

I’ve triple-checked these configurations, but I’m still hitting the 403 error in the pipeline environment only. Does anyone have any ideas about what might be causing this discrepancy between local and pipeline execution, or suggestions for additional troubleshooting steps?

Thanks for any insights!

Answer 1

Hi @Csanád Tabajdi,

Welcome to Microsoft Q&A, thanks for posting your query.

The 403 you are facing suggests that there might be an issue authentication setup within your pipeline environment. Here are few troubleshooting steps to follow.

Virtual Network Configuration:
If your storage account is setup with a private endpoint or restricted to a virtual network, the pipeline might not have access. This is common for the environment where network access to cloud resource is controlled. Verify that the pipeline has permissions to connect to the storage account by reviewing any VNET or firewall restrictions on the storage account.

Verify "Allow trusted Microsoft services to access this storage account" is enable in your storage account's networking settings.

https://docs.microsoft.com/en-us/azure/storage/common/storage-private-endpoints

Service Principal authentication:

Check authentication in the pipeline if the service principle is missing permissions or due to any authentication problem, confirm whether it is setup correctly in the pipeline. This could involve using a managed identity or confirming the service principal’s permissions on the storage account.

Check weather all the environment variables are defined properly in the pipeline.

Service connections - Azure Pipelines | Microsoft Learn

Define variables - Azure Pipelines | Microsoft Learn

Authentication and Access:

Certificate Path: The Certificate path and permissions that work locally might not be valid in the pipeline. Make sure that the certificate is correctly uploaded, accessible, and has the right permissions in the pipeline.

Azure DevOps offers a “secure files” feature where you can securely upload files like certificates. Try adding your certificate this way to ensure its safely available during the pipeline run.

Storage Blob Data Contributor Role:

Confirm whether the Storage Blob Data Contributor role is assigned directly to the storage account or the specific resource group containing the storage account. Make sure that there is no delay in role assignment propagation. Azure built-in roles - Azure RBAC | Microsoft Learn

Add the detailed logging to your pipeline to capture the values of environment variables, authentication tokens, and storage URIs. This can help identify differences between the local and pipeline environment. Troubleshoot Azure RM service connection issues - Azure DevOps | Microsoft Learn

You could add an Azure CLI command (like az storage blob list) in the pipeline to check if it can access the storage account. This helps confirm if authentication and storage access are set up correctly before running the main task.

By following the above steps, you can identify and resolve the differences between your local and pipeline environments. Please let us know if you have any further queries. I’m happy to assist you further. 

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.