Unable to list blobs with BlobServiceClient Python

Question

Unable to list blobs with BlobServiceClient Python

Anuj Pachawadkar 65

I have written an Azure Function which should enlist all the blobs under specific container's specific directory. Below is the code:

import logging
import azure.functions as func
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient

pipelineParams = {}
blob_cont_client = ''

def main(req: func.HttpRequest) -> func.HttpResponse:
    logging.info('Python HTTP trigger function processed a request.')
    
    global pipelineParams
    pipelineParams['container_name'] = '<container_name>'
    pipelineParams['pipeline_name'] = '<pipeline_name>'
    pipelineParams['runid'] = '<runid>'
    pipelineParams['storage_account'] = '<storage_account>'
    pipelineParams['managed_id'] = '<managed_id>'

    token_credential = DefaultAzureCredential(managed_identity_client_id=pipelineParams['managed_id'])

    blob_service_client = BlobServiceClient(
        account_url="https://*******.blob.core.windows.net/",
        credential=token_credential
    )

    global blob_cont_client
    blob_cont_client = blob_service_client.get_container_client(pipelineParams['container_name'])

    # Getting list of all blobs under runs directory
    generator = blob_cont_client.list_blobs(pipelineParams['container_name'], prefix="runs/" + pipelineParams['pipeline_name'] + "/" + pipelineParams['runid'] + "/", delimiter="")
    logging.info(generator)

    for blob in generator:
        logging.info(blob)

    name = req.params.get('name')

    if name:
        return func.HttpResponse(f"Hello, {name}. This HTTP triggered function executed successfully.")
    else:
        return func.HttpResponse(
             "This HTTP triggered function executed successfully. Pass a name in the query string or in the request body for a personalized response.",
             status_code=200
        )

For this I am getting following error on output:

HTTP response code: 504 Gateway Timeout

HTTP response content:

<!DOCTYPE html PUBLIC '-//W3C//DTD XHTML 1.0 Transitional//EN' 'http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd'>
<html xmlns='http://www.w3.org/1999/xhtml'>

<head>
    <meta content='text/html; charset=utf-8' http-equiv='content-type' />
    <style type='text/css'>
        body {
            font-family: Arial;
            margin-left: 40px;
        }

        img {
            border: 0 none;
        }

        #content {
            margin-left: auto;
            margin-right: auto
        }

        #message h2 {
            font-size: 20px;
            font-weight: normal;
            color: #000000;
            margin: 34px 0px 0px 0px
        }

        #message p {
            font-size: 13px;
            color: #000000;
            margin: 7px 0px 0px0px
        }

        #errorref {
            font-size: 11px;
            color: #737373;
            margin-top: 41px
        }
    </style>
    <title>Service unavailable</title>
</head>

<body>
    <div id='content'>
        <div id='message'>
            <h2>Our services aren't available right now</h2>
            <p>We're working to restore all services as soon as possible. Please check back soon.</p>
        </div>
        <div id='errorref'>
            <span>20230818T064345Z-4ntdg2p3ux0srd3a8n8r16byn4000000043g00000000p8us            </span>
        </div>
    </div>
</body>
</html>

And following on the console:

2023-08-17T12:10:25Z   [Warning]   DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
2023-08-17T12:10:49Z   [Warning]   DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
2023-08-17T12:11:48Z   [Warning]   DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
2023-08-17T12:12:09Z   [Warning]   DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
2023-08-17T12:12:48Z   [Warning]   DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
2023-08-17T12:12:48Z   [Error]   Executed 'Functions.HttpTrigger1' (Failed, Id=0fd13d77-fc27-4f3a-ad28-37960c83fbd3, Duration=76752ms)

Up to below line it doesn't break:

    generator = blob_cont_client.list_blobs(pipelineParams['container_name'], prefix="runs/" + pipelineParams['pipeline_name'] + "/" + pipelineParams['runid'] + "/", delimiter="")
    logging.info(generator)

once we add for loop for the iterator returned with the list_blobs() method it breaks and gives the mentioned error.

Can anyone help here to understand or resolve the issue?

Thanks in advance!

MuthuKumaranMurugaachari-MSFT 22,441 Reputation points Moderator

2023-08-18T15:47:22.4966667+00:00

Anuj Pachawadkar Thanks for posting your question in Microsoft Q&A. Based on the exception and error message, it appears that there were some outages that affected the service during that time. Are you still facing the issues? If so, I suggest you open a ticket with our support team to restore the service.

If you need any help, let me know.

Your answer

MuthuKumaranMurugaachari-MSFT 22,441 Reputation points Moderator

2023-08-18T15:47:22.4966667+00:00

Anuj Pachawadkar Thanks for posting your question in Microsoft Q&A. Based on the exception and error message, it appears that there were some outages that affected the service during that time. Are you still facing the issues? If so, I suggest you open a ticket with our support team to restore the service.

If you need any help, let me know.

Share via

Unable to list blobs with BlobServiceClient Python

Your answer