list blobs of different directory in Azure blob Storage

Question

We have below folder structure in which we have stored data in Azure data lake.

I want to list all the blobs which are present in the Data directory. ie. abc.txt,xyz.txt,lmn.txt,abc1.txt,xyz1.txt,lmn1.txt,abc2.txt,xyz2.txt,lmn2.txt,gh.txt,kim.txt,tae.txt,....

I used list blobs but it is returning like below which is not useful for me. example: Email/Contact, Email/Contacts/2021 Email/Contacts/2021/November

How can I get all the blobs which are in Data Directory only?

I am using c# language.

Answer 1

Hello @Vaishali Vyas ,
Please check out the below PowerShell code which will list out only the required data.

//connect to the subscription
Connect-AzAccount
//Get the Storage Account
$strAccount = Get-AzStorageAccount -ResourceGroupName "ResourceGroupName" -Name "StorageAccountName"
//Get the context
$context = $strAccount.Context
//Get the corresponding container emails data which has the path data.
$blobs = Get-AzStorageblob -Container "emails" -Blob "data" -Context $context
//Iterate through the blobs to extract only names of the blobs
foreach($blob in $blobs)
{
$temp = $blob.Name
$blobFinaloutput = $temp.Substring($temp.LastIndexOf("/") + 1)
write-host $blobFinaloutput
}

Let us know if that helps !

Regards,
Shiva.

Answer 2

Hi @shiva patpi

I am using console app(c#) to achieve this.

Answer 3

Hello @Vaishali Vyas ,
Here is the C#.net code to achieve the same result, tested locally and working for me to extract only the blob names.
I am using the same ListBlobs API , but the blob.util.segement.last() can give you the desired result.

 public static void ListOnlyBlobNames()  
    {  
        try  
        {  
            string connection = "DefaultEndpointsProtocol=https;AccountName=accountname;AccountKey=key;EndpointSuffix=core.windows.net";  
            string strContainer = "emails";  
            CloudStorageAccount backupStorageAccount = CloudStorageAccount.Parse(connection);  
            var backupBlobClient = backupStorageAccount.CreateCloudBlobClient();  
            var backupContainer = backupBlobClient.GetContainerReference(strContainer);  
            var list = backupContainer.ListBlobs(useFlatBlobListing: true);  
            var listOfFileNames = new List<string>();  

            foreach (var blob in list)  
            {  
                var blobFileName = blob.Uri.Segments.Last();  
                listOfFileNames.Add(blobFileName);  
            }  
            foreach(var item in listOfFileNames)  
            {  
                Console.WriteLine(item);  
            }  
        }  
        catch (Exception ex)  
        {  
            Console.WriteLine(ex.ToString());  
        }  
    }