How to query the content of a blob using .net

Question

I would like to know how can i query the content of a csv file using .net. I know how to do it using Python (query_blob function). I would like a minimal example if it is possible.

Answer 1

As of today, the Azure SDK for .NET does not support the blob query feature directly like the Python SDK does. A workaround to this would be to use Azure Data Lake Storage which supports querying blobs. Alternatively, you would need to download the file from Azure Blob Storage, then use Linq to query the data but depending on the size of file, you might run into issues (memory and/or performance). See: .NET code samples of the Azure Blob storage

Answer 2

Hello vrige,

Thank you for posting your query here!

Please check the code below to query the content of a CSV file using Azure .Net SDK.

using System.Globalization;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;
using CsvHelper;
using CsvHelper.Configuration;

namespace QueryCsvExample
{
    class Program
    {
        static async Task Main(string[] args)
        {
            string connectionString = "xxxxxxx";
            string containerName = "demo";
            string blobName = "currency.csv";

            var blobServiceClient = new BlobServiceClient(connectionString);
            var containerClient = blobServiceClient.GetBlobContainerClient(containerName);
            var blobClient = containerClient.GetBlockBlobClient(blobName);
            await QueryHemingway(blobClient);
        }

        static async Task QueryHemingway(BlockBlobClient blob)
        {
            string query = @"SELECT Name from BlobStorage where Code in ('AED','AFN')";
            await DumpQueryCsv(blob, query, false);
        }

        private static async Task DumpQueryCsv(BlockBlobClient blob, string query, bool headers)
        {
            try
            {
                var options = new BlobQueryOptions()
                {
                    InputTextConfiguration = new BlobQueryCsvTextOptions()
                    {
                        HasHeaders = true,
                        RecordSeparator = "\n",
                        ColumnSeparator = ",",
                        EscapeCharacter = '\\',
                        QuotationCharacter = '"'
                    },
                    OutputTextConfiguration = new BlobQueryCsvTextOptions()
                    {
                        HasHeaders = true,
                        RecordSeparator = "\n",
                        ColumnSeparator = ",",
                        EscapeCharacter = '\\',
                        QuotationCharacter = '"'
                    },
                    ProgressHandler = new Progress<long>((finishedBytes) =>
                        Console.Error.WriteLine($"Data read: {finishedBytes}")
                    )
                };
                options.ErrorHandler += (BlobQueryError err) => {
                    Console.ForegroundColor = ConsoleColor.Red;
                    Console.Error.WriteLine($"Error: {err.Position}:{err.Name}:{err.Description}");
                    Console.ResetColor();
                };

            
                using (var reader = new StreamReader((await blob.QueryAsync(
                        query,
                        options)).Value.Content))
                {
                    using (var parser = new CsvReader(reader, new CsvConfiguration(CultureInfo.CurrentCulture) { HasHeaderRecord = true }))
                    {
                        while (await parser.ReadAsync())
                        {
                            Console.WriteLine(String.Join(" ", parser.Parser.Record));
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Exception: " + ex.ToString());
            }
        }
    }
}

Source: https://stackoverflow.com/questions/77845024/how-to-query-the-content-of-a-blob-using-net

Please let us know if you have any further queries. I’m happy to assist you further.

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.