question

Vamshi-5948 avatar image
0 Votes"
Vamshi-5948 asked SaiKishor-MSFT answered

Azure blob container - Improve performance in fetching pdf files from Azure blob container.

Hi Team,

Below is my code to fetch the pdf files from Azure blob container. Most of the times the blob container contains up to 5000 pdf files.

Issue i am facing: Performance is very slow and taking more time to fetch when there are more than 500 pdf files in the blob container.

Need inputs and help to correct my code to improve the Performance. Expecting to see if there is any way to improve the code to fetch all the files in 5 seconds..


List<WorkflowDocumentListModel> documentListModel = new List<WorkflowDocumentListModel>();
if (request.RequestModel.WorkflowStatusId == (byte)WorkflowStatus.RCV) {
IList<PdfFileMetaDataResponse> RCVFileList = await _blobManager.GetAllFilesMetaDataAsync(UserContext.AzureBlobContainerWorkflowQueue, cancellationToken).ConfigureAwait(false);
foreach (PdfFileMetaDataResponse item in RCVFileList ) {
documentListModel.Add(new WorkflowDocumentListModel {
Id = 0,
Name = item.FileName,
AssignedTo = item.CustomUserName,
AssignedToId = item.CustomUserId,
AppDate = item.LastModified?.ClientTimeToUtc(UserContext.TimeZone),
IsGrabbedByUser = userId == item.CustomUserId
});
}

  public async Task<IList<PdfFileMetaDataResponse>> GetAllFilesMetaDataAsync(string containerName, CancellationToken cancellationToken)
         {
             BlobServiceClient _blobServiceClient = new BlobServiceClient(UserContext.AzureBlobStorageConnection);
             BlobContainerClient containerClient = _blobServiceClient.GetBlobContainerClient(containerName);
             List<PdfFileMetaDataResponse> responseList = new List<PdfFileMetaDataResponse>();
             await foreach (BlobItem file in containerClient.GetBlobsAsync(cancellationToken: cancellationToken))
             {
                 BlobClient blobClient = containerClient.GetBlobClient(file.Name);
                 BlobProperties blobProperties = await blobClient.GetPropertiesAsync(cancellationToken: cancellationToken).ConfigureAwait(false);
    
                 DateTime? customLastModifiedDate = ConvertWebApiFileLastModifiedMillisecondsToDateTime(
                                                 blobProperties.Metadata?.Where(i => i.Key.ToUpperInvariant() == CustomBlobMetadataLastModifiedMilliseconds.ToUpperInvariant())
                                                                                 .Select(i => i.Value)
                                                                                 .FirstOrDefault());
                 string customUserName = blobProperties.Metadata?.Where(i => i.Key.ToUpperInvariant() == CustomBlobMetadataUserName.ToUpperInvariant())
                                                                                 .Select(i => i.Value)
                                                                                 .FirstOrDefault();
    
                 bool hasCustomUserID = int.TryParse(blobProperties.Metadata?.Where(i => i.Key.ToUpperInvariant() == CustomBlobMetadataUserId.ToUpperInvariant())
                                                                 .Select(i => i.Value).FirstOrDefault(), out int customUserID);
    
                 PdfFileMetaDataResponse response = new PdfFileMetaDataResponse()
                 {
                     FileName = file.Name,
                     CustomUserName = customUserName,
                     CustomUserId = hasCustomUserID ? customUserID : (int?)null,
                     LastModified = customLastModifiedDate ?? file.Properties.LastModified?.UtcDateTime
                 };
    
                 responseList.Add(response);
             }
    
             return responseList;
         }


Thank you.

dotnet-csharpazure-blob-storagedotnet-aspnet-core-mvcdotnet-aspnet-core-webapidotnet-entity-framework-core
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

SaiKishor-MSFT avatar image
1 Vote"
SaiKishor-MSFT answered

@Vamshi-5948 I see your question was answered on SO- https://stackoverflow.com/questions/73349120/azure-blob-container-improve-performance-in-fetching-pdf-files-and-files-detai

If you need any further assistance, please do let us know. Thank you!

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.