VERY slow blob download w/ SharedAccessSignature

Question

I'm getting well below acceptable performance downloading a few blobs from a storage account (Storage v2, standard/hot, ra-grs, East US / West US) SSL, Shared Access Signature

File, Size Bytes, Seconds to download (run in succession)
File1.zip, 35,747,096, 409
File2.zip, 30,304,760, 266
File3.zip, 34,124,219, 299
File4.zip, 30,281,327, 11
File5.zip, 16,046,615, 3

That's over 16 minutes for 145 mb. Which arguably should be <30 seconds given the performance of the last file. My internet is 500mb down, and definitely not heavily loaded. I am in the Netherlands, so I don't expect ping times under 100ms, but consistent throughput should not be a problem.

As for my code, .Net Core 3.1, using WebClient.DownloadFileTaskAsync() to download the zip files to a temp folder. I've tried a few combinations of running them in parallel / serial to see if anything helps. Invariably at least one (but usually 3+) of the 5 files takes 300+ seconds. That's 100kB/s.. I'm pretty sure old school floppy drives are faster.

I've also tried downloading the files directly in chrome, and while they may start out with a burst, eventually it slows down considerably.

Any thoughts on where to start?

-Peter

Answer 1

@Peter Drier Just for testing purpose can try to download the same Files using Azcopy and or its core library Azure Storage Data Movement Library, which supports super-fast transferring for blobs, and the transferring can be paused & resumed. This library is based on the core data movement framework that powers AzCopy, which also provides you high-performance uploading, downloading.

There was a similar thread discussion in Microsoft Q&A Forum, can you refer to the troubleshooting steps mentioned over-there and let me know if the issue still persists.

Additional information: There is no throttling on the Azure side below the published 60 MB/s scalability target for a single blob. If the network and client machine can handle the traffic then we will send it.

Hope this helps! Kindly let us know if the above helps or you need further assistance on this issue.

---------------------------------------------------------------------------------------------------------------------------

Please don’t forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

Hi there, thanks for the quick reply..

I skipped ahead and just ran the azcopy benchmark against my container (with a sas) in download mode, and got what's below.. TL;DR: 8mbit per second.. I'm actually suspecting my internet provider is having troubles, as I'm getting wildly inconsistent ping/tracert times along the route to the blob storage server. I'm investigating on the ISP side for now.. Thanks. -Peter

INFO: The benchmark feature is currently in Preview status.
INFO: Benchmarking downloads from https://XXXX.blob.core.windows.net/anker.
INFO: Scanning...
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
INFO: Trying 4 concurrent connections (initial starting point)

Job 4489d7a2-e313-4d4a-4a07-a70d544a4ee9 has started
Log file is located at: C:\Users\peter\.azcopy\4489d7a2-e313-4d4a-4a07-a70d544a4ee9.log

INFO: Trying 16 concurrent connections (seeking optimum)
INFO: Trying 4 concurrent connections (backing off)
INFO: Trying 8 concurrent connections (seeking optimum)
INFO: Trying 16 concurrent connections (seeking optimum)
INFO: Trying 8 concurrent connections (backing off)
INFO: Trying 9 concurrent connections (seeking optimum)
INFO: Trying 8 concurrent connections (at optimum)
INFO:
INFO: Automatic concurrency tuning completed. Recording of performance stats will begin now.
INFO:
INFO: *** After a minute or two, you may cancel the job with CTRL-C to trigger early analysis of the stats. ***
INFO: *** You do not need to wait for whole job to finish.                                                  ***
INFO:
99.3 %, 55 Done, 0 Failed, 1 Pending, 0 Skipped, 56 Total,


Job 4489d7a2-e313-4d4a-4a07-a70d544a4ee9 summary
Elapsed Time (Minutes): 14.8226
Number of File Transfers: 56
Number of Folder Property Transfers: 0
Total Number of Transfers: 56
Number of Transfers Completed: 56
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 1271491799
Final Job Status: Completed

Performance benchmark results:
Note: The benchmark feature is currently in Preview status.

Main Result:
  Code:   NetworkIsBottleneck
  Desc:   Network bandwidth appears to be the key factor governing performance.
  Reason: No other factors were identified that are limiting performance, so the bottleneck is assumed to be available network bandwidth. The available network bandwidth is the portion of the installed bandwidth that is not already used by other traffic. Throughput of 8 Mega bits/sec was obtained with 8 concurrent connections.

This benchmark tries to find optimal performance, computed without touching any local disk. When reading
and writing real data, performance may be different. If disk limits throughput, AzCopy will display a message on screen.