I am getting performance issue when I use azcopy

Question

Hi Team,

I am trying to copy a large file from one Azure storage to to azure storage using azcopy.

Initially it started well with high speed. Now the speed is decreased and am not sure whether it will complete by today.
The file size is about 400gb.

If am not wrong, the transfer happens at MS backbone right within Azure.

Ciao

Answer 1

@Sky High Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

• May I know how are you calculating the speed: https://www.azurespeed.com/Azure/Upload? (There could be several reason, for the performance issue (Network bandwidth, intermittently issue may occur).
• Are you uploading the file in the same region? I assume you are using the latest version of the azcopy?

For now you can use the AzCopy v10, there's a parameter that lets you directly specify a cap on the amount of bandwidth it uses. E.g. --cap-mbps 200 will limit it to a max of 200 Mbps.

Using Azcopy , you can investigate what it tells you. E.g. network error rate, CPU contention etc.

Instructions on running AzCopy benchmarks are here: https://learn.microsoft.com/en-us/azure/storage/common/storage-ref-azcopy-bench
AzCopy can be downloaded here: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10

About the best performance settings. You can run “azcopy bench” and it will tell you what the optimal numbers for the various settings are given your setup. But again,

Additional information: Here's a guide for troubleshooting high E2E latency: https://learn.microsoft.com/en-us/azure/storage/common/storage-monitoring-diagnosing-troubleshooting#metrics-show-high-AverageE2ELatency-and-low-AverageServerLatency
There is no throttling on the Azure side, If the network and client machine can handle the traffic then we will send it.

Choose an Azure solution for data transfer

Please let us know if you have any further queries. I’m happy to assist you further.

----------

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.