Frequent az storage blob upload timeout error on macOS

Question

This seems to primarily happen with the larger of the files we upload, but I've had it happen randomly on some smaller ones I believe.
The files in question are .dmg files and only weigh in around 42MB.
Similarly sized files uploading via similar scripts on Windows do not yield this problem, think i've never had it fail me on windows.
I also tried adding --timeout 6000 as suggested elsewhere for these timeouts, but still seems to timeout.

Script is basically calling this:
az storage blob upload --timeout 6000 --account-name $Account --auth-mode login -c builds --file $Path --name $Name

End up after a while getting: ERROR: ('Connection aborted.', TimeoutError('The write operation timed out'))

Uploading via Azure Storage Explorer on the mac worked fine, just seems to struggle on some files when using the AZ CLI. This has been going on for months.

Answer 1

@Jeremy Knope Thank you for reaching out to Microsoft Q&A.

I understand that you are having issues with Timeout error on your MACOS when doing az storage uploads using Azure CLI whereas don;t see the issues with your windows machine or when using storage explorer. Please correct me otherwise.

These issues can sometime be caused due to Latency, Network bandwidth/Memory and High CPU issues from the client side.

To understand what may be causing these issues, please check the end to end (E2E) latency and server latency for "Put Blobs API". Here are details regarding how to verify the latency from the Portal- https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-scalable-app-verify-metrics

Please confirm if you are seeing high latency here. Please note that- A Put Blob operation is permitted 10 minutes per MiB to complete. If the operation is taking longer than 10 minutes per MiB on average, the operation will timeout. Also verify if there are any High CPU or low memory issues on the mac that you are having issues with and if possible, try using a different device and see if you face similar issues.

I also wanted to add that since you are using Storage Explorer and not seeing any issues with it, you could also make use of Storage Explorer for the upload operations as this is well suited for your environment. You can also try to use Azcopy - https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-v10. Is there a specific reason you want to use the CLI?

Please let us know if you have any more questions in the meanwhile and we will be glad to assist you further. Thank you!

Remember:

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.