is multipart upload for large file supported in BlockBlob storage ?

virender kaushik 1 Reputation point

i have large file of 250GB to upload from my own premises HDFS to azure block blob storage using
distcp command, i am using below command

hadoop distcp \"DEBUG" \<account_key> \
-Dmapreduce.task.timeout=0 -overwrite -i -numListstatusThreads=1 \
/user/test/250gb_file \

Firstly, i am not able to upload file more than size of 195GB. how can we upload the file of size more than 195Gb using distcp command

Secondly, file is large in size so how can we upload file in parallel because sequentially its taking long
time to upload. how can we support multipart in this distcp while uploading file in azure block blob

Not Monitored
Not Monitored
Tag not monitored by Microsoft.
37,663 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Leon Laude 85,721 Reputation points


    Q&A currently supports the products listed in right-hand pane (more to be added later on).

    You can reach the experts in the dedicated Azure Storage forum over here:

    (please don't forget to mark helpful replies as answer)

    Best regards,

    0 comments No comments