Speeding up data copy service from TrueNAS to Azure Databox

Question

We are transferring around 300 TB of data from a TrueNAS storage appliance to multiple Azure DataBox devices.

We're using the Copy data (Preview) feature to transfer the data, but it's fairly slow (from our perspective). We're getting maybe 2 TB of transfer over 24 hr per data copy service job that's running.

We have four databoxes running 2-3 data copy service jobs all at the same time and none of the jobs seem to negatively impact the speed of each other. When we ran the first job overnight, by itself, the speed was the same.

Questions

1. Are there any tips to speed the transfer of a data copy service up?
2. Why do multiple data copy services run at the same speed?
3. Is the transfer software installed on the Databoxes the bottleneck?

Answer 1

@@Randy Burgess Thanks for providing the required information:

For (1), I was also trying to understand what type of physical media (like HDD or SSD etc). So, RAID 5+0 with how many disks? 

For (4), I don’t think it’s a good idea for same databox to have multiple jobs do the same hierarchy copy. What I mean is:

Job 1: Copy from \\truenas\dataset to \\databox\blockblob_archive\dataset

Job 2: Copy from \\truenas\dataset to \\databox\blockblob_archive\dataset

Etc

Instead, better one would be to do a separation:

Job 1: Copy from \\truenas\dataset\datadir1 to \\databox\blockblob_archive\dataset1

Job 1: Copy from \\truenas\dataset\datadir2 to \\databox\blockblob_archive\dataset2

 For (5), is 220MBps sustained over time from the storage for reads? i.e. for a larger copy.

 
Anyways, if this issue is still persistent, can you please contact support, so If you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support. In this case, could you send an email to AzCommunity[at]Microsoft[dot]com referencing this thread as well as your subscription ID. Please mention "ATTN subm" in the subject field. Thank you for your cooperation on this matter and look forward to your reply. and work on getting the support package from the device? Given that, it has been few weeks, are you finished copying the data? Or is this a PoC that they are trying to do?

Please let us know if you have any further queries. I’m happy to assist you further.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.