How to handle seeding data to Data Drives and then implementing delta sync after the seeded data has been placed into Azure?

Pranay Kumar 1 Reputation point
2022-02-17T08:21:43.12+00:00

Hello,

How to handle seeding data to Data Drives and then implementing delta sync after the seeded data has been placed into Azure?

What we want to do -

  1. Move 100 TB of data in which 80TB will be placed in File Share; the other 20TB will be from SQL Server.
  2. We are planning to use 2 data boxes to seed the 100 TB
  3. After seeded in Azure, delta sync needs to be implemented for the changes to the data after the backup was performed on the data disks.
  4. We currently have 100MB connection to the internet to support the delta syncs.

Questions

  1. Can the data boxes be used in this scenario? If not what are alternatives to support the above?
  2. How does the data need to be configured in Azure to support delta sync from On-prem?
  3. What tools can be used to manage the sync between On-prem SQL / File Share to the Data Boxes?

If data boxes are not supported then what are alternative options to meet the above requirements?

Azure Data Share
Azure Data Share
An Azure service that is used to share data from multiple sources with other organizations.
43 questions
Azure SQL Database
Azure Data Box
Azure Data Box
A family of appliances and solutions for offline data transfer to Azure​.
38 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Ronen Ariely 15,096 Reputation points
    2022-02-17T10:22:31.127+00:00

    Hi,

    1. Can the data boxes be used in this scenario? If not what are alternatives to support the above?

    As much as I understand your description the answer is: partially

    data boxes are physical devices, which you use to move data from and to the Azure by sniping the device.

    Basically it is like using disk-on-key or external disk to move files from one machine to another by upload the data to hard disk in one location and and ship it (not using network but physically ship the disk) to the second location.

    This is great solution for handle seeding data but probably less fit for keeping your data sync between two places.

    If you plan to sync the data once every X days and you are OK with the latency of shipping the devices, then it can fit as well, but this not a common scenario probably.

    If you need to keep the data synchronized more frequently and the amount of delta sync data is not large, then using this solution does not make sense, and you can use the network to pass the data. If you need to keep the data synchronized at all time, then this is solution do not do the work at all (your description is not clear regarding the synchronization you need).

    1. How does the data need to be configured in Azure to support delta sync from On-prem?

    The following document include the full explanation about the procedure

    https://learn.microsoft.com/en-us/azure/databox/data-box-deploy-ordered?WT.mc_id=DP-MVP-5001699

    1. What tools can be used to manage the sync between On-prem SQL / File Share to the Data Boxes?

    Speaking about File Share do you mean Azure files?

    Your scenario is not fully clear to me regarding SQL Server!

    data box has nothing to do with sync databases but sync data on disk. I do not understand what is the direct relation to SQL Server. Obviously you can use shared folder (like Azure files) for example to configure transactional replication between Azure SQL Managed Instance and SQL Server, but in this case the files are pushed by the server using the network.

    You can manually move backup files from local to shared folder in the Azure in order to manually sync database. IN theory in this case you might want to move the files using data box if this fit the golden roles I mentioned above in question 1

    If you need to sync files between on-premises disk and Azure files then you can use the Azure File Sync agent which enables data on a Windows Server to be synchronized with an Azure File share.

    https://learn.microsoft.com/en-us/azure/storage/file-sync/file-sync-deployment-guide?WT.mc_id=DP-MVP-5001699

    Again, without more information I cannot know if this is what you need but this is the common scenario of keeping local folder sync with Azure files.

    If data boxes are not supported then what are alternative options to meet the above requirements?

    As I mentioned data box fit for part of the scenario like your first moving of the files but for sync files Azure File Sync agent should fit better in most cases and for sync database a replication is probably the best solution

    ----------

    14150-image.pngRonen Ariely
    Personal Site | Blog | Facebook | Linkedin