How to load data from DB to adls/Blob partitioned

Question

Hi Experts,

I have a database in on-prem that contains data for 15 years and not used frequently. I want to store this data in ADLS or Blob Stoarge in azure to reduce cost. The requirement is that the data should be migrated from on-prem to azure cloud and should be loaded in cool tier, in a way that if user wants to extract only one day or one week data,they need to be able to extract only required data, instead of taking all data from cool tier to hot tier.

Can anyone please suggest how to achieve this?

1. Do I need to read and load data per each day (for past 15 years) and load it in a partitioned storage?
2. Can I load all data from on-prem DB to adls and then can distribute it as per business date?
3. Out of ADLS and blob storage ,which one is better for this use case ? Any suggestion will be really appreciated l.

Answer 1

Hello Saha,

PFB the response/Queries:

1. Do I need to read and load data per each day (for past 15 years) and load it in a partitioned storage?
2. Can I load all data from on-prem DB to adls and then can distribute it as per business date?

[NH] : What is the overall size of data? This would help us determine the approach

1. Out of ADLS and blob storage ,which one is better for this use case ? Any suggestion will be really appreciated

Based on your above use case, I would prefer leveraging ADLS, as you can create folders pertaining to days which you can easily map /query via serverless/notebooks.

Note: you need to ensure that the tier is hot initially while copying data from On prem to sink and post that turn it into cold