PFB the response/Queries:
- Do I need to read and load data per each day (for past 15 years) and load it in a partitioned storage?
- Can I load all data from on-prem DB to adls and then can distribute it as per business date?
[NH] : What is the overall size of data? This would help us determine the approach
- Out of ADLS and blob storage ,which one is better for this use case ? Any suggestion will be really appreciated
Based on your above use case, I would prefer leveraging ADLS, as you can create folders pertaining to days which you can easily map /query via serverless/notebooks.
Note: you need to ensure that the tier is hot initially while copying data from On prem to sink and post that turn it into cold