Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,553 questions
Best Archival Strategy for delta parquet in ADLS ?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 95%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sagar, Sawan
0
Reputation points
We have a situation where we will be loading SAP data into ADLS layer using SAP Datasphere in delta parquet format. Now, we want to implement a robust archival framework for this data. Can you please guide/help me with the correct/MS recommended approach to deal with this. I would really appreciate if you can help me with some documentations on this.
Thanks,
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vinod Kumar Reddy Chilupuri 3,745 Reputation points Microsoft External Staff2025-04-16T10:36:44.73+00:00 Hi Sagar, Sawan,
Azure Blob Storage lifecycle management policies enable the automatic movement or deletion of data based on its age or usage patterns, making them particularly effective for efficiently archiving Delta Parquet files in Azure Data Lake Storage (ADLS).
Automatic Tier Transitioning:
- Older files can be automatically moved from Hot to Cool or Archive storage tiers based on their last modified or last accessed timestamps. This feature helps reduce storage costs while ensuring that data remains accessible when needed.
Data Expiration & Deletion:
- You can establish rules to delete obsolete files after a specified period. This capability is beneficial for compliance purposes and aids in optimizing storage usage.
Metadata-Based Filtering:
- Policies can be applied to specific files using prefixes or blob index tags, allowing for more granular management of different datasets. This feature facilitates the organization and handling of data according to specific criteria.
By leveraging these lifecycle management features, organizations can implement a robust and cost-effective archival strategy for their Delta Parquet files in ADLS.
Configure a lifecycle management policy
Optimize costs by automatically managing the data lifecycle
Hope the above suggestion helps! Please let us know do you have any further queries.
Please do consider to “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
-
Vinod Kumar Reddy Chilupuri 3,745 Reputation points Microsoft External Staff
2025-04-17T12:50:20.5566667+00:00 Hi Sagar, Sawan,
Just checking to see if the suggestion provided helped you to solve your issue, please let us know if you have any further queries. If the above suggestion helps you to solve your issue, please Up-vote, this can be beneficial to other community members.
-
Vinod Kumar Reddy Chilupuri 3,745 Reputation points Microsoft External Staff
2025-04-21T10:13:05.7466667+00:00 Hi Sagar, Sawan,
Just checking to see if the suggestion provided helped you to solve your issue, please let us know if you have any further queries. If the above suggestion helps you to solve your issue, please Up-vote, this can be beneficial to other community members.
Sign in to comment
Sign in to answer