What is best way to move/archive files from one folder to another with a retention of one week in Azure Data lake Gen 2 storage?

Vish Bhanage 41 Reputation points
2021-09-29T07:28:31.343+00:00

On way is Is Copy and Delete Activity in Azure Data factory pipeline?

Or use Azure Data Lake Storage archive tier and is possible to set retention as well with it?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,335 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,489 questions
0 comments
Accepted answer
  1. Samy Abdul 3,366 Reputation points
    2021-09-29T07:59:17.033+00:00

    Hi @Vish Bhanage , If I understand you correctly ,the best practice is to have a layer tiers in Azure Data Lake

    GEN2 and have slicing window or time series partitioned that is for example year=2021 month=Sept and so on.

    This approach also makes data navigation easier as we could able to locate the required data set quite easily.

    Thank you

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest