What is best way to move/archive files from one folder to another with a retention of one week in Azure Data lake Gen 2 storage?

Vishal Bhanage 41

On way is Is Copy and Delete Activity in Azure Data factory pipeline?

Or use Azure Data Lake Storage archive tier and is possible to set retention as well with it?

Accepted answer

Samy Abdul 3,376 Reputation points

2021-09-29T07:59:17.033+00:00

Hi @Vishal Bhanage , If I understand you correctly ,the best practice is to have a layer tiers in Azure Data Lake

GEN2 and have slicing window or time series partitioned that is for example year=2021 month=Sept and so on.

This approach also makes data navigation easier as we could able to locate the required data set quite easily.

Thank you
Please sign in to rate this answer.
Vishal Bhanage 41 Reputation points

2021-10-19T22:35:31.737+00:00

Setting a year folder would force to update the paths in pipeline every year?
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

0 additional answers

Your answer