How to set storage tier for individual files in Data Lake storage

Mathieu Longtin 76 Reputation points Microsoft Employee
2023-04-17T18:30:51.59+00:00

Why does the REST API for Data Lake not expose storage tier? If I list a data lake directory, if I want the storage tier information, I then need to repeat the operation with the blob API.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,349 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,715 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Sina Salam 3,876 Reputation points
    2023-04-18T05:44:04.1733333+00:00

    @Mathieu Longtin Welcome to Microsoft Q&A, thank you for posting your here! Regarding your question on how to set storage tier for individual files in Azure Data Lake storage? First, you can set the storage tier when you create the storage account. In the Azure portal, select Create a resource->Storage->Storage account. Select from the options of Hot or Cool for the Access tier (default) setting<sup>1</sup>. If you want to change the access tier of an existing blob, you can use Azure Storage Explorer or Azure PowerShell<sup>2</sup>. I’m not sure why the REST API for Data Lake does not expose storage tier information at your end. However, you can use the blob API to get the storage tier information for individual files. I hope this helps! Let me know if you have any other questions. Regards, Sina

    0 comments
  2. PRADEEPCHEEKATLA-MSFT 77,751 Reputation points Microsoft Employee
    2023-04-19T05:15:45.2266667+00:00


    Mathieu Longtin     - Thanks for the question and using MS Q&A platform.

    Azure Data Lake Storage Gen2 does not currently support setting the storage tier for individual files. However, you can set the storage tier for the entire storage account or container using the Azure portal, Azure PowerShell, or Azure CLI. To set the storage tier for the entire storage account or container, follow these steps:

    1. Open the Azure portal and navigate to the storage account or container that you want to set the storage tier for.
    2. Click on the "Configuration" tab and select "Access tier".
    3. Choose the desired access tier (Hot, Cool, or Archive) and click "Save".

    This will set the storage tier for the entire storage account or container. All new files added to the storage account or container will be stored using the selected storage tier.

    Regarding the REST API for Data Lake, you are correct that it does not currently expose the storage tier information. If you need to retrieve the storage tier information for individual files, you can use the Azure Blob Storage REST API to retrieve the properties of the blob that corresponds to the file in Data Lake Storage Gen2. You can do this by using the Get Blob Properties operation in the Azure Blob Storage REST API.

    I hope this helps you set the storage tier for your Data Lake Storage Gen2 account or container. If you have any further questions or concerns, please let me know.