Retaining all files attributes like ACLs, File Owner & etc in Blob Storage

LoganIN 61 Reputation points
2022-01-28T17:09:37.397+00:00

Hi,

We are using Azure Data Factory to copy the objects from local server to Azure Blob storage. We were able to copy the objects successfully using the copy-activity in ADF, but the file attributes like ACLs, TimeStamp, File Owners are missing . I couldn't see any article which says the file attributes retention, Can someone please confirm the same and if yes, which tool we can use to preserve the same.

Thank you in advance!

Regards,
Logan

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,679 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,426 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,520 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. MartinJaffer-MSFT 26,021 Reputation points
    2022-01-28T22:25:48.837+00:00

    Hello @LoganIN and welcome to Microsoft Q&A.

    There is an option for preserving the metadata you mentioned. See Here. This option applies only to binary dataset because other types allow mapping, which can change contents, and make the metadata invalid.

    When you configure source as Data Lake Storage Gen1/Gen2 with binary format or the binary copy option, and sink as Data Lake Storage Gen2 with binary format or the binary copy option, you can find the Preserve option on the Settings page in Copy Data Tool or on the Copy Activity > Settings tab for activity authoring.

    Thanks, Martin

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

  2. MartinJaffer-MSFT 26,021 Reputation points
    2022-02-28T17:24:53.187+00:00

    File Owner falls under the same issue as with ACL.

    Created date is tied to the file system, at least on a low level. At the most base level, created date is not "When was this document created?" , but instead "When was this section of disk memory allocated to hold data?". The difference between these two questions, is the difference between talking about the sheet of paper a document is written on versus talking about the content of the document. When copying data from one system to another, you are creating a new document with contents identical to the old one. This the created date in the destination is when the copy was made.

    I did chat with my colleague, and he suggested using Azure Files and AzCopy rather than Azure Blob and Data Factory, as this may have the capabilities you want. See discussion thread. @Loganathan V

    0 comments No comments