Azure ADF copy activity - Copy file from local file system to ADLS

Question

Azure ADF copy activity - Copy file from local file system to ADLS

Raghavendra 25

Hello, I am trying to copy a bunch of files from my local machine to ADLS2.0 using copy activity. Though this functionality works as expected, the original file timestamp from source (local file) is not retained. Post file copy, the timestamp is overwritten with the time file is uploaded. How can I retain the file timestamps even after the copy data activity?

Anthony Shaw 5 Reputation points

2025-05-15T21:47:55.3166667+00:00

Not a direct answer, but a workaround maybe. Zip the file and then upload it.

My ¢2
Chandra Boorla 13,790 Reputation points Microsoft External Staff Moderator

2025-05-15T22:10:59.65+00:00
@Raghavendra

It sounds like you're trying to copy files from your local system to Azure Data Lake Storage (ADLS) using Azure Data Factory (ADF), but the original timestamps (created, modified) of the files aren't being preserved during the copy process. This is expected because ADF does not natively support retaining original file timestamps during file uploads.

How to Preserve File Timestamps in ADF

Preserve Metadata for Lake Migration - When you migrate data from one data lake to another, you can choose to preserve the file metadata along with the data. Copy activity supports preserving customer-specified metadata and specific system properties during the data copy process. The system properties that can be preserved include attributes like contentType, contentLanguage, contentEncoding, contentDisposition, and cacheControl. The service automatically replaces invalid characters in metadata keys when preserving metadata.

Configuration Example (JSON) - If you are working with JSON configuration (ARM templates or API calls), you can configure it like this:

{ "name": "CopyAndPreserveMetadata", "type": "Copy", "typeProperties": { "source": { "type": "FileSystemSource" }, "sink": { "type": "AzureBlobFSink" }, "enableMetadataPreserve": true } }

How This Works - When enabled, ADF will attempt to preserve the source file's original metadata (including timestamps) when copying to ADLS Gen2. This works best when copying between compatible storage types (like local file system to ADLS Gen2).

Considerations

The "Preserve Metadata" option may not work for all source types. Ensure that your local file system metadata is accessible.

If this does not work directly due to unsupported source types, you can use a workaround:

Capture the file's original timestamps (created, modified) using a pre-copy script (PowerShell, Python).

Store these timestamps as metadata on the files in ADLS Gen2 using a post-copy script or an Azure Function.

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Raghavendra 25 Reputation points

2025-05-15T23:25:45.61+00:00

Thanks! I do not see any option "Preserve Metadata". Based on what I read, it shows only when you copy between ADLS locations and not local file to ADLS. About using pre and post copy scripts, please share an example.
Chandra Boorla 13,790 Reputation points Microsoft External Staff Moderator

2025-05-16T18:09:51.5033333+00:00
@Raghavendra
Thank you @Nandan Hegde

I sincerely apologize for the confusion in my previous responses. You are absolutely right, since you are copying files from your local machine to Azure Data Lake Storage Gen2 (ADLS Gen2) using Azure Data Factory (ADF), the "Preserve Metadata" option is not available. I made an error in my response, and I genuinely apologize for any confusion this may have caused.

Understanding Your Scenario

Since ADF treats files uploaded from a local file system as new objects, the original timestamps (created, modified) are not retained.

These timestamps become the time of upload, and they cannot be changed directly in ADLS, as they are read-only system properties.

Recommended Solution:

As suggested by @Nandan Hegde, if you need to retain the original timestamps, here is an effective workaround:

Use the "Get Metadata" activity in ADF to capture the original file timestamps (created, modified) from your local system before the Copy Activity.

In the Copy Activity, dynamically rename the target files to include the original timestamp.

Example: If the file is report.csv and the original modified time is 2025-05-16T08:30:00, you can copy it as report_20250516T083000.csv.

I hope this information helps.

Thank you.
Raghavendra 25 Reputation points

2025-05-16T18:28:09.1766667+00:00

Thank you everyone for the suggestions. It is helpful to think about various solutions to move forward.

Accepted answer

0 additional answers

Your answer

Anthony Shaw 5 Reputation points

2025-05-15T21:47:55.3166667+00:00

Not a direct answer, but a workaround maybe. Zip the file and then upload it.

My ¢2
Raghavendra 25 Reputation points

2025-05-15T23:25:45.61+00:00

Thanks! I do not see any option "Preserve Metadata". Based on what I read, it shows only when you copy between ADLS locations and not local file to ADLS. About using pre and post copy scripts, please share an example.
Chandra Boorla 13,790 Reputation points Microsoft External Staff Moderator

2025-05-16T18:09:51.5033333+00:00

@Raghavendra
Thank you @Nandan Hegde

I sincerely apologize for the confusion in my previous responses. You are absolutely right, since you are copying files from your local machine to Azure Data Lake Storage Gen2 (ADLS Gen2) using Azure Data Factory (ADF), the "Preserve Metadata" option is not available. I made an error in my response, and I genuinely apologize for any confusion this may have caused.

Understanding Your Scenario

Since ADF treats files uploaded from a local file system as new objects, the original timestamps (created, modified) are not retained.

These timestamps become the time of upload, and they cannot be changed directly in ADLS, as they are read-only system properties.

Recommended Solution:

As suggested by @Nandan Hegde, if you need to retain the original timestamps, here is an effective workaround:

Use the "Get Metadata" activity in ADF to capture the original file timestamps (created, modified) from your local system before the Copy Activity.

In the Copy Activity, dynamically rename the target files to include the original timestamp.

Example: If the file is report.csv and the original modified time is 2025-05-16T08:30:00, you can copy it as report_20250516T083000.csv.

I hope this information helps.

Thank you.
Raghavendra 25 Reputation points

2025-05-16T18:28:09.1766667+00:00

Thank you everyone for the suggestions. It is helpful to think about various solutions to move forward.

Answer 1

Nandan Hegde 35,831 MVP Volunteer Moderator

Unfortunately based on my understanding, it is not possible to preserve the lastmodifiedtime for files during file movements from file systems to Azure blob storage.

And once the file is uploaded in ADLS/Blob, that property is read only and cannot be updated/modified.

So in case if you do need the original value, you can use get meta data activity initially before copy activity and capture the original timestamp and then while copying the file, append that timestamp to the file name at the need.

Share via

Azure ADF copy activity - Copy file from local file system to ADLS

0 additional answers

Your answer