how to get a last modified timestamp when copying file from SFTP to ADLSGEN2

Naga Nekkanti 20 Reputation points

Hi team,

I am looking to get a last modified date for my source file when copying source file form SFTP to adlsgen2.

Instead of getting current timestamp in ADLS gen2 I need last modified timestamp.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
993 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
1,833 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
7,172 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
4,936 questions
{count} votes

Accepted answer
  1. Sedat SALMAN 5,320 Reputation points

    There isn't a built-in way to preserve the "last modified" timestamp when copying files from SFTP to Azure Data Lake Storage Gen2 (ADLS Gen2) with Azure Data Factory

    As a workaround

    When you are copying the file from SFTP, you can also retrieve the last modified timestamp and store it in the metadata of the blob in ADLS Gen2. This method requires using an additional service like Azure Functions or Logic Apps to retrieve the last modified date from the SFTP source and set it as metadata on the blob in ADLS Gen2.


    If you are using Azure Databricks, you could potentially write a script that copies the file and retrieves the last modified date from the SFTP source, then stores that date in the metadata of the blob in ADLS Gen2.

    0 comments No comments

0 additional answers

Sort by: Most helpful