How to load or skip locked files loading through datafactory?

Prashanth T 26 Reputation points Microsoft Employee
2020-06-17T16:55:08.99+00:00

I have to load gatewaylog files from VM to Datalake through ADF. Some files will get locked while loading.

  1. How to load or skip them in Azure data factory?
  2. How can I load skipping files after lock get removed?
  3. Is there any option to check file is locked or not in ADF?
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,127 questions
{count} votes

1 answer

Sort by: Most helpful
  1. ChiragMishra-MSFT 956 Reputation points
    2020-06-25T07:31:25.807+00:00

    HI @PrashanthT-2849,

    Sorry for the delayed response. Yes, you can use the getMetadata activity to get the lastModified dates for each file in a folder/container. You can then use a Filter activity to filter files based on the lastModified date. Alternatively you can use an If condition activity to do the filtering for you.

    Hope this helps.