If file exist in the source .. other wise copy activity does not execute

hemanth Kumar loka 226 Reputation points

Hi Community,

I had question is
I have a SFTP(source) server which is run every day.

Am receiving files from SFTP to ADLS the file contain (IF file today is File 1 _24_04_2022.csv) Etc…. I want to copy the file which contains today date only those file copy today other wise don’t copy, if we are file is not found, send an email

Please help me

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,381 questions
{count} votes

Accepted answer
  1. HimanshuSinha-msft 19,196 Reputation points Microsoft Employee

    Hello @hemanth Kumar loka ,
    Thanks for the question and using MS Q&A platform.
    As we understand the ask here is how to copy a file from SFTP to ADLS GEN2 and checking the filename to be todays date , please do let us know if its not accurate.

    This is what I suggest .

    1. Create a variable and contruct the file name so that ADF can dynamically create the filename everyday . You will have to use Dynamic expression .
    2. Please use the Metadat data actvity to check if the file exists . https://learn.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity


    1. Use a IF statemnt to read if the file exists ( from the output of metadat actvity ) and if it does , inside the IF statemnt add add the copy activity ( I think you already should have one )
    2. At this time ADF does not an email activity , but i think you can always use logic apps and call that from ADF . The other route could to enable monitoring on ADF and enable email alerts . In the event there is NO file you can fail a child pipeline which will generate a email .

    Please do let me if you have any queries.

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
      • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    0 comments No comments

0 additional answers

Sort by: Most helpful