Share via

Copy latest files from SAP Server using Azure Data Facotry and File System Linked Service and store it in Azure Data Lake

Mariya Susnerwala 21 Reputation points
Sep 9, 2021, 6:37 PM

I have a task to copy the newly added files everyday from SAP Server and store it in ADLS. There are 2 types files on the server (Recurring and Upfront) appended with date. I need to store the files in separate folder and everyday add the latest file from the SAP Server to the ADLS.

File name format: R_07292021.orc Recurring_08312021.orc U_07292021.orc Upfront_08312021.orc

Below are the steps I have taken so far

  1. Get Metadata Activity to get the list of files from the server

130856-image.png

  1. Use filter activity to separate the files based on the names, so filtering with the initial letter

130844-image.png

  1. I tried using the Foreach activity and If Condition, but it doesn't seem to be working.

I am not sure how to proceed from here. Any help is appreciated. Thank you!!

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,360 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,245 questions
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,506 Reputation points Microsoft Employee
    Sep 10, 2021, 10:51 AM

    Hi @Mariya Susnerwala ,

    Thank you for posting query on Microsoft Q&A Platform.

    Similar case to get latest file discussed in below links. Kindly check them.
    https://learn.microsoft.com/en-us/answers/questions/136815/lates-file-extraction-azure-data-factory.html

    You can refer below video also which explains same case step by step.
    https://www.youtube.com/watch?v=QHeG36oqkPQ

    Hope this is helpful. Please let us know if any further queries.

    --------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.