How to autmatically copy the files to another folder in same storage account ?

anil kumar 1,641 Reputation points
2022-01-25T07:02:50.37+00:00

Hello,

We have a data lake with folders "Landing Zone" and "Target Zone". Whenever a new blob is added in "Landing Zone". we want that blob to be copied into "Target Zone" automatically. We know we can do that using Azure Data Factory - but is there any other way to do that ? Something like trigger on data lake/storage account to copy newly added file(s) to another container/folder. We are exploring Azcopy also.

Appreciate your insightful response. Thank you!!

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,338 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,248 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,526 questions
Accepted answer
  1. Samy Abdul 3,366 Reputation points
    2022-01-25T12:48:25.01+00:00

    You can schedule event based trigger as explained below:

    https://www.youtube.com/watch?v=RXEHrET9dUc

    https://www.cathrinewilhelmsen.net/triggers-azure-data-factory/

    Logic Apps: Please go through series of videos on logic apps here:

    https://www.youtube.com/watch?v=KxkiE2JC0RU&list=PLMWaZteqtEaIWwpz64BwOBytNDPka700J Thanks

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Samy Abdul 3,366 Reputation points
    2022-01-25T08:54:07.073+00:00

    Hi @anil kumar , do you mean datafile added in to the landing zone blob? If that is the case you can use event based trigger so that as soon as file

    being arrived in landing zone blob , the event based trigger can be kicked off. You can also explore logic apps to monitor the change and eventually

    pipeline gets trigger when the change being takes place. Thanks