DYNAMICALLY READING FILES FROM BLOB STORAGE

Anandazure 46 Reputation points
2021-11-22T09:39:32.04+00:00

How to read files dynamically from azure datalake storage gen2?

daily new files coming how to read daily updating files

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
6,331 questions
No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Sander van de Velde 15,241 Reputation points MVP
    2021-11-22T10:22:51.183+00:00

    Hello @Anandazure ,

    you want to react to incoming blobs in your Azure data lake (the blob storage).

    Azure Functions is a great way to execute code based on incoming blobs.

    The Azure Function is just a piece of code triggered by some event that happens in Azure. Even blob storage can trigger an Azure function.

    An example of a blob storage trigger is seen here.

    If you only want to execute some code once in a while, the timer trigger is a very good solution.

    MS Learn offers multiple modules on how to work with Azure Functions. I recommend checking these out, for example this one.