DYNAMICALLY READING FILES FROM BLOB STORAGE

Anandazure 46

How to read files dynamically from azure datalake storage gen2?

daily new files coming how to read daily updating files

1 answer

Sander van de Velde 28,706 Reputation points MVP

2021-11-22T10:22:51.183+00:00

Hello @Anandazure ,

you want to react to incoming blobs in your Azure data lake (the blob storage).

Azure Functions is a great way to execute code based on incoming blobs.

The Azure Function is just a piece of code triggered by some event that happens in Azure. Even blob storage can trigger an Azure function.

An example of a blob storage trigger is seen here.

If you only want to execute some code once in a while, the timer trigger is a very good solution.

MS Learn offers multiple modules on how to work with Azure Functions. I recommend checking these out, for example this one.
Please sign in to rate this answer.
Anandazure 46 Reputation points

2021-11-22T12:31:53.307+00:00

@markus.bohland@hotmail.de ('capcon/',substring(utcnow(),0,4),'/',substring(utcnow),5,2),'/',substring(utcnow(),8,2))

what is the meaning of this function and in which situation we can use it

Sander van de Velde 28,706 Reputation points MVP

2021-11-22T15:33:01.837+00:00

Hello @Anandazure ,

I'm not sure where this line of code coming from.

What it does seems simply building up a file path form parts of the current date (year? month? day?).

Are you now able to read new files on a daily basis?

HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2021-11-22T23:33:03.197+00:00

Hello @Anandazure ,
Thanks for the ask and using Microsoft Q&A platform .

The utcnow() function returns the utc time

eg : 2018-04-15T13:00:00.0000000Z
The dynamic expression
@markus.bohland@hotmail.de ('capcon/',substring(utcnow(),0,4),'/',substring(utcnow),5,2),'/',substring(utcnow(),8,2))
will return as string like
capcon/2018/04/15

Please read about the functions more here . https://learn.microsoft.com/en-us/azure/data-factory/control-flow-expression-language-functions#utcNow

Please do let me know how it goes .
Thanks
Himanshu

-------------------------------------------------------------------------------------------------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2021-12-03T03:46:04.8+00:00

Hello @Anandazure ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .In case if you have any resolution please do share that same with the community as it can be helpful to others . Otherwise, will respond back with the more details and we will try to help .
Thanks
Himanshu
Sign in to comment