question

antothomas-9485 avatar image
0 Votes"
antothomas-9485 asked MartinJaffer-MSFT commented

Does validation activity in ADF accepts wildcard paths for folder name ?

I would like to check the existence of an .ack file in a folder which will be having name with date and time. (eg: folder name = 2021-03-31-01-19-21).

The day day part in the folder can be set, but we are not sure about the time part in the folder name, it depends on the time of creation of that folder. Tried to add this path for validation task in ADF but there I was not able to provide wildcard path for the time part, since we are not known what is exact value.

Is it possible to provide the folder path like [ @concat('temp/folder',(formatDateTime(utcnow(), 'yyyy-MM-dd'),'-??-??-??') ]
Is there any alternative way to achieve this? I need to check for the existence of .ack file, every 10 mins for 2hrs.
(get metadata activity alone is not helping to recheck the existence of file every 10 mins till 2hrs and also not able to pass wildcard file path)

azure-data-factoryazure-data-lake-storage
· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

@antothomas-9485 did my response answer your question or help at all? If it solved your issue, please mark as accepted answer. If you need more assistance., please tell me how I may better assist you.

0 Votes 0 ·

@antothomas-9485 are you still facing the issue? If you found your own solution, could you please share it here with the community?

0 Votes 0 ·

1 Answer

MartinJaffer-MSFT avatar image
0 Votes"
MartinJaffer-MSFT answered

Hello @antothomas-9485 and welcome to Microsoft Q&A.

I tried with a binary blob dataset, placing * for the filename. Unfortunately this did not work, it looked for a file named * rather than treating as a wildcard. In addition, in the blob dataset documentation, next to filename it says:

The file name under the given container and folder path. If you want to use wildcard to filter files, skip this setting and specify that in activity source settings.

Unfortunately validation activity has no relevant source setting. So, it looks like the answer is no.

However, have you considered using an event trigger? An event trigger accepts wildcards, and will start the pipeline when the file is created. This way, you will not need to check for the existence of the file, the file will start the pipeline!

5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.