Get Latest folder name (Azure Data factory)

Question

In azure blob storage, a binary is uploaded in a folder which is created with date (yyyyMMdd) See example belpw

20231124/test.jar

20231122/test.jar

20231121/test.jar

20231120/test.jar

• We have an ADF pipleine which required to be triggered every day once only if a new folder (yyyyMMdd) is available in the path.

Is there a way to achieve this?

Answer 1

Hi @Siddareddy, Bharath

You can use two pipelines, one to check the foldername. If the foldername matches current date, you can call your child pipeline where you have the actual logic to process.

Steps:

In the parent pipeline, you can have a storage event trigger which will get triggered whenever a folder is created\ added to blob\adls. Upon triggering parent, you can use below steps.

Use get meta data activity to get all child items (folder names)

Use an if condition with below expression

@contains(activity('Get Metadata1').output.childItems,
  json(concat('{"name":"',formatDateTime(utcNow(),'yyyyMMdd'),
			'/test.jar","type": "Folder"
		}')))

This will check if there is a folder name with todays date. In true part, call the child pipeline using execute pipeline activity. In False part, you need not do anything.

Please try and let us know if you have questions.

Thanks