ADF copy source file path with wildcard in dynamic parameter

Question

Hi

We have ADF copy and the source file path which has date in it. The date is getting resolved with an input pipeline parameter. The problem is input parameter date has yyyyMMddHH and the actual path has yyyyMMddHHmmss. So in input we don't get minutes and seconds.

for example, in input parameter we get 10/20/2022 05:00:00, which will be get formatted to 2022102005. And the actual paths look like these

a/b/c/20221020051356/1.parquet, 2.parquet...
a/b/c/20221020065434/1.parquet, 2.parquet...
a/b/c/20221020072241/1.parquet, 2.parquet...

How can I create a dataset with wild card, which can resolve the above folders ?

Here is our dataset example. Can you please give me an example of how the wildcard file path can be used here?

Thanks

Answer 1

Hello @Unknown-2795 ,
Thanks for the question and using MS Q&A platform.
As we understand the ask here is how to create a path like YYYYMMDDHH* ( eg ( 2022201005* ) , please do let us know if its not accurate.
I see that since you are already paramterizing the dataet , i think you are going in the right direction .

I was able to male it will in two steps .

Step 1 . Use the split function to read the parameter and split the string at '/' and then also replace the black which i see after the YYYY ( eg /2022 05 )

@split(replace(substring(pipeline().parameters.DateAsParameter,0,13),' ',''),'/')

Note the Split function always writes back to a variable type array ( see the screenshot below ) .

Step 2 .
Use the concatenate function

@markus.bohland@hotmail.de (substring(variables('var1')2,0,4),variables('var1')1,variables('var1')[0],substring(variables('var1')2,4,2),'*')

Note : I am using the variables('var1')2 twice to get the YYYY and also HH .

I did tested the solution for one of the date which you posted .

{
"name": "NewPath",
"value": "2022201005*"
Your path is
20221020051356

}

Please do let me if you have any queries.
Thanks
Himanshu

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators