Use "Move File after completion" with dynamic dataset in data flow

Question

I have a Data Flow that has a source (obviously). The source's dataset is pointing to a blob storage container. However, the folder path and file name are dynamic (i.e. at one run it could be maincontainer/X/123, and during another run it's maincontainer/Y/789).
The behavior works as expected. Now, I'm planning to use the "After Completion" feature under Source Options, with "Move" as the selected option. Problem is, I have no idea how to set the "From" path dynamically. I want it to move the specific file it runs against, and I don't have the folder path or the file name since it's dynamically sent to the data flow. Is there a way to do the requested?

Answer 1

Hello @Wassim Al Mokdad ,

Thanks for the question and using MS Q&A platform.

As you mentioned that you are passing the filename and folder name dynamically , you can use the same parameters to create a valid path where you want to move the file.

Let me know how it goes .

Thanks

Himanshu

Please accept as "Yes" if the answer provided is useful , so that you can help others in the community looking for remediation for similar issues. 

Answer 2

I found a way. The pipeline feeds the dataset with the value of the parameters. So, by using the same logic, I've created a new parameter in the Data Flow, called it FilePath, and used it in the "From" path of the "After completion" action. Then, from the pipeline editing page, I supplied the value to the Data flow. That worked for me