Create a derived column based on a value within the file name (Azure data flow)

Question

Hello,

I'm trying to map between json files backing up in a blob storage and a sql database. I have a need to add a column with the same file name.

Is there a way to do this processing with data flows in azure data factory please?

Answer 1

Hi @khouloud Belhaj ,

Thank you for posting query in Microsoft Q&A Platform.

We can achieve this in two ways. One is using copy activity and another is using Data flows.

I explained both ways below in detailed. kindly check them.

Approach 1: Using Copy Activity
Inside, copy activity we can make use of "additional column" setting under source tab to add a column with file name. Kindly check below gif to understand better

Approach 2: Using Data Flow
Step1: GetMetadata activity to get file Name

Step2: Create a data flow with FileName Parameter in it.

Step3: Source transformation which points to your source file and derived column transformation to add new Column with fileName and Sink Transformation to load data in to target

Hope this will help. Thank you.

---

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Answer 2

There is much simpler way to do this in dataflow. Use "Column to store file name" option in the source and you don't need to pass parameters around to handle this.