ADF - How to copy file value into pipeline variable

Question

ADF - How to copy file value into pipeline variable

Abhilash Sadhale 21

I am getting two files from source system,

One file with actual data
Second file with number of rows in first file

The second file will be used for file validation purpose. I need to read second file which only has number of records value. For example, if first file has 1000 records second file will only have number '1000'. I need to read this number from second file which client has sent and then need to compare this number with count of data loaded in target table. The comparison will take place in stored proc, so I need to send this number '1000' as a parameter.

Is there any way to assign the file data value into a pipeline variable?

Accepted answer

1 additional answer

Your answer

Answer 1

ShaikMaheer-MSFT 38,551 Microsoft Employee Moderator

Hi @Abhilash Sadhale ,

Thank you for posting query in Microsoft Q&A Platform.

You can read content of your file using Lookup activity and then use Set Variable activity to set the value of count in to your variable.

Below example implements same.

Lookup activity to read file:

Set Variable activity:

Hope this will help. Please let us know if any further queries.

---------
Please consider hitting Accept Answer button. Accepted answers helps community as well.

Answer 2

Nandan Hegde 36,156 MVP Volunteer Moderator

Hey, You can use lookup activity to read that value from file and assign the output of lookup activity to a variable via set variable activity

https://stackoverflow.com/questions/69444885/in-azure-data-factory-how-to-see-output-data-for-a-lookup-activity-in-debug-mod

https://stackoverflow.com/questions/67404483/azure-data-factory-not-able-to-retrieve-json-lookup-activity-out-put-when-jso

mssqltips.com/sqlservertip/6185/azure-data-factory-lookup-activity-example/

Share via

ADF - How to copy file value into pipeline variable

1 additional answer

Your answer