Azure data factory reporting

Question

Hello

I'm using ADF in order to develop an ETL process, at the end of the process I need to have a txt file with these kind of numbesrs:
number of files processed
Number of records processed (per file)
Number of records not processed (per file)

there is a way to do it?? maybe creating an activity to read number of records?? If someone know or deal with a similar problem could help me?

many thanks

Answer 1

You could make use of Stored Procedure activity. Run SPs after the activities to log the data to table. Refer below blog to implement something similar. It has to be customized based your requirement

https://www.mssqltips.com/sqlservertip/6320/logging-azure-data-factory-pipeline-audit-data/

---

If the response helped, do "Accept Answer" and upvote it -- Vaibhav

Answer 2

Hello @CORONADO GRANADOS Diana Milena ,

We can always capture the number of the records inserted on the sink side by using the @{activity('Copy-Table').output.rowsCopied}' , but in case if want to capure how many records were NOT processed , we can use take the difference between of @{activity('Copy-Table').output.rowsRead} and the rows copied( expression shared above ) .

Hope this helps .

Thanks
Himanshu