Azure data factory bulk copy

Question

I copy multiple tables in bulk IN lookup for each activity. So how can i get total number of records and size of data for all the tables that process for each.

Answer 1

@Snv, Sudhir K Thanks for using Microsoft Q&A forum and posting your query.

As per your requirement my understanding is that you have a lookup activity to get the table names for which data needds to be copied and then followed by a ForEach activity which has copy activity inside it and you would want to get the total number of records copied at the end.

Assuming that your forEach is sequential, after completion of the copy activity for each iteration load the dataWritten and rowsCopied from Copy activity output to a staging table in SQL including the current pipelinerunID. Once all the iterations are completed, outside of ForEach activity, have a Stored procedure activity which will have a query that points to the staging table and do the math for addition of all dataWritten and rowsCopied and return that as desired output and from there you can have them as per your business need.

Below is a sample copy activity looks like:

Hope this info helps.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.