foreach - set number of iterations

arkiboys 8,371 Reputation points

in ADF pipeline, there is a lookup activity which is looking at .csv with over 200 rows...
Then there is foreach activity.
inside foreach activity, I am building a string with comma separated items pulled from .csv...
in foreach, I have set variables... which use @fr to build the comma separated string...

Question, I want to only build the strings with up to 50 items at a time
For example:
inside the foreach to build up to 50 items then do some work
Then pass in the next 50 items to build a new string with those items
and so on. perhaps I need a count mechanism in there to proceed to the copy activity once the loop has passed say 50 items...

Hope you know what I mean.
Thank you

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
7,187 questions
0 comments No comments
{count} votes

Accepted answer
  1. AnnuKumari-MSFT 23,506 Reputation points Microsoft Employee

    Hi @arkiboys ,

    Thankyou for using Microsoft Q&A platform and posting your query.

    As I understand your ask here, you want to build string with comma separated values for a group of 50 items each. Please let me know if my understanding is incorrect.

    For this purpose, you need to make use of range function to refer as the items in the forEach activity . Please follow the below steps to achieve the above requirement:

    Step1: Use Look Up activity to fetch the content of .csv file.
    Step2: Create 4 variables var1, var2, var3, var4 of type - Array
    Step3: Use multiple ForEach activities ( in your case 4 ForEach activities ) connected to the lookUp Activity specifying the item as @range(0,50) , @range(50,50) , @range(100,50) , @Rover (150,50)respectively. **Step4**: Inside each ForEach activity, Use Append variable pointing to each variables and provide value as@activity('Lookup1').output.value[item()].prop_0. **Step5**: Create 4 morevarString1, varString2, varString3,varString4variable of Type String to use set Variable activity to store the final results as string. **Step6**: Connecting to each ForEach activity, add a Set Variable activity to convert the array to string using join function with the expression such as:@fr (variables('var1'),',')`

    Here is the video for your reference:

    Additionally, you can go through the following video which talks about how to use join function to convert items of an array to string :

    Hope this will help. Please let us know if any further queries.


    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

0 additional answers

Sort by: Most helpful