loop in ADF

Question

loop in ADF

arkiboys 9,711

Hello,
in my ADF, There is a lookup activity which reads a column from a csv file.
After the lookup, there is a loop activity which loops through each of the records in the csv.
Inside the loop activity there is a copy activity which calls an api by building a url string as follows:
https://site/valuefromLoop...ID eq (''', dataset().ID, ''')')

Question:
As you see above, the url is being built one ID at a time...
How is it possible so that I use several, say 10 IDs at a time, i.e. something like:
https://site/valuefromLoop...ID IN ('ID1', 'ID2', 'ID3', 'ID4', etc.)')

Thank you

0 comments

Answer accepted by question author

0 additional answers

Your answer

Answer 1

Hi @arkiboys ,

Thankyou for using Microsoft Q&A platform and posting your query .

As per my understanding about your question, it looks like you want to know if there is any possibility to hit multiple REST API URLs at once using IN clause. Please let me know if my understanding about your query is not correct.

There are possibilities of using optional parameters in the URL provided the source URL supports the same. This filter is applied on top of the result set we get out of the API URL.

As in your case ,You want the web activity to be parallelly triggered specifying the list of IDs in the URL itself . It doesn't seem to be possible. Best possible solution would be to use ForEach block without checking Sequential option so that it would parallelly execute web activities for all the IDs coming as the lookUp output.

Below is a simple use case where this functionality of using optional parameter would work for getting the list of resource name with a particular tagName or list of resources within specified Resource group lists etc.

Resources - List - This is the article which talks about using optional URI parameters such as $expand , $filter, $top etc.

$filter=startswith(tagName, 'depart') : This query will return all resources with a tag name prefixed by the phrase depart (i.e.department, departureDate, departureTime, etc.)
$filter=resourceGroup eq 'annu-rg' or resourceGroup eq 'adf-rg' : This will return all the resources present in any one of the provided resource group name.

Hope this will help. Please let us know if any further queries.

------------------------------

Please don't forget to click on or upvote button whenever the information provided helps you.
Original posters help the community find answers faster by identifying the correct answer. Here is how
Want a reminder to come back and check responses? Here is how to subscribe to a notification
If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

arkiboys 9,711 Reputation points

2022-04-18T11:53:28.147+00:00

Hello,
I do not quite follow what you are suggesting...
Summary of the question:
instead of passing each id, one at a time inside the for each loop, i.e. ...
http://...id='xyz'
http://...id='urt'
...

is it possible to take all teh ids and pass them like this?
http://...id in ('xyz', 'urt', ...)

?

Thank you
AnnuKumari-MSFT 34,571 Reputation points Microsoft Employee Moderator

2022-04-19T05:45:21.197+00:00

Hi @arkiboys ,

Passing multiple Ids can be done with the help of 'Or' condition . Example: http://site/valuefromLoop $filter=Id eq 1 or Id eq 2 or Id eq 3 . But it depends on the source URL if it supports the filter parameter or not.

You can try checking your source URL accepts filter parameter or not using the example mentioned above. However, using 'In' will not work in the API URL.
arkiboys 9,711 Reputation points

2022-04-19T06:27:27.377+00:00

Hello,
The filter url accepts in keyword as well as or as per your suggestions.
What I would like to find-out is as follows:
1- which of the above two methods are more efficient or doe sit not make a difference?
2- if I continue using one id per iteration in th eforeach loop, I notice there is parallel processing. So is that efficient to use? perhaps not as api is being called multiple times rather than once?

Thank you

Share via

loop in ADF

0 additional answers

Your answer