parallel processing

Question

There is a url which accepts ids, i.e.
https://someaddress/id in {@IDS_test }

example1:
https://someaddress/id in {'vxyz'}
https://someaddress/id in {'ITYG'}
https://someaddress/id in {'FTRT'}
https://someaddress/id in {'OOOO'}

or I can send all ids in one url as follows:
example2:
https://someaddress/id in {'vxyz','ITYG','FTRT','OOOO',...'}

looking at the above urls, the one which has all of the ids will be quicker than passing each id individually...

In ADF pipeline I have a foreach loop which basically does example1

Question:
When I run the pipeline, I see or what it looks like that multiple RestAPI activities are running at the same time inside the foreach activity. So I guess the urls with different ids are being run in parallel and so efficiency in processing?
If that is the case in the foreach loop activity then I guess it is similar to me having only one RestAPI activity and running example2?

Thank you

Answer 1

Hello @arkiboys ,
Thanks for the question and using MS Q&A platform.
As we understand the ask here is how is it different calling a API with just an ID vs calling out all of them together with a IN clause , please do let us know if its not accurate.
When we call a API , we have an agreement with the API owner as to what are the inputs(parameter ) , which we need pass and what will be response .
In your use case , if there are two API one whch takes only the id and then other which takes more then one ID , you can always use use both of them . But the API are designed to be as granular and as fast as possible , so API with a bunch of ID does not look to be a good in general but I agree we may have exceptions .

You are right in foreach loop you call the API and it looks like they all are called in a parallel ( FYI you can change this behaviour by setting the sequential to "yes") .

Going back to the ask if calling the API with individual ID is effecient . The API are designed we do not as granular as possible , so thats a good design .

Please do let me if you have any queries.
Thanks
Himanshu

---

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
  • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators