Throttling Webservice Requests

Question

Hi, Everybody...

I have an "internal" API that tracks packages (plural). This "internal" API accepts a collection of tracking numbers, iterates through the collection and invokes the "external" (i.e., third-party vendor) track package (singular) API. This effectively "blasts" a number of requests at the "external" (i.e., third-party vendor) track package (singular) API. Apparently, the vendor API cannot handle multiple concurrent requests. I need to somehow "throttle" the requests one at a time (i.e., invoke the "external" (i.e., third-party vendor) track package (singular) API, wait for the response, process it and then invoke the same "external" (i.e., third-party vendor) track package (singular) API with the next tracking number, until there are no more tracking numbers).

I imagine instantiating a singleton and have it instantiate a queue upon starting the webservice. The singleton and the queue will persist until the webservice is shutdown. The "internal" API that tracks packages (plural) will simply enqueue the tracking numbers. A worker thread would dequeue a tracking number, invoke the "external" (i.e., third-party vendor) track package (singular) API, wait for the response, process it and repeat until all the tracking numbers have been dequeued.

Also, is it possible to make a queue "eventful" (i.e., raise an event when the queue count becomes non-zero and another event when the queue count becomes zero)? Thus, the former event would create a worker thread and the latter event would release/dispose the worker thread.

Also, will I need to perform some housekeeping when the webservice is shutdown?

Is this a viable approach?... Or is there a better way?

Note: I viewed the following. None were applicable specifically to my situation.

• throttling api requests
• Azure Translation API - Throttling client requests
• Monitor webservice requests?

Answer 1

you need to better understand the api limits. is max concurrent requests (denial of service attack detection), max requests per second, max requests per minute, max requests per hour, max requests per day?

you only need as many threads a con-current requests are allowed. to implement the queue use:

https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentqueue-1?view=net-7.0

you can use semaphores to wakeup the background queue processes when adding to the queue, or the they can use a timer if the queue is empty.

Answer 2

ASP.NET Core 7 already includes built-in rate limiting support,

https://github.com/dotnet/aspnetcore/issues/29933

https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-7.0

If you are using ASP.NET 4.x, then you need to find an alternative for that legacy framework.