Unblocking Flow Throttles

Last time, we were looking at how to control flow through a channel pump by introducing the concept of a throttle. The implementation of the throttle was a semaphore, allowing us to limit the number of channels active in the system at any point in time. This lead to a straightforward approach; the channel pump stopped to acquire the semaphore each time before starting new work.

 ChannelPump
BEGIN LOOP
  AcceptChannel
   AcquireThrottle
 DoWork
END LOOP

 DoWork
MessagePump
CloseChannel
ReleaseThrottle

The straightforward approach to throttling has the drawback that a thread of execution is blocked sitting in the ChannelPump loop. We can have several instances of DoWork running simultaneously by executing DoWork asynchronously. We can similarly use asynchronous execution to straighten out the ChannelPump loop. This produces an approach where execution ping-pongs between two asynchronous methods.

The key to the ping-pong approach is that the asynchronous execution can wait to start until a condition is satisfied. Let's change AcquireThrottle to return immediately from the call rather than blocking. If AcquireThrottle returns true, then it means that we acquired the semaphore. If AcquireThrottle returns false, then it means that we weren't successful but in the future a method will be executed once the semaphore is available. Let's call that method GotThrottle.

We can now bounce execution from ChannelPump to GotThrottle and then back to ChannelPump in the future.

 ChannelPump
BEGIN LOOP
   AcceptChannel
   IF AcquireThrottle THEN
     DoWork
  ELSE
        STOP
END LOOP

 GotThrottle
DoWork
ChannelPump

 DoWork
MessagePump
CloseChannel
ReleaseThrottle

There's no longer any need to wait for the semaphore to become available. There is some code waiting to start running pending availability of the semaphore but it doesn't have to be assigned any resources. We have to be careful now though about being very precise with asynchronous execution. Since ChannelPump is calling itself through an intermediary, synchronous execution of those calls would lead to building up an unlimited number of stack frames. We have also created some new complexity for ourselves in the form of state management between the calls.

That's all for the channel pump this week. I'll have some more articles in this series in the future.

Next time: ContractNamespaceAttribute