MSMQ and Poison Messages

Last time we looked at the idea of poison messages in queues- messages that are permanently unprocessable. If we don't handle a poison message carefully, then we will be locked into a permanent cycle of requesting the message from the queue, failing to process the message, and returning the message to the queue. There are many different strategies that can be applied to reduce or eliminate the resource waste of these futile cycles.

I'll first look at the primary strategy that MSMQ uses for poison messages. Tomorrow, we'll finish off the series by looking at a comparison of this approach with various other strategies.

First, how do we detect that a message is permanently unprocessable? The application may be able to tell the queue this, but we can't guarantee that the application will recognize every potential poison message. We can approximate the idea of being permanently unprocessable by saying that there is a threshold for number of failures beyond which we expect this failure pattern to continue indefinitely. On the MSMQ binding, that threshold is controlled by the ReceiveRetryCount.

Now, assume we've got a poison message. The most common way of dealing with a poison message is to move it out of the main queue and into some other queue where no processing will take place. That other queue is commonly called a dead-letter queue. Up to version 3 of MSMQ, the dead-letter queue is the end of the line. A downside of the dead-letter queue is that some administrator has to flush the messages out in case they can be corrected or in case we have accidently classified a temporary processing failure as a permanent processing failure.

Version 4 of MSMQ introduces an optional intermediary step between the main queue and the dead-letter queue. Remember that while we were spinning doing retries on the poison message, other messages were not getting processed. The addition to the strategy is to have a retry queue. Like the dead-letter queue, the retry queue does not process messages. However, after a period of time, set by the RetryCycleDelay, the message gets moved from the retry queue back into the main queue. This process can repeat a number of times, up to the MaxRetryCycles. The retry queue allows us to attempt processing many more times without impeding the progress of the service as much.

Next time: More Poison Message Handling