Priority Queue Pattern

Prioritize requests sent to services so that requests with a higher priority are received and processed more quickly than those of a lower priority. This pattern is useful in applications that offer different service level guarantees to individual clients.

Context and Problem

Applications may delegate specific tasks to other services; for example, to perform background processing or to integrate with other applications or services. In the cloud, a message queue is typically used to delegate tasks to background processing. In many cases the order in which requests are received by a service is not important. However, in some cases it may be necessary to prioritize specific requests. These requests should be processed earlier than others of a lower priority that may have been sent previously by the application.

Solution

A queue is usually a first-in, first-out (FIFO) structure, and consumers typically receive messages in the same order that they were posted to the queue. However, some message queues support priority messaging; the application posting a message can assign a priority to a message and the messages in the queue are automatically reordered so that messages with a higher priority will be received before those of a lower priority. Figure 1 illustrates a queue that provides priority messaging.

Figure 1 - Using a queuing mechanism that supports message prioritization

Note

Most message queue implementations support multiple consumers (following the Competing Consumers pattern), and the number of consumer processes can be scaled up or down as demand dictates.

In systems that do not support priority-based message queues, an alternative solution is to maintain a separate queue for each priority. The application is responsible for posting messages to the appropriate queue. Each queue can have a separate pool of consumers. Higher priority queues can have a larger pool of consumers running on faster hardware than lower priority queues. Figure 2 shows this approach.

Figure 2 - Using separate message queues for each priority

A variation on this strategy is to have a single pool of consumers that check for messages on high priority queues first, and then only start to fetch messages from lower priority queues if no higher priority messages are waiting. There are some semantic differences between a solution that uses a single pool of consumer processes (either with a single queue that supports messages with different priorities or with multiple queues that each handle messages of a single priority), and a solution that uses multiple queues with a separate pool for each queue.

In the single pool approach, higher priority messages will always be received and processed before lower priority messages. In theory, messages that have a very low priority may be continually superseded and might never be processed. In the multiple pool approach, lower priority messages will always be processed, just not as quickly as those of a higher priority (depending on the relative size of the pools and the resources that they have available).

Using a priority queuing mechanism can provide the following advantages:

• It allows applications to meet business requirements that necessitate prioritization of availability or performance, such as offering different levels of service to specific groups of customers.
• It can help to minimize operational costs. In the single queue approach, you can scale back the number of consumers if necessary. High priority messages will still be processed first (although possibly more slowly), and lower priority messages may be delayed for longer. If you have implemented the multiple message queue approach with separate pools of consumers for each queue, you can reduce the pool of consumers for lower priority queues, or even suspend processing for some very low priority queues by halting all the consumers that listen for messages on those queues.
• The multiple message queue approach can help to maximize application performance and scalability by partitioning messages based on processing requirements. For example, vital tasks can be prioritized to be handled by receivers that run immediately while less important background tasks can be handled by receivers that are scheduled to run at less busy periods.

Issues and Considerations

Consider the following points when deciding how to implement this pattern:

• Define the priorities in the context of the solution. For example, “high priority” could mean that messages should be processed within ten seconds. Identify the requirements for handling high priority items, and what other resources must be allocated to meet these criteria.
• Decide if all high priority items must be processed before any lower priority items. If the messages are being processed by a single pool of consumers, it may be necessary to provide a mechanism that can preempt and suspend a task that is handling a low priority message if a higher priority message becomes available.
• In the multiple queue approach, when using a single pool of consumer processes that listen on all queues rather than a dedicated consumer pool for each queue, the consumer must apply an algorithm that ensures it always services messages from higher priority queues before those from lower priority queues.
• Monitor the speed of processing on high and low priority queues to ensure that messages in these queues are processed at the expected rates.
• If you need to guarantee that low priority messages will be processed, it may be necessary to implement the multiple message queue approach with multiple pools of consumers. Alternatively, in a queue that supports message prioritization, it may be possible to dynamically increase the priority of a queued message as it ages. However, this approach depends on the message queue providing this feature.
• Using a separate queue for each message priority works best for systems that have a small number of well-defined priorities.
• Message priorities may be determined logically by the system. For example, rather than having explicit high and low priority messages, they could be designated as “fee paying customer”, or “non-fee paying customer.” Depending on your business model, your system might allocate more resources to processing messages from fee paying customers than non-fee paying ones.
• There may be a financial and processing cost associated with checking a queue for a message (some commercial messaging systems charge a small fee each time a message is posted or retrieved, and each time a queue is queried for messages). This cost will be increased when checking multiple queues.
• It may be possible to dynamically adjust the size of a pool of consumers based on the length of the queue that the pool is servicing. For more information, see the Autoscaling Guidance.

When to Use this Pattern

This pattern is ideally suited to scenarios where:

• The system must handle multiple tasks that might have different priorities.
• Different users or tenants should be served with different priority.

Example

Microsoft Azure does not provide a queuing mechanism that natively support automatic prioritization of messages through sorting. However, it does provide Azure Service Bus topics and subscriptions, which support a queuing mechanism that provides message filtering, together with a wide range of flexible capabilities that make it ideal for use in almost all priority queue implementations.

A Azure solution can implement a Service Bus topic to which an application can post messages, in the same way as a queue. Messages can contain metadata in the form of application-defined custom properties. Service Bus subscriptions can be associated with the topic, and these subscriptions can filter messages based on their properties. When an application sends a message to a topic, the message is directed to the appropriate subscription from where it can be read by a consumer. Consumer processes can retrieve messages from a subscription using the same semantics as a message queue (a subscription is a logical queue).

Figure 3 illustrates a solution using Azure Service Bus topics and subscriptions.

Figure 3 - Implementing a priority queue with Azure Service Bus topics and subscriptions

In Figure 3 the application creates several messages and assigns a custom property called Priority in each message with a value, either High or Low. The application posts these messages to a topic. The topic has two associated subscriptions, which both filter messages by examining the Priority property. One subscription accepts messages where the Priority property is set to High, and the other accepts messages where the Priority property is set to Low. A pool of consumers reads messages from each subscription. The high priority subscription has a larger pool, and these consumers might be running on more powerful (and expensive) computers with more resources available than the consumers in the low priority pool.

Note that there is nothing special about the designation of high and low priority messages in this example. These are simply labels specified as properties in each message, and are used to direct messages to a specific subscription. If additional priorities are required, it is relatively easy to create further subscriptions and pools of consumer processes to handle these priorities.

The PriorityQueue solution in the code available with this guidance contains an implementation of this approach. This solution contains two worker roles projects named PriorityQueue.High and PriorityQueue.Low. These two worker roles inherit from a class called PriorityWorkerRole which contains the functionality for connecting to a specified subscription in the OnStart method.

The PriorityQueue.High and PriorityQueue.Low worker roles connect to different subscriptions, defined by their configuration settings. An administrator can configure different numbers of each role to be run; typically there will be more instances of the PriorityQueue.High worker role than the PriorityQueue.Low worker role.

The Run method in the PriorityWorkerRole class arranges for the virtual ProcessMessage method (also defined in the PriorityWorkerRole class) to be executed for each message received on the queue. The following code shows the Run and ProcessMessage methods. The QueueManager class, defined in the PriorityQueue.Shared project, provides helper methods for using Azure Service Bus queues.

public class PriorityWorkerRole : RoleEntryPoint{  private QueueManager queueManager;  ...  public override void Run()  {    // Start listening for messages on the subscription.    var subscriptionName = CloudConfigurationManager.GetSetting("SubscriptionName");    this.queueManager.ReceiveMessages(subscriptionName, this.ProcessMessage);    ...;  }  ...  protected virtual async Task ProcessMessage(BrokeredMessage message)  {    // Simulating processing.    await Task.Delay(TimeSpan.FromSeconds(2));  }}

The PriorityQueue.High and PriorityQueue.Low worker roles both override the default functionality of the ProcessMessage method. The code below shows the ProcessMessage method for the PriorityQueue.High worker role.

protected override async Task ProcessMessage(BrokeredMessage message){  // Simulate message processing for High priority messages.  await base.ProcessMessage(message);  Trace.TraceInformation("High priority message processed by " +    RoleEnvironment.CurrentRoleInstance.Id + " MessageId: " + message.MessageId);}

When an application posts messages to the topic associated with the subscriptions used by the PriorityQueue.High and PriorityQueue.Low worker roles, it specifies the priority by using the Priority custom property, as shown in the following code example. This code (which is implemented in the WorkerRole class in the PriorityQueue.Sender project), uses the SendBatchAsync helper method of the QueueManager class to post messages to a topic in batches.

// Send a low priority batch. var lowMessages = new List<BrokeredMessage>();for (int i = 0; i < 10; i++){  var message = new BrokeredMessage() { MessageId = Guid.NewGuid().ToString() };  message.Properties["Priority"] = Priority.Low;  lowMessages.Add(message);}this.queueManager.SendBatchAsync(lowMessages).Wait();...// Send a high priority batch.var highMessages = new List<BrokeredMessage>();for (int i = 0; i < 10; i++){  var message = new BrokeredMessage() { MessageId = Guid.NewGuid().ToString() };  message.Properties["Priority"] = Priority.High;  highMessages.Add(message);}this.queueManager.SendBatchAsync(highMessages).Wait();

Related Patterns and Guidance

The following patterns and guidance may also be relevant when implementing this pattern:

• Asynchronous Messaging Primer. A consumer service processing a request may need to send a reply to the instance of the application that posted the request. The Asynchronous Messaging Primer provides more information on the strategies that can be used to implement request/response messaging.
• Competing Consumers Pattern. To increase the throughput of the queues, it’s possible to have multiple consumers that listen on the same queue, and process the tasks in parallel. These consumers will compete for messages, but only one should be able to process each message. The Competing Consumers pattern provides more information on the benefits and tradeoffs of implementing this approach.
• Throttling Pattern. You can implement throttling by using queues. Priority messaging can be used to ensure that requests from critical applications, or applications being run by high-value customers, are given precedence over requests from less important applications.
• Autoscaling Guidance. It may be possible to scale the size of the pool of consumer processes handling a queue depending on the length of the queue. This strategy can help to improve performance, especially for pools handling high priority messages.

More Information

• The article Priority Queue Pattern on the Cloud Design Pattern website.
• The article Enterprise Integration Patterns with Service Bus on Abhishek Lal’s blog.

This pattern has a sample application associated with it. You can download the "Cloud Design Patterns – Sample Code" from the Microsoft Download Center at https://aka.ms/cloud-design-patterns-sample.

Next Topic | Previous Topic | Home | Community