Advantages of Service Broker
Service Broker's features provide a number of significant benefits to database applications. These features and benefits include:
- Database integration enhances application performance and simplifies administration.
- Message ordering and coordination for simplified application development
- Loose application coupling provides workload flexibility.
- Related message locking allows more than one instance of an application to process messages from the same queue without explicit synchronization.
- Automatic activation allows applications to scale with the message volume.
Database Integration
Service Broker's integrated design provides benefits for application performance and administration.
Integration with SQL Server allows transactional messaging without the added overhead and complexity of an external distributed transaction coordinator. An application receives one or more messages, processes the message or messages, and sends a reply message within a single database transaction. If the transaction fails, all work rolls back, and the received message is returned to the queue so that another attempt can be made to process it. No actions take effect until the application commits the transaction. The application remains in a consistent state.
Administration is easier when data, messages, and application logic are all in the database, because administration for the application (disaster recovery, security, backup, and so on) then become part of the routine administration for the database, and the administrator does not have to manage three or four separate components.
With traditional messaging systems, the message store and the database can become inconsistent. For example, when one component is restored from a backup, another component must also be restored from a backup taken at the same time, or else the information in the message store does not match the information in the database. Because Service Broker keeps messages and data in the same database, inconsistency is not an issue.
A common development environment is also a benefit of database integration. The messaging part of an application and the data part of an application can use the same SQL Server languages and tools in a Service Broker application, taking advantage of the developer's familiarity with database programming techniques for message-based programming. Stored procedures that implement a Service Broker service can be written in either Transact-SQL or one of the common language runtime (CLR) languages. Programs outside the database use Transact-SQL and familiar database programming interfaces such as ADO.NET.
Additionally, database integration enables automatic resource management. Service Broker runs in the context of the SQL Server instance, so Service Broker monitors all messages ready to be transmitted from all databases in the instance. This allows each database to maintain its own queues while helping Service Broker to manage resource usage across the entire SQL Server instance.
Ordering and Coordinating Messages
In traditional messaging systems, the application has been responsible for ordering and coordinating messages that may arrive out of order. For example, Application A sends messages 1, 2, and 3. Application B receives and acknowledges messages 1 and 3, but experiences an error with message 2. Application A resends message 2, but now the message is received after messages 1 and 3. In the past, a developer either had to write the application so that message order didn't matter, or else temporarily cache message 3 until message 2 arrived so that the application could process the messages in the correct order. Neither solution is straightforward or simple.
Another problem with traditional systems has been duplicate delivery. In the previous example, if Application B receives message 2, but the acknowledgement message back to Application A is lost, Application A will resend message 2, causing Application B to receive message 2 twice. Application code had to be able to either identify and discard the duplicate, or reprocess the duplicate without negative effects. Again, both approaches were difficult to implement.
Message coordination has also traditionally been a difficult issue to handle. For example, an application may submit hundreds or thousands of requests to a service. The service processes the requests in parallel and returns a response as soon as the service finishes processing the request. Because each request takes a different amount of time to process, the application receives responses in a different order from the order in which the application sent the outgoing message. However, to correctly process responses, the application must associate each response with the correct outgoing message. In traditional messaging systems, each application managed this association, which added to the cost and complexity of developing the application.
Service Broker solves these problems by handling message order, unique delivery, and conversation identification automatically. Once a conversation is established between two service broker endpoints, an application receives each message only once, and in the order in which the message was sent. Your application can process messages exactly once, in order, without additional code. Finally, Service Broker automatically includes an identifier in every message. An application can always tell which conversation a particular message belongs to.
Loose Coupling and Workload Flexibility
Service Broker provides loose coupling between the initiating application and the target application. An application can send a message on a queue and then continue with application processing, relying on Service Broker to ensure that the message reaches its destination. This loose coupling provides scheduling flexibility. The initiator can send out multiple messages, and multiple target services can process them in parallel. Each target service processes messages at its own pace, depending on the current workload.
Queuing also allows systems to distribute processing more evenly, reducing the peak capacity required by a server. This can improve overall throughput and performance in database applications. For example, many database applications have a surge in transactions at certain times of the day, increasing resource consumption and slowing response times. With Service Broker, these applications need not perform all of the processing for a business transaction when it is submitted to the application. Instead, the application uses Service Broker to send information about the transaction to applications that perform background processing. Those applications process the transactions reliably over a period of time, while the main entry application continues to receive new business transactions.
If the destination is not immediately available, the message remains in the transmission queue for the sending database. Service Broker retries the message until the message is sent successfully, or until the lifetime of the conversation expires allowing a reliable conversation to continue between two services even if one service is temporarily unavailable at some point during the conversation. Messages in the transmission queue are part of the database; Service Broker delivers the message even if the instance fails over or restarts.
Related Message Locking
One of the most difficult things to accomplish in a traditional messaging application has been to allow multiple programs to read in parallel from the same queue. In traditional messaging systems, messages can get processed out of order when multiple programs or multiple threads are reading from the same queue. Service Broker prevents this situation from occurring through conversation group locking.
Consider a traditional order processing application. Message A, containing instructions about creating the order header, and Message B, containing instructions about creating the order line items, are both received on the queue. If both of these messages are dequeued by separate application instances and processed at the same time, the order line item transaction might attempt to commit first, and fail because the order does not yet exist. The failure in turn causes the transaction to roll back and the message to be requeued and processed again, wasting resources. Traditionally, programmers solved this problem by combining the information from Message A and Message B into a single message. While this approach is straightforward for two messages, this approach scales poorly to systems that involve coordinating dozens or hundreds of messages.
Service Broker solves this problem by associating related conversations in a conversation group. Service Broker automatically locks all messages in the same conversation group, so that these messages can only be received and processed by one application instance. Meanwhile, other instances of the application can continue to dequeue and process messages in other conversation groups. This allows multiple parallel application instances to work reliably and efficiently, without requiring complicated locking code in the application.
Automatic Activation
One of the most useful features of Service Broker is activation. Activation allows an application to dynamically scale itself to match the volume of messages that arrive in the queue. Service Broker provides features that allow both programs running inside the database and programs running outside the database to take advantage of activation. However, Service Broker does not require that an application use activation.
Service Broker monitors the activity in a queue to determine whether an application is receiving messages for all of the conversations that have messages available. Service Broker activation starts a new queue reader when there is work for the queue reader to perform. To determine when there is work for a queue reader, Service Broker monitors activity on the queue. When the number of queue readers matches the incoming traffic, the queue periodically reaches a state where either the queue is empty, or all messages in the queue belong to a conversation that is being processed by another queue reader. If a queue does not reach this state for a period of time, Service Broker activates another instance of the application.
Applications use different activation approaches for programs that run inside the database and programs that run outside the database. For programs within the database, Service Broker starts another copy of the stored procedure specified by the queue. For programs that run outside the database, Service Broker generates an activation event. The program monitors this event to determine when another queue reader is needed.
Service Broker does not stop a program started through activation. Instead, activated applications are written to shut down automatically after a certain period of time without any messages arriving to be received. An application designed this way can allow the number of application instances to grow and shrink dynamically as the traffic to the service changes. Also, if the system shuts down or is rebooted, applications are automatically started to read the messages in the queue when the system restarts.
Traditional messaging systems lack this behavior, and frequently end up having either too many or too few resources dedicated to a particular queue at any given time.
See Also
Other Resources
Service Broker Architecture
Introduction to Service Broker Programming