In a post below, and in several samples, I use sm:header() to quickly navigate to the header collection in order to determine if a particular header is present.  With this data, you can also go ahead and start to write other XPath functions that riff off of these.  Have at it!

-Matt

“Can the Routing Service do load balancing?” is one of the most common questions I’m asked.  The immediate answer I give is almost universally disappointing: No.  It’s actually hard to say that Load Balancing (as most people mean it) should be done at the Routing Service.  But why?  It seems like since I have this intermediary that load balancing is one of the obvious applications.  Here’s why it doesn’t work as well or as easily as it should.

How would I build a filter that allowed me to do something simple like Round Robin load balancing?  Well, it would seem that I would have a filter, one for each destination that I wanted to send to, and as each message came through, only one of the selected filters would respond that it matched.  So a setup like this:

RoundRobinFilter1 –> Endpoint1

Round RobinFilter2 –> Endpoint2

RoundRobinFilter3 –> Endpoint3

Which we then fed 3 messages would result in message 1 going to Endpoint1, message 2 to Endpoint 2, etc.

But building this is hard :) Why?  You immediately run into two issues:

1. How do these MessageFilters exchange information?  Basically how does each filter know that this time it’s the one that needs to match (and the other two know that they’re not)?
2. How does this information stay around between messages?  Remember that MessageFilters are designed to be stateless, meaning that they don’t normally have any information and would always match a given message the same way.

We actually have a sample out that shows how this would work, but it relies on a “creative” use of the MessageFilter API.  The hint is that MessageFilters can be a used as a factory to create the MessageFilterTable that they belong to.  Another key point is that MessageFilterTables can contain child MessageFilterTables (usually of a specific type).  An example of this in the framework today is the XPathMessageFilter.  When you add a XPathMessageFilter to the Routing Service’s MessageFilterTable, that’s actually not what happens.  Instead, this is the structure you end up with:

Where did this middle XPathMessageFilterTable come from?  Reflector gives us the answer.  When the XPathMessageFilter is added to the parent MessageFilterTable (red), the parent MFT actually calls CreateFilterTable on the XPathMessageFilter, at which point the XPathMessageFilter creates a new XPathMessageFilterTable and returns it to the parent MessageFilterTable to use.

When the parent MessageFilterTable is asked by the RoutingService to return a list of endpoints that match a particular message, the parent MessageFilterTable in turn delegates this message to its own internal MessageFilterTables and MessageFilters, building the full result set and returning it to the Routing Service.

The sample RoundRobinMessageFilter inside the Routing Service’s Advanced Filters sample (over here) does much the same thing as the XPathMessageFilter.  Now that we have a custom MessageFilterTable in the mix, we can use that to store state and coordinate the responses from the individual MessageFilters that ultimately gets returned to the parent MessageFilterTable.  So can you build things this way?  Absolutely.  Is it the best answer? Probably not.  At the end of the day you’re still stuffing state into the MessageFilter model, where it doesn’t belong.  Furthermore, implementing custom MessageFilterTables is a pain because it requires you to stub in a lot of method calls that won’t ever be used.

Another answer is a different custom MessageFilter, which wouldn’t do RoundRobin specifically, but would send messages to a particular endpoint 1/n of the time, on average, where n is the number of possible destinations.  This is also buildable, and could even be stateless, but could also be tricky because you’d have to figure out how to gaurentee that only one filter matched any given message.  Just having say four filters that all compute a rand() between 0 and 1 and then determine if it’s in a particular range isn’t going to cut it, since multiple filters could independently report that they matched.  I leave this one as an exercise to the reader, but one solution is to predefine your ranges and use some piece of message data (such as say, the time the message was sent, some random int that the client injected, or a hash of some other piece of data) to pick which filter matches.  You could also get really creative with your use of the DynamicUpdate functionality to periodically switch out which endpoints the Routing Service was pointed to, but this seems a kludge.

The final solution, and the one I’m going to recommend, is to use a different extensibility point within the Routing Service.  Note that the MessageFilterTable that we use is a MessageFilterTable<IEnumerable<ServiceEndpoint>> (something I alluded to a while ago).  Why not write your own custom IEnumerable<ServiceEndpoint> and plug that into the right hand side of the MessageFilterTable?  That way your filter selection could be simple (you could even use just a MatchAll if you didn’t need any other semantics).  Here’s what that would probably look like:

   class RoundRobinEndpointList : IEnumerable<ServiceEndpoint>

    {

        ServiceEndpoint[] endpoints;

        int head;

        public RoundRobinEndpointList(params ServiceEndpoint[] endpoints)

        {

            if (endpoints == null || endpoints.Length == 0)

            {

                throw new ArgumentException("One or more ServiceEndpoints must be provided", "endpoints");

            }

            //Copy the list to ensure it doesn't change on us (and so we can lock() on our private copy)

            this.endpoints = endpoints.ToArray();

        }

        public IEnumerator<ServiceEndpoint> GetEnumerator()

        {

            int currentHead;

            lock (this.endpoints)

            {

                currentHead = this.head++;

                if (this.head == this.endpoints.Length)

                {

                    //Wrap back to the start

                    this.head = 0;

                }

            }

            //If one just wanted to return 1 endpoint as opposed to a list with

            //backup endpoints this 'yield' is all that would be needed.

            //yield return this.endpoints[currentHead];

            //return results [current] ... [last]

            for (int i = currentHead; i < this.endpoints.Length; i++)

            {

                yield return this.endpoints[i];

            }

            //return wrap-around (if any) [0] ... [current-1]

            for (int i = 0; i < currentHead; i++)

            {

                yield return this.endpoints[i];

            }

        }

        IEnumerator IEnumerable.GetEnumerator()

        {

            return this.GetEnumerator();

        }

    }

Overall, this is a much more pleasing implementation of RoundRobin, and could easily be extended to other load balancing patterns as well.  It has the distinction of being 1) quite small, 2) easy to understand (especially compared to the other implementations) 3) thread safe, and 4) performant (since we only need to lock on a copy of a list instead of the actual list that other MessageFilters might need to be using).

 Here's another Routing Service project, this time from Rochard Seroter, who put together a "multi-type" Routing Service hosted in WAS (the easy way this time).  Here's the link.

Richard also had an interesting question: "Do the contracts of all the services behind the routing service need to implement the same contract?  Otherwise, where do you do the format transformation?  In a custom behavior?  Or would you recommend that the service accept an untyped object and cast it to the known type later?"

This is a great question as it points to one of the strengths of the Routing Service, which is its typelessness.  The question now is what happens when the "types" presented at the front of the Routing Service don't map to the ones that the backend services are expecting?

To answer Richard's question:  No, they don’t have to have the same contract.  By default the Router doesn’t touch the message body: we take whatever was presented by the client or returned from the service (there's one notable exception to this related to processing Faults).  If your client is presenting messages that your remote services can’t understand (are incompatible) then yeah, you’ll need a transformation of some sort.  I’ve seen this done a few ways:
 
1)      A generic XSLT behavior, attached at either the inbound service endpoint (applied to all messages) or the outbound endpoint (transformation specific to this destination).
2)      The use of something like automapper in a behavior.  The downside of Automapper is that is presumes you're working with strongly typed objects, which you aren't by default at the Routing Service (though you could write some code that translated the XML message into your strongly typed objects and then converted it with AutoMapper.  (I'd probably just write the XSLT, but that's a less robust solution).
 
Going untyped is a third option, but is only necessary if you’re going to do the mapping/transformation/casting later. 


Let's keep those Router sample projects coming!

 -Matt

Go check it out!

• Routing on message body data (routeOnHeadersOnly=false)
• WAS Hosted Routing Service
• use of the Namespace table to introduce custom namespaces used in the messages

 Nicely done :)

 For reference: the fully qualified name of the Routing Service is:

"System.ServiceModel.Routing.RoutingService, System.ServiceModel.Routing, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" 

This would be the path you referenced either in the Service tag of your SVC file, and in some situations also when referencing the service name inside your Web.config files.

 -Matt

 Enjoy!

-Matt

Dynamic reconfig is the result of questions like the following:

• When self or web hosting how do I get updates into the Router?
• During runtime what is it even safe to change?
• What if my deployment changes (services move)?
• What happens if I need to add a new rule to my Router to capture new traffic?
• What happens if I notice some error in my configuration that I want to adjust?
• How do I get my rules into the Routing Service if I decide to store them somewhere else?

When building the Routing Service, it was initially (and still is primarily) configuration driven.  If you were self hosted, and you wanted to change your configuration, you’d have to throw together some file watcher and be able to know how not to interrupt messages and connections that were in flight before shutting down the host to pick up the new config.  This is non-trivial, especially outside of the service implementation proper where user code would live.  If you were webhosted (in IIS) you’d run into the “my config changed and now I have a (seemingly arbitrary) set amount of time before my host recycles and I lose any connections that were in play”.  Since sorting out these questions is tricky enough, we felt like we could add some value by not making our customers deal with them.  The solution we arrived at is Dynamic Reconfig. 

Let’s rewind for a second and remember how we configure the Routing Service normally through code:

            //set up some communication defaults

            string clientAddress = "http://localhost:8000/servicemodelsamples/service";

            string routerAddress = "http://localhost:8000/routingservice/service";

            Binding routerBinding = new WSHttpBinding();

            Binding clientBinding = new WSHttpBinding();

            //add the endpoint the router will use to receive messages

            serviceHost.AddServiceEndpoint(typeof(IRequestReplyRouter), routerBinding, routerAddress);

            //create the client endpoint the router will route messages to

            ContractDescription contract = ContractDescription.GetContract(typeof(IRequestReplyRouter));

            ServiceEndpoint client = new ServiceEndpoint(contract, clientBinding, new EndpointAddress(clientAddress));

            //create a new routing configuration object

            RoutingConfiguration rc = new RoutingConfiguration();

            //create the endpoint list that contains the service endpoints we want to route to

            //in this case we have only one

            List<ServiceEndpoint> endpointList = new List<ServiceEndpoint>();

            endpointList.Add(client);

            //add a MatchAll filter to the Router's filter table

            //map it to the endpoint list defined earlier

            //when a message matches this filter, it will be sent to the endpoint contained in the list

            rc.FilterTable.Add(new MatchAllMessageFilter(), endpointList);

            //attach a new RoutingBehavior with this RoutingConfiguration to the service host

            serviceHost.Description.Behaviors.Add(new RoutingBehavior(rc));

Note how most of the logic is actually encapsulated in one object, the RoutingConfiguration.  The RoutingConfiguration object describes the message filter table that the Routing Service should use, which includes the message filters and client endpoints.

Dynamic config works by allowing you to hand the Routing Service a new RoutingConfiguration during runtime, which the Routing Service then picks up and uses.  In doing so, we make sure that messages and connections that were in flight continue to use the RoutingConfiguration (and thus filters) that was in place when they started being processed.

As the diagram indicates, the API for applying a new RoutingConfiguration to the Router during runtime is (creatively) called “ApplyConfiguration”. It takes a RoutingConfiguration object, and is located on a ServiceExtention attached to the Routing Service called the RoutingExtension.  We used a ServiceExtension so that you could write your own ServiceExtension code that is able to find our extension and call this method.  Your ServiceExtension would have some code similar to that above: newing up a RoutingConfiguration, creating some MessageFilters and Endpoints, and then calling ApplyConfiguration on our extension. 

But where do you get the information for building these filters?  Well, it could be anywhere, but the key is that you probably want it to be event driven: something happens, a callback fires, and that data is used to create a new set of filters.  What is that event?  Well, it could be a file watcher watching some external configuration store, it could be a Discovery Announcement, it could be a SQL or Velocity Cache event notification, whatever.  Note that since this can happen whenever you want, it can happen immediately on service startup, meaning that you’re free to leave the Routing Service’s App or Web.config entirely out of the process and do it all in memory dynamically.

As far as the performance of applying new rules, it’s pretty snappy, since internally it’s basically a pointer move.  The far more complex step is the construction of the rules themselves.  What about the impact to the memory footprint of having all these extra sets of rules hanging around?  Well luckily MessageFilters and MessageFilterTables are pretty small, so by themselves they’re not a big problem.  Furthermore, since a RoutingConfiguration is only referred to by the sessions that are using it, once all those sessions are closed, that RoutingConfiguration gets GC’d.  In some ad-hoc tests we did where we opened connections that never died and also kept calling ApplyConfiguration() with a new RoutingConfiguration each time, we ran out of TCP connections on the box before we ran out of memory.  Unless your RoutingConfiguration’s MessageFilterTable is exceptionally huge, you’re very unlikely to see any issues as a result of this feature (besides, hopefully, an increase in the flexibility of the solutions you’re able to build).

In the last post I discussed the Error Handling capabilities of the Routing Service.  In that post I showed how error handling can be added to lots of common scenarios in order to provide a more robust experience for clients.  These scenarios are useful, but they don’t illustrate the strengths of the error handling capabilities, which is what happens when the Routing Service is dealing with more complex messaging patterns and capabilities, such as multicast, sessions, and transactions.  In this post, we’ll examine some of these patterns and how the Routing Service deals with them when handling errors. Lets say you have the Routing Service configured as a part of the following scenario:

In this example, you have some inbound queue into which “work messages” are delivered (either by clients, other services, etc). You’re using the Routing Service to make sure that a record of that message is delivered to exactly one of your Service Queues (to make sure the work gets done) and to exactly one of your Logging Queues (to make sure the work is recorded).   The routing configuration would look something like this (presuming all the queue endpoints had already been defined):

  <routing>

      <filters>

        <!-- Create a match all filter -->

        <filter name="MatchAllMessageFilter1" filterType="MatchAll" />

      </filters>

      <filterTables>

        <filterTable name="multicastTable">

          <!-- Add two entries to the filter table so that when a message comes in it is multicasted to both lists of queues -->

          <add filterName="MatchAllMessageFilter1" endpointName="primaryServiceQueue" backupList="backupServiceEndpoints"/>

          <add filterName="MatchAllMessageFilter1" endpointName="primaryLoggingQueue" backupList="backupLoggingEndpoints"/>

        </filterTable>

      </filterTables>

      <!-- Define the backup service and logging endpoint lists -->

      <backupLists>

        <backupList name="backupServiceEndpoints">

          <add endpointName="backupServiceQueue" />

        </backupList>

        <backupList name="backupLoggingEndpoints">

          <add endpointName="backupLoggingQueue" />

        </backupList>

      </backupLists>

    </routing>

What you don’t want is for work to be done that isn’t recorded somewhere (that’s how customers don’t get billed) or to record that work was done that actually wasn’t (that’s how customers get billed incorrectly and end up upset).  Thus, this configuration, with the Routing Service should successfully move the message out of the inbound queue only if the action of inserting the message into exactly one of the Service Queues AND exactly one of the logging queues.  In order to do this, the Routing Service utilizes both the Receive Context features of .NET 4 and transactions.

Receive Context is used to obtain a temporary lock (peek-lock) on the message inside the inbound queue so that other readers from the queue (potentially other Routing Services) don’t attempt to operate on the message while this Routing Service is.  Next the Routing Service reads the message, and feeds it into the Message Filter Table defined here, which matches the message to the two filter entries (the primaryServiceQueue/backup list and corresponding logging endpoints).  Because there were multiple matches, the message is multicast to both destinations (in this case the primaryServiceQueue and the primaryLoggingQueue).  Because there are messaging guarantees that the Routing Service is trying to meet, it first creates a transaction.  The Routing Service will attempt to do the necessary work inside this transaction, and then will only commit the transaction if everything succeeds.

When there’s nothing wrong, the result of placing a message in the inbound queue looks like this:

If we turn on tracing and get a look at the events that are coming out of the Routing Service we get a good idea of what’s going on:

So that’s the happy path, but what happens when the various queues are unavailable/unreachable or the Routing Service encounters errors when trying to insert the message into the destination queues (say they don’t exist anymore for some reason or that the destination host is unreachable)?  Clearly if the primary service or logging queue is down, the Routing Service will use the backups instead.But what’s going on inside the Routing Service when this happens?

We see that when the initial send to the primary Service Endpoint fails, the Routing Service automatically knows that the current transaction is no longer any good, and thus it spins up a new transaction for sending the message to the backup service queue.  When these sends succeed, the second transaction is committed. (Not pictured is the prior transaction being discarded).   This pattern of discarding the current transaction, moving the message to a backup endpoint, and then creating a new transaction and trying again will be repeated until the message is successfully sent to all of the destinations indicated and the transaction is able to commit.

What about when the Routing Service is unable to deliver the message with the transactional guarantees in the way that it was configured to do so (say in a case where we run out of backup endpoints to send the message to)?  In this case, the ultimate behavior is configurable as a setting with MSMQ, but a common action is to move the message out of the inbound queue and into a DLQ.  In order to do this, the Routing Service, after trying to create and use a transaction to send the message to some functional combination of the specified destinations, calls Abandon on the Receive Context, indicating to the inbound queue that it was unable to process the message correctly.

And here’s the trace:

This kind of advanced processing works even when there are sessions (or sets) of messages that are being processed.  In this case, as the session of messages passes through the Routing Service, the Routing Service temporarily caches each message so that it can be replayed to the last destination it was successfully routed to.  Only when all messages are successfully sent to a endpoint and successfully received can the transaction be committed and the messages marked as successfully sent.

Whew! So what does this all mean?  This means that we’ve baked a lot of complex state tracking into the Routing Service, so that even when your message patterns, filter rules, and transactional/sessionful requirements change, we’ve got you covered on the robustness front.  The Routing Service is going to take a lot of pains to deliver messages and handle this logic so that you don’t have to write it into your clients.

First: what do we mean when we say errors?  In this case, we're talking primarily about CommunicationExceptions and TimeoutExceptions, two classes of Exception that (can) occur commonly in a distributed system.  Communication Exceptions are things like EndpointNotFoundExceptions that generally indicate that the service that you were trying to talk to was down, didn't provide a correct response, or didn’t understand the message you sent it.  TimeoutExceptions are just what they sound like: you've been stuck waiting for a response for too long.  Either of these could happen for a variety of reasons: temporary network outage, messages that get lost on the network, glitches or bugs in the configuration of the services involved, implementation issues of the remote service, etc.

Since the Routing Service is acting on behalf of the client, we will sometimes see these errors.  Now, we could just forward these exceptions back to the client and let the client application deal with them.  However, since we're already participating in the message flow, there's an opportunity for us to do something else, which can help your application be more robust and tolerant of these common (and frequently temporary) errors.  We allow you to configure a set of alternate endpoints that the Routing Service should use as backups when a particular endpoint returns an exception.  In these cases the Routing Service will "fail-over" to the backup endpoint and try to deliver your message again.

Aside from a slightly slower response (after all, the Routing Service does need to wait for the exception to occur before taking action), the client is none the wiser. 

As we’ve seen before, when defining an entry in the MessageFilterTable, I normally map a MessageFilter to a particular destination endpoint like this:

<add filterName="someFilter1" endpointName="someDestinationEndpoint1" />

When defining backup endpoints, this changes slightly, as we have to both define the backup list, and then reference it in the filter table entry. First, to create the backup list, we define a new section inside of the <routing> section called, appropriately “backupLists”.  This is where we create a particular list and add endpoints to it. 

<backupLists>

     <backupList name="backupList1">

          <add endpointName="backup">

          <add endpointName="backup2">

     </backupList>

</backupLists>

To actually use the list of alternates that we built, add a reference to the list to the filter table entry, like so:

<add filterName="someFilter1" endpointName="someDestinationEndpoint1" backupList="backupList1"/>

In code, setting up backups would look like this:

//assume the endpoint objects and the filter objects already exist

      List<ServiceEndpoint> endpointList = new List<ServiceEndpoint>(){someDestinationEndpoint1, backup, backup2};

RoutingConfiguration rc = new RoutingConfiguration();

      rc.FilterTable.Add(someFilter1, endpointList);

The order that the endpoints are added to the list are the order in which they will be attempted.  With this configuration, first the endpoint referenced by the endpointName would be tried.  Should the connection to that endpoint fail or time out, the first endpoint in the backup list (“backup”) would be tried.  Only if communication to that destination failed would the message be sent to backup2.  Should backup2 fail, the client would get the same exception/error they normally would have had there been no alternates specified.  In this way we add reliability without changing the underlying model expected by the client. 

Error handling can be added to pretty much any messaging pattern or protocol: one-way, request response, duplex, multicast, sessionful, etc.  The logic present knows how to handle everything from single-message request-response messages (basicHttp) to sessionful, transactional messages (MSMQ).  These more advanced error handling semantics will be the topic of the next post, since talking through their state can be somewhat tricky.

Enjoy!

-Matt