Multicasting Messages with AppFabric (Service Bus & Access Control) – Publish / Subscribe Sample Application – Part 1 of X

This sample demonstrates using the NetEventRelayBinding binding on the Windows Azure platform AppFabric Service Bus. This binding allows multiple applications to listen to events sent to an endpoint; events sent to that endpoint are received by all applications. This blog is just my personal notebook as I learn the platform. The source code will be presented as the project unfolds.

This blog entry has been a fun one since we had the biggest storm of the season. The app presented is all about weather and the Service Bus. 

As you read this, don’t forget you can click on an image to see it more clearly.

Pub / Sub – What is it ( I like the Wiki definition )

• Publish/subscribe (or pub/sub) is an asynchronous messaging paradigm where senders (publishers) of messages are not programmed to send their messages to specific receivers (subscribers).
• Rather, published messages are characterized into classes, without knowledge of what (if any) subscribers there may be.
• Subscribers express interest in one or more classes, and only receive messages that are of interest, without knowledge of what (if any) publishers there are.
• This decoupling of publishers and subscribers can allow for greater scalability and a more dynamic network topology.

Why we like Pub / Sub

• You can't help but hear the terms "loose coupling" or "tight coupling" in almost any discussion on software design.
• Coupling among classes or subsystems is a measure of how interconnected those classes or subsystems are.
• Tight coupling means that related classes have to know internal details of each other, changes ripple through the system, and the system is potentially harder to understand.

We all like decoupling and all the good things that derive from it:

• Why Decoupling can be Good
  • Reduces compile times
  • Increases reusability of components
  • Makes for easier unit testing
  • Reduces bugs from copied code
  • Makes systems composable from parts

MulticastClient / Multicast Host

• MulticastClient is our "publisher"  or “client” or “producer” or “sender”
• MulticastHost is our "subscriber"  or “servicehost” or “listener” or “consumer”
• The client communicates with the service by executing methods residing in that service
  • -   The client passes parameters to those methods when calling a web service that is listening
    -   Both the client and the host share a common interface (interface IMulticastContract)
    -   Behind the scenes the Access Control part of the AppFabric is managing security

 

Understanding our architecture

• Weather reports are being send to the cloud endpoint by MulticastClient
• Weather reports are being read from the cloud endpoint by MulticastHost

"Weather Station Client"

• is the publisher
• sends weather reports

"Weather Station Billboard"

• is the subsciber
• reads weather reports

You can have any number of "Weather Station Client" and "WeatherStation Billboard." Some limitations are addressed later in this post.

 

Notice our solution in Visual Studio 2010

There are 3 projects in 1 solution

• The first project is "DataContracts." It contains our shared data structures and contracts.
• The second project is MulticastClient. It is a WPF application that sends weather reports.
• The third project is MulticastHost. It is a WPF application that reads weather reports.

 

The extended blog post is all about learning a new AppFabric  Binding - netEventRelayBinding

• NetEventRelayBinding represents a binding that supports one-way multicast eventing and allows N event publishers and M event consumers to rendezvous at the same endpoint.

The Bottom Line – Why you would read this blog entry

NetEventRelayBinding supports multiple listeners on the same URI. To set up an event rendezvous point (also called topic), the clients and listeners merely need to agree on a shared URI within the scope of a solution and send to or listen on it. Messages that get sent by any of the clients get distributed to all listeners

This blog includes:

• Why we are doing this?
• Getting started
  • Setup on Web Portal
    • ServiceName, IssuerName, IssuerSecret
  • Install c:\Program Files\Windows Azure platform AppFabric SDK
  • Visual Studio 2010
  • Source Code
• Big picture
• Sample code run
• The Code

Why Are We Doing This?

This sample demonstrates using the NetEventRelayBinding binding on the Windows Azure platform AppFabric Service Bus. This binding allows multiple applications to listen to events sent to an endpoint; events sent to that endpoint are received by all applications.

Who wouldn’t want the ability to send messages to multiple endpoints?

• -auto selects the AutoDetect mode. In this mode, the Service Bus client automatically switches between TCP and HTTP connectivity.
• -tcp selects the Tcp mode, in which all communication to the Service Bus is performed using outbound TCP connections.
• -http tells the application to use the Http mode, in which all communication to Service Bus is performed using outbound HTTP connections.

N-to-N Is Key to Many Architectures

The ability to easily implement N-to-N multicast capabilities has been wanted. The ability for two or more computers behind a firewall to broadcast messages to other computers behind other firewalls has been difficult at best. As explained in previous blog entries, the reason is that there is a lot of plumbing that is need to make it work, from the relay service in the cloud to the authenticating service, to supporting both tcp and http - the list goes on.

Workflow Patterns

• Workflow models are often built using multicast technologies. Workflow often needs some form of "eventing," because there is always the situation when one computer needs to announce changes or "events" to other computers that are involved in the workflow.
• There has also been a lot of talk about event-driven architecture as a technique to build more scalable and maintainable systems.
• Things have come a long way in the Microsoft world. In the old days there was NET Remoting, MSMQ and HTTP - nothing like today's advanced tooling.

How the problem has been solved in the past

In the past, we had to write custom code:

• Hosting the queue listeners
• Encoding and decoding messages
• Dealing with reliability and managing subscriptions

Microsoft has given us MSMQ and BizTalk to facilitate such patterns and architectures. Microsoft has also given us WCF, which provides support through peer-to-peer messaging techniques. The industry has given us REST, which allows communication to take place over an http protocol.

The Move To Internet Scope

In order for N-to-N multicasting to work, you need some core infrastructure:

• Identity and access control
• Naming  for your services
• Service registries

Federated Security – A difficult challenge for the Pub / Sub Model

• But perhaps one of the biggest problem to solve is Federated Security.
• Federated Security is all about a person's user information (or principal), stored across multiple distinct identity management systems.
• Access to resources on the network is gained through the use of the common token, usually the user name with some extra credential information. 
• The most difficult security challenge is implementing authentication process across multiple IT systems or even organizations.

As of January 2010, the following facts are true

• Limitations on concurrent listeners

  • For the current release, the number of listeners is constrained to fit departmental scenarios with fewer than 20 concurrent listeners. Messages sent through this binding are constrained to at most 60KB in wire size.

• No Assurances for Message Arrival

  • There are no assurances about messages arriving in sequence or being delivered at all.

• Default settings for NetEventRelayBinding

  • With its default settings, the NetEventRelayBinding supports SOAP 1.2 messaging over TCP using .NET Framing and .NET Binary Serialization.

• The “sb” URI Scheme

  • Services and Clients using the NetTcpRelayBinding register their endpoints on the Service Bus using the "sb" URI scheme, regardless of the active connectivity mode.

Getting Started

In order to work with the code and a live example, there are a few things you need to do:

• Sign up and get the info you need (ServiceName, IssuerName, IssuerSecret)
  • portal.ex.azure.microsoft.com
• Install the “Windows Azure platform AppFabric SDK”
  • Currently here: https://www.microsoft.com/downloads/details.aspx?FamilyID=39856a03-1490-4283-908f-c8bf0bfad8a5&displaylang=en
  • Else, look it up.
• I used Visual Studio 2010 Beta 2. It will be easier if you download it. Right now it is free.

The workflow is pretty simple

• Step 1: Run Weather Station Billboard and click "Connect to Service." You need to run the listener first. This will use the issuer secret code to make sure nobody unauthorized gets through.
• Step 2: Run Weather Station Client and click "Connect to Service." Client also uses the issuer secret code to connect.
• Step 3: In Weather Station Client  type in a Zip Code and click "Send Weather Report." This calls a method at Weather Station Billboard passing some .NET objects with weather information. These same objects will be de-serialized automatically by the .NET runtime once they get to Weather Station Billboard, which will popup a new XAML windows to display the weather data that was sent by the client.
• Step 4: Watch Weather Station Billboard display the weather send by Weather Station Client. This happens all magically with security through the Service Bus. No password – no weather.
  • Note: It is important to note that Weather Station Billboard will display a new window for each message it receives.

 

Sample screenshots

• Here is a list of the screens in both the publisher and the subscriber
  • Weather Station Client
  • Weather Station Billboard
  • Output for Weather Station Billboard

Figure: Weather Station Client

The figure above is the MulticastClient application that is already connected.

Figure: Disconnected Icon

 

Figure: Weather Station Billboard

The figure above shows MulticastHost, which is the listener. It is also the consumer and receiver of events. We can debate my loose use of terminology but nuances can be learned about later.

Figure: Weather Station Billboard Weather Display Window

The MulticastHost also has this window above. You can see that I’m about to get hit by rain !

Figure: Visual Studio 2010 with all the code loaded up

 

Defining the data structures

•     All in packaged in a separate project along with the needed interfaces which act as contracts
  • • Note the DataContracts.cs file. That is where all the data structures live. It is also where the interfaces and contracts reside.
    • These data structures are in a separate project because they are shared by both:
      • Weather Station Client
      • Weather Station Host
    • • Note
        • These are the main data objects (business objects) for our application. They inherit from IExtensibleDataObject so we can serialize easiliy.
        • WeatherData – holds main weather data for display in Weather Station Billboard
        • MyLatLon – holds latitude and longitude for a given zip code
        • WeatherInfo – holds WeatherData and acts as a container

Defining the data structures-continued

•     Better separation is still possible
  • You can make this code better. Please do and email me (Bruno at bterkaly@microsoft.com)
  • I changed my design a few times and haven’t fully refactored
•     Data comes from the National Weather Service in XML format that we need to parse. We need to learn how to do this step.