Article
08/13/2015

October 2009

Volume 24 Number 10

Service Station - Building RESTful Clients

By Jon Flanders | October 2009

In this installment of the RESTful Service Station, I'll discuss building clients against RESTful services. Building clients is perceived as difficult (mostly because of the lack of automatic client generation from metadata, à la SOAP and WSDL), but in reality it's like any other type of code that you write: At first there is some ramp-up time as you get used to a particular programming paradigm. Then, once you have the hang of that paradigm, writing code against it becomes easier and easier. I've found this to be true for writing clients against RESTful services, as have many other developers I've interacted with.

Client Basics

I'm going to show the basics of interaction between a client and a service using REST. Just a quick recap about REST before we dig in. REST is an architectural style that builds on the same principles that provide the foundation of the World Wide Web. Clients interact with services by making HTTP requests, and services respond with HTTP responses, which often include a representation of a resource that client code can use to drive an application.

Imagine I have a service that exposes information and functionality related to Hyper-V (the virtualization technology built into Windows Server 2008). If the "entry" URI of this service was https://localhost/HyperVServices/VMs.svc/, as a client I'd make an HTTP GET request to retrieve a representation of the resource identified by this URI. In this case, the resource is formatted as XML and represents a list of all the virtual machines (VM)s installed on a particular machine.

Figure 1 A Simple HTTP GET Request

As I showed in the first of my Service Station articles about REST, in the January 2009 issue of MSDN Magazine (msdn.microsoft.com/magazine/2009.01.servicestation.aspx), one of the small benefits of using REST is that you can use HTTP request tools to make your initial test requests against a service. (Using these tools for debugging problems is also very useful.) In this case, I'm using a free tool called Fiddler (fiddlertool.com) to make an HTTP GET request for the resource that represents all my virtual machines. See Figure 1. Of course, tools like Fiddler are valuable, but to build an application you need to write code against the service. I'm going to start with the basics of making an HTTP request and then get into the issue of actually reading the resource.

The .NET Framework has always offered a basic HTTP API that can be used to interact with RESTful services. At the center of this API are the HttpWebRequest and HttpWebResponse types. To make an HTTP request, you create and configure an HttpWebRequest instance and then ask for the HttpWebResponse. Figure 2 shows an example. Making a request to a RESTful service is a simple set of steps:

Make sure you are using the correct URI.
Make sure you are using the correct method (HTTP verb).
Process the response if the status code is 200 OK.
Deal with other status codes as appropriate (more on this later).

Because steps 1 and 2 are pretty easy, generally speaking step 3 (and sometimes step 4) end up being the more difficult of the steps.

The majority of the work of step 3 is processing the resource representation in some reasonable way. Since XML is still the most typical resource response, I'm going to cover that in this article. (The other most likely media type would be JSON.) When the resource is XML, you need to parse that resource and write the code that can extract the data you need from that resource.Using .NET, you have a few options for parsing XML. There is the tried and true XmlReader class. The XmlDocument class is also a possibility, and by using either of those types you can manually parse or use XPath to navigate your way around the XML. With the advent of .NET 3.0, you also get the XDocument class, which, when combined with LINQ to XML, has become the de facto choice for processing XML. Figure 3 shows an example of code processing the list of VMs using XDocument.

Figure 2 Simple HttpWebRequest Code

string uri = "https://localhost/HyperVServices/VMs.svc/";
var webRequest = (HttpWebRequest)WebRequest.Create(uri);
//this is the default method/verb, but it's here for clarity
webRequest.Method = "GET";
var webResponse = (HttpWebResponse)webRequest.GetResponse();
Console.WriteLine("Response returned with status code of 0}",
webResponse.StatusCode);
if (webResponse.StatusCode == HttpStatusCode.OK)
ProcessOKResponse(webResponse);
else
ProcessNotOKResponse(webResponse);

LINQ to XML, along with anonymous types, provides a nice, easy way to deal with turning XML into objects that an application can process, and, of course, you can also use a predefined type instead of an anonymous type.

Another approach that is popular when programming against SOAP-based and REST-based services is to cause the responses to be automatically deserialized into .NET types. In the case of SOAP, this generally happens in the WSDL-generated proxy. With Windows Communication Foundation (WCF) and REST, this can be accomplished in a couple of ways. One way (which I don't really recommend but am mentioning for completeness) is to use the symmetrical nature of WCF to use a WCF service contract definition on the client. In fact, the WCF support for REST includes a type named WebChannelFactory that can be used to create a client channel against a service contract definition. I don't recommend this type of client programming for two reasons. First, creating the client becomes a very manual and error-prone operation. Second, using a strongly typed service contract creates a tight coupling between your client and the service. Avoiding tight coupling is one of the main reasons that the Web has succeeded, and we want to continue that trend when using the Web programmatically.

Another way to use XML serialization is to use the HttpWebResponse.GetResponseStream method and deserialize the XML into the object manually. You can do this by using either the XmlSerializer or the WCF DataContract serializer. In most cases, the XmlSerializer is the preferable approach because it deals with more variations in XML documents (for example, attributes and nonqualified elements) than the DataContract serializer does.

Figure 3 Processing a RESTful Response Using XDocument

var stream = webResponse.GetResponseStream();
var xr = XmlReader.Create(stream);
var xdoc = XDocument.Load(xr);
var vms = from v in xdoc.Root.Elements("VM")
select new { Name = v.Element("Name").Value};
foreach (var item in vms)
{
Console.WriteLine(item.Name);
}

The problem still seems to circle back to the fact that RESTful services generally don't expose metadata that can be used to autogenerate some or all of this code. Although many in the RESTful camp don't see this as a problem (again, anything that autogenerates against service definitions can be seen as the evil agent of tight coupling), there certainly are times when not having these tools is a hindrance to the adoption of REST.

One interesting approach taken by the WCF REST Starter Kit (https://aspnet.codeplex.com/wikipage?title=WCF%20REST&ProjectName=aspnet), which is an enhancement from Microsoft of the REST programming model in .NET 3.5, is to provide a partial autogeneration feature. If you install the starter kit, you will have a new menu item in Visual Studio 2008 under the Edit menu, as you can see in Figure 4.

Figure 4 Paste XML as Types Menu Item

The usage model of this command is pretty simple. You copy the XML resource representation onto the Clipboard (either from some human-readable documentation the service exposes or by making a request with a tool like Fiddler). Once you do this, a set of XmlSerializable types is created, which you can then use to turn the Stream from the HttpWebResponse into an object. See Figure 5 (the body of the generated types isn't shown for brevity's sake).

Figure 5 Code Generated from Using Paste XML as Types

var stream = webResponse.GetResponseStream();
//in a real app you'd want to cache this object
var xs = new XmlSerializer(typeof(VMs));
var vms = xs.Deserialize(stream) as VMs;
foreach (VMsVM vm in vms.VM)
{
Console.WriteLine(vm.Name);
}
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType = true)]
[System.Xml.Serialization.XmlRootAttribute(Namespace = "",
IsNullable = false)]
public partial class VMs
{
private VMsVM[] vmField;
/// <remarks/>
[System.Xml.Serialization.XmlElementAttribute("VM")]
public VMsVM[] VM
{
get
{
return this.vmField;
}
set
{
this.vmField = value;
}
}
}
/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Xml", "2.0.50727.4918")]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.Xml.Serialization.XmlTypeAttribute(AnonymousType = true)]
public partial class VMsVM
{
//omitted
}

The REST Starter Kit not only simplifies the use of the XmlSerializer, but it also provides a very nice API on top of the HttpWebRequest/WebResponse API. The following code shows the simple GET request from Figure 1 rewritten using the REST Starter Kit's HttpClient API:

string uri = "https://localhost/HyperVServices/VMs.svc/";
var client = new HttpClient();
var message = client.Get(uri);
ProcessOKResponse(message.Content.ReadAsStream());

The HttpClient class greatly simplifies (and makes explicit what verbs you are using) the use of the .NET HTTP API. Also, as you can see in the following code, it simplifies the use of the Paste XML As Types generated feature:

var vms = XmlSerializerContent.ReadAsXmlSerializable<VMs>(
message.Content);

Although the REST Starter Kit isn't officially supported by Microsoft yet, it does illustrate one very important point: that REST client programming can be simplified and partially automated without a RESTful service having complete metadata à la a WSDL file.

Using HTTP

One way that a client and a service can take advantage of HTTP is to properly use status codes and headers. More information about status codes can be found at w3.org/Protocols/rfc2616/rfc2616-sec10.html.

As I've gone around the world speaking about REST, one of the advantages of RESTful services that I often point out is that GET requests can be cached. Make no mistake, the scalability and the success of the Web is largely because of HTTP caching.

One way to take advantage of HTTP caching is to use conditional GET. Conditional GET enables the client ("user agent" in HTTP lingo) to make a GET request for a resource that the user agent already has a copy of. If the resource hasn't changed, the server tells the user agent that the resource is exactly the same as the version already held by the user agent. The efficiency benefit of conditional GET is a reduction in bandwidth usage of the network between the server and the user agent, freeing up the bandwidth to be used by requests for newly created or modified resources. In addition, it saves the additional processing time required to serialize the resource, although not the processing time to generate or retrieve the resource (since you need a copy of the current resource to compare it with the information sent by the user agent with the conditional GET).

Figure 6 Server-Side Implementation of Conditional GET

[OperationContract]
[WebGet(UriTemplate = "/{name}")]
public VMData GetOne(string name)
{
VMManager.Connect();
var v = VMManager.GetVirtualMachine(name);
var newVM = FromVM(v);
string etag = GenerateETag(newVM);
if (CheckETag(etag))
return null;
if (newVM == null)
{
OutgoingWebResponseContext ctx =
WebOperationContext.Current.OutgoingResponse;
ctx.SetStatusAsNotFound();
ctx.SuppressEntityBody = true;
}
SetETag(etag);
return newVM;
}
private bool CheckETag(string currentETag)
{
IncomingWebRequestContext ctx =
WebOperationContext.Current.IncomingRequest;
string incomingEtag =
ctx.Headers[HttpRequestHeader.IfNoneMatch];
if (incomingEtag != null)
{
if (currentETag == incomingEtag)
{
SetNotModified();
return true;
}
}
return false;
}
string GenerateETag(VMData vm)
{
byte[] bytes = Encoding.UTF8.GetBytes(vm.ID +
vm.LastChanged.ToString());
byte[] hash = MD5.Create().ComputeHash(bytes);
string etag = Convert.ToBase64String(hash);
return string.Format("\"{0}\"", etag);
}
void SetETag(string etag)
{
OutgoingWebResponseContext ctx =
WebOperationContext.Current.OutgoingResponse;
ctx.ETag = etag;
}

Like most of the other "advanced" HTTP concepts, WCF doesn't support conditional GET automatically because the implementation of conditional GET is highly variable among service implementations. However, as it does for other "advanced" HTTP concepts, WCF provides the tools to implement conditional GET. There are two approaches to accomplish this: using the time the resource was last modified, or using a unique identifier known as an ETag.

ETags have become the more popular way to implement condition GET. In Figure 6, you can see the code to implement an ETag-based conditional GET on the VM service. Please note that this is the server-side implementation; I'll get to the client in a moment.

The basic flow is for the server to look for the ETag in the If-None-Match HTTP header sent by the client, and to try to match it against the current ETag generated for the resource. In this case, I'm using the unique ID of each VM plus the last-modified time stamp (converted to bytes and then made into an MD5 hash, which is a fairly common implementation). If the two values match, the server sends back a 304 not modified HTTP header with an empty body, saving serialization time as well as bandwidth. The client gets a much quicker response and knows it can use the same resource it already has.

Figure 7 A Simple WPF VM Client

Imagine a client application like the one shown in Figure 7.This application shows the name of each VM, plus the image of the current status of the VM. Now imagine that you want to update this application to match the current state of each VM. To accomplish this, you would have to write some code in a timer and update each record if it was different from your local record. And it's likely that you'd update everything upon every iteration just to simplify the application, but this would be a waste of resources.

If you instead used conditional GET on the client, you could poll the service for changes by sending a conditional GET request, which would use an If-None-Match HTTP header to indicate the ETag of the resource. For the collection of VMs, the service can use the most recently changed VM to generate the ETag, and the client will update only if one or more of the VMs have changed their state. (If you had lots of VMs, you might want to do this for each VM. But since we are databinding to the collection in this application, I'm OK with updating the whole collection).

Now, implementing this logic isn't terribly difficult, but it is one of the features that the REST Starter Kit implements for you. Included in the code samples is a file named PollingAgent.cs, which has an automatic conditional GET client that polls a RESTful endpoint on an interval that you define. When the PollingAgent determines that the resource has changed (because the service is no longer returning a 302), it fires a callback.

So in my simple WPF application, I just take the result of that callback (which is the HttpResponseMessage object) and rebind my controls to the new data. Figure 8 shows the code to implement this application using the PollingAgent.

Following Hypertext

Before leaving the subject of clients, I want to touch on another of the important constraints of REST: using hypermedia as the engine of application state (known as HATEOAS).

I find that HATEOAS is easier to understand in the context of the human Web. When using the human Web, sometimes we have bookmarks to sites we know we want to visit. Within those sites, however, we generally follow hyperlinks based on our needs of the day. I might have a bookmark to www.amazon.com, but when I want to buy something, I follow the links on each page (resource) to add an item to my cart. And then I follow the hyperlinks (and at times forms) on each subsequent page to process my order. At each stage of order processing, the links in the page represent the current state of the application (in other words, what can I, as the user agent, do?).

One important consideration when building RESTful clients (although part of this is incumbent on RESTful service implementers as well) is to keep "bookmarks" to a minimum. What this means practically is that clients should have as little knowledge as possible about the URI structure of your representations, but as much as possible they should know how to "navigate" the "hyperlinks" inside those resources. I put "hyperlinks" in quotes because any piece of data in your resource can be a relative URI for building the next "link" for the client.

Figure 8 Conditional GET Using the REST Starter Kit Polling Agent

public Window1()
{
InitializeComponent();
string uri = "https://localhost/HyperVServices/VMs.svc/";
var client = new HttpClient();
var message = client.Get(uri);
var vms = XmlSerializerContent.ReadAsXmlSerializable<VMs>(
message.Content);
_vmList.DataContext = vms.VM;
var pa = new PollingAgent();
pa.HttpClient = client;
pa.ResourceChanged += new EventHandler<ConditionalGetEventArgs>(
pa_ResourceChanged);
pa.PollingInterval = new TimeSpan(0, 0, 5);
pa.StartPolling(new Uri(uri), message.Headers.ETag, null);
}
void pa_ResourceChanged(object sender, ConditionalGetEventArgs e)
{
var vms = XmlSerializerContent.ReadAsXmlSerializable<VMs>(
e.Response.Content);
_vmList.DataContext = vms.VM;
}

In my example, each VM's name is really a link, because a client can ask for a particular VM's data by asking for the name as part of the URI. So https://localhost/HyperVServices/VMs.svc/MyDesktop (where MyDesktop is the value of the Name element in the VM's element inside the collection) is the URI for the MyDesktop VM resource.

Imagine that this RESTful service allows for provisioning and starting VMs. It would make sense to embed inside of each VM resource hyperlinks to the various statuses that a VM could be put in at a particular time, rather than letting the client start a VM that hasn't been provisioned properly yet.

Using HATEOAS helps to verify the current state of the application, and it also encourages more loosely coupled clients (since the clients don't have to know as much about the URI structure of the service).

Easier Than You Think

There is no doubt that building RESTful clients is harder to start doing than building SOAP-based clients supported by WSDL metadata. As I wrote in my previous column, "REST is simple, but simple doesn't necessarily mean easy. SOAP is easy (because of WSDL), but easy doesn't always mean simple."With good coding practices and tools like the REST Starter Kit, building RESTful clients can be easier than you think. And in the end I think you'll find that the advantages you get from using the architectural style will more than make up for any temporary delay in building clients.

Jon Flanders is an independent consultant, speaker and trainer for Pluralsight. He specializes in BizTalk Server, Windows Workflow Foundation and Windows Communication Foundation.

Service Station - Building RESTful Clients

Client Basics

Using HTTP

Following Hypertext

Easier Than You Think

Additional resources