Service Station

Improving Web Service Interoperability

Aaron Skonnard

Contents

Interoperability Requires Teamwork
Web Services Profiles
Messaging
Service Description
Service Publication, Discovery, and Security
Profile Conformance
Conformance Testing Tools
Sample Applications
Where Are We?

If interoperability is the main promise of Web services, why is it that so many developers and organizations have a difficult time achieving it in practice? With all due respect to our hard-working standards bodies, the primary culprits are the imperfect specifications guiding today's implementations. Ambiguities and too many choices often lead to differing interpretations, resulting in incompatible implementations.

Hence, responsibility lies with the developer for early identification of problem areas that are likely to produce interoperability issues. Once identified, developers should avoid them religiously in their designs. Most developers know that achieving their scalability, reliability, or performance goals depends largely on their work during the design phase. The same holds true for achieving interoperability goals. This month I'll discuss some of these problem areas and how to pursue a design focused on interoperability.

Interoperability Requires Teamwork

Identifying problem areas is a team effort that takes time. It requires the input of numerous developers representing the different vendors involved. It requires thorough interoperability testing. This is where all of the vendors come together in a single room and run a suite of interoperability tests across their implementations. The group of developers document the encountered problems and discuss resolutions. Once they've worked through the interoperability issues at hand, they publish their guidance in what's known as a "profile." Because this type of work requires a significant amount of time, coordination, and political cooperation, a special organization, called the Web Services Interoperability Organization (WS-I), was formed in February 2002. All organizations interested in promoting interoperability are encouraged to join the effort.

The WS-I complements other standards bodies such as the W3C and the IETF. Instead of producing new specifications, the WS-I provides guidance around existing specifications. The WS-I organizes Working Groups (WG) that focus on specific Web services specifications and deliverables. Each WG is chartered to produce a specific deliverable that will assist developers in their quest for interoperability. The types of deliverables produced by the WS-I include profiles, usage scenarios and use cases, sample applications, and testing tools (see Working Group Deliverables for more information on this topic).

Profiles provide the design guidance needed to avoid the most obvious interoperability problems, while the usage scenarios demonstrate how to apply the guidance to your business scenarios. Samples demonstrate how to implement a fully WS-I-compliant application and the testing tools allow you to verify conformance in your own applications. A suite of samples and testing tools will be provided for each WS-I profile once they solidify. When used properly, these collective resources offer helpful insight into your Web services designs.

Web Services Profiles

In many ways, the WS-I is tasked with constraining agreement around different groups of Web services specifications. The WS-I completed their first profile, called Basic Profile (BP) version 1.0, in April 2004. The BP states that conforming Web services use SOAP, Web Services Description Language (WSDL), and Universal Description, Discovery, and Integration (UDDI), and it provides valuable guidance on using them properly.

In conjunction with the Basic Profile guidance, the WS-I has also released the WS-I Usage Scenarios and a complete sample application based on a supply-chain management scenario (complete with use cases, design, and implementation files). In addition, the WS-I has provided a suite of testing tools for verifying BP conformance in your own applications. The work on the Basic Profile has already helped to improve interoperability across the industry's first-generation implementations.

The WS-I is also actively working on additional profiles that focus on higher layers of functionality and different sets of specifications. The WS-I has published a Working Group Draft of the Basic Security Profile 1.0, which focuses on guidance around Web services security. The Basic Security Profile includes coverage of WS-Security, WS-Security UsernameToken Profile, WS-Security X.509 Certificate Token Profile, XML Signature, XML Encryption, and other security-related topics. The WS-I also recently released an Approval Draft for the Attachments Profile 1.0, which provides guidance around using SOAP Messages with Attachments, XML 1.0 + Namespaces, and MIME for attaching non-XML resources to SOAP messages.

These additional profiles (and future ones) will likely take some time to complete; however, it's never too early to begin learning about the interoperability issues addressed by each profile. And once they solidify, the WS-I will provide samples and tools like they did for the BP. Serious Web services architects should frequent the WS-I Web site to internalize the profile guidance as it becomes available.

The guidance provided by the BP is organized into four categories: messaging, service description, service publication and discovery, and security. However, the bulk of the guidance is focused on how to properly design WSDL definitions for service description. Since most toolkits map between WSDL definitions and methods, incompatibilities at this level can (and usually do) cause significant problems. Let's briefly take a look at some of the design guidance provided in each of these areas.

Messaging

The guidance on SOAP messaging clarifies some of the ambiguities that exist in the SOAP and HTTP-related specifications. For example, the guidance states that receivers must be able to process SOAP messages that contain the Unicode byte order mark (BOM), and that all messages must use either the UTF-8 or UTF-16 character encodings. It states that messages should not contain processing instructions or DTDs. And it clarifies how to properly structure the SOAP fault element and use the soap:mustUnderstand and soap:encodingStyle attributes in SOAP messages.

The messaging guidance also clarifies how to use HTTP with SOAP. For example, the guidance states that senders should prefer to use HTTP 1.1, but must use at least HTTP 1.0. It also states that messages must use HTTP POST in all cases and that the SOAPAction header value must always be quoted. And the guidance clearly defines which HTTP status codes to use in different situations (such as always use 500 Internal Server Error for a SOAP fault, and so on).

Most of this guidance is intended for implementing Web services frameworks and isn't something that you typically need to be concerned with in your designs and implementations. This section only provides a few pieces of behavioral guidance that are directly applicable to all Web services developers.

The first one has to do with properly processing mandatory header blocks. According to the profile "this requirement guarantees that no undesirable side effects will occur as a result of noticing a mandatory header block after processing other parts of the message. The Profile requires that receivers generate a Fault when they encounter header blocks that they do not understand targeted at them. When a Fault is generated, no further processing should be performed."

You actually have to write some code in ASMX 1.x in order to adhere to this behavioral requirement because the ASMX framework doesn't check to see if unknown headers were understood until after the method finishes processing. The ASP.NET team wanted to make it possible for your method to walk through the collection of unknown headers and make a decision about "understanding it" at run time.

However, the unfortunate reality of this situation means side effects can occur even when the infrastructure decides to throw a SOAP fault when it determines a header wasn't understood by your code. In order to make your ASMX code compliant with this Basic Profile requirement, you must add code to each of your WebMethods to check for unexpected mandatory header blocks, as illustrated in Figure 1.

Figure 1 CheckUnknownHeaders

public class MyService { public SoapUnknownHeader[] unknownHeaders; [WebMethod] [SoapHeader("unknownHeaders", Required=false)] public string Method1() { CheckUnknownHeaders(); ... // normal Web service processing here } [WebMethod] [SoapHeader("unknownHeaders", Required=false)] public string Method2() { CheckUnknownHeaders(); ... // normal Web service processing here } public static void CheckUnknownHeaders( SoapUnknownHeader[]unknownHeaders) { if(unknownHeaders != null) { foreach(SoapUnknownHeader hdr in unknownHeaders) { if(hdr.MustUnderstand == true) { hdr.DidUnderstand = false; throw new SoapHeaderException( "Header was not understood", SoapException.MustUnderstandFaultCode); } } } } }

Using cookies to implement stateful Web services interactions is another area that wasn't explicitly defined in the original SOAP messaging specifications. The HTTP State Management Mechanism ("Cookies") allows the creation of stateful sessions between Web clients and servers. The BP doesn't disallow cookies, but rather limits their use. The guidance basically states that cookies should not be required for proper operation. Instead, when used, they should serve as a hint that can be used to optimize processing without directly affecting the execution of the Web service. And consumers should not have to understand cookie values—they should only need to pass them along in future requests.

Service Description

The BP states that WSDL must be used to describe Web services. The WSDL specification was originally designed to accommodate a wide variety of scenarios and development styles. WSDL was intentionally designed to be open and flexible with numerous options and choices available to developers. As a result of this general philosophy, WSDL has emerged as the sore point of Web services interoperability. The WS-I BP has addressed this situation by removing many of the choices and reducing the number of things we need to agree on in the specification.

Figure 2 describes the various clarifications made by the BP to the WSDL specification. For example, it clarifies many of the ambiguities revolving around the proper use of WSDL and XML Schema "import" elements and "targetNamespace" attributes. It clarifies how to deal with such things as parameter order, one-way operations, response message wrapper elements, fault descriptions, and message validation.

Figure 2 Basic Profile Service Description Clarifications

Figure 3 describes all of the BP restrictions imposed on the WSDL specification. For starters, the BP disallows the use of type systems other than XML Schema. It also disallows solicit-response and notification operations, operation name overloading, MIME and HTTP GET/POST bindings, the user of transports other than HTTP, encodings (including the SOAP encoding), mixing styles within a single binding, and requiring custom WSDL extensions. All of these BP restrictions have a recent history of negatively impacting interoperability.

Figure 3 Basic Profile Service Description Restrictions

As you can see in Figure 3, this guidance can have a significant impact on your WSDL design. However, most Web services frameworks (including ASMX) don't encourage developers to start with WSDL, but rather with classes and methods. Such frameworks typically generate an appropriate WSDL definition from the method signatures and type definitions at run time. Although this approach shields you from WSDL complexities, it also leaves you completely dependent on the framework to get your WSDL design right with respect to BP guidance.

By default, ASMX does a good job of conforming to the BP rules related to WSDL. However, you can easily configure an ASMX service to violate the BP (say, by applying the [SoapRpcService] attribute). The Microsoft® Patterns & Practices group has published "Building Interoperable Web Services: WS-I Basic Profile 1.0", which provides clear guidance on how to apply the various BP rules when implementing and consuming Web services with ASMX.

In general, a WSDL-first development model can increase interoperability since it gives you complete control over WSDL design. Starting with WSDL allows you to validate that you've followed all of the BP WSDL guidance before you generate implementation code. See "Place XML Message Design Ahead of Schema Planning to Improve Web Service Interoperability" in the December 2002 issue of MSDN®Magazine for more details on this approach. Otherwise, you need to intimately understand your tool's support for WSDL and the BP.

Service Publication, Discovery, and Security

When you need to publish and discover Web services, the BP outlines how to use UDDI for this purpose. Support for UDDI is not required to conform with the BP since many services simply don't need the functionality. Figure 4 summarizes the BP's guidance on using UDDI.

Figure 4 Profile Publication and Discovery Clarification

The BP also provides guidelines for using HTTPS to secure Web services today. It basically states that Web services may require HTTPS at a particular endpoint. With HTTPS, Web service endpoints can guarantee server authentication, integrity, and privacy for all message exchanges. The profile also states that Web services can require client (mutual) authentication. And since HTTPS happens to be a mature and widely available standard, using it to provide basic Web services security features can be a big win for promoting interoperability.

Profile Conformance

When a Web service conforms to the BP, your chances for interoperability go up. So how do you know if a given Web service is conformant? Conformance is really something that has to be determined and asserted by the Web services developer. Testing tools can assist in this determination by validating WSDL designs and runtime SOAP messages. However, the tools cannot validate every BP rule since some of them are related to external specifications or runtime behaviors. This is why the WS-I has not attempted to produce official certification criteria. The testing tools should be viewed as a good indicator of conformance and not an official certification tool.

Now that the Basic Profile is finished, vendors are likely to provide more assistance in the area of conformance. All developers should follow the implementation guidance provided by their vendor surrounding their tools (such as the Microsoft Patterns & Practices document mentioned earlier). ASP.NET 2.0 is going to make BP conformance even easier with its internal understanding of the BP rules. The ASP.NET 2.0 framework provides a ConformanceClaims property on the WebServiceBindingAttribute that influences an endpoint to behave in a way that's conformant with the BP. The following example illustrates how it's used:

[WebServiceBinding(ConformanceClaims = WsiClaims.BP10)] public class MyService { ••• }

Applying this property also causes the WSDL definition to contain an assertion that the service conforms to the BP. You can still do things in your code to break conformance with the BP, but you'll get runtime exceptions once you interact with the service thanks to its built-in conformance validation. Also, when generating Web services proxies with wsdl.exe, it will check for conformance assertions in the WSDL and complain if it finds anything that violates the Basic Profile.

Conformance Testing Tools

The WS-I Testing Tools provide a great resource to developers designing and implementing Web services. You can download the testing tools from the WS-I site and choose either a C# or Java-language version. Using these tools during design and development can assist in conformance. In order to install the C# version, all you need to do is unzip the files into a directory.

The architecture of the two main tools you'll be using, the monitor and the analyzer, is illustrated in Figure 5. The monitor uses a simple port-forwarding technique to intercept and log SOAP messages traveling between a sender and a receiver. The analyzer is then used to analyze the log file along with the corresponding WSDL definition and UDDI entries. The analyzer produces a report describing BP conformance.

Figure 5** WS-I Testing Tool Architecture **

Both the monitor and the analyzer rely on configuration files to specify settings, such as ports and locations. The tools come with sample configuration files but you'll need to modify them to fit your needs. Once you have the configuration files in place, type monitor.exe from a command prompt and it will begin listening for requests on the port specified in monitorConfig.xml (9090 by default). You'll also need to specify where to forward the messages (such as https://localhost:80).

Then you need to update your Web services calling code to send the requests to the specified port (like 9090). Here's an example of doing this in C#:

MathService svc = new MathService(); svc.Url = "https://localhost:9090/myservice/service1.asmx"; double sum = svc.Add(3, 4); Console.WriteLine("3 + 4 = {0}", sum);

When you run this code, you'll notice the monitor displays information about the SOAP messages it has processed. After you've finished invoking all of the operations you want to monitor and analyze, shut down monitor.exe. At this point, the monitor writes the log file to disk, and then you can send the log file through the analyzer. Before running the analyzer, however, you'll need to modify some of the settings in analyzerConfig.xml to specify the locations of the log file, the WSDL definition, and the UDDI entries (if there are any). When you run analyzer.exe, it will produce a human readable report file.

Figure 6** Analyzer Report Showing General Failure **

The report will give you a summary of BP conformance and indicate whether it passed or failed (see Figure 6). It will also give you a thorough breakdown of each BP rule and whether it passed or failed (see Figure 7). In this example, I analyzed a WebMethod that had been configured with the SoapRpcMethodAttribute—this causes it to use an rpc/encoded binding in violation of the BP.

Figure 7** Analyzer Report Showing Specific Failure **

For a complete walk-through of using these tools in .NET, see "Using the WS-I Test Tools". You may also want to check out the Testing Tools v1.0 User Guide for more details on installation and configuration.

Sample Applications

A few of the deliverables that accompany the BP are part of a complete sample application that illustrates a conforming design and implementation. The WS-I has published a document called "Supply Chain Management Use Cases", which presents a high-level definition of a Supply Chain Management (SCM) application. It has also published a document called "Sample Application Supply Chain Management Architecture", which details the technical design and implementation of the sample application. The architecture includes standardized XML Schema and WSDL definitions that illustrate proper contract design. The sample application was modeled after the SCM use cases and is intended to demonstrate how conformant Web services might be designed, implemented, and deployed.

Numerous vendors (including Microsoft, IBM, BEA, Sun Microsystems, Novell, Oracle, SAP, and others) have provided implementations of this well-defined sample application. They are available for download from the WS-I site. The Microsoft implementation was built using ASP.NET 2.0. Most of the ASMX code was generated from the predefined WSDL definitions provided with the architecture document. The sample illustrates the importance of proper WSDL design and the benefits of using a WSDL-first development approach.

Figure 8** SCM Sample Application Configuration **

Once you download the sample, you'll need to follow the setup instructions in the accompanying document files to configure the necessary virtual directories and SQL Server™ databases. When you first run the sample, you'll need to configure it and specify which SCM endpoints you'd like to use (see Figure 8). You can use the Microsoft endpoints that come with the sample or endpoints provided by other vendors, or even your own. This allows you to experiment with interoperability in a real-world business scenario. Once configured, you can place orders that will be processed through a supply chain consisting of several warehouses and you can track the order as illustrated in Figure 9.

Figure 9** SCM Sample Application Tracking Orders **

If you're designing Web services for your organization, you should definitely take some time to download the test tools along with the WS-I sample application. Check out the various WSDL definitions and walk through the code. Doing so will show you by example how to apply the BP guidance discussed in this column.

Where Are We?

Interoperability is not a given when using Web services. It requires research, thought, and a good understanding of the issues surrounding the various Web services specifications. Luckily, the WS-I does the heavy lifting and provides summarized guidance in the form of profiles, samples, and testing tools. Taking advantage of these resources during design and implementation will greatly improve your interoperability success, which ultimately saves you time and money during integration.

Send your questions and comments for Aaron to  sstation@microsoft.com.

Aaron Skonnard is a co-founder of Pluralsight, an education and content creation company, where he focuses on XML and Web services technologies. He's a contributing editor to MSDN Magazine and the author of several books, including Essential XML Quick Reference (Addison-Wesley, 2001).