You can download this release from the project site or one of the mirrors.  I hope you will take a look and let us know what you think. We know it's not everything you could possibly hope for, so please report issues and make suggestions. Do these code samples help you understand how to solve the problems you might have had building SOA applications with the web services standards and toolkits?  What's missing? What challenges are you facing that a multi-platform sample application and best practices project could better illustrate how to solve?

This is only the first step toward Stonehenge becoming a full-fledged Apache project.  Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful Apache Software Foundation projects. This milestone shows evidence that the community and codebase is up and running, but both will have to become more mature before the project can be fully endorsed by the ASF.

The first milestone release builds from code contributed by WSO2 and Microsoft late last year.  Today  Sun Microsystems contributed an implementation of the StockTrader sample application services built with Metro, the web services stack in Glassfish. The announcement came during Microsoft's keynote address at the JavaOne confernce.  As co-presenter Steven Martin put it:

This is important for two reasons. First, it means that Stonehenge will deliver even more value by providing best practice guidelines and reference implementations across an even broader range of scenarios and platforms, including Java, .NET, PHP, etc. The more samples and real world guidance we can give the community the better since it gives customers the ability to choose the best ones for their specific business requirements. It also makes it easier to pinpoint potential interoperability problems.

In addition, it represents another step forward in our ongoing work with Sun. As we all know, today’s IT environments are heterogeneous; whether it’s a single organization that runs both.NET and Java apps or multiple organizations that seek to collaborate with each other.

Going forward, it would be great to get even more individuals, projects, and companies involved in Stonehenge.  The focus so far has been on the StockTrader services-oriented sample application originally developed for IBM WebSphere but now has interoperating versions running on several technologies including .NET, PHP, and a couple of Java SOA frameworks. Future work can both deepen and broaden what we've started with.  For example, there has been much joint Sun / Microsoft work to  support interoperability across our identify federation products, and it would be good so see this in Stonehenge.    It would also be great to get implementations for a broader set of platforms and frameworks ... Apache CXF?  Oracle Fusion?  A new improved WebSphere implementation at Apache?  How about showing us how to implement this security-focused scenario with REST?  Or maybe one or more of the emerging Cloud platforms?  All would be welcome. 

But we hope there will be more to Stonehenge than just StockTrader.  We're thinking about what other realistically complex and challenging -- but manageable! -- service-oriented scenarios would lend themselves to a set of interoperable sample implementations.  Please join the conversation on the Stonehenge mailing list or issues tracker.

So, what is Stonehenge and why am I participating?  This project is championed by Paul Fremantle, co-founder and CTO of WSO2, which has been a great partner in helping to improve and demonstrate the interoperability of the WS-* standards across platforms.  For example, at TechEd 2008, Jonathan Marsh of WSO2 and Greg Leake of Microsoft demonstrated how separate WSO2 and Microsoft components implementing a mutlti-tier stock trading application can interoperate and be substituted for one another.  

StockTrader is just the starting point for the broader goals of Stonehenge.  As the project proposal puts it:

The aim of the Stonehenge project is to develop a set of sample applications to demonstrate seamless interoperability across multiple underlying platform technologies by using currently defined W3C and OASIS standard protocols.

We are proposing this incubator project because we believe that a project that includes a set of sample applications, with multiple language and framework implementations will become a useful and important part of the SOA landscape. It will:

· illustrate and develop best practice for interoperable applications that communicate via distributed protocols,

· demonstrate interoperability between platforms,

· provide sample code upon which SOA developers can build,

· help identify interoperability issues and their solutions, and

· build confidence in cross-platform deployment of SOA technologies

More generally, we believe that Stonehenge can help wire up the "last mile" between the standardized web services infrastructure that is now implemented across key platforms, and a new generation of service oriented applications that will span them. Existing WS-* interoperability work such of the sort done by WS-I and in our "plugfests" will continue to solidify the platform-level interoperability.  The new work, exemplified by Apache Stonehenge, should attract a wider community of users who can exploit the hard standardization and platform interoperability work without having to wallow in as many nasty details as in the past.

We've gotten clear direction from customers that sample applications based on real-world scenarios and challenges will help them realize the potential of these technologies which have been developed and standardized for the last 8 years or so. Likewise, the initial response from the Apache community has been quite favorable.   I have a personal commitment to invest in helping make Stonehenge a success, and look forward to digging in.

For more information on Stonehenge and other Microsoft work with the Apache Foundation, see:

• Sam Ramji's ApacheCon keynote and the writeup at eWeek
• Paul Fremantle's blog post on Stonehenge, Kamaljit Bath's report from ApacheCon, and the Apache Feathercast featuring the two of them.
• Subscribe to the mailing list stonehenge-dev-subscribe@incubator.apache.org

Not only does Microsoft closely track the standard version of specifications, Microsoft invests in helping other companies do similarly.  One way is by sponsoring  “plugfests” where implementers can get together and test what works and what doesn’t while products are still being developed.  Don’t believe me, take a look at accounts of the recent web services plugfest from other “coop-etitors” in the SOA industry.  For example Stefan Tilkov  interviewed Harold Carr to assess the state of interoperability between Sun’s Metro product and WCF.  Harold notes:

.NET 3.5 (which is released) is based on the standard versions. Metro 1.x (which will ship later in 2008) will be based on the standard versions also.

Harold's own blog has details on exactly which of the ratified standards were used successfully to interoperate with .NET 3.5. 

These ongoing  Plugfests are a great way to increase real world interoperability, and we hope to see even broader participation in the future. See the scenarios and frequently asked questions page for details on how to participate. All interested parties are invited irrespective of which platform they develop for and whatever their business relationship (or not!) with Microsoft might be, with no legal agreements to be signed.

Here is a list of Web Services standard specifications (i.e. the versions of these specs that are OASIS standards or W3C Recommendations) that are shipping today in WCF, Microsoft’s application platform framework: 

• SOAP 1.2
• W3C Web Services Addressing 1.0 - Core, SOAP binding, WSDL binding
• MTOM
• WS-ReliableMessaging 1.1
• WS-Security 1.1 (also SAML Token Profile 1.1)
• WS-Policy 1.5 / WS-PolicyAttachment 1.5
• WS-SecurityPolicy 1.2
• WS-SecureConversation 1.3
• WS-Trust 1.3
• SAML 1.1 Token Format
• WS-Addressing 1.0
• WS-Coordination 1.1
• WS-AtomicTransaction 1.1
• UDDI 2.0
• WS-I Basic Profile 1.1
• WS-I Basic Security Profile 1.0

Other ratified standards are supported elsewhere in the Microsoft product line, e.g. the DMTF WS-Management standard is supported in Vista.

It’s fair to point out that not all potentially relevant specs that can claim to be standards are supported in .NET (or the native Windows XML libraries for that matter).  The clearest example of a ratified version of a spec that Microsoft and many others chose not to implement is XML 1.1.  This was a difficult decision, since some changes in XML 1.1 make it better aligned with the evolving Unicode standard than XML 1.0 is, but others  are “breaking changes” that would actually degrade real world interoperability.  In this case, principled pushback on a standards committee has led to a proposed new edition of XML 1.0 that has improved Unicode alignment without sacrificing interoperability.

We encourage implementers to participate in the ongoing work to demonstrate and improve real world WS-* interoperability, and for customers to let us know about additional scenarios and specifications where we need to focus in the future in order to make cross-platform services ever more valuable.

 A couple of points in reply to comments on the earlier post: 

• It overlaps in functionality with WSDM, another family of specs, and this overlap is a source of friction between the MS and IBM web services stories.  I won't get into the history or politics here other than to speculate that WS-Management is a classic example of the Pareto Principle -- it does quite a bit less than WSDM (maybe 20% of the functionality and complexity of WSDM), but its relative simplicity makes it easier to implement while being good enough for 80% of the important use cases.  So, even in the allegedly pathological WS ecosystem, evolution favors the simple and functional over the the more highly but complexly designed.
• Why didn't something even simpler and more RESTful evolve, which Patrick Logan's comment suggests would have been easy?  I don't know the history, but it's clear that REST concepts were in fact leveraged to keep the spec relatively simple.  Patrick also pushed back that that it is a "shell of a solution with little agreement on what goes into the shell." REST is also a "shell of a solution", and WS-Management adopts the same basic approach to allow late binding to the resources in a catalog rather than tight coupling to a predefined set of objects.
• Why not just use HTTP rather than WS-Transfer?  I created a bit of contention by asserting that WS-Man is designed for use "in situations where the web-scale alternatives really don't fit, such as deep within operating systems or in the firmware of chips".  A number of people pointed out (probably correctly, but I don't really know) that an HTTP implementation would "fit" in the same amount of code or memory as a WS-Transfer implementation.  That being as it may, I meant "fit" in the architectural sense -- I thought the common REST argument that HTTP is an application protocol indicated that it would not "fit" so deep down in the stack as the firmware on a chip or device.  More significantly, HTTP does "fit" on top of TCP/IP, but WS-Management was designed for use in situations where only UDP network services are available.  
• Whether or not the designers were simply riding the WS bandwagon rather than thinking hard about RESTful alternatives,  WS-Management sits on a number of WS specs, not just WS-Transfer.  By building on WS-* they could leverage services that HTTP does not provide.  WS-Eventing provides asynchronous "push"  notifications, WS-Enumeration supports the traversal of collections of resources, and WS-Security, WS-Trust, and WS-SecureConversation provide a security infrastructure.

WS-Management leverages some key benefits of SOAP such as its protocol neutrality while adopting some of the REST abstractions to avoid the complexity of an RPC API for all possible management service invocations.  In other words, it exemplifies what Jon Udell calls WS-JustRight for its intended domain: "I stand by what I’ve been saying all along, in a variety of places including this InfoWorld cover story: there’s WS-Heavy, there’s WS-Lite, and for every situation there’s a WS-JustRight that may rely on elements of one, the other, or both. The Richardson/Ruby book brings much-needed clarity to the WS-Lite end of the tolerance continuum, and that’s a great thing. But when we celebrate one end of the continuum, why must we deprecate the other?"

Let's consider one example, WS-Management, which I had barely heard of when I started in CSD Interoperability. It's stated purpose is:

To promote interoperability between management applications and managed resources, this specification identifies a core set of Web service specifications and usage requirements that expose a common set of operations central to all systems management. This comprises the abilities to
• Get, put (update), create, and delete individual resource instances, such as settings and dynamic values,
• Enumerate the contents of containers and collections, such as large tables and logs,
• Subscribe to events emitted by managed resources, and
• Execute specific management methods with strongly typed input and output parameters.

At first glance this appears to duplicate widely deployed bits of the web.  For example, it depends on the oft-criticized WS-Transfer spec, and some are advocating using Atom and the Atom Publishing Protocol rather than WS-* for describing collections and subscribing to notifications of their contents.  On closer examination, WS-Management is widely used today in situations where the web-scale alternatives really don't fit, such as deep within operating systems or in the firmware of chips.    For example:

For IT Professionals, Windows Server and Microsoft Operations Manager will enable the management of heterogeneous software and hardware systems using WS-Management. For consumers, Windows Vista will support the discovery of and interaction with Web services-enabled devices, such as printers, digital cameras, and home control systems.

Those devices may or may not be "on the Web"; increasingly support is built into the firmware of the devices. For that matter, the heterogenous software systems being managed may include HTTP servers.  Furthermore, this is not a Windows-specific technology, it is actively developed and supported across platforms by HP, Intel, and others. 

Another example where WS standards and technologies are quietly taking root is in the area of identity management.   Both the major identity management stacks (WS-Trust / WS-Federation and Liberty) depend on Web services technologies, and there is no credible "pure Web" alternative. The "identity metasystem" supported by  Windows Cardspace and several other vendors and open source projects that to some extent bridges these stacks is getting real adoption, inching WS-* up toward the "plateau of productivity."

Perhaps we are seeing a repetition of the pattern that made XML ubiquitous.  It was originally conceived as "SGML for the Web", but that vision has never materialized -- non-wellformed HTM, Javascript,  and proprietary formats such as PDF and Flash predominate on the Web itself. XML has become pervasive as a convenient format for less glamorous tasks, such as configuration files, summaries of site contents (RSS and Atom), and lists of lists (OPML).  Likewise, at the peak of the hype cycle the web services technologies were promoted as part of a grand vision.  The post-Vista world may not look exactly like the 2001-vintage Longhorn vision, but WS technologies are doing the unglamorous work for which they were actually designed. 

In short, from what I have learned recently, the trajectory of WS-* isn't pointing toward oblivion, it looks headed toward the same sort of pragmatic ubiquity that XML has achieved.  That's not to say all is rosy; there is lots of confusion and dissension, again just like there was in the early days of the Web and XML.   Likewise, "ubiquity" doesn't mean that the WS technologies are the best or the only option across the board,  but that it they are increasingly available as a very viable option when developers need protocol-neutrality, security, identity, management capability, etc.

I saw The Secret of Apple Design and was struck by some similarities with how LINQ to XML came about. The article quotes extensively from design guru and former Apple exec Donald Norman.  For example:

"[Jobs] had a single, cohesive image of the final product and would not allow any deviation, no matter how promising a new proposed feature appeared to be, no matter how much the team complained. Other companies are more democratic, listening to everyone's opinions, and the result is bloat and a lack of cohesion.

Thie quote points to the first similarity -- the existence of a designer with the skill to make hard decisions and the authority to make them stick. In LINQ, the Steve Jobs role is played by Anders Hejlsberg. Superficially, the LINQ design teams  look a lot like W3C working groups, with regular meetings where requirements are interpreted and fine-tinued, and proposals for alternative ways of achieving the requirements are presented and discussed, sometimes for months.  In sharp contrast to the designed-by-committee XML technologies such as DOM, however, LINQ has an architect who makes decisions, not a chair who drives consensus.  In the LINQ design meetings, all opinions were considered, and previous decisions open for reconsideration, but there is no pretense of democracy.  When it came time end a discussion, there wasn't a vote, or weasel-wording to paper over disagreements, there was a decision.

The quote also highlights the importance of conceptual integrity. Anders began what is now called LINQ to XML with a clear vision for how a modern XML API could work, with C# functional constructors and LINQ queries providing an essentially functional approach to working with XML.  Lots of ideas were proposed, and lots of people complained when they were removed or rejected, but the goal of internal cohesion and alignment with LINQ was paramount.  Of course, XML reality intervened in various unavoidably ugly ways, but much design effort was put into minimizing and isolating the ugliness.

The third similarity I noted is an insistence on simplicity over feature richness.  Quoting again from the article:

"The hardest part of design ...is keeping features out." Simplicity, [Norman] says, is in itself a product differentiator, and pursuing it can lead to innovation.

Rolston agrees. "The most fundamental thing about Apple that's interesting to me," he says, "is that they're just as smart about what they don't do. Great products can be made more beautiful by omitting things."

 A year ago, I wrote a post on what LINQ to XML would not do, such as guarantee that a tree in memory is well-formed, support XML 1.1, offer more than bare minimum support for DTDs, round-trip entity references, and so forth.  Likewise, the design team decided to remove the streaming input API that had been extensively discussed and prototyped.  In many such cases, Anders decided that the functionality benefit wasn't worth the complexity cost.

Of course, the real secret to successful design is the skill and tenacity of the designers and the implementers; giving a chief designer the authority to purse cohesiveness and simplicity just enables success, it doesn't assure it. Likewise, LINQ's technical success doesn't guarantee real world success. For example, for all  its aesthetic flaws, DOM has been an undeniable success story in helping make XML and dynamic HTML reasonably usable by developers, and has helped the world move from mostly proprietary to interoperable data formats over the last 10 years.   It is possible that the same ugly compromises that promote committee consensus actually help overall adoption.  Likewise, just as Apple's design aesthetic pays off because it appeals to an upscale market willing to pay a premium for form and fashion on top of functionality, perhaps LINQ to XML will appeal mostly to a niche in the developer community.  We shall see ... but I bet a couple of years on this and I'm convinced that this technology will be widely adopted on the Microsoft platform and will spark a round of competitive innovation on other platforms.  ]]> https://docs.microsoft.com/archive/blogs/mikechampion/accelerating-evolution-linq-news-from-mix-2007 Wed, 02 May 2007 22:49:42 GMT https://blogs.msdn.microsoft.com/mikechampion/2007/05/02/accelerating-evolution-linq-news-from-mix-2007/ There is a lot of interesting (and once confidential) stuff that came out of the Mix conference this week.

Jon Udell's  "Watching Anders Hejlsberg reinvent the relationship between programs and data" ... offers an enthusiastic summary:

A lot of the time, when we use the web, we’re effectively performing joins among data sources. You visit one site to look up some data, then you grab some of it and plug it into another site. If you’re lucky, somebody will have built a mashup, on a third site, to facilitate that join. But what if your browser had the data manipulation chops to help you do that mashup directly? I’m hoping that technologies like Silverlight and LINQ will enable things like that to happen.

Jon doesn't elaborate a whole lot on what "reinventing the relationship between programs and data" means.  Fortunately, Anders' LINQ presentation itself is now online.  Some points he makes about the new relationship between programs and data include:

Data != Objects  - Now you can query data without it being in a database ... LINQ offers "native support for queries as a first class concept". C# and VB now offer the same expresive power as SQL and XQuery built right into the languages.  LINQ unifies and brings to a higher level of abstraction the whole notion of data-driven programming. 

Imperative->Declarative - Programmers can focus on "what now how" without having to learn a new language such as SQL or XQuery.  Imperative approaches have run their course; we won't see 10x productivity improvements anytime soon, nor will compilers be able to automatically adapt imperative code to new hardware.  But the declarative approach will support 10x improvements in the performance of data-driven applications and the "what not now" approach allows compilers to adapt to multicore architectures and so forth.  The key is to adapt programming styles to a more declarative, query-oriented style.  LINQ supports this.

OO programming languages have no notion of projection, you have to do it with tedious declarations and imperative transformations.  LINQ makes this much easier, with anonymous types and functional construction.  In the demo, Anders usees this approach to transform XML data into business objects with very little code, and no schemas or code generators.

I'd elaborate a bit on the last point.  Lots of technologies exist that leverage some explicit mapping from raw data to programming objects; that mapping might be a conceptual model, a schema, an ontology, or an adapter.  For example, XQuery is used to integrate and query over diverse data sources, but only after the data has been explicitly transformed from its native representation (e.g. SQL tables) to an XQuery data model.  LINQ to Objects and LINQ to XML, on the other hand exploit the IEnumerable<T>, aka monads/monoids, that is implicit in almost all collections of data, or exposed as axes in XML documents.  To me, that's the essence of LINQ's reinvention of the relationship between programs and data: conceptualizing data in an object graph, database, XML document (or Amazon.com catalog,  LDAP directory,  ad infinitum) allows developers to write smaller, cleaner, more parallelizable, etc. programs with much less concern for the plumbing.

Putting all this into a realistic application, Aaron Dunnington and Tim Scudder of the Data Programmability / XML team gave a Mix Presentation that exploits the XML features in Silverlight clients and LINQ, etc. on the server.  Their demo is called "The Socializer", showing how Silverlight applications can bring all sorts of social networking data together and display it in a visually appealing way. Technologies used in the demo include Silverlight, C# 3.0,  LINQ to Objects, LINQ to XML, RDF, FOAF and more. 

Needless to say, all this Silverlight love from unexpected parties isn't going unchallenged.  The most popular statement of the opposing side seems to be the fine rant from Mark Pilgrim. His most trenchant bits are unquoteable, but the gist of it seems to be that Adobe and Microsoft (and Sun's "alternative to Ajax") indicate the return to the bad old days of proprietary technology and vendor lockin.  Astonishingly enough I see it differently: The purely standards-based web is quite functional and interoperable for relatively static content, but the "Web 2.0" demands for browser-centered applications that are dynamic, attractive, and portable are very difficult to meet with currently standardized technologies.  As Dan Ingalls of Sun put it:

AJAX sort of deals with all of the old way of doing things. It makes it simpler, which is great, but underneath it’s still all this junky HTML, Document Object Model, CSS, all that stuff, where 30 years ago, we knew how to do that stuff cleanly with a dynamic programming language and a simple graphics model.

That's NOT to say that Ajax is dead.  As far as I know, there was a lot of Microsoft Ajax support announced at Mix, and more in the pipeline.  I hope and pray we (inside and outisde MS) don't repeat the IE6 mistake of declaring "junky HTML, DOM, CSS, all that stuff" obsolete and stop maintaining the standards and implementations.  It will live on, and improve for some time... while LINQ/Sliverlight/Apollo/Flair/whatever mature and potentially drive a new generation of web standards.  But a new generation of web standards won't be invented by committees, but by developers, then polished  by competition ... then committees can come along and tidy up.  Just like Web 1.0 came about.

It's interesting watching various pundits and analysts read the tea leaves of the numerous LINQ subprojects, Silverlight versions, and incubation projects such as Volta to figure out the "Microsoft" master plan for pulling it all together.  Just as customer pain with today's technology drives diverse innovation across the industry, so it does within Microsoft, and the process resembles evolution in action more than intelligent design by a supreme architect.  I don't think we're seeing a silly season so much as a Cambrian explosion of new ideas from all sorts of places, and Father Darwin alone knows how it will end up.

"Huh?" you might ask.  Why voluntarily join the doomed crew of WS-Deathstar?  Or maybe the WS-* crowd are not evil, they're  weenies who get sand kicked in their faces? As with XML itself, the point isn't whether "any damn fool could produce a better data format" it's whether the web services specs provide a pragmatic foundation for actual interoperabiliy. Just as XML quietly became pervasive while its numerous limitations were being flamed by the cognoscenti, the WS technologies are quietly taking root in all sorts of places where interoperable identity, security, reliability, management, etc. concerns trump geek aesthetics.  There's no disputing that skillful people can implement systems with all these properties without WS-*, but there is much to be said for building these capabilities into the infrastructure of Windows, .NET, Websphere, Java, Apache, etc., in a standardized and interoperable way. I don't pretend that goal has been achieved, but I'm convinced its achieveable.

 Another part of my reason for moving is that LINQ to XML is close enough to being done that I can declare victory and move on, hoping that its success will rebalance my karma after the beating it suffered from my role in DOM :-).  I'm still involved in the project to some extent, backing up Ralf as he assumes the role of LINQ to XML PM and doing my bit to evangelize the LINQ XML story in Orcas and Silverlight.

But there are some more substantive attractions to WS-Darkness.  First, the overall WS-* framework doesn't deserve its bad reputation among XML and Web geeks. It's hard to call anything designed by that many committees elegant, but there is an underlying conceptual integrity in it that I've come to appreciate over the years. That's not exhibited by all the specs, to be sure ...don't ask me to defend XSD or WSDL!  There's no disputing that current interoperability above the SOAP level can be problematic, but much of this can be traced to incorrect implementations and problems mapping XSD to underlying object systems. It will take longer to get full interoperability at the higher levels of the WS stack, but I'm convinced that it will be much more practical to complete the work that is underway than to start over with some new paradigm, or to return to the bad old days of one-off data contracts and armies of consultants needed for everything.

 The stock phrase in the WS-* specs is that they are "designed to be composed with each other to provide a rich messaging environment".  I think the term "Aspect-oriented messaging" better captures the underlying architectural principle that the WS specs can work separately or be composed together to provide a variety of message services: 

[SOAP 1.2] provides a much simpler and more powerful aspect-oriented mechanism for extending the SOAP processing model. New aspect-oriented features can be added to SOAP by defining additional SOAP headers and how to process them. Such features can range from transport-level features such as security and reliable messaging to business-level features such as spending-policy enforcement to personalization and differentiated service. Bottom Line: Users who consider SOAP to be just a low-level RPC mechanism should review SOAP v1.2 to understand its potentially profound architectural implications for distributed computing.

"Ironick" indeed, that's by the same Nick Gall whose Web of Services for Enterprise Computing whitepaper got so much link love from the WS-skeptics. Gall may or may not have come to regret his previous enthusiasm for SOAP's architectural principles, but in any event the main point of his position paper isn't  isn't so much about WS architecture as it is about W3C's philosophy: "While there is nothing wrong with [the various WS-* specs]  --in fact there is great value in having the major middleware vendors finally agree on middleware standards after so many years of proprietary middleware protocols--the problem is that such work has nothing to do with web architecture or the W3C."  That may be a defensible position, but the fact remains that the W3C management and staff have worked hard to give at least some of the more fundamental and web-related WS specs a home at that organization.  

Which leads to my second point:  "It's a tool, not a religion".

I have spent time around job sites. I can promise you that you'll find circular saws, chop saws, and table saws. Is there overlap in functionality and potential application of these saws? Absolutely. Why have all three then? Well, because each is particularly good for certain types of cutting chores and less optimal for others. Contractors, like mechanics, want a broad set of tools available so they can have the optimal tool for each task.

Why then do we, as software engineers, have to work so hard to reduce our toolbox to the ultimate tool? There is this tendency to look for the Swiss Army language with supporting framework that will be the optimal solution for every problem in the world. The simple reality is that such a language and framework can never be created.

I've always been annoyed by the zealotry manifested in the "SOAP vs REST" debate.  The overhyped WS marketing and the RESTifarian pushback in 2001-2002 or so had the effect of people "flipping the bozo bit" on each other wholesale.  We're just now seeing some exploration of the sensible middle position -- the native Web technologies are powerful and deserve first class support in the services-oriented tools and specs, and at least some of the WS technologies are useful on the Web as well as the enterprise, especially in the realm of identity and security.  People on the WS side are finally understanding the the potential of the SOAP GET binding, and maybe the REST folks are starting to understand the oft-maligned WS-Transfer  ("HTTP over SOAP over HTTP") is not quite so pointless in a multi-hop, multi-protocol world.

Finally, the supposed coup de grâce in most skewerings of WS-* is to point out the complexity of the web of specs and frameworks.  The best response I've seen is by Charles Zedlewski:

But I think the kvetching over the WS-* complexity is pretty overblown. What struck me looking at Innoq’s diagram is how similar it is to the Java class library (or any other major language’s class library for that matter).

...  you don’t have to learn all the inner workings of either standard to make good use of it. You either:
1) Learn and choose to leverage only the most useful bits of the standard that are relevant for your project. ... Or you:
2) Utilize some combination of tools and platforms that hide the complexity of the underlying standards.

Anyway, I'm having fun getting back up to speed on all this and getting back in the blog debates I swore off awhile ago. 

Our unexpectedly intense snow yesterday was created by a Puget Sound Convergence Zone:

Those northwest winds will collide with the Olympic Mountains. Part of the air flow will be deflected east down the Strait of Juan de Fuca, while the other part will be deflected down the western side of the Olympics. When the northern branch reaches the I-5 Corridor and the Cascade Mountains, it will then be forced to the south. Meanwhile, when the southern branch reaches the I-5 corridor and Cascade Mountains past the southern side of the Olympics, it will then turn to the north.

Eventually, the south-flowing branch and the north-flowing branch will converge. When that happens, the air has nowhere to go but up. Rising air will lead to convection. That will lead to cloud and storm development.

That seems like a good metaphor for why technology prediction is hard.  The interesting stuff happens in the "convergence zones" where different streams of ideas and technologies collide and create upward convection currents (driven partly by hot air and vapor I suppose).  The World Wide Web arose out of a convergence of the internet and SGML-ish markup;  the Internet Bubble arose out of a convergence of this enabling technology and the vast numbers of PCs in every home and office; XML arose out of a convergence of the capability for universal interoperability spawned by the SGML and the Web,  and the demand for a Web-like experience in a whole range of document and data products that was accelerated by the Bubble. 

But not every wind / technology convergence creates a convergence zone.   See the "Meteorological Mumbo-Jumbo" section in this article about our weather yesterday for an explanation of how this weather pattern was different than the other 20 recent occasions where the winds were about right:

This time, the cold air is, at least initially, coming from the Gulf of Alaska. Normally, that pattern doesn't make for much snow because the air has a long time to moderate over the warmer ocean waters. But the air mass up there is SO cold ... the air was still cold enough to snow by the time it made it here.

In technology, lots of things -- technologies, personalities, zeitgeist -- have to come together at just the right time.   Why was "Dynamic HTML" a bit of a yawner in the late '90s but "AJAX" a big hit in 2006? Why did the Newton flop and then the Palm become a hit a few years later?  Why is YAML a backwater but JSON the Next Big Thing this year? I don't pretend to know.  I do think, however, that we know enough about the XML technology landscape and the forces that interact with it and each other to at least talk about some likely convergences and divergences.

First, as the weather truism goes, the best prediction of tomorrow's weather is today's weather.  2007 will look a lot like 2006.  Duh.  Overall, XML has become a fairly mature and slow-changing technology because there is a 10-year legacy holding back any radical change, e.g. a mass migration to RELAX NG, JSON, LINQ, or whatever.  We will see a gradual migration to XSLT 2.0, growing interest and support for XQuery (as a query language!), continued improvements in XML Schema 1.0 interoperability and growing interest in 1.1 as it solidifies, and so on.

Second, there's a strong prevailing wind that puts XML almost everywhere, even places where it might not belong (e.g., multi-gigabyte log files!). But the pervasiveness of XML means that the "XML community" has less and less in common.   The streams that collided with Mt. NonInteroperable and reconverged to create XML 10 years ago have shifted because Mt. NonInteroperable has been greatly eroded.  People care more about interoperable documents OR data OR messages than interoperable documents AND data AND messages, so the fact that XML underlies all of their interop stories is not particularly interesting anymore. What predictions does this pattern imply?

•  Assuming that nothing pops up to reconverge these separate streams, we'll probably see a continuing decline in the popularity in generic XML forums, books, conferences, etc., and more integration of domain-focused XML topics into more specialized venues.
• Likewise, we'll probably see an increase in the use of niche XML-related technologies, such as RELAX NG for purely textual documents and JSON for web applications, in the domains for which they are well-suited. 
• Conversely, we'll see a decline of interest in "one size fits all" approaches.  For example, it looks (from the outside) like the W3C Efficient XML Interchange people are focusing on one problem -- the fact that XML is too bloated for many limited-bandwidth scenarios -- and not trying all that hard to come up with something that is smaller / faster / cheaper / more universal than XML for all scenarios. 
• Finally, the breadth of XML's adoption comes at the expense of depth. This suggests that we'll see more of the pattern that Jon Bosak decried in his XML 2006 keynote: "business users will never adopt a solution that depends on an additional XSLT pass because it would require them to learn something new (never mind that this “new” thing had been widely employed in other contexts for years)." Those vendors may know their customers, and suspect that they really don't want to learn yet another XML technology just because it would offer an elegant solution to a somewhat peripheral problem.

Third, like the wind XML is mostly invisible -- it's hidden behind the firewall, embedded in ZIP files, or just called something else. What's more, most people really don't want to see the XML that surrounds them. They want to see readable documents, updated feeds, processable objects, and delivered services.  The better the XML is hidden, the more the bulk of the world likes it. Being out of sight, it's also out of mind.  Few noticed that "Asynchronous Javascript And XML" actually refers to the XmlHttpRequest API much more than XML content, so the substitution of JSON for XML as the typical bits on the wire format caused little concern. Actually that understates the case -- most people are happy to use more familiar technologies instead of XML or in front of XML.  What might we predict from this pattern?

• I'll make a (self serving!) prediction that the LINQ approach of focusing on what is common across data formats will get more mainstream traction than will the notion that users want specialized tools for their XML data.
• Tools that make it relatively easy to consume XML directly into programming objects such as LINQ to XSD will continue to mature technically and be adopted by pragmatic developers.

What about some of the XML technologies that have been swirling around out there ...  What landfalls may we expect in 2007? Again, the whole point of the argument here is that only incremental change can be foreseen unless XML technologies get tangled up in some larger crosscurrents.  So, the Semantic Web will remain highly domain-specific (e.g., the biomedical arena where well-established taxonomies and ontologies really can be leveraged by OWL inferences, SPARLQ query processors, etc.).  I can't think of any larger social forces that might collide to provide upward convection for the semantic web technologies other than some low-probability event such as a major terrorist attack being thwarted as a direct result of Homeland Security's semantic technology investment.  How about SVG, or XML 1.1, or some other major XML technology that hasn't gotten mainstream support (ahem, partly because of decisions in Redmond)?  I don't foresee any major shifts in the winds here ... but then again I didn't know about the Puget Sound Convergence Zone the other day until the streets near home were iced over.

What great technological, economic, or political forces can you envision changing the XML climate in the next year?

• I came to the software industry rather late in life after a mis-spent youth in Political Science at the University of Michigan.  I discovered that it was more fun writing the code than writing the dissertation, not to mention the fact that the job market for junior programmers was immensely larger and more lucrative than the market for assistant professors circa 1980.  Furthermore, programming gave me the luxury of staying in Ann Arbor for 25 more years.
• My longest lasting hobby is playing the acoustic guitar. I  learned much of what I know from Frank Salamone, who sadly is no longer able to play. I spent the accrued vacation pay from my last job on a Martin 000-28 when I started at Microsoft.  The piece I most enjoy playing is Leo Kottke's The Fisherman.  I try, with minimal success, to play various stuff by John Renbourn.
• My musical tastes run mostly toward the classical/traditional. A survey of the dying iPod (Apple seems to have rediscovered the virtues of Planned Obsolescence) doesn't show a whole lot of written after 1759 (when Handel died) and only a handful  after 1827 (Beethoven).   All time favorite recordings include Parkening Plays Bach, Trevor Pinnock's Goldberg-Variationen, and Derek Bell's Carolan's Receipt.
• My current favorite "leisure" activity is, as one of my kids puts it, getting torn up by nasty thorny things.  I'd give it a more pretentious label, "natural area restoration", which around here involves a lot of struggle with Himalayan Blackberry on behalf of the Sammamish River Stewards. Back in Ann Arbor, is was a struggle with European Buckthorn on behalf of the Friends of Dicken Woods. 
•  Based on the countries I've actually visited, if I had my life to live over I might do it in either Norway or Australia.

Being paranoid of this kind of response, I won't inflict the meme on anyone else.