XQuery support in SQL Server (soon) and .NET (maybe later)

This is Mike Champion at the controls of the XMLTeam Weblog this morning.  I have introduced myself on my own MSDN weblog, if you want to know my background and role on the team.

One of the things that people are a bit confused about is the role of XQuery in the next generation of Microsoft products.  To be honest, we haven't done a great job of communicating recent changes in thinking, as  Jeff Julian   noted recently. The reason we had to change course is that neither XQuery nor XSLT are going to be W3C Recommendations in the timeframe of the Whidbey release of the .NET framework and Visual Studio.  Microsoft  has learned the hard way (by supporting a draft of XSLT in IE5) that supporting draft specifications in core technology causes far more pain for everyone over the long run than it gives back in benefit. My colleague Arpan Desai does a nice job of explaining the realities from the XML Team's point of view:

When we do decide support something, it should have a pretty long lifetime. XQuery 1.0 and XSLT 2.0 are huge undertakings. Writing an implementation is, surprisingly enough, is one of the simpler factors. Making sure it's scalable makes things a little harder. Getting the quality high enough so it's enterprise-ready (btw, this is much, much more than passing XYZ test suite) is pretty difficult. Getting tooling support like query generators, debuggers, and profilers is a substantial amount of work on top of everything else. Investing this much into a couple of unfinished standards? Maybe not the best idea.

XQuery is, however, so critical for powering the XML datatype in the next version of SQL Server that a subset of XQuery, which we expect to be stable, will be supported there.  SQL Server 2005's support for XML and XQuery has been seriously underestimated and misunderstood in a recent article by an industry analyst, and Michael Rys has set him straight:

SQL Server 2005 provides an XML datatype that also provides Infoset/XQuery data model fidelity which is basically an extended SQL-2003 standard XML type. So based on the information provided in that article, I would not say that either of the two vendors are "well behind the curve". Especially not, since the IBM offering talks about an alpha version, whereas both Oracle's and SQL Server offerings are being used today in production systems. ...We may however get excited at reports that are somewhat clueless and imply that we are behind, while we are ahead.

Furthermore, as Arpan notes, the XQuery experience has offered overall benefits to our support for XML in .NET:

So was XQuery a black hole? Maybe, but some good came out of it. For example, we came up with a new query architecture to support both XSLT and XQuery. Our Whidbey XSLT implementation is built on this architecture and the performance of this implementation, well, blows the socks off XslTransform. Another major benefit was our ability to provide feedback to the XQuery/XPath working groups in finding issues in the standard. This aspect of early implementation is often trivialized, but it's an important step in validating a standard.

Looking past the current release cycle, all I can say is that multiple options are on the table and we are waiting for completion of the XPath2 and XQuery Recommendations and for feedback from the user community before making any irrevocable decisions.  Microsoft is committed to assisting with completing the XQuery Recommendation in a timely manner, and indeed that is one of the major reasons for creating the position I hold. But that is not a blind commitment to XQuery as the One True XML Paradigm: XML gurus have a wide variety of opinions on whether XSLT or XQuery provides a better declarative approach to XML manipulation, which APIs provide the best support for procedural programmers, and whether ideas such as C Omega might be the inspiration for new approach.

Despite widespread assumptions to the contrary, the XML WebData team cannot even think about supporting everything that is purported to be a standard or jump on every new idea that comes along.  We hope to hear from the folks in the trenches who are actually building XML-based solutions with Microsoft tools to learn what challenges you today, and what visions inspire you about the future.

Mike Champion
Program Manager - XML Standards