If you’re wondering why I’d be at a WordPress event, the answer is that I’ve moved to a new role here at Microsoft on the Interoperability team, and in this role I’m working with a variety of open-source technologies that run on Windows Azure, including WordPress. If you didn’t know that WordPress runs on Windows Azure, check out this article on the InteroperabilityBridges website: “How to deploy WordPress using the Windows Azure SDK for PHP WordPress scaffold.” We’ve also published guidance on deploying Wordpress Multisite, and we’ll be continuing to work with the WordPress community to improve the Windows Azure deployment experience for them going forward.

My colleague Ben Lobaugh, the WordPress expert on our team, gave a presentation at WordCamp entitled “The Proof is in the Pudding – Yep it works with Microsoft.” You can find Ben’s slides here, and he covered several topics of interest to WordPress developers including WebMatrix and the BizSpark and WebsiteSpark programs that help developers and designers get free tools, support and visibility for building websites and other applications. I learned about WebMatrix’s support for WordPress from Ben last week, and used WebMatrix to design a new personal blog that I deployed to Mahugh.com during this trip to WordCamp. The experience was simple and straightforward – WebMatrix handles installing and configuring everything you need, so that you can just concentrate on the look and feel and content of your site.

It was fun to meet so many interesting people, and I’m looking forward to returning to WordCamp next year. Thanks to Paul Holmes for organizing and running a great event!

Here are a few photos from the event, including the spectacular ferry rides between Seattle and Victoria (by way of Bainbridge Island and Port Angeles) …

As Rob notes, many people from around the world representing many organizations contributed to ODF 1.2. Here’s a Wordle of the attendees of every ODF Technical Committee (TC) call in the final year of ODF 1.2’s development:

Microsoft has supported ODF since Office 2007 Service Pack 2 and formally joined the ODF TC in early 2008.  . I’ve attended many meetings myself, and my colleague Cherie Ekholm, Senior Standards Professional, has been active in the ODF TC as well as the OIC (ODF Interoperability and Conformance) TC since 2009.

We’ve also had some great support from others in broader Microsoft Office team who have made significant contributions to ODF 1.2:

• Eric Patterson, a Senior Program Manager on the Excel team, has been a regular attendee and voting member in the ODF TC for more than three years, working with the Open Formula Subcommittee(SC) to finalize the specification of ODF spreadsheet formulas. Eric submitted 45 issues related to functions, referencing and syntax, worked with the SC to resolve those issues, provided detailed information about Microsoft Excel behavior in response to various requests, and worked with other implementers to improve formula interoperability.
• David LeBlanc, Security Architect for Office, has worked with the ODF TC to ensure that XAdES digital signatures are specified in an interoperable manner. He helped ensure that the standard is consistent with the XMLDsig and XAdES standards and worked to establish an excluded path, so that implementers may maintain unsigned metadata in the archive without breaking the signature. He also helped tighten up the standard to avoid allowing overly broad transforms that can cause security problems.

Going forward, we’ll continue partnering with the ODF TC on the next version of ODF, informally referred to as ODF v-Next. Another colleague of mine, John Haug, Standards Professional,  has been working with the Advanced Document Collaboration SC for the last nine months to help figure out how to address change tracking in ODF v-Next, and he has written a proposal for extending the existing ODF change tracking, including many specific examples of how to handle the use cases that the SC has defined for change tracking.

Even as work on ODF v-Next proceeds in OASIS, ODF 1.2 is expected to be submitted to ISO/IEC for consideration as a new ISO/IEC standard.  This will provide the global standards community with an opportunity to review the standard out of OASIS, provide any comments and suggestions for improvement they may have, and ultimately decide whether to ratify it.  ODF 1.2 may be modified by this work, but we have every reason to be optimistic that ODF 1.2 will be ratified by ISO/IEC at the end of the process.

At Microsoft, we are committed to ensuring that Office customers are able to choose the format that best meets their needs.  As a result, we will continue to give them a range of format options, including ODF.  We are also sensitive to our customers’ need for long term usability and minimal cost when it comes to using new versions of the formats we support.  That is why we are active in the maintenance of ODF, PDF and OpenXML and tend to want to support stable versions of formats that have emerged from the standardization process.

We look forward to continuing our participation in the evolution of ODF. Congratulations to the entire ODF community on the approval of ODF 1.2 as an OASIS standard!

You can find general information about DII on the Open Specifications site, as well as a recap of past interoperability events. (We’ll also be posting some information about upcoming events soon, and I’ll cover that in a separate blog post.)

The implementer notes for Office’s implementation of Open XML and ODF used to be available in an interactive format on the DII site, but we heard feedback from implementers that they wanted them published in an all-in-one PDF that would be easier to work with offline. Those changes are all done, and the PDFs are available for free download on the Microsoft Office Protocols website. There you can find implementer notes for ECMA-376, ISO/IEC 29500, and ODF.

Finally, if you have technical questions about any of the content on these sites, you can get help from our support engineers and others from the implementer community over on the Interoperability Forums.

BFFValidator runs as a command line without a dependency on the Office product. BFFValidator is targeted towards binary file format developers and generates prescriptive failure information when a given file does not conform to the Open Specifications. The tool will generate an XML log file that will correlate the issue with a specific section of documentation, the stream offset where the failure occurred, and other pertinent debug information. To learn more, please check out the BFFValidator reference file on the dowload page and stay tuned for a "How to guide" to be published on MSDN.

If you're working with the binary formats, this is a great tool for assuring the best possible interoperability with other implementations. Check it out, and if you have questions about the tool you can get those answered at the Microsoft Open Specification support alias, dochelp@microsoft.com.

Historically, there have been two popular .NET APIs for Open XML development: System.IO.Packaging (which first appeared in .NET 3.0) and the Open XML SDK, released in early 2007. There’s also a COM-based native packaging API available for non-.NET Windows developers.

The libOPC API is roughly analogous to System.IO.Packaging, in that it’s focused on the details of OPC and MCE (parts 2 and 3 of IS 29500), but doesn’t provide higher-level abstractions for WordprocessingML, SpreadsheetML or PresentationML (as covered in parts 1 and 4 of IS 29500).  I say “roughly” because libOPC doesn’t yet address some of the things that System.IO.Packaging handles (e.g., digital signatures) but does include some more advanced capabilities not available in System.IO.Packaging, such as the opc_generate functionality described below, which is essentially the same as the document reflector functionality of the Open XML SDK.

The key new feature in libOPC is its cross-platform capabilities. If you’re working on a non-Microsoft platform, or working with embedded systems that have limited OS support for XML and ZIP, you now have a very fast, simple API that you can use to implement Open XML read and write capabilities in your applications. And libOPC is designed from the ground up to be wrapper-friendly, for use from programming languages other than C.

The coordinator of the libOPC project is Florian Reuter, a well-known voice in the document formats community through his years of work with the OASIS ODF TC, Ecma TC45, OpenOffice.org and LibreOffice. Florian has worked with both ODF and Open XML, as a Sun employee, a Novell employee, and (now) an independent developer, and this gives him a well-rounded perspective on what sorts of tools can help developers to be more productive when working with XML-based standardized document formats. Microsoft is sponsoring Florian’s contributions to libOPC, and I can’t think of a better person to be leading this project.

The first release of libOPC is mostly about providing abstractions that make it easy for developers to work with OPC packages. There are several basic tasks that you need to do over and over when working with OPC packages, such as finding a relationship by type, determining the target of a relationship, retrieving the content of a part in a stream, and so on. The libOPC API provides simple methods for carrying out these tasks, and the developer doesn’t need to think about details such as how a package’s relationship tree is serialized into various XML parts.

There’s a video on the libOPC Codeplex site in which Florian goes over the basics of what’s available in version 0.0.1. One of the things he demonstrates is opc_generate – a tool that takes an Open XML document as input and generates a cross-platform C program that will re-create that document through use of libOPC. This is a very powerful tool for building customized document assembly systems, because the code it generates can provide a big head start for certain types of projects.

In addition to enhancing developer productivity, opc_generate can enable better collaboration between graphic designers and developers. A graphic designer can create a beautiful document without giving any thought to how to generate that document programmatically, and then a programmer can generate source code for that document instantly, then focus on customizing that code to attach it to data sources as appropriate. My colleague John Haug did an informal test of opc_generate by taking all of the documents in our internal SharePoint document library and generating libOPC code, then running the code to re-generate the original documents. They all worked great, including some quite complex documents.

On the documentation page, you’ll find information about how to use libopc on Linux, MacOSX, Windows, iOS, and Android, as well as demo videos showing libopc in action with WebKit and the iPhone. The WebKit demo uses the Open XML standard itself as an input, so it’s a great example of the raw performance that libOPC delivers.

If you’re familiar with the Open XML SDK, you may wonder whether libOPC will eventually include the sorts of higher-level functionality that the Open XML SDK provides. This is an interesting question, and I’m really looking forward to seeing how this project evolves going forward. Just as .NET developers started with the System.IO.Packaging API and then the Open XML SDK was built on top of it, we may see other higher-level APIs and tools built on top of libOPC. If you have thoughts in this area, or are interested in contributing, please get involved and join the libOPC project on Codeplex.

Eric White posted yesterday release V2 of his simple Open XML document generation system, a high-performance tool for generating customized documents from any data source that can be serialized as XML.

This tool is part of Eric’s ongoing series of posts on generating Open XML WordprocessingML documents. In his approach, you configure the document generation process by creating a template document, then inserting content controls in the template document, and then entering appropriate XPath expressions in the content controls. Those XPath expressions specify where to find the data that will populate the content controls for each custom-assembled document at runtime.

For an example of this approach in action, check out the two-minute screencast on Eric's blog post, which demonstrates generation of 3000 documents in less than 30 seconds. The source code for this system is less than 230 lines total, and provides a great starting point for those who want to do something more complex. For many common applications, however, this example already includes everything you need.

On a related note, Eric has recently started writing how-to articles for the OpenXMLDeveloper.org web site, so if you’re doing Open XML development you’ll want to follow those as well. Many of his articles are inspired by questions posted on the Forums section of that site, so be sure to post your Open XML development questions there, and if the answer merits a more comprehensive response he’ll write up an article illustrating the concepts involved.

I attended the XML Prague conference over the weekend, where I had the opportunity to see some great presentations, meet some interesting people, and drink plenty of Czech beer. The latter being a requirement for any trip to Prague, of course!

The conference was scheduled on the weekend between meetings of the W3C XSL working group last week and the ISO/IEC JTC 1/SC34 working group and plenary meetings that are taking place this week, which made it convenient for SC34 members to attend as part of the same trip to Prague. Today at the SC34 WG4 meeting we have several attendees who were also at XML Prague, including Jirka Kosek (Czech Republic), Mohamed Zergaoui (France), Murata Makoto (Japan), and my colleague Chris Rae and I, who are participating in SC34 as members of the Ecma TC45 delegation.

As you can see from the conference agenda, there were presentations on a wide variety of XML topics over the two days. As noted in the closing keynote (more on that below), there was a general theme of “XML in the browser” that cut through many sessions, as well as sessions on other topics such as EPUB, JSON, DITA, and others. The room was packed, with nearly every seat taken and extra chairs lining the aisle, for the opening session by Norm Walsh (above) that covered the history of HTML and XML and some of the use cases that have been considered by the W3C TAG HTML/XML Task Force. And the room stayed packed, all weekend long, with none of the drop-off in attendance many conferences experience.

After a full day of presentations on Saturday, we all walked up the hill behind Prague Castle to the Strahov Monastery brewery, for dinner, beers, and a demo competition. The view of Prague from the monastery was spectacular:

Sunday’s agenda was kicked off by Michael Kay’s “XML in the browser” presentation, which covered the history of XSLT and proposed how XSLT's declarative event-driven approach could be used to deliver the sorts of interactive functionality that has traditionally been the domain of JavaScript. Saxon CE, which allows you to run XSLT 2.0 in any web browser released in the last year, is now available in alpha release on the Saxonica web site.

There were many other great sessions, and you can see the videos when they’re posted on the web site, and/or download the conference proceedings to get all of the details. WG4 convenor Murata Makoto, wearing his EPUB hat (he is co-convenor of the SC34 ad-hoc group looking into EPUB standardization), provided an overview of EPUB and covered the current status of EPUB3.

Michael Sperberg-McQueen’s closing keynote was a masterly speech. He started with a story of Vaclav Havel characterizing the core of a crisis as being caused by cold impersonal machinery that was set against the aspirations of free-willed individuals. He then used this story as an allegorical backdrop for an exposition of the tension between the cold impersonal interests of the browser vendors and the original aspirations of many in the XML community, building up to an exhortation to put XML in the browser “whether they want it there or not.” I can’t describe it in a way that does it justice, but the video should be on the XML Prague web site soon, and it’s worth checking out.

Overall, XML Prague was a great learning experience for me, a 2-day crash course in the state of XML technologies and various creative ideas about how those technologies will evolve going forward. Thanks to Mohamed and Jirka for suggesting we attend!

I mentioned in that post that “we would welcome the opportunity to work with other TC members to improve ODF’s ability to handle tracked changes.” I reiterated that later the same year, when I spoke at OOoCon 2009 in Orvieto – in response to questions as to whether Microsoft would take an active role in helping to improve change tracking in ODF “v-Next.”

The time has come, thanks to the creation of the ADC SC (Advanced Document Collaboration sub-committee) of the OASIS ODF technical committee. The Statement of Purpose for the ADC SC emphasizes the importance of change tracking:

The initial and highest priority for the Subcommittee will be change tracking. Reliable and user-friendly revision management is critical for professional document workflows in corporate and public sector environments, and as such an important feature of Open Document Format.

Change tracking has indeed been the primary topic of discussion within the ADC SC, and per the “Roadmap for ADC SC work” on the ADC SC wiki, proposals may be submitted by SC members through the end of March.

Over the last couple of months, my colleague John Haug has been working closely with change tracking experts on the Word team to put together a proposal, and today we have submitted that proposal to the ADC SC. As stated in the Introduction to that proposal:

The basic idea behind this proposal is to preserve the existing ODF change tracking and extend it to support missing use cases. We believe this best preserves backward compatibility with existing implementations and makes best use of existing patterns and code. It also maintains the current model of ignoring changes for applications that do not support change tracking. A relatively small number of additions to the existing capability can enable a core set of previously unsupported use cases.

This proposal focuses on the need for robust change tracking in text documents, the primary user scenario based on customer feedback. In addition to supporting new use cases as described below, we propose expanding the prose that describes the current change tracking support to make its intended use and scope more explicit.

Our proposal is informed by our experience in building change tracking functionality over the course of several product releases, as well as our participation in the ADC SC discussions to date and our analysis of the existing ODF change tracking functionality when we implemented ODF support. We have identified the specific use cases that we feel need to be addressed, and for each of them we’ve provided examples of the ODF markup before and after the change, along with an explanation of our proposed approach. Of course additional use cases can also be addressed as the ADC SC’s work proceeds. And we look forward to digging in and understanding all other proposals submitted to the SC – the quality of the final work product will benefit from considering every submission carefully.

We have also included questions to the SC, and multiple options in some cases. As covered in the proposal itself, we see it as a starting point:

We believe that a robust discussion among implementers is necessary to create a good final design, we are open to changes in everything from design and applicable scope to element and attribute names and this document is provided as a vision for the general architecture and a seed for such a discussion. We expect that the mechanical details can be worked out in a straightforward manner once the subcommittee agrees on a broad architectural approach.

I’m excited to see the change tracking discussion taking place in the Advanced Document Collaboration SC, and like Rob Weir I hope that it will be every bit as successful as the Open Formula SC and Metadata SC have been in improving the ODF standard. It’s great to see the ODF community collaborating on addressing this critical area, and I look forward to some good discussions in the SC as the process moves forward.

Please get involved if you have an interest in this area, or expertise to contribute. Any OASIS member may participate in the SC, and if you’re not an OASIS member it’s easy to join.

Standards maintenance processes add new functionality slowly and carefully, but the products that implement those standards may progress much more rapidly at times, to address new needs in the marketplace, new technologies, and ever-changing market dynamics. Is it possible to “loosely couple” a standard and its implementations in a way that allows implementers to deliver innovations while remaining interoperable and conformant to the standard?

I recently had the opportunity to work with a few of my Microsoft colleagues to put together a whitepaper (attached below) that touches on some of these issues. The paper was delivered at an interoperability event in Beijing earlier this month, and it provides an overview of MCE (Markup Compatibility and Extensibility, Part 3 of ISO/IEC 29500) and how it has been used to enable interoperability between various versions of the standard and its implementations.

There are several other strategies that have also been used to address this challenge in document formats:

• A standard may allow foreign markup to appear inline. This gives implementers the freedom to innovate in any way they’d like, but it also means that documents can’t be validated against the published schemas in the standard. Removing the foreign markup is a necessary first step, and this can be difficult to do efficiently in some programming environments, or may be handled inconsistently across multiple implementations.
• A standard may have defined extension points: places in the schema where an implementer can add non-standardized content to address new requirements, perhaps in the context of an xsd:any element in an XML schema. This approach makes it possible to validate the document against the schemas in the standard, but innovation can only occur in the places where extension points are defined.
• A standard may not allow any extensions at all, and all innovation must occur through the standards maintenance process. This solves the validation problem, but is not always a realistic approach because of the slow pace of standards maintenance.

MCE is designed to address the limitations of some of these alternative approaches, and it has shown great promise for the types of issues that arise in the evolution of document formats. Each implementer can create their own MCE-based extensions, and documents containing those extensions can be gracefully handled by products that support varying sets of extension functionality (or none at all). MCE is not an unproven theory, but rather an approach informed by the things Office engineers learned from many years of dealing with these same sorts of issues in the binary file formats. We used MCE for all of the new innovations in Office 2010, and this has enabled interoperability between Office 2010 and other applications that don’t have that new functionality, as covered in the attached whitepaper. (For further information on Office’s use of MCE, see also the Microsoft Office File Formats Documentation on MSDN.)

As time goes on, of course, any new features used by more than one implementation should probably be integrated into the standard. The question of how and when to standardize these is one that WG4 have been discussing a lot this year. In October, the Japanese National Body submitted a proposal for a new multi-part standard containing standardized MCE-based extensions to IS 29500. Allowing standardized extensions inside MCE will permit implementers of the core IS 29500 standard to remain conformant while still allowing new features to be standardized.

Document Format Compatibility and Extensibility.pdf

What to Expect:

• Presentations on the Binary File Format Open Specifications (Word, Excel, PowerPoint, PST, Graphics, Document Cryptography etc.)
• 1:1 Sessions with product development team members
• Access to Office's Binary File Format validation tool for Word, Excel, and PowerPoint
• Interaction with Microsoft Support
• Breakfast, lunch, and dinner will be provided on both October 19th and 20th

Space is limited, so register early. To learn more about the event or to register, please contact Michael Bowman.