Office Adopts New Windows Display Technology
Office 2013 has undergone a substantial shift to a relatively new display facility, Direct2D, and a new text facility, DirectWrite. These are the display facilities that are used on Windows Phone 8, the new Windows RT slates, and optionally on Windows 7 & 8. Up through Office 2010, the Windows Graphics Device Interface (GDI) and complex script support by Uniscribe were used. RichEdit 8.0 interfaces to both kinds of display technologies, but a given RichEdit instance uses only one of them. This post describes some of the advantages of using the new display facilities along with some of the problems we encountered in supporting them. One of the challenges was ensuring that math display wasn’t broken in the Direct2D/DirectWrite world. Another was to implement higher-level BiDi protocols using DirectWrite, which was designed to support the Unicode Bidi Algorithm (UBA) only.
Direct2D and DirectWrite are modern components that can take advantage of graphics acceleration hardware. DirectWrite supports better text placement using subpixel, device independent layout along with ClearType or other antialiasing methods. So in principle, it’s a big win to move forward to these components. Meanwhile GDI and Uniscribe have been around for years and have been taught to handle all sorts of nonobvious problems. It’s not easy to produce new code with their generality. Furthermore, RichEdit needs to be able to use either DirectWrite Version 1.0, which ships with Windows 7, or the more powerful DirectWrite Version 2.0 that ships with Windows 8. As a result, it was a bit of a tour de force to generalize RichEdit to interface smoothly to these three different display models. The factorization we’ve done has simplified getting RichEdit onto other platforms as well, such as the Android and iOS operating systems.
As an example of GDI tricks, if you specify the charset in creating a font object, GDI will ensure that you get a font that handles that charset. Admittedly charsets cover only a subset of the world’s languages (no Indic, Syriac, etc.), but they do cover many important languages. Another trick is if a character is an end-user-defined character (EUDC) in the Unicode Private Use Area, GDI will ensure that you see a glyph by searching through registered EUDC fonts. These characters are not defined in the Unicode Standard, so you can’t use them reliably for text interchange. But they are popular in CJK locales and a given machine may have fonts with the glyphs that the user wants.
Uniscribe was developed alongside LineServices to support both the UBA and rich-text higher-level protocols. This approach is important for RichEdit rich-text instances, since they use the keyboard and math higher-level BiDi protocols. In addition, digit substitution support requires breaking out numeric runs, which Uniscribe does. DirectWrite creates runs giving the same script value to alphabetic characters and numeric digits that are considered by context to be part of that script. The method we found to identify the digits in an Arabic run given by DirectWrite, for example, is that they are given a BiDi level of 2, whereas the strong RTL characters have a BiDi level of 1. We’ve dreamt up a number of other ways to ensure that Latin letters are rendered LTR (except in plain text with an RLO override embedding). And then you get little problems such as two Thai characters surrounding the Arabic comma U+060C. Since U+060C is considered common to several scripts, DirectWrite happily puts it in the Thai script run. But a Thai font cannot handle it. Font fallback is one way to see a glyph; RichEdit breaks the run and uses an Arabic font.
For the new immersive environment on slates and the Windows Phone 8, not only are GDI and Uniscribe absent, so are the functions handled by the venerable user.dll and OLE dll’s. The user.dll program library includes the OS SendMessage, MessageBox, CreateWindow, etc. In the immersive environment, RichEdit is only available in windowless mode. The client can still send RichEdit messages via the ITextServices::TxSendMessage() method. The advantage of dropping user.dll and the OLE dll’s is the relative simplicity of the model and the correspondingly reduced size. But doing so omits significant functionality, at least in the initial version. At the same time, the touch functionality is dramatically improved on Windows 8 and in the immersive environment.
OneNote 2013 is the first Office application that has a fully immersive version, thanks, in part, to the fact that RichEdit 8.0 can run in a fully immersive environment. OneNote 2013 has the math display facility and you can edit math using alt+= to insert math zones and the linear format to enter equations. One problem with OneNote 2013 is that for some zoom factors, horizontal lines are snapped vertically to the wrong pixel boundary. There wasn’t enough time to fix this problem for OneNote 2013, so you may notice some square roots with misaligned bars.
Another side effect of these huge changes is that we have had no time to improve the Office math editing and display facility for Office 2013. The main accomplishment math-wise is not to lose any math functionality, aside from misaligned bars in OneNote. Hopefully next time we can add some cool math features.
Comments
Anonymous
July 29, 2012
It is really sad to hear there are no improvements in the math department. When you say next version, do you refer the official release or Office 14? Thanks.Anonymous
July 29, 2012
No equation numbering in Office 2013... Why am I not surprised??Anonymous
July 29, 2012
Wow, the year is 2012 and we still can't number an equation. Maybe I missed it, but the first thing I did after downloading the preview was to excitedly open Word and bring up the Equation Ribbon. No Dice. I dream that maybe my grandchildren will enjoy such a feature. Office 2047?Anonymous
July 30, 2012
At this point I am really lost. Equation numbering is not some nice, optional feature. The lack of it makes the otherwise fantastic equation editor in Word pretty much useless for any semi-serious work. I am convinced that way more than half of all documents that have equations in them actually number them. If you type equations, most people have to number them. I did have some understanding that MS didn't manage to squeeze it in for the initial version. But not even TWO releases later? Six years later?? Looking back, it seems the decision to include equations in OneNote was a big, big strategical mistake. Clearly by now the team is so busy with just keeping up support in various versions that there is no room left to make the thing useful in the app that arguably needs it most, Word. Do people use equations in OneNote a lot? It is really frustrating to see a team that produced such a wonderful piece of code be hold back by the typical MS tax of needing to integrate with things that no one needed. It would have been WAY better to first fix equations in Word (by giving them numbers) before adding them to all the other apps in Office. Major, major fail on the part of the program management of this feature. I should also give you my perspective: I teach at a large research university in the US. I have a lot of contact with a large and diverse group of people that uses equations. Every single person I talked to in the last couple of years said they aren't using the build in equation editor because it doesn't do numbers. This goes from students, grad students to professors.Anonymous
July 30, 2012
I personally know how important equation number is, having coauthored over a 100 papers in theoretical physics and three books on laser physics. I pushed really hard to get equation numbering into Word 2007, 2010, and 2013 and hopefully it'll happen next time. I think people here think it's harder than it really is, what with most of the infrastructure already in place. By the way, in OneNote 2013 have you tried typing into a math zone something like E=mc^2#(3)<enter> ? You might be surprised! Many thanks for your feedback and that of many others who have commented or contacted me. It's comments like David's that convince people to adjust priorities.Anonymous
July 30, 2012
I find Equations in Power Point and One Note equally important. MS has enough programmers to make it right in every application. I wish the did it in Office 2013.Anonymous
August 05, 2012
If I understand it correctly, this version of Office 2013 is "only" the Consumer Preview (=Beta). Later, there should be the Release Preview (=Release Candidate) and then finaly the RTM version. So it still time for progress and improvements (for adding features). But unfortunately, it seems that the Office Word team has no effort to add new features (which are connected to text/math typography...). For comparison, the Internet Explorer team added many new features in each new Preview of Internet Explorer each 12 weeks!! The Internet Explorer team has shown the real progress of IE10 development and improvements. So, it is good reason to release the new version of Internet Explorer (IE10). And I will upgrade from IE9 to IE10. But what is the reason to release the Word 2013? Without new (typographical) features (and equation numbering!!) I do not need to upragde from Office 2010 to Office 2013. Sorry, but this is my opinion.Anonymous
August 06, 2012
A good summary of Word 2013 enhancements is given in the Word blog post blogs.office.com/.../introducing-word-2013.aspx. One very cool thing is the great integration with the cloud. With it, you can access your documents from anywhere and collaborate better than ever before. Also the touch and Windows 8 integration is excellent.Anonymous
August 13, 2012
I have known about the enhancements of Word 2013 desribed on the Word blog. Primarily I though that the main improvements of MS-Word (as a text and word processing application) should be in the area of typography. After Word 2007 introduced the new and advanced math editing and displaying feature, I believed that after similar improvements of ordinary text displaying and editing the MS-Word would be better for scientific text preparation than LaTeX. But it seems that the priority has changed..., changed to the cloud and so on. I work with sensitive data so I will not used cloud for saving the MS-Office documents. The math editing and displaying facility of Word shows the great potential of this application in the field of math typography and this advanced philosophy should not be limited only to math.Anonymous
August 17, 2012
Microsoft announced this week that Office 2013 can open PDF files in Word and save them as Word documents (see [1]). Will math expressions in PDF docs be opened/saved as OMML expressions? Can you say more about it? [1] blogs.office.com/.../the-new-office-expands-file-format-options.aspxAnonymous
August 20, 2012
PDF Reflow, the official name for the PDF to DOCX converter, currently leaves formulas as plain text or renders them as images. The PDF format usually does not contain information indicating when text is mathematical. All that is stored in PDF is absolutely positioned text, images, and graphics. Moreover sometimes PDF does not even contain the underlying Unicode values for mathematical text, so that when you copy text from the PDF, the result is messed up, although the original PDF looks correct. PDF Reflow tries to detect mathematical text and to render it with the correct semantics; the formula structure is preserved, but the text cannot be edited. When mathematical text is not recognized as such, PDF Reflow imports the same text that you get otherwise by copying the content from the PDF. A blog post was written recently on the Word blog related to PDF Reflow: blogs.office.com/.../unlock-pdfs-with-the-new-word.aspx. Ideally mathematical text in PDFs would be recognized and imported as OMML (Office MathML). Then you could edit it and use it in symbolic and numerical computations. Maybe that will happen some day. The team that made PDF Reflow belongs to the same organization that made the Windows Math Input Panel, which recognizes hand-written equations and converts them to MathML. Conceivably some of the Math Input Panel's logic could be applied to recognizing formulas in PDFs.Anonymous
October 21, 2012
The comment has been removedAnonymous
October 21, 2012
OK, I have no correct myself, no greyscale AA in OneNote, but now why in Word, hopefully just a bug?Anonymous
March 18, 2013
The comment has been removedAnonymous
April 16, 2013
Please, can I have a question? What is the purpose of CTRL+ALT+ENTER keyboard shortcut in MS-Word? What should it do? Thank you.Anonymous
April 16, 2013
I don't think anything is associated with CTRL+ALT+ENTER. You can use it for your own purposes. Here's a list of Word hotkeys: allhotkeys.com/microsoft_word_hotkeys.htmlAnonymous
January 29, 2014
Given that OneNote is a RichEdit control why does it not support copying and pasting rtf? Thank you.Anonymous
January 30, 2014
OneNote uses RichEdit for editing and display on a paragraph level. It doesn't use RichEdit for list numbering or tables. And it doesn't use RichEdit for file I/O since a file needs to deal with list numbering, tables, and other features of a OneNote page. So OneNote uses HTML and its own XML format. But it could in principle support RTF, especially at the paragraph level. I'll ping my OneNote friends.Anonymous
March 29, 2014
I always wondered why the text in Onenote 2013 looks so much better than Word 2013. That could explain it. Is there no way to turn directwrite on for Word 2013 ?Anonymous
June 01, 2014
The comment has been removed