Murray Sargent: Math in Office

Math Autocomplete

MurrayS3 — Fri, 14 Jun 2019 20:59:06 GMT

Users of OfficeMath note that entering math symbols using a keyboard can be hard because they don't know the symbol keywords to type and/or the control words are long. To ease these problems, we create a math autocomplete listbox that displays the control words that match what the user types. The user can then use ↑↓ and Enter, mouse or touch to enter the desired symbols. For example, \in inserts ∞. You might not know that the TeX control word for ∞ is \infty, rather than \infinity, but you don’t have to know this with the autocomplete listbox. This functionality has been shipping in the Office RichEdit in a sharable way for a while. It creates the autocomplete lists using the RichEdit internal math autocorrect table, so it doesn’t show entries added by the user to the Office math autocorrect dictionary. An example using our handy test program reitp2.exe is Here the most common entry is selected when the listbox is created so that you can insert it directly using the Enter key. This works well for control words for math operators and Greek letters. In addition, Unicode contains many math alphabets as described in Section 2.2 of Unicode Technical Report #25, Unicode Support for Mathematics. It’s not practical to display all 52 control words for a given math alphabet. Instead, the listbox contains a single entry, such as “\scriptH ℋ”when the user enters \sc. This aids discovery of the math-alphabetic control words. If a letter other than ℋ is desired, the user can type the whole control word, such as \scriptQ, whereupon the listbox entry becomes “\scriptQ 𝒬”. In addition to \scriptX, where X is any one of A..Z and a..z, there are \frakturX, \doubleX, \boldX, \ssX (sans serif X), and \ttX (monospace, “typewriter type”). The math autocomplete feature strives to improve discoverability and input speed. Accordingly, to speed up entry of the math alphabetic symbols, the facility offers the abbreviated versions \bolX for \boldX, \scX for \scriptX, \douX for \doubleX, and \frakX for \frakturX, where X is any letter other than the corresponding letter in the full name. The abbreviated versions correspond to cases where the only partial match is a math alphabetic, which is a useful user cue to use an abbreviation. Math-bold and math-italic alphabets can also be accessed using the usual bold and italic hot keys and ribbon buttons. There are the additional math-alphabetic control words \ssbX (math sans-serif bold),\ssiX (math sans-serif italic), and \ssbiX (math sans-serif bold italic) that work but aren’t currently included in the math autocomplete feature. Here’s an example of the alphabets for the letter H.

\boldH	𝐇
\doubleHHH	ℍ
\frakturH	ℌ
\scriptH	ℋ
\ssH	𝖧
\ssbH	𝗛
\ssiH	𝘏
\ssbiH	𝙃
\ttH	𝙷

There are no control words for math-italic characters since they are used by default if the italic button is active. This is usually the case for math typography. In chemical formulas, upright letters are usually used as in C₆H₁₂O₆ for glucose. There are several math Greek alphabets: bold, italic, bold-italic, sans-serif bold, and sans-serif bold italic. But there aren’t any alphabet control words for these alphabets or for the bold, double-struct, sans-serif, sans-serif bold, and monospace digits. You can Alt+x them into Word or RichEdit if you know their Unicode code points (U+1D6A8..U+1D7FF). By default, lower-case Greek letters are math-italic in math zones and upper-case Greek letters appear upright. If you enter \th, the listbox includes the preselected entry "\theta θ" and you can hit Enter to get 𝜃. There are other math alphabets that need to be added to Unicode, such as “calligraphic”. For that alphabet we can use \calX. But currently \scriptX may display a calligraphic glyph as in the Cambria Math font or a fancy script glyph as in the STIX Math font. So \scriptX is ambiguous and we’ll need a new control word to specify fancy script unambiguously.

Syncing Braille Cell buttons with OfficeMath

MurrayS3 — Wed, 29 May 2019 01:40:55 GMT

A typical refreshable braille display has a row of braille cells with a button above each cell as illustrated in the figure of a HIMS Braille Edge 40. The purpose of a button is to move the insertion point (IP) to the character or abbreviation represented by cell below the button. This post discusses how pressing a cell button associated with a math braille cell can move the text selection to the closest matching character position (cp) in the corresponding math zone. The algorithm accounts for the many-to-many relationship between braille cells and OfficeMath character positions (cp’s). A single math symbol may be represented by multiple braille cells (‘=’ is represented by the four braille cells “⠀⠨⠅⠀”) and characters appear in math zones that aren’t shown on a braille display (there’s a start delimiter for 𝑎² that has no counterpart in the corresponding braille “⠁⠘⠆”. This is discussed further in Math Braille UI and below. And there are other cp differences such as hidden text as discussed in the post Using MathML-Based Speech to Edit Math in Different Math Models. In math braille, abbreviations are not used, a simplification that allows Nemeth math braille to be embedded in the braille for any language.

Synchronization algorithm

The algorithm given here to synchronize the math-zone insertion point with a braille-cell button is analogous to the algorithm for finding the cp from a mouse-button hit on a line of text. For the latter, you measure the text until its width equals or passes the hit position. For the braille-cell button press, you

Get the braille for a whole math zone or some valid selection therein. Note that if you select the start delimiter of a math object, such as a fraction, the selection is automatically extended to include the entire fraction.
Press a braille cell button and get its offset into the math braille string. I’ve found that Narrator reports the offset via ITextRangeProvider::Move() and Select() calls.
Recreate the same math braille noting the math-zone cp where the length of the evolving braille string equals the braille offset for the braille-cell button.
Adjust that cp so that it is a valid IP in the math zone.
Set the math-zone IP equal to the adjusted cp.
Once again recreate the math braille to update the braille cell IP (⣀) corresponding to the new math-zone IP.

Braille-cell buttons versus ←/→ key navigation

The left and right arrow keys can traverse every valid cp in an OfficeMath math zone, while the braille-cell buttons only have access to cp’s that have braille counterparts. Let’s consider the two cases mentioned in the introduction. The ‘=’ sign is represented by “⠀⠨⠅⠀”. If you press the button above the starting braille space (U+2800), the corresponding math-zone cp should be in front of the ‘=’. Remember that insertion points in rich text like math zones are in between characters, not on top of characters. If you press a button above any of the remaining three braille characters, the math-zone cp (IP) should follow the ‘=’. Meanwhile a single → key press in front of the ‘=’ moves past the ‘=’. Now consider 𝑎². In a math zone this is represented by {𝑎|2}, where { stands for the superscript-object start delimiter, | is the argument separator, and } is the superscript-object end delimiter. In RichEdit these delimiters are given by the Unicode characters U+FDD0, U+FDEE, and U+FDEF, respectively. The start delimiter has no counterpart in the corresponding superscript braille string “⠁⠘⠆”. So, pressing the braille-cell button above the ‘⠁’ could set the IP in front of the superscript object or at the start of the superscript-object base (in front of the 𝑎). Structured navigation that selects the whole superscript object or just the ‘⠁’ could be used to choose between these two possibilities. Meanwhile a single → key press in front of the superscript object moves to the start of the superscript-object base. The subscript object 𝑎₂ also appears as {𝑎|2} in the OfficeMath backing store. The object type is specified in the character formatting of the start delimiter. But in Nemeth math notation, 𝑎₂ is given simply by “⠁⠆”. The subscript operator ‘⠰’ is implied. In computer braille notation, this is “A2”. So, for such simple subscript objects, the start, middle, and end points are all ambiguous. For example, in the string “⠁⣀⠆”, with ‘⣀’ marking the insertion point, the next character typed could be at the end of the subscript base (first argument) or at the start of the subscript (second argument). Math Braille UI disambiguates the two by including a dot 8 (‘⠠’) in the braille-cell character(s) in the object argument that contains the insertion point. If the IP is at the start of the subscript, the math braille for 𝑎₂ is “⠁⣀⢆”, whereas if it’s at the end of the base, the math braille is “⢁⣀⠆”. Similar cases include the ends of integrands, summands, accents and math-function objects as well as optional arguments like missing integral limits and radical indices (a square root has a missing radical index). A braille user should have at least three kinds of OfficeMath navigation: braille-cell buttons and vertical scrolling, structured navigation, and the ←/→ keys. Currently, I have to use a keyboard for structured navigation and the arrow keys. But we should be able to assign keys on refreshable braille displays to perform such navigation in addition to the braille-cell button navigation.

Math Zone Navigation

MurrayS3 — Tue, 30 Apr 2019 18:51:20 GMT

The post Using MathML-Based Speech to Edit Math in Different Math Models discusses the need to navigate math zones in the edit space rather than in a MathML copy of the edit space. This need arises for editing since the navigation location must be synchronized with the edit selection. It’s also important for defining the selection bounding rectangles even if editing isn’t allowed. The present post compares the math-zone edit navigation in Microsoft Office apps to the structured child/parent/next/previous tree navigation provided by MathPlayer and other systems. Combining both approaches results in rich navigation and editing experiences for blind and sighted users alike.

Edit navigation

The OfficeMath math zone represents math as a “flattened” tree, in which nested math objects such as fractions and matrices have start and end delimiters. This allows character navigation to traverse every character in a math zone. Typing the → key at the start of a fraction moves into the numerator. Typing → at the end of the numerator moves into the denominator. Typing → at the end of the denominator moves out of the fraction. The → and ← keys let you “escape” from a math object (go to the next higher level in the tree) and to enter a math object (go down a level into the tree) as well as moving past characters. Typing Ctrl+→ moves by siblings. So, typing Ctrl+→ at the start of a fraction moves past the fraction to whatever follows the fraction. A Ctrl+→ at the end of the denominator moves out of the fraction, so like the → key, Ctrl+→ can escape from down inside an object (go to the next higher level in the tree). But it cannot go down into a lower level of the tree. Both Ctrl+→ and → leave the selection as an insertion point (IP), ready for inserting more characters and math objects. Shift+Ctrl+→ and Shift+→ select the next object or character. The Ctrl+← and ← keys work the same ways but move toward the start of the math zone. (Note that Word hasn’t yet implemented this Ctrl+→ and Ctrl+←behavior, but OneNote and PowerPoint have). All eight editing hot keys ([Shift+][Ctrl+]←/→) move out of the math zone when they come to the end of the math zone. The keys are usually geometric but are logical as well in that → moves from the end of a numerator to the start of the denominator. Such movement isn’t geometric since the motion is down and to the left rather than to the right. At any time, the user can enter and delete characters. If the selection is nondegenerate, entering a character replaces the selection and results in an insertion point (IP). If the selection is already degenerate, e.g., an IP following Shift-less key motion, the character is entered at the IP. With OfficeMath speech, you can use this kind of navigation while entering and editing equations with a keyboard. You don’t need to see the screen. As far as I know, the Office apps Word, Outlook, PowerPoint, and OneNote are the only major apps that let blind users enter and edit equations using only speech and a keyboard.

Structured navigation

Contrast these edit movements with structured Parent/FirstChild/LastChild/Next/Previous tree navigation. This structured navigation always selects a character or object unless it ends up inside an empty argument. At the start of a fraction, FirstChild selects the numerator and LastChild selects the denominator. If the numerator is selected, Next selects the denominator. But unlike Ctrl+→, another Next does nothing since a fraction only has two arguments. To get out of the fraction, you enter Parent and then Next to go to the next sibling of the fraction. If an argument is only partially selected, then Next goes to the next sibling in the argument. If the last sibling in an argument is selected, Next does nothing. Entering Parent selects the whole argument and another Parent selects the math object. The highest level in the tree is the whole math zone and structured navigation commands won’t move outside the math zone. If a selection consists of multiple characters, e.g., the “sin” in the math function object “sin 𝑥”, FirstChild selects the ‘s’ and LastChild selects the ‘n’. Next/Previous select the next/previous character, respectively, unless the last/first character is selected (in which case Next/Previous do nothing). In this way, you can examine every character in the math zone using structured navigation. Possible hot keys for Parent, FirstChild, LastChild, Next, and Previous are Alt+↑, Alt+↓(or Alt+Home), Alt+End, Alt+→, and Alt+←, respectively. Since these differ from [Shift+][Ctrl+]←/→, both kinds of navigation can be used together. It’s desirable to have corresponding navigation buttons on refreshable braille displays. While structure navigation takes more keystrokes than edit navigation, it’s valuable for understanding complicated equations, particularly if you cannot see the equations easily and are navigating with the help of math speech.

Conclusions

Ideally for editors like Word, Outlook, PowerPoint, and OneNote, both kinds of navigation would be available. If using structured navigation, you want to enter a character at the end of the current selection, type → to collapse the selection to an IP at the end and type the character. If you want to enter a character at the start of the current selection, type ← to collapse the selection to an IP at the start and type the character. With these techniques, visually impaired users can have a rich navigation experience and all users can enter and edit equations easily.

Using MathML-Based Speech to Edit Math in Different Math Models

MurrayS3 — Thu, 28 Mar 2019 02:14:29 GMT

This post discusses how an Assistive Technology program (AT) can use Presentation MathML to create consistent speech for editing equations created with different math models, such as OfficeMath and MathType. A goal is to make the speech and editing experience be as similar as possible, even though the underlying math models differ in significant ways. An important aid for editing is that the editor handle navigation so that the insertion point (IP) and speech are synchronized. When navigation occurs in a MathML copy of the math zone, editing isn’t possible unless there’s a way to convert a location in the MathML copy to the corresponding character position (cp) in the document. Such synchronization is also needed in calculating the bounding rectangles of the math being spoken. Math keyboard input is facilitated by sophisticated input methodology, such as special hot keys, autocorrection, and autocompletion. The AT should not attempt to handle math keyboard input. The post Speaking of math… describes two granularities of math speech: coarse-grained (navigate by words—siblings), which speaks math expressions fluently in a natural language, and fine-grained (navigate by characters), which reveals the content at the insertion point (IP) in enough detail to enable unambiguous editing. It seems clear that an AT can generate the same coarse-grained math speech from the MathML for an equation regardless of the underlying math model. The question arises as to whether the fine-grained math speech can also be the same for different math models.

Math speech generality

To create math speech for all math models, the MathML and the speech generated therefrom need to be rich enough semantically to describe the union of all arguments of all math objects (fractions, subscripts, integrals, math functions, etc.) of the various math models. The post Integrands, Summands, and Math Function Arguments compares some math objects for OfficeMath, Presentation MathML, Content MathML, [La]TeX, MathType/Equation Editor, and Nemeth math braille. To illustrate one difficulty, Presentation MathML, LaTeX and Nemeth braille don’t have explicit N-ary elements, while the others do. If the same fine-grained math speech is to work for all models, they all need to supply MathML that lets the AT announce that the insertion point is at the end of an integrand, for example. Another case is the OfficeMath math-function object which has a function-name argument and an argument for the function. For example, in memory sin 𝑥 is stored as a math-function object with the name “sin” and the math argument 𝑥. It’s important for fine-grained speech to announce, “end of function name” when the user navigates past the ‘n’ and “end of argument” when the user navigates past 𝑥 but not yet out of the math-function object. That notifies the user that subsequent keyboard input will be in the function name or argument, respectively. The user might want to change sin 𝑥 to sin 𝑥², for which the math-function argument is 𝑥² and thereby not mean (sin 𝑥)². Math semantics are important for correct math speech. For example, most superscripts represent raising a base to a power, so that speaking 𝑎² as “a squared” is correct. But in tensor analysis, superscripts are used as indices and 𝑎² should be spoken as “a superscript 2” or “a sup 2”. Presentation MathML 3.0 doesn’t have a way to distinguish between these cases. Adding new MathML attributes could provide a concise way to convey the semantics for speech. Alternatively, the tag could be included with the corresponding Content MathML, but that approach is probably too involved for most ATs.

Differences in MathML and OfficeMath models

To illustrate differences in computer math models, consider how MathML, MathType, and OfficeMath represent sin 𝑥. In the PowerPoint and RichEdit OfficeMath memory layouts, sin 𝑥 appears as sin𝑥. Here the Unicode character is the math-object start delimiter, the is the argument separator, and the is the math-object end delimiter. Word also has such delimiters but with different values. Starting at the , each → arrow key moves past one of these characters. In a math model that doesn’t have the math-function object, no → arrow key is needed to move to the start of the function name or to move out of the math-function argument. That’s a basic difference in UI between models that affects fine-grained math speech. It doesn’t affect coarse-grained math speech, which in English is “sine x” for all math models. Ideally the Presentation MathML for sin 𝑥 is

sin&2061;𝑥

How do you relate a position in this MathML to the corresponding position in the OfficeMath memory? Do the and each have a character position (cp)? They do for OfficeMath, but not for MathType. Are sin and 𝑥 separated by a character? They are in OfficeMath, but not in MathType. MathType represents the difference between the sin and the 𝑥 by character formatting and doesn’t use object delimiters for math functions. Another cp mapping example is ab. In MathML, there’s nothing between the numerator a and the denominator b, while in the OfficeMath backing store, there’s the U+FDEE argument separator. In a MathML model, when at the start of a the → arrow key might move directly into the denominator, while in OfficeMath it moves to the end of the numerator, allowing the user to insert characters there. It takes an additional → arrow key to move to the start of the denominator. In addition to location differences in MathML and math-zone spaces, there are text-length changes resulting from automatic conversions of ASCII and lower-case Greek letters to math-italic letters, hidden text, revision marks, and special objects not representable in MathML such as images, hyperlinks, and fields. So, mapping from MathML space to editing space needs special assistance.

Navigating in MathML space

MathPlayer uses a MathML representation of an equation and navigates in the abstract MathML space, not the editing space. Hence it needs a way to transfer MathPlayer locations to the user selection for inserting/deleting/selecting text and displaying bounding rectangles. In principle, the MathML writer can create a cp array indexed by the MathML-tag index. Every MathML tag would have an entry in the array, including all closing tags. Then a new UIA method could allow an AT navigating in the MathML space to set a client selection end to the cp for the n^th tag. In particular, the AT could set the edit insertion point and bounding rectangles corresponding to locations in the MathML space. This approach requires that the AT keep track of the MathML tag indices. The post Math Accessibility Trees compares a display tree to a semantic tree for the equation

This equation appears in Nemeth braille as

⠹⠂⠌⠆⠨⠏⠼⠮⠰⠴⠘⠆⠨⠏⠐⠹⠨⠈⠈⠙⠨⠹⠌⠁⠬⠃⠀⠎⠊⠝⠀⠨⠹⠼⠀⠨⠅⠀⠹⠂⠌⠜⠁⠘⠆⠐⠤⠃⠘⠆⠐⠻⠼

There are nodes in both trees to attach the MathML tag indices to. Each node needs to cache the tag indices for the start and end tags that delimit the node. This approach isn’t likely to be implemented by Microsoft Word since Word creates MathML by converting the OMML for the requested math to MathML using OMML2MML.xsl, a process that doesn’t keep track of MathML tag cp’s. Word would have to create a native MathML writer to implement a tag-cp mapping array. Such an array could be implemented in the RichEdit native MathML writer, thereby enabling tag-cp mapping in PowerPoint and OneNote. The RichEdit MathML writer was created before the OfficeMath build up/down facility was written. That facility uses a subset of the TOM interfaces that is implemented by Word and OfficeArt enabling them to use the facility. The RichEdit MathML reader also uses the TOM subset and in principle could be used by Word instead of converting MathML to OMML using MML2OMML.xsl. In contrast, the RichEdit MathML writer is pretty RichEdit-specific. Math zones can be copied from Word into RichEdit, but the original Word cp’s aren’t copied.

Navigating in editing space

If navigation is done in the editing space, the post Editing Math using MathML for Speech describes two ways for an AT to produce fine-grained math speech from MathML: 1) the MathML contains an element that gives the explicit math speech for the content at the insertion point, or 2) the MathML represents the math object in which the insertion point is located and includes an element identifying the insertion point. The first approach typically leads to different speech for different programs. Character navigation in the editing space depends on the order in which math-object arguments appear in memory and how the arrow keys are handled. All math models put the numerator of a fraction before the denominator and can in principle be traversed using the ← and → arrow keys. MathType puts the integrand of an integral object first, followed by the lower limit and then the upper limit, while OfficeMath puts the limits first followed by the integrand, which is the visual order. Also, MathType uses ↑ and ↓ to navigate between N-ary object arguments vertically, perhaps because the arguments aren’t located in visual order and the ← and → arrow keys are strictly geometric and don’t traverse every character in an equation. In OfficeMath, the ← and → arrow keys move logically, rather than purely geometrically, and traverse every character in an equation. For example, the → arrow key at the end of a numerator moves to the start of the denominator. The MathML puts the index after the radicand, while OfficeMath puts it before (in display order). The most straight-forward approach is to navigate in the editing space as done with character and word (sibling) navigation with OfficeMath speech and Narrator. Then the insertion point is always synced to the speech. Characters are entered and deleted correctly, and information that MathML doesn’t represent is retained. MathPlayer was designed to explore equations and wasn’t intended to be used for creating and editing equations. The MathType editor has no accessibility support so editing equations using speech wasn’t a design scenario. In contrast, it was an essential design scenario for OfficeMath. It wouldn’t be hard to duplicate the richer MathPlayer navigation experience in OfficeMath but by navigating in the editing space rather than in a virtual MathML space. One might ask whether it’s desirable to have a one-size-fits-all fine-grained editing experience. It might be unexpected or even confusing to say “end of argument” after the 𝑥 in sin 𝑥 in environments that don’t have a math-function object. That coupled with the differences in the way math text is laid out in memory makes the goal of identical fine-grained speech for all math models seem impractical. But for coarse-grained speech, these details are hidden, and it should be possible to have the same coarse-grained math speech for all math models. It’s a pleasure to thank Doug Geoffray, Ron Parker, and Neil Soiffer for very helpful discussions on these topics.

Using Math Alphanumerics in Code and Web Pages

MurrayS3 — Wed, 27 Feb 2019 22:04:01 GMT

The post UTF-8 RTF shows how much easier it is to read the rich text format (RTF) with Unicode characters instead of the RTF \uN notation. You see the real characters instead of signed 16-bit decimal numbers and two such numbers for characters above the BMP. In fact, UTF-8 RTF is remarkably readable. The same readability improvement occurs in computer programs and web source. This post illustrates how much easier it is to read C++ programs and MathML using UTF-8. Contemporary compilers, editors and browsers support UTF-8 and there’s no need to use ASCII numeric character codes like \x222B. The use of such ASCII is so last century. If you want ∫, use ∫ instead of \x222B. UTF-8 is the default encoding for HTML and XML and is the most widely used encoding on the web. This includes excellent plane-1 support thanks in part because most emoji are there. Hence the support for plane-1 math alphanumerics is excellent everywhere as illustrated by the math alphanumerics in this post.

C++

Microsoft Office apps run on the Windows, Mac, iOS, and Android platforms. The compilers and editing environments for these platforms all support UTF-8 source files. For example, my math unit tests include string literals like L"𝑥=(−𝑏±√(𝑏^2−4𝑎𝑐))/2𝑎", which is the UnicodeMath for the solution to the quadratic equation. Using last-century notation (ASCII with hexadecimal literals), that string can be written as

    L"\xD835\xDC65=(\x2212\xD835\xDC4F\x00B1\x221A(\xD835\xDC4F^2\x2212"
    L"4\xD835\xDC4E\xD835\xDC50))/2\xD835\xDC4E"

That’s hard to read, while L"𝑥=(−𝑏±√(𝑏^2−4𝑎𝑐))/2𝑎" is easy to read and comprehend. Another example is an entry in the RichEdit Nemeth braille conversion tables

    {L"⠀⠈⠱⠨⠅⠀", 0x2245 },     // ≅ Approximately equal to

Here the braille string in L"⠀⠈⠱⠨⠅⠀" is encoded in the Unicode braille block U+2800..U+28FF and represents the Unicode character ≅, which has the code U+2245. You can use the character literal L’≅’ instead of the 0x2245, but it’s handy to know what the Unicode value is. Typically, with character literals, I include both the hexadecimal character code and the Unicode character itself. For years I’ve wanted to use legitimate Unicode operators like ≤ and ≠ in C++ programs (see Section 6 of UnicodeMath). While the C++ compilers still don’t recognize Unicode operators other than those in the ASCII subset, you can fake them using fonts like Fira Code that have ligatures for common ASCII operator pairs and sequences. For example, with Fira Code, <= and != display as ≤ and ≠, respectively. The widths of the ligatures are the same as the underlying operator pairs, so that the column alignment is unchanged. Try it, you might like it! One operator Fira Code doesn’t have a ligature for is ->. It should display →. Maybe someday. You can also use the Unicode math alphanumerics in program variable names. For example, the Visual Studio C++ and Clang compilers (used for Apple and Android programs) accept code like

for (int 𝑛 = 0; 𝑛 <= 𝑁; 𝑛++)
     sum += 𝑎[𝑛];

Of course, it’d be even nicer to write this asBack in the 1980’s, I coded C++ using mathematical notation as described in Section 6.2 of the UnicodeMath paper. That approach used my PS technical word processor for editing. Maybe we can add an option to Visual Studio C++ to do something similar!

MathML

In MathML the need for math-italic letters is reduced substantially due to the convention that a single ASCII letter inside … is automatically converted to math italic for display. So, x represents 𝑥 (U+1D465). Would be nice if the reverse were true! But other math alphanumerics (script, Fraktur, bold, bold-italic, etc.) don’t have such a convention. They can be specified via the mathvariant attribute as in L for the character ℒ. But for this it’s easier to read ℒ , which is also valid MathML. The mathvariant attribute was added to MathML before the Unicode math alphanumerics were encoded in Unicode 3.1.0 (March, 2001). But now it’s only needed for reading existing documents that contain it. In implementing MathML math variants, the Unicode math alphanumerics should be used. The character-level machinery of editors and browsers handle the math alphanumerics. There’s no need to use a higher-level protocol such as CSS to handle them. The reason the math alphanumerics were encoded in Unicode is to support math characters in plain text since different math styles have different semantics. ℋ is a different variable from 𝐻. If you ignore the difference, you convert the Hamiltonian formula

into an integral equation! Something with an entirely different meaning. Using CSS in MathML may also limit the resulting content to environments that support CSS. That excludes many math programs and CSS isn’t supported in the Office MathML readers and writers. So, while CSS can be used “under the sheets” to implement MathML display, it shouldn’t be part of MathML content or, at least, ignoring it shouldn’t change the meaning of the content.

Entering UTF-8 characters in programs

In contemporary document editors such as Word, you can enter Unicode characters in a variety of ways, ranging from an Insert Symbol dialog and math-ribbon galleries to the hex hot key Alt+x (type 222B Alt+x and you get ∫). At present, the Visual Studio IDE editor doesn’t offer such input methods, although it should. To make up for this, you can enter them in Word or RichEdit and copy them into your C++ programs and web pages.

RichEdit 9 Additions

MurrayS3 — Fri, 18 Jan 2019 22:25:16 GMT

Six years have past since the post RichEdit 8 Feature Additions and a lot has happened in between. Along the way, several versions have shipped, but we might as well call the current one RichEdit 9, This covers RichEdit up through Office 2019 and includes some features of more recent Office 365 versions. The latter versions are shipped almost every month with fixes and improvements and, hopefully, no regressions! Much of the new functionality is described in posts, so I point to them as well as mention other improvements. Excel has switched to RichEdit 9 both for in-cell editing and formula-bar editing. This switch required numerous Excel-specific additions to RichEdit in order to maintain Excel’s existing behavior. Most notably, a lot of effort was spent on the ruby text feature. Many changes were small enough to be named bug fixes, although some added functionality. Substantial effort has been devoted to keeping up with Unicode, particularly with emoji, but also with new scripts.

Cross Platform

Perhaps the most impressive change is that RichEdit 9 ships in many forms: Win32, Universal, Mac, iOS, Android, and server! To support all these versions, we write unit tests for all new features. The extensive test infrastructure for older features runs on Windows, and some of those tests have been converted to unit tests that run on other platforms. Fortunately, the implementations of most features are platform independent and can be tested adequately on Windows alone. Part of porting RichEdit to other platforms depended on porting LineServices and DirectWrite to those platforms as well. The remainder of this post has links to features that have shipped. Please click on the links that you’re interested in.

RichEdit Property Sets

Variable fonts are the latest addition to the OpenType specification. RichEdit 9 supports both OpenType features and variable fonts. December 22, 2018

UI Automation Math Text Support

Microsoft products expose their contents for accessibility purposes via a set of interfaces known as UI Automation (UIA). Currently UIA has no special support for math text. Either the assistive technology program (AT) has to figure out if math is involved or the application has to return math-specific speech text as done with Office math. RichEdit 9 supports two ways for an AT to retrieve math zones in a specific format, e.g., MathML. November 16, 2018

Converting Microsoft Equation Editor Objects to OfficeMath

As discussed in the post Editing equations created using the Microsoft Equation Editor, the Microsoft Equation Editor 3.0 (MEE) was removed from Office installations because it has security problems and no maintenance. Microsoft doesn’t have access to the MEE source code and MEE’s author, Design Science, doesn’t maintain it, instead offering the more powerful, upward-compatible MathType. Users can invoke the converter built into RichEdit 9 in Word and PowerPoint to convert MEE and MathType math objects to OfficeMath. August 31, 2018

RichEdit Animated GIFs

The post RichEdit 8.0 Image Support describes how RichEdit supports popular image formats, such as jpeg’s, png’s and GIF’s. RichEdit 8.1 added direct support for jpeg’s and png’s in the Rich Text Format (RTF) instead of using RichEdit’s proprietary blob format. Even so, GIFs were treated as second-class images in two ways. First, they were converted to png’s when persisted in RTF. Since png’s are limited to a single frame, only the initial frame of a multiframe GIF was saved. Second, only the initial frame was displayed, so the animated GIFs that are seen frequently on the web and in texting programs weren’t animated. This post describes how RichEdit 9 fixes both problems. February 21, 2018

LaTeX Math in Office

For a while now it’s been possible to switch Word’s math input mode from UnicodeMath to LaTeX. We didn’t advertise this highly requested feature since it needed more work. We fixed most of the problems and this post describes how you can use [La]TeX as a math input method in Word, PowerPoint, OneNote, and Outlook. The LaTeX converters are shared code built into RichEdit 9. July 30, 2017

Microsoft Office Math Speech

Microsoft Office math-aware applications can now speak math in over 18 different languages! Try it out with native math zones in Word by enabling Narrator (type Windows+Ctrl+Enter) and navigate a math zone as described in the post Speaking of math… There are two math-speech granularities: coarse-grained (navigate by words), which speaks math expressions fluently in a natural language, and fine-grained (navigate by characters), which explains the content at the insertion point (IP) in enough detail to enable editing. The math speech engine is shared code built into RichEdit 9. February 27, 2017

Nemeth Braille—the first math linear format

The 6-dot Nemeth braille encoding was created by Abraham Nemeth for mathematical and scientific notation and is general enough to encode almost all Microsoft Office math notation. RichEdit 9 supports Nemeth math braille input and output. This capability hasn’t yet been integrated into Office apps. July 31, 2016

RichEdit Language Tag Handling

For years, many applications have used the locale ID (LCID) to identify the language and locale for text and other data. For example since 1997 (RichEdit 2.0), RichEdit’s character formatting has included CHARFORMAT2::lcid. The LCID can, in fact, describe most language/locale combinations in use as far as text is concerned. However, the more general BCP-47 language tag is based on international standards and it can describe virtually any language/locale combination and it can include additional subtags such as for script and private-use. Since myriad existing documents use LCIDs, text engines still need to support LCIDs. This post describes how RichEdit 9 supports both properties gracefully in a way that doesn’t require tearing existing programs apart. October 19, 2015

Color Fonts

The popularity of emoji symbols has encouraged a variety of technological innovations, notably fonts with multicolor characters. This is different from just having text color, which is described in RichEdit Colors. While some colored glyphs were part of the original Japanese emoji standards, colored glyphs got much more elegant when Apple introduced its proprietary color emoji font. This post describes how RichEdit 9 displays text formatted with Microsoft color fonts. September 24, 2015

Posts for previous versions

Here are some posts that describe older RichEdit versions and history.

RichEdit 8 Feature Additions

The time has come to summarize the features added in RichEdit 8, which shipped with Windows 8 and Office 2013. Since so much was added, I wrote a number of blog posts over the last twelve months about the larger RichEdit 8 features. The present post lists those features and then describes some smaller features… September 7, 2013

Some RichEdit History

A couple of comments have raised the question of people outside Microsoft using the various versions of RichEdit. Specifically, Teis Johansen asks, “Just to be sure. Can I redistribute RichEdit 6.0 with my application?” and Kyle Alons asks, “So what’s the point of listing these features without documenting how to use them? Just to make… October 19, 2006

RichEdit Versions Update to 7.0

The original RichEdit Versions post covered RichEdit versions 1.0 through 6.0, since 6.0 was the latest version at the time. RichEdit 7.0 will ship with Office 2010, so here’s an update describing what that version adds. Most additions involve math editing/display and play a central role in the math features of OneNote 2010, PowerPoint 2010,… September 5, 2009

Math Handwriting Recognition

Starting with Windows 7, Windows includes a cool applet called the Math Input Panel. This applet lets you enter mathematical text using a pen or a mouse. It recognizes what you enter and displays the result using a special private version of RichEdit 6. It also lets you copy the results to Word, Mathematica, or any… May 6, 2009

RichEdit Versions 1.0 through 3.0

Digging through old doc files, I ran across the following summary of RichEdit up through Version 3.0. It’s more detailed than my post on RichEdit Versions, so it might be of interest to history buffs, anyhow. And it does describe the riched20.dll that still ships with Windows, mostly for purposes of backward compatibility. I wrote… January 12, 2010

RichEdit versions

Recurring questions are what RichEdit’s are available, where they are installed and what features they have. A relatively new question is which RichEdit’s support the new Office math editing and display. So this post attempts to answer these questions. To answer the last question first, only RichEdit 6.0 has the Office math facility, although RichEdit… October 13, 2006

RichEdit Property Sets

MurrayS3 — Sat, 22 Dec 2018 16:46:32 GMT

RichEdit has many character-format properties, most of which are documented for ITextFont2 and CHARFORMAT2. Nevertheless, the OpenType specification defines many more character-format properties called OpenType features consisting of a 32-bit identifier (id) and a 32-bit value. For example, the Gabriola font has stylistic set 6, which displays “Gabriola is graceful” as Variable fonts are the latest addition to the OpenType specification and the variable-font axis coordinates are also specified by an id-value pair. For example, the experimental HoloFont font has three axes, ‘wght’, ‘wdth’, and ‘opsz’, the first two of which are illustrated in

HoloFont was designed by John Hudson and Ross Mills of Tiro Typeworks Ltd. You can try out variable fonts by checking out this site and you can find myriad variable-font articles and talks here. Variable fonts present a user-interface (UI) challenge. One technique is to use a slide bar to choose an axis coordinate. AI might provide good default values. If the traditional font drop downs are used, you can be confronted with a zillion choices. HoloFont has 9 weights × 5 widths × 6 optical sizes = 270 entries which all appear in the current Word drop-down font list! And that’s tiny compared to the continua of possible axis coordinate values. To illustrate this quandary, here are the first few entries in the HoloFont font drop-down list

Narrow Thin Narrow ExtraLightmmm Narrow Light Narrow SemiLight Narrow Narrow SemiBold Narrow Bold Narrow ExtraBold Narrow Black

SemiNarrow Thin SemiNarrow ExtraLightmmm SemiNarrow Light SemiNarrow SemiLight SemiNarrow SemiNarrow SemiBold SemiNarrow Bold SemiNarrow ExtraBold SemiNarrow Black

Thin ExtraLightmmm Light SemiLight Regular SemiBold Bold ExtraBold Black

SemiWide Thin SemiWide ExtraLight SemiWide Light SemiWide SemiLight SemiWide SemiWide SemiBold SemiWide Bold SemiWide ExtraBold SemiWide Black

Clearly such detailed font drop-down lists are impractical, so maybe we should use slide bars or drag selected text handles. OpenType properties that are used in shaping complex scripts like Arabic are invoked automatically by DirectWrite and Uniscribe. But many other OpenType properties including these examples are discretionary and must be present in the backing store to work. In addition, it’s desirable to be able to add other kinds of properties. The CHARFORMAT2::dwCookie allows a client to attach one 32-bit value to a text run, but there’s need to attach multiple properties such spelling, grammar, and other proofing-error annotations along with other client properties. To handle all these properties, the latest Office 365 RichEdit implements property sets as described in the remainder of this post. The D2D/DirectWrite RichEdit mode (but not the GDI/Uniscribe mode) displays the OpenType properties as illustrated in the figures above. The following, admittedly technical, discussion describes the property-set object model, the RTF and binary file format additions for property sets, how to display variable-font and other OpenType features using DirectWrite, and the OpenType variable-font (fvar) table.

Kinds of Properties

The kinds of RichEdit character format properties are summarized in the table

ID Range	Usage
0..0xFFFF	Properties not in property sets
0x10000..0x1FFFF	RichEdit temporary properties such as proofing errors
0x20000..0x2FFFF	Client temporary properties
0x30000..0x3FFFF	RichEdit persisted properties
0x40000..0x2020201F	Reserved; returns E_INVALIDARG if used
0x20202020..0x7E7E7E7Emmm	OpenType features/axis (if 0x80808080 mask = 0; else invalid)
0x7E7E7E7F..0xFFFFFFFF	Reserved; returns E_INVALIDARG if used

There are no persisted client properties since they are client-specific and could be misinterpreted if read by a different client.

Property Set Object Model

The client APIs for setting and getting properties are ITextFont2::SetProperty (id, value) and ITextFont2::GetProperty (id, pvalue). The id’s for these methods are given by xxxx, where xxxx is an OpenType feature tag, an OpenType variable-font axis tag (see MakeTag() below) or an annotation id defined in the table at the end of the preceding section. Since OpenType x’s belong to a limited set of ASCII characters in the U+0020..U+007E range, there’s plenty of room in the 32-bit id space to define other properties. Common properties like font weight are already represented as CCharFormat::_wWeight and in principle don’t need to be members of a property set. Since by default there are no properties in a property set, calling ITextFont2::SetProperty(id, tomDefault) deletes the property id if it exists. Note that id values < 0x10000 are reserved for other purposes, such as tomFontStretch (0x33E) to define a font’s stretch value. These values are well below the first possible OpenType id 0x20202020 (4 spaces). The largest OpenType tag is 0x7E7E7E7E, which gives 94⁴ = 78,074,896 tags, although most of them will never be used or are used for other purposes such as ‘MATH’ for the math table. This leaves 256⁴ − 94⁴ = 4,294,967,296 − 78,074,896 = 4,216,892,400 IDs for other purposes. OpenType tags are constructed in the order given by the macro

#define MakeTag(a, b, c, d)   (((d)<<24) | ((c)<<16) | ((b)<<8) | a)

For example, the variable-font weight axis tag ‘wght’ has the value 0x74686777. Internally it’s useful to mark OpenType feature tags with a bit (tomOpenTypeFeature—0x00800000) to distinguish them from variable-font axis tags. This bit cannot be confused with annotation id’s which have values of 0x3FFFF or less. The feature tags are defined by the DWRITE_FONT_FEATURE_TAG enum defined in dwrite.h. The variable-font axis tags are defined by the font’s fvar table discussed below and in principle can be any combination of ASCII letters. So, if a tag isn’t a feature tag, we assume that it’s a variable-font axis tag and let DirectWrite accept or reject it.

Property Set RTF

In RTF, property sets are encoded similarly to the {\colortbl…} for colors and have the form

{\*\propsets id value…; …}

Here the id and value are 32-bit values that are encoded for all properties in a property set. Each property set is ended by a semicolon. This format is repeated for all property sets used in the text. If an id starts with an ASCII letter and consists of 4 ASCII letters, it is written as a character string. For example, the id ‘wdth’ is written as such for the 32-bit id value 0x68746477. If any byte in the id isn’t an ASCII letter, the id is written as a 32-bit integer. These choices make it easier to read property IDs. A value with no fractional part is written as an integer. A value with a fractional part is written as a decimal fixed-point number, e.g., 123.545. Any other combination is invalid and ends reading the RTF stream. The property set table {\*\propsets …} is stored in the RTF header following {\fonttbl …} and {\colortbl …} (if they are present). An example with two property sets containing variable-font id’s is

{\*\propsets wght 800 wdth 104;wght 400;}

This syntax is a slightly simplified version of the variable-font CSS syntax used in web applications. In the RTF body, a reference to the Nth property set in the \propsets table is given by \psN (like \crN for choosing the Nth color in the \colortbl). Here N is 0-based, that is, \ps0 refers to the property set immediately following \propsets.

Property Set Binary Format

The property id-value pair is written in the binary format as opyidProperty (0x8A), optProperty (opt8Bytes) followed by the 32-bit id and value. CPropertySet is written as opyidPropertySet (0x89), optPropSet (optArray) followed by the set’s opyidProperty’s. The array of property sets CPropertySets is written as opyidPropertySets (0x88), optPropertySets (optArray) followed by the opyidPropertySet’s. These constants are defined in rebinary.h.

Rendering Variable-Fonts and OpenType Features

In addition to backing-store enhancements, the display routines need to pass active variable-font axis coordinates and OpenType features to DirectWrite. See OpenType Variable Fonts for information about the DirectWrite APIs for this. To create a font specified in part by axis coordinates, RichEdit gets an IDWriteFontFace5 (see dwrite_3.h) with the desired axis coordinates in place of the usual IDWriteFontFace. It does this by calling IDWriteFontFace::QueryInterface() to get an IDWriteFontFace5 interface, calling IDWriteFontFace5::GetFontResource() to get an IDWriteFontResource interface, releasing the IDWriteFontFace5 and calling IDWriteFontResource::CreateFontFace() to get a new IDWriteFontFace5 with the desired axis coordinates. Then it uses this IDWriteFontFace5 instead of the original IDWriteFontFace. To pass OpenType features to DirectWrite, copy them into a std::vector and pass them to IDWriteTextAnalyzer1::GetGlyphs() and IDWriteTextAnalyzer1::GetGlyphPlacements(). Some font features, such as Gabriola’s stylistic set 6 ‘ss06’ introduce glyphs with ascents and/or descents that exceed the standard typo ascents and descents as discussed in High Fonts and Math Fonts. To display such large glyphs with no clipping, the rendering software needs to calculate the line ascent and descent from the glyph ink, rather than from the usual font values. This is the approach used with the LineServices math handler.

OpenType Variable Font Axes

The variable font axes are defined in the OpenType fvar table, which has the header

struct FvarHeader             // Variable font fvar table header
{
   OTUint16 majorVersion;     // Major version of fvar table (1)
   OTUint16 minorVersion;     // Minor version of fvar table (0)
   OTUint16 axesArrayOffset;  // Byte offset from table start to first VariationAxisRecord
   OTUint16 reserved;         // Permanently reserved (2)
   OTUint16 axisCount;        // Count of VariationAxisRecord's
   OTUint16 axisSize;         // BYTE count of VariationAxisRecord (20 for this version)
   OTUint16 instanceCount;    // Count of InstanceRecord's
   OTUint16 instanceSize;     // BYTE count of InstanceRecord
};                            //  (axisCount*sizeof(DWORD) + (4 or 6))

Types like OTUint16 that begin with OT describe 4-byte, big-endian quantities that need reverse ordering to work with our little-endian machine architecture. The header is followed by axisCount VariationAxisRecord’s defined by

struct VariationAxisRecord
{
   OTUint32 axisTag;          // Tag identifying axis design variation
   OTFixed  minValue;         // Minimum coordinate value (16.16 format)
   OTFixed  defaultValue;     // Default coordinate value
   OTFixed  maxValue;         // Maximum coordinate value
   OTUint16 flags;            // Axis qualifiers (hidden if 1)
   OTUint16 axisNameID;       // ID for 'name' table entry that provides axis display name
};

The axisTag’s have the same MakeTag() form as the regular OpenType tags. Since they are accessed via the OpenType fvar table, they are in a different namespace from the regular OpenType tags. We don’t know of any tag conflicts between the two name spaces, so it’s probably okay not to mark the axis tags differently. But internally we mark OpenType feature tags by setting the high bit of byte 2 (OR in tomOpenTypeFeature), since the tags consist of ASCII symbols in the range 0x20..0x7E. This marking avoids sending OpenType tags to the wrong DirectWrite APIs. The VariationAxisRecord’s are followed, in turn, by the InstanceRecord’s defined by

struct InstanceRecord
{
   OTUint16 subfamilyNameID;       // ID for 'name' table entry giving subfamily name
   OTUint16 flags;                 // Reserved for future use (0)
   OTFixed coordinates[axisCount]; // instanceSize coordinates
   OTUint16 postScriptNameID;      // Optional. ID for 'name' table entry giving PostScript name
};

At some point, it might be worth dealing with the InstanceRecord’s, but it’s certainly easier to use axis coordinates than handle myriad localizable font names (see Holofont discussion in the introduction). RichEdit could export a facility for translating between the two, but probably such a facility should be delegated to the font picker. The localizable font names are designed to help end users recognize the nature of a variable font instance, but they aren’t efficient at the RichEdit level. They also aren’t usable for variable-font animations, since such animations vary axis coordinates continuously.

4-28 Decimal Floating-Point Format

The OpenType “fvar” table described in the previous section defines the min, max, and default variable-font axis coordinate values using the OpenType 16.16 numeric format. The integer part of the value is given by shifting right 16 bits, i.e., dividing by 65536. If the fractional part is nonzero, store the value in a floating-point variable and divide by 65536. In applications, coordinates are easier to read when the fractional part is 0 if only the integer part is displayed. Since purely fractional coordinates (values < 1) are useless, if the absolute value is less than 65536, the value can be understood to be an integer without a fractional part. The OpenType 16.16 format is a binary fixed-point format that may encounter roundoff when converted to decimal, e.g., 800.1 → 800.100006. This roundoff is ugly in RTF, CSS, and dialog boxes. So we need a decimal floating-point format that doesn’t have such roundoff. The IEEE 754-2008 decimal floating-point encoding defines decimal32 with 20 bits of precision, a sign bit and the large exponent range of 10¹⁹². OpenType variable-font axis coordinates need at most four decimal places. The sign bit is used for the slant (slnt) standard axis and can be used for custom axes. If the value has no fractional part, we store it as a standard 2’s complement integer rather than in the high word of 16.16 for readability in RTF, CSS and dialog boxes. To convert it to the 16.16 format, multiply by 65536. But if the value has a fractional part, we use the following signed 4-28 decimal floating-point format

s	n			significand
31	30	29	28	27	26	25	24	23	22	21	20	19	18	17	16	15	14	13	12	11	10	9	8	7	6	5	4	3	2	1	0

If the number is negative, the sign bit 31 is 1. Bits 0..27 are the significand. The decimal divide value n is defined by

n	divide significand by
000mm	(not floating point)
001	10
010	100
011	1000
100	10000
101	100000
110	1000000
111	(not floating point)

n must have at least one 0 bit to distinguish the format from a negative 2’s complement integer and at least one 1 bit to distinguish it from a positive integer. This gives 28 bits of precision with a maximum value of (10²⁸ – 1)/10 = 26843545.5 with one decimal place and a minimum value of 0.000001 with six decimal places. These limits are beyond the values used for OpenType variable font-axis coordinates, which typically range between 1 and 1000. The 4-28 decimal floating-point format is easy to use and displays the original fixed-point values with no round-off error. To convert it to the 16.16 format, store the 28-bit significand field in a double variable, divide by the number corresponding to n, multiply by 65536 and round to the nearest integer. For the DWrite APIs, store the 28-bit significand field in a double, divide by the number corresponding to n and cast the result to a FLOAT. In C, the 4-28 decimal floating-point format of the value x is recognized by the function IsDecimalFloat(x) defined by

#define IsDecimalFloat(x)       IN_RANGE(3, (x >> 28) & 7, 6)

where IN_RANGE() is defined by

#define IN_RANGE(n1, b, n2)     ((unsigned)((b) - (n1)) <= unsigned((n2) - (n1)))

The divide factor in the n table is given by pow(10, (x >> 28) & 3) or (x >> 28) & 3 can be used as a table index.

UI Automation Math Text Support

MurrayS3 — Fri, 16 Nov 2018 23:57:53 GMT

Microsoft products expose their contents for accessibility purposes via a set of interfaces known as UI Automation (UIA). Currently UIA has no special support for math text. Either the assistive technology program (AT) has to figure out if math is involved or the application has to return math-specific speech text as done with Office math speech. To support math more generally, UIA needs three additions:

A document (ITextProvider) property that specifies the AT’s desired default math format (MathML, LaTeX, UnicodeMath, …) for math-zone text returned by calling ITextRangeProvider::GetText()
A parameter that specifies the math format for an individual ITextRangeProvider::GetText() call
Ways to navigate and select math zones

An AT like NVDA that handles all math accessibility (speech and braille of various verbosities and options) would use addition 1. With MathML containing the entity for UI, NVDA could generate math speech and braille enabling both speaking and editing of math. An AT like Narrator that doesn’t understand MathML would use addition 2, getting math speech with one call and math braille with another. This post describes these ways for improving UIA math support.

Recognizing Math Text

For the first two additions, it’s easy for an AT to recognize most math formats in a text string returned by UIA, especially since the AT chooses which format to return and can be looking for it. MathML math zones are XML strings that start with and end with . A LaTeX inline math zone starts with “$” or “$“ and ends with “$” or “$”. A LaTeX display math zone starts with “$$” or “\[“ and ends with “$$” or “\]”. A UnicodeMath math zone starts with “⁅” (U+2045) and ends with “⁆” (U+2046). Math braille is given by characters in the Unicode U+2800..U+28FF braille block. Non-math text uses other Unicode characters since braille engines can braille natural languages. Math speech supplied by Office apps usually doesn’t have start and end math speech text. It might be worthwhile to have a math speech format type that includes the delimiters and . These delimiters wouldn’t be spoken but could be cues to speak the text as is and afterward to call for math braille if brailling is active. If a XML element is added, it’d be worthwhile to support the Speech Synthesis Markup Language (SSML) more generally so that math character styles could be spoken with a different pitch, for example. Another possibility is to add to SSML. The sections below define methods that provide this UIA math functionality. Note that unless Narrator wants to take advantage of Office Nemeth math-braille capabilities, Windows doesn’t need to do anything other than document the new methods and include them in UIAutomationCore.h. Math speech already works well with Narrator, although it doesn’t offer math verbosity options (which differ from natural-language verbosity options).

Document Math Text Format Property

Typically, UIA doesn’t have UIA state properties. The properties it exposes are properties of the source content. But to define which math format ITextRangeProvider::GetText() should use by default, UIA needs to set a document property that specifies the math format. Accordingly, we define ITextProvider3 as follows

MIDL_INTERFACE("242A2469-3CAB-403E-9DA6-FAF1327C7FC6")
ITextProvider3 : public ITextProvider2
{
   public:
   virtual HRESULT STDMETHODCALLTYPE get_Property(
   /* [in] */ int Type,
   /* [retval][out] */ int *pValue) = 0;

   virtual HRESULT STDMETHODCALLTYPE set_Property(
   /* [in] */ int Type,
   /* [in] */ int Value) = 0;
};

Type specifies the property type. For now, there’s only the default math format: TextProperty_MathFormat (1). Its values are given by

enum MathFormatType
{
   MathFormatType_Default     = 0, // Same as GetText
   MathFormatType_MathML      = 1, // Math zones in MathML
   MathFormatType_Nemeth      = 2, // Math zones in Nemeth braille (U+2800 block)
   MathFormatType_LaTeX       = 3, // Math zones in LaTeX
   MathFormatType_UnicodeMath = 4, // Math zones in UnicodeMath
   MathFormatType_Speech      = 5, // Math zones with speech
};

Another property type could be TextProperty_MathVerbosity. To get an ITextProvider3 interface, the client calls

ITextProvider::QueryInterface(__uuidof(ITextProvider3), (LPVOID *)ppTextProvider3)

If this call fails, the program doesn’t have math-format support.

Range Math Text Property

To enable getting more than one math format, e.g., speech and braille, we define the range-level interface

MIDL_INTERFACE("724258C8-8A0D-407D-9622-E5E75D307513")
ITextRangeProvider3 : public ITextRangeProvider2
{
   public:
   virtual HRESULT STDMETHODCALLTYPE GetText2(
   /* [in] */ int maxLength,
   /* [in] */ int Flags,
   /* [retval][out] */ __RPC__deref_out_opt BSTR *pRetVal) = 0;
};

The arguments maxLength and pRetVal are the same as for ITextRangeProvider::GetText(maxLength, pRetVal). The low four bits of Flags are given by the MathFormatType enum above. The AT calls

ITextRangeProvider::QueryInterface(__uuidof(ITextRangeProvider3),
      (LPVOID *)ppTextRangeProvider3)

If this call fails, the program doesn’t have range-level math-format support.

Math Zone Navigation

There are two general kinds of math-zone navigation: 1) from one math zone to another, and 2) within a math zone. The latter can be accomplished with existing functionality, typically by following the program selection changes or by moving by UIA TextUnit_Character and TextUnit_Word. To navigate up to a math zone or skip onto the next math zone, UIA needs to have a math-zone unit. UIA annotation and attribute values are distinct from the defined UIA TextUnit’s, since attributes are in the 40000 range and annotations are in the 60000 range, while the TextUnit’s are < 10. If we enable the ITextRangeProvider ExpandToEnclosingUnit() and Move(), etc., methods to treat AnnotationType_Mathematics as another unit, then an AT could move by math zones, expand to a math zone, etc. If the AT calls for moving by TextUnit_Format, a math zone would be a format break point. If the QueryInterface for an ITextProvider3 succeeds, then a client could expect that navigation by AnnotationType_Mathematics would work. If a call returns an HRESULT error, navigation and selection by AnnotationType_Mathematics isn’t supported.

Math Control

People have thought about an alternative approach that uses a UIA math control like a UIA hyperlink or table control. This approach is discussed in Math Accessibility Trees. While it makes sense theoretically, math zones can be numerous and are often very small such as the variable 𝑥, which is only a single character. So, it’s simpler and more efficient to treat math zones as text with a math-zone attribute. AnnotationType_Mathematics seems to fit the bill and it has been defined for several years. Note that the Text Object Model (TOM) uses a math-zone attribute (tomMathZone—see tom.h) as a unit for navigation and selection. Version 2 of TOM has advanced support for math text processing some of which is described in the post Setting and Getting Math Speech, Braille, UnicodeMath, LaTeX… For UIA purposes, the relatively simple approaches given here seem to suffice.

OfficeMath UI

MurrayS3 — Wed, 17 Oct 2018 00:49:19 GMT

The post OfficeMath describes the history, model, file format, typography and math font of the native math facility introduced in Office 2007. That post refers to the present post for discussion of OfficeMath user interfaces (UI). OfficeMath UI can be grouped into keyboard, menu/ribbon, ink, and accessibility categories. Let’s consider each of these in turn. The keyboard, menu/ribbon, and ink categories are discussed in Chapter 6 of Creating Research and Scientific Documents with Microsoft Word.

Editing Math with Keyboard

A succinct summary of entering and editing math with a keyboard is given in this blog’s first post, Formula Autobuildup in Word 2007 . Basically, type the hot key Alt+= to insert a math zone and then type math using TeX control words for symbols. For example, in UnicodeMath mode, typing a/b=c insertsThe UnicodeMath syntax is like that used in programming languages. Naturally there’s much more to math than symbols and fractions, and the keyboard input methods are described in UnicodeMath for the Unicode input method and in LaTeX/TeX input method for the LaTeX/TeX input method. In UnicodeMath mode, build up to the “Professional” format is automatic as described in When Formula Autobuildup Occurs. In Word’s LaTeX mode, you must request build up. Enter Ctrl+= to build up a math zone into “Professional” format and Shift+Ctrl+= to build the math zone down into the current linear format (UnicodeMath or LaTeX). Or you can click on the corresponding options of the math-zone acetate rectangle. In addition to the LaTeX/TeX control words, there are operator shortcuts described in Math Keyboard Shortcuts, Negated Operators, Keyboard Operator Shortcuts, Entering Unicode Characters, and Klinke’s Streamlined Math Input Notation. For example, /= autocorrects to ≠ and <= to ≤. Subscripts and superscripts are entered using _ and ^, respectively as discussed in Section 2.2 of UnicodeMath and in Keyboard Entry of Subscripts and Superscripts. Nice things to add include making the leading backslash optional and having an autocomplete drop-down menu of possible control words once you’ve entered the first few characters. For example, many control words start with \left and it would be nice to be able to select the desired one after typing \le rather than having to type in the whole word like \leftrightarrow for ↔. In LaTeX mode, the subscript, superscript, numerator, and other math arguments are single entities. An entity can be a character or control word for a character like \alpha for α, or it can be an expression in curly braces like {a+b}. In UnicodeMath mode, the argument can be a sequence of alphanumeric characters. You can see such a difference by comparing what a^12 becomes: in LaTeX you get 𝑎¹2 and in UnicodeMath you get 𝑎¹². To get the latter in LaTeX input mode, enter a^{12}. Unicode has many math characters (see Section 2 of Unicode Technical Report #25, Unicode Support for Mathematics). The post Math Symbol Hierarchy divides the math operator symbols into basic, intermediate, and full Unicode math categories. Most technical papers use the symbols in the basic and intermediate categories. The remaining characters are very specialized, e.g., ⪑, so you’ll probably never need them. Built-up math zones convert alphabetic characters to math alphabetic characters, e.g., ‘a’ becomes ‘𝑎’, which is given by the Unicode character U+1D44E. Conversion to math alphabetic is overruled for special situations like trigonometric function names and can be overruled for arbitrary text. Also it doesn’t occur for Greek upper-case letters as noted in Math Greek Letters. Math spacing is important and User Spaces in Math Zones explains how UnicodeMath build up may remove a space that’s automatically inserted by math spacing rules. In LaTeX mode, spaces are ignored except to terminate control words. You can navigate through a math zone Using Left/Right Arrow Keys in Mathematical Text or you can use a mouse. Math Selection is similar to selection of ordinary text, but if you select a math object start/end/separator delimiter, the whole object is selected. Up and down-arrow keys try to go to the logical target, e.g., up arrow in the denominator of a fraction goes to the numerator. In navigating and selecting text, it’s useful to understand the concept of the Text Insertion Point. The insertion point is in between characters, not on top of a character. You can enter accented characters as discussed in Math Accents and in Representation of Math Accents. You can enter matrices as discussed in Entering Matrices . If you want to line up two or more equations just right, see Equation Arrays . In OfficeMath, empty numerators, denominators, subscripts, superscripts, and other essential arguments, etc., display the place-holder character ⬚. If you want to hide the ⬚, insert a “zero-width space” given by the Unicode character U+200B as discussed in The Invisibles . In OneNote you can edit optional arguments. These arguments are normally not shown, but you can move inside them by using the left/right arrow keys. When the IP is inside an optional argument, the ⬚ is displayed and you can enter characters. For example, you can convert a square root into an n^th root by navigating into the root’s index argument and typing n. To make such changes in Word or PowerPoint, you need to use a context-menu option.

Editing Math using Ribbon, Dialogs, Context Menus

If you become familiar with keyboard entry, you’ll probably find that the fastest way to enter math (see also the Ink section next). But admittedly, it’s not obvious how to enter many things. The math ribbon displays lots of math objects in readily clickable form. As such it provides easily discoverable ways to enter common mathematical expressions. For a comparison of keyboard and ribbon, see Math Ribbon Entry of Subscripts and Superscripts . Math Context Menus provide context-sensitive ways to modify math objects, such as changing a stacked fraction into a slashed fraction, or aligning a set of equations at their equal signs. See also More on Math Context Menus. You can use the Office Insert Symbol Dialog to insert any Unicode character including all Unicode math symbols. The more common math symbols can be inserted using the symbol galleries on the math ribbon. Smart phones running OfficeMath don’t sport a math ribbon, but a math on-screen keyboard could let you enter lots of math entities easily. Think of exposing math symbols instead of emoji and using surround menus. Also, smart phones can work with ink…

Ink

You can enter equations with a pen as described in OneNote Math Assistant and the links therein. Microsoft’s math ink recognition first shipped in Windows 7 with the applet called the Math Input Panel. This applet lets you enter mathematical text using a pen or a mouse. It recognizes what you enter and displays the result using a private version of RichEdit. It also lets you copy the results to Word, Mathematica, or any other application that reads Presentation MathML. Many people may find that writing equations by hand is the easiest and fastest way to enter them into a computer. Since I’ve made similar claims for UnicodeMath entry, a colleague of mine and I decided to have a race. I chose nine equations from theoretical physics and we started entering. The colleague entering via hand writing beat me by a nose, but had two errors, whereas I had none. But really, we both won, since we demonstrated that we could enter equations into Word remarkably fast.

Accessibility

Math accessibility falls into two categories: speech and braille. Microsoft Office Math Speech shipped in over 18 languages in January 2017. As described in Speaking of math…, math speech has two granularities: coarse-grained for fluent speech and fine-grained for editing. Together with touch typing on a keyboard, this combination enables a blind, nondeaf person to consume and edit math, both elementary and advanced. The OfficeMath speech capability could be extended in useful ways such as offering alternate speech as discussed in Speaking Subscripts, Superscripts, and Fractions. Also, the facility “spoon feeds” the math speech to UI Automation. Some Assisted Technologies (ATs) such as NVDA and JAWS would like to get MathML for math zones and generate the math speech (and braille) themselves. Ways to do this will be the subject of a future post. Interestingly MathML can, in principle, be used both for generating math speech and for editing math as discussed in Editing Math using MathML for Speech. Key infrastructure for math braille shipped in August 2017, namely the RichEdit build up/down machinery used by OfficeMath applications added support for entering and editing math using Nemeth Braille—the first math linear format. More work is needed for applications to expose math braille to end users. The main reason for using Nemeth math braille is given in Braille for Math Zones, which points out that the usual braille digit code ambiguities don’t exist in math zones, which is where the math is. Specifically, braille contractions aren’t used in math zones, so digits can be represented unambiguously using computer braille codes; no numerical indicator is needed for digits in Nemeth math zones (aside from an obscure case). Nemeth braille in math zones works with all languages (is globalized), whereas braille in ordinary text is localized to the language being used. Other posts describing work on math braille include Unicode – Nemeth Character Mappings, which discusses extending the Nemeth specification to include many Unicode math symbols not in the current Nemeth specification and Nemeth Braille Alphanumerics and Unicode Math Alphanumerics, which relates how the Unicode math alphanumerics can be represented using Nemeth braille. The post Math Braille UI describes ways to reveal the math insertion point (IP) using a refreshable braille display. The braille IP location is complicated relative to that for ordinary text in that math structure characters described in OfficeMath aren’t always represented by a Nemeth code. For fractions, they are, but the start delimiter of a subscript object, for example, isn’t present in the Nemeth code. Math dictation would be another math input method for blind and sighted users alike. Imagine, you can say 𝑎² + 𝑏² = 𝑐² faster than you can write or type it! Math dictation would work with all devices, computers, tablets, and phones. Hopefully someday…

OfficeMath

MurrayS3 — Sun, 30 Sep 2018 16:55:30 GMT

Microsoft Word 2007 and RichEdit 6.0 introduced the native Office math facility. PowerPoint, Excel, and OneNote followed suit in 2010, and Mac Word followed in 2011. But ironically the native math facility hasn’t had a recognizable name. “Microsoft Equation Editor” (MEE) seems natural, but it’s the name of the Design Science math editor that shipped first in Office on Windows and the Mac in 1992 and was recently discontinued due to security problems. In fact, the post Converting Microsoft Equation Editor Objects to OfficeMath needs a name for the native math facility since it describes how you can convert MEE OLE math objects into native math zones. Sometimes people refer to the native math facility as OMML (Office Math Markup Language), which is the XML file format that encapsulates the in-memory math model and is used in docx, pptx, and xlsx files. But the facility is more than a file format since it has TeX typographical quality, is based on Unicode, has user interfaces (UI) and works with an OpenType math font. “Microsoft Math” is a possible name, but other companies might want to ship something similar. None of its specifications are proprietary. A good name is OfficeMath. “Office” alludes to Microsoft Office but needn’t be exclusive. “Office” suggests a high-quality level (okay, maybe I’m biased ☺). OfficeMath might suggest calculations rather than math text, but documentation can resolve that ambiguity, which also exists for the linear formats AsciiMath and UnicodeMath. The heart of OfficeMath is its in-memory model, named “Professional” in the OfficeMath UI. This model is mirrored in the OMML file format. It features N-ary structures such as integrals with limits and integrands, subscripts, superscripts and accents with well-defined bases, and math functions with function names and arguments. This level of detail is ordinarily reserved for content math formats such as Content MathML and OpenMath. OfficeMath incorporated these structures to support high-quality math typography, with the nice side effect of facilitating symbolic manipulations and graphing (OneNote Math Assistant). This post summarizes OfficeMath’s history, model, file format support, interoperability, math font, and math formatting, and includes links to further information in OfficeMath-oriented posts in Math in Office. OfficeMath UI is discussed in a separate post.

History

A fun place to learn about the origins of OfficeMath is the post LineServices, which tells how the LineServices line-layout component came to be and how it evolved to yield TeX-quality math typography. OfficeMath depends on other technologies as well, including the creation of the math-font OpenType standard described in High-Quality Editing and Display of Mathematical Text in Office 2007 and OpenType Math Tables. For older history, the post How I got into technical WP describes the first math display program (Scroll, 1970) and predecessors of UnicodeMath. OfficeMath was based on Unicode from the start. Unicode 3.2 (March, 2002) already had most of the current Unicode math character set. The Unicode Technical Committee is committed to including all attested math symbols in the Unicode Standard, so Unicode makes an ideal foundation on which to build math functionality. It also streamlines incorporation into Microsoft Office applications, since they are based on Unicode.

Math Model

As with [La]TeX, MathML, MathType, and other math presentation programs, OfficeMath puts all math expressions and equations into math zones. Math-zone typography differs from the typography of ordinary text (see the section on Formatting below). In the OfficeMath in-memory "Professional" format, mathematical objects like fraction and subscript are represented by a start delimiter, the first argument, an argument separator if the object has more than one argument, the second argument, etc., with the final argument terminated by an end delimiter. For example, the fraction 𝑎/𝑏 is represented in built-up format by {_frac 𝑎|𝑏} where {_frac is the start delimiter, | is the argument separator, and } is the end delimiter. Similarly, the subscript object 𝑎_𝑏 is represented by {_sub 𝑎|𝑏}. The start delimiter is the same character for all math objects as are the separator and end delimiters. In RichEdit, these delimiters are given by the Unicode characters U+FDD0, U+FDEE, and U+FDEF, respectively. In OMML, the start delimiter is represented by an container element, such as for fraction and arguments appear within argument element containers, such as … for a numerator. The type of object is specified by a character-format property associated with the start delimiter. In plain text, the built-up forms of the fraction and subscript are identical if the fraction arguments are the same as their subscript counterparts. In the example here, a plain-text search for {_frac 𝑎|𝑏} matches {_sub 𝑎|𝑏} as well as {_frac 𝑎|𝑏}. Searching for OfficeMath equations involves plain-text searches like this together with comparison of the object types as discussed in Math Find/Replace and Rich Text Searches. The OfficeMath math objects are listed in the table in the next section along with their OMML and Presentation MathML representations. The objects are represented by prefix notation: the character formatting of the object start delimiter contains the object properties (see ITextRange2::GetInlineObject()). This differs from infix notation like 𝑎/𝑏, which needs to be parsed. The OfficeMath in-memory format is a “built-up” format as distinguished from linear formats like UnicodeMath and LaTeX.

Supported File formats

The OMML format is the XML format that encapsulates the OfficeMath in-memory “Professional” format. When OfficeMath was designed, Presentation MathML 3.0 was nearing publication. But Presentation MathML is missing two important elements which therefore require emulations to represent OfficeMath. Specifically, Presentation MathML doesn’t have an explicit N-ary element, nor does it have an explicit math-function element. Furthermore, OfficeMath needs to embed client (Word, PowerPoint, Excel, …) XML easily into the math XML. The MathML element can embed such information, but it’s awkward. Accordingly, OMML was created to describe the OfficeMath in-memory format naturally. With best practices, MathML without the element can be used to round-trip OfficeMath equations apart from non-math formatting like revision markings and embedded objects. Here is a listing from MathML and Ecma Math (OMML) of the OMML elements and exact or approximate MathML counterparts

Built-up Office Math Object...	OMML tag...	MathMl
Accent	acc	mover/munder
Bar	bar	mover/munder
Box	box	menclose (approx)
Boxed Formula	borderBox	menclose
Delimiters	d	mfenced
Equation Array	eqArr	mtable (with alignment groups)
Fraction	f	mfrac
Math Function	func	mrow with FunctionApply (2061) mo
Left SubSup	sPre	mmultiscripts (special case of)
Lower Limit	limLow	munder
Matrix	m	mtable
N-ary	nary	mrow msubsup/moverunder with N-ary mo
Phantom	phant	mphantom and/or mpadded
Radical	rad	msqrt/mroot
Group Char	groupChr	mover/munder
Subscript	sSub	msub
SubSup	sSubSup	msubsup
Superscript	sSup	msup
Upper Limit	limUpp	mover

Other OMML references are Extracting OMML from Word 2003 Math Zone Images and OMML Specification, Version 2. More MathML discussion is given in MathML 3.0, Improved MathML support in Word 2007, Rendering MathML in HTML5, and MathML on the Windows Clipboard. Mathematical RTF is essentially OMML in RTF syntax. See also Office Math RTF and OMML Documentation and Updated RTF Specification. Linear Format Notations for Mathematics include UnicodeMath and LaTeX Math in Office. See also Recognizing LaTeX Input in UnicodeMath Input Mode.

Interoperability

Major interoperability is afforded via Presentation MathML and [La]TeX math. In addition, the Design Science MEE and MathType equations can be converted to OfficeMath as described in Converting Microsoft Equation Editor Objects to OfficeMath. MathType can convert OfficeMath to MathType equations. These equation facilities are compared in Equation-Editor Office-Math Feature Comparison and Other Office Math Editing Facilities. The latter also compares them to the Microsoft Word EQ Field. With a bit of effort, equations can be imported into Office applications from Wikipedia articles as described in Copying Equations from Wikipedia into Office Applications. You can create HTML documents with equations in them as described in Creating Math Web Documents using Word 2007.

Math Font

A basic part of OfficeMath is the Unicode OpenType math font. The first such font, Cambria Math, and the OpenType math tables were developed together with the Office 2007 math software, each influencing the other to obtain high quality results. Some history is given in the post High-Quality Editing and Display of Mathematical Text in Office 2007. The font contains extensive math tables, glyph variants and glyphs for most of the Unicode math character set. The tables were incorporated into the OpenType standard as noted in OpenType Math Tables. Posts elaborating on the math font are Special Capabilities of a Math Font and High Fonts and Math Fonts. Cambria Math and Cambria are serifed fonts designed to look good on digital displays. As such, the stem widths never get skinny, in contrast to Times Roman fonts. If you prefer, the STIX math font is a Times Roman font that includes the OpenType math table support and works with OfficeMath. This font is discussed further in Math STIX Fonts 2.0 and UTR #25 Updates.

Formatting

This section discusses how OfficeMath handles math formatting involving math spacing, math styles, and alignments, and gives links to posts with further information. A math zone is defined by the math-zone character-format effect, an effect like bold or italic. As such, this is a non-nestable property, unlike math objects like fractions, which can be nested arbitrarily deeply. Adjacent math zones automatically merge into a single math zone. An essential part of good math typography is math spacing. Within a math zone, OfficeMath follows the math spacing rules given in Appendix G of The TeXbook plus some enhancements that weren’t added to TeX for reasons of archivability. Section 3.16 of UnicodeMath summarizes the rules for the most common situations. Also see User Spaces in Math Zones for ways that OfficeMath autocorrects typical user input spacing errors. Two Math Typography Niceties shows how phantom objects can improve math spacing beyond the standard spacing rules. Math bold and math italic define different math variables in math zones (𝐚 ≠ 𝑎 ≠ a ≠ 𝒂), while in ordinary text, bold and italic are used for emphasis. In math zones, math bold and math italic characters are different Unicode alphanumeric characters, while in ordinary text, bold and italic are character format attributes with no change in character codes. For example, 𝐚 is U+1D41A, 𝑎 is U+1D44E, a is U+0061, and 𝒂 is U+1D482. Even though the math and ordinary-text uses of bold/italic are unrelated semantically, the user can control these math styles using the usual bold and italic UI as described in Using Math Italic and Bold in Word 2007 . There are other math styles that yield still different mathematical variables, such as open-face, script, Fractur, and sans serif (see Section 2.2 of Unicode Technical Report #25). In general, character formatting is controlled in math zones as described in Restricted Math Zone Character Formatting . In informal documents, people may want to use sans-serif characters instead of serif characters for aesthetic reasons rather than for defining different variables. Currently OfficeMath doesn’t support this choice, but maybe it should. Occasionally one needs to embed ordinary text, such as words, into math zones. OfficeMath defines a character format attribute “ordinary text” for this purpose. Text with this attribute uses standard character formatting for italic, bold, etc. Unless the “ordinary text” attribute is active, the bold and italic settings only affect math alphanumerics; ASCII digits, punctuation, operators, and non-math characters are all rendered nonbold and upright. In addition, OfficeMath has a “no-build-up” attribute to treat operator characters literally rather than use them in build-up translations. For example, if ‘/’ is marked with this attribute, build up in UnicodeMath mode leaves it as the character ‘/’ rather than converting it with the arguments around it into a built-up “stacked” fraction. Since math zones are one level deep, you can embed ordinary text into a math zone, but you can’t nest a math zone within that ordinary text or elsewhere within the math zone. This hasn’t proven to be a limitation, although TeX can embed ordinary text inside math zones and nested math zones inside the ordinary text. It always seems to be possible to unwrap such nested math-zone scenarios into unnested math zones. It’s useful to be able to define math properties for an entire document, rather than specify them for each math zone. This is described in Default Document Math Properties . A new property could be defined to use sans-serif math characters instead of serif characters. There are two kinds of math zones: inline and display. For example, an inline math zone in TeX has the form $...$ and a display math zone has the form $$...$$. Inline math zones use reduced spacing and character sizes to make expressions fit better in line with normal text. In OfficeMath a display math zone starts at the start of a document or follows a hard or soft paragraph end (U+000D or U+000B, respectively) and ends with a hard or soft paragraph end. In some cases, it would be useful to apply display math-zone formatting to inline math zones, but this isn’t currently available. Inter-equation alignment and line breaking involve multiple lines. To handle these cases and equation numbering, OfficeMath has the Math Paragraph , while MathML uses tables and MathType uses PILEs. A math paragraph is a sequence of one or more display math zones separated by soft paragraph ends (U+000B). Line breaking can be automatic or manual as described in Breaking Equations into Multiple Lines . Background on paragraph formatting is given in Paragraphs and Paragraph Formatting . In a document with more than a few equations, it’s useful to number equations referred to from elsewhere in the document. The math paragraph has elegant equation-number support, but it hasn’t been exposed beyond prototyping. The earliest way to handle equation numbering is described in Cool Equation Number Macros for Word 2007 . Later ideas are in More on Equation Numbering and equation numbering using equation arrays is described in Equation Numbering in Office 2016. This last approach isn’t quite as convenient as the ideal math-paragraph equation numbering, but it can handle virtually all cases.