TextPattern and Embedded Objects Overview

This overview describes how Microsoft UI Automation exposes embedded objects, or child elements, within a text document or container.

In UI Automation an embedded object is any element that has non-textual boundaries; for example, an image, hyperlink, table, or document type such as an Microsoft Excel spreadsheet or Microsoft Windows Media file. This differs from the standard definition where an element is created in one application and embedded, or linked, within another. Whether the object can be edited within its original application is irrelevant in the context of UI Automation.

This topic contains the following sections.

• Embedded Objects and the UI Automation Tree
• Expose Embedded Objects Using TextPattern and TextPatternRange
• Common Scenarios
• Related Topics

Embedded Objects and the UI Automation Tree

Embedded objects are treated as individual elements within the control view of the UI Automation tree. They are exposed as children of the text container so that they can be accessed through the same model as other controls in UI Automation.

 

---

Example of a Text Container with Table, Image, and Hyperlink Embedded Objects

 

---

Example of the Content View for a Portion of the Preceding Text Container

Expose Embedded Objects Using TextPattern and TextPatternRange

Used in conjunction, the TextPattern control pattern class and the TextPatternRange class expose methods and properties that facilitate navigation and querying of embedded objects.

The textual content (or inner text) of a text container and an embedded object, such as a hyperlink or table cell, is exposed as a single, continuous text stream in both the control view and the content view of the UI Automation tree; object boundaries are ignored. If a UI Automation client is retrieving the text for the purpose of reciting, interpreting, or analyzing in some manner, the text range should be checked for special cases, such as a table with textual content or other embedded objects. This can be accomplished by calling GetChildren to obtain an AutomationElement for each embedded object and then calling RangeFromChild to obtain a text range for each element; this is done recursively until all textual content has been retrieved.

 

---

Example of a text stream with embedded objects and their range spans

When it is necessary to traverse the content of a text range, a series of steps are involved behind the scenes in order for the Move method to execute successfully.

1. The text range is normalized; that is, the text range is collapsed to a degenerate range at the Start endpoint, which makes the End endpoint superfluous. This step is necessary to remove ambiguity in situations where a text range spans TextUnit boundaries; for example, "{The U}RL https://www.microsoft.com is embedded in text" where "{" and "}" are the text range endpoints.

2. The resulting range is moved backward in the DocumentRange to the beginning of the requested TextUnit boundary.

3. The range is moved forward or backward in the DocumentRange by the requested number of TextUnit boundaries.

4. The range is then expanded from a degenerate range state by moving the End endpoint by one requested TextUnit boundary.

 

---

Examples of how a text range is adjusted for Move() and ExpandToEnclosingUnit()

Common Scenarios

The following sections present examples of the most common scenarios that involve embedded objects.

Legend for the examples shown:

{ = Start

} = End

Hyperlink

Example 1 - A text range that contains an embedded text hyperlink

{The URL https://www.microsoft.com is embedded in text}.

 

Method Called	Result
GetText	Returns the string "The URL https://www.microsoft.com is embedded in text".
GetEnclosingElement	Returns the innermost AutomationElement that encloses the text range; in this case, the AutomationElement that represents the text provider itself.
GetChildren	Returns an AutomationElement representing the hyperlink control.
RangeFromChild where AutomationElement is the object returned by the previous GetChildren method.	Returns the range that represents "http://www.microsoft.com".

 

Example 2 - A text range that partially spans an embedded text hyperlink

The URL http://{www}.microsoft.com is embedded in text.

 

Method Called	Result
GetText	Returns the string "www".
GetEnclosingElement	Returns the innermost AutomationElement that encloses the text range; in this case, the hyperlink control.
GetChildren	Returns null since the text range doesn't span the entire URL string.

 

Example 3 - A text range that partially spans the content of a text container. The text container has an embedded text hyperlink that is not part of the text range.

{The URL} https://www.microsoft.com is embedded in text.

 

Method Called	Result
GetText	Returns the string "The URL".
GetEnclosingElement	Returns the innermost AutomationElement that encloses the text range; in this case, the AutomationElement that represents the text provider itself.
Move with parameters of (TextUnit.Word, 1).	Moves the text range span to "http" since the text of the hyperlink is comprised of individual words. In this case, the hyperlink is not treated as a single object. The URL {http}://www.microsoft.com is embedded in text.

 

Image

Example 1 - A text range that contains an embedded image

{The image is embedded in text}.

 

Method Called	Result
GetText	Returns the string "The is embedded in text". Any ALT text associated with the image cannot be expected to be included in the text stream.
GetEnclosingElement	Returns the innermost AutomationElement that encloses the text range; in this case, the AutomationElement that represents the text provider itself.
GetChildren	Returns an AutomationElement representing the image control.
RangeFromChild where AutomationElement is the object returned by the previous GetChildren method.	Returns the degenerate range that represents "".

 

Example 2 - A text range that partially spans the content of a text container. The text container has an embedded image that is not part of the text range.

{The image} is embedded in text.

 

Method Called	Result
GetText	Returns the string "The image".
GetEnclosingElement	Returns the innermost AutomationElement that encloses the text range; in this case, the AutomationElement that represents the text provider itself.
Move with parameters of (TextUnit.Word, 1).	Moves the text range span to "is ". Because only text-based embedded objects are considered part of the text stream, the image in this example does not affect Move or its return value (1 in this case).

Table

Table used for examples

Cell with Image	Cell with Text
	X
	Y
Image for Z	Z

 

Example 1 - Get the text container from the content of a cell.

 

Method Called	Result
GetItem with parameters (0,0)	Returns the AutomationElement representing the content of the table cell; in this case, the element is a text control.
RangeFromChild where AutomationElement is the object returned by the previous GetItem method.	Returns the range that spans the image .
GetEnclosingElement for the object returned by the previous RangeFromChild method.	Returns the AutomationElement representing the table cell; in this case, the element is a text control that supports TableItemPattern.
GetEnclosingElement for the object returned by the previous GetEnclosingElement method.	Returns the AutomationElement representing the table.
GetEnclosingElement for the object returned by the previous GetEnclosingElement method.	Returns the AutomationElement that represents the text provider itself.

 

Example 2 - Get the text content of a cell.

 

Method Called	Result
GetItem with parameters of (1,1).	Returns the AutomationElement representing the content of the table cell; in this case, the element is a text control.
RangeFromChild where AutomationElement is the object returned by the previous GetItem method.	Returns "Y".

See Also

Tasks

Access Embedded Objects Using UI Automation
Expose the Content of a Table Using UI Automation
Traverse Text Using UI Automation

Reference

TextPattern
TextPatternRange
ITextProvider
ITextRangeProvider

Other Resources

TextPattern Search and Selection Sample