Reading PowerPoint Slide Show content with UI Automation

This topic describes the UI Automation elements, annotations, and events that PowerPoint exposes during Slide Show, and how UI Automation clients can read slide content automatically and in sync with entrance and exit animations.

1. Summary

  • Starting in version 2609, in PowerPoint for Windows in Slide Show view, the UI Automation tree and each non-decorative element's Text pattern reflect the current animation state.
  • Content that has not yet appeared is not in the tree.
  • As the presenter advances the Slide Show and objects and text appear from entrance animations, the corresponding elements and text are added to the tree.
  • As the presenter navigates back and objects and text disappear, the corresponding elements and text are removed from the tree.
  • Similarly for exit animations, the corresponding elements and text are removed from the tree when advancing and added to the tree when navigating back.
  • For entrance animations that reveal text by word or by character, UI Automation exposes the containing paragraph as a unit. The full paragraph appears in the tree when its first word or character appears, and one event and annotation represent that paragraph-level change.
  • Each animation step raises UI Automation events that clients can use to know when content has been added or removed.
  • The text that appeared in the most recent animation step has a custom CurrentAnimationStep annotation, so a client can announce just the new content instead of re-reading the entire slide.

2. What the UI Automation tree in Slide Show looks like

During a Slide Show, each visible text box or shape that contains text exposes a Text control pattern (UIA_TextPatternId). The pattern's document range is scoped to the text that is currently visible, except that a by-word or by-character entrance is exposed at paragraph granularity as described in the summary.

Shapes marked as decorative don't expose a Text pattern in Slide Show. Their shape, list, list-item, and equation elements are excluded from the UI Automation control and content views. Interactive hyperlink children remain available so that clients can still discover and invoke them. Events will still be raised for list, list-item, equation, and link child elements that are added or removed for a decorative shape.

Bulleted text is structured as a UI Automation list:

Content UI Automation exposure
Bulleted text block UIA_ListControlTypeId element
Each bullet UIA_ListItemControlTypeId child, with UIA_PositionInSetPropertyId, UIA_SizeOfSetPropertyId, and UIA_LevelPropertyId set
Math equations A custom element named "Equation" (UIA_CustomControlTypeId), parented under its bullet's list item
Hyperlinks UIA_HyperlinkControlTypeId element, parented under its bullet's list item

Only bullets, equations, and hyperlinks that have animated in are present. Each list item exposes its own Text pattern scoped to that bullet's text, so walking the list reads each bullet exactly once.

3. Events to listen to

When the presenter advances or navigates back one animation step, PowerPoint raises the following UI Automation events on the affected shape element. A client that wants to track the slide automatically should subscribe to both, with TreeScope_Subtree on the slide element so it receives events from any shape:

3.1 UIA_Text_TextChangedEventId

Raised on the shape's element whenever the visible text changes because of an animation step.

Upon receiving this event, the client can re-query the element's Text pattern (IUIAutomationTextPattern::get_DocumentRange) to read the text. Because the document range is scoped to currently-visible content, the text returned is exactly what is on screen after advancing.

3.2 UIA_StructureChangedEventId

Raised when the set of child elements changes — for example a new bullet, equation, or hyperlink element appears or disappears. The structure-change type is one of:

  • StructureChangeType_ChildAdded — a newly-revealed element (forward step).
  • StructureChangeType_ChildRemoved — an element hidden by stepping backward.
  • StructureChangeType_ChildrenInvalidated — the subtree should be re-walked.

Upon receiving this event, the client can update the affected part of its cached tree. A removed child might no longer be available to query, so use the structure-change type and runtime identifier supplied with the event rather than assuming that the removed element can still be retrieved.

Note: Multiple events may be raised in a single animation step. For example, when a bullet appears, both a StructureChanged ChildAdded event and a TextChanged event are raised. When 2 objects appear, 2 StructureChanged ChildAdded events are raised.

3.3 Suggested subscription

// pUIA: IUIAutomation*  |  pSlide: IUIAutomationElement* for the current slide
CComPtr<IUIAutomationCacheRequest> pCache; // optional; cache Name/ControlType/Text
pUIA->AddAutomationEventHandler(
    UIA_Text_TextChangedEventId, pSlide, TreeScope_Subtree, pCache, pTextChangedHandler);
pUIA->AddStructureChangedEventHandler(
    pSlide, TreeScope_Subtree, pCache, pStructureChangedHandler);

4. The CurrentAnimationStep custom annotation

When an animation step causes new text to appear, the client can determine which text was added by checking for the CurrentAnimationStep custom annotation on the element associated with the event. This allows the client to read just the text that appeared.

If a single step reveals multiple paragraphs at once , every paragraph revealed by that step carries the annotation, not just the last one.

4.1 How it is exposed

The annotation is surfaced through the Text pattern's annotation text attribute. A text range that intersects the current step's revealed range returns the registered annotation type id in its UIA_AnnotationTypesAttributeId value (an array of int annotation type ids).

4.2 How a client reads it

Custom annotation types are correlated between provider and client by GUID. The provider registers the GUID with the UI Automation annotation registrar and receives a runtime annotation type id; a client that registers the same GUID receives the same id for the life of the session. The client then looks for that id in the AnnotationTypes text attribute.

// 1) Register the same GUID once to obtain the matching annotation type id.
CComPtr<IUIAutomationRegistrar> pRegistrar;
CoCreateInstance(CLSID_CUIAutomationRegistrar, nullptr, CLSCTX_INPROC_SERVER,
                 IID_PPV_ARGS(&pRegistrar));

UIAutomationAnnotationTypeInfo info{};
// {A7C3B9E1-4D82-4F6A-B14E-7D0C39E8A2F5}
info.guid = { 0xa7c3b9e1, 0x4d82, 0x4f6a,
              { 0xb1, 0x4e, 0x7d, 0x0c, 0x39, 0xe8, 0xa2, 0xf5 } };
info.pLocalizedName    = L"Current Animation Step";

int currentAnimationStepTypeId = 0;
pRegistrar->RegisterAnnotationType(&info, &currentAnimationStepTypeId);

// 2) On a text range, read the AnnotationTypes attribute and look for the id.
CComVariant vAnnotationTypes;
pTextRange->GetAttributeValue(UIA_AnnotationTypesAttributeId, &vAnnotationTypes);

bool fIsCurrentStep = false;
if (vAnnotationTypes.vt == (VT_ARRAY | VT_I4))
{
    LONG lBound = 0, uBound = -1;
    SafeArrayGetLBound(vAnnotationTypes.parray, 1, &lBound);
    SafeArrayGetUBound(vAnnotationTypes.parray, 1, &uBound);
    for (LONG i = lBound; i <= uBound; ++i)
    {
        int typeId = 0;
        SafeArrayGetElement(vAnnotationTypes.parray, &i, &typeId);
        if (typeId == currentAnimationStepTypeId) { fIsCurrentStep = true; break; }
    }
}

Putting it together, a client that reads a slide automatically and in sync with the animation can use the following loop:

  1. On entering the slide, read the visible text from each shape's Text pattern and announce it. This is the content already on screen.
  2. Subscribe to UIA_Text_TextChangedEventId and UIA_StructureChangedEventId on the slide with TreeScope_Subtree (see §3.3).
  3. On each event caused by the presenter advancing or navigating back:
    1. Re-query the shape's Text pattern document range to get the now-visible text.
    2. Walk the document range and test the UIA_AnnotationTypesAttributeId attribute for the CurrentAnimationStep annotation id (see 4.2).
    3. Announce only the annotated range(s) — that is the content the step just revealed. This avoids repeating previously-read bullets.
    4. Announce only objects that just appeared by listening to StructureChanged events.
    5. For text that has disappeared, there is no CurrentAnimationStep annotation. To determine the text that has disappeared, keep track of the previous text and compare with the current text.

Requirements

Microsoft 365 Version 2609 (Build 20430.xxxxx)

Windows 11

  • Last updated on