IE9’s Assistive Technology Interface Enhancements

IE9 introduces improvements to the interfaces used by assistive technologies (ATs) to enhance the accessibility of Internet Explorer. These changes relate to IE9’s switch to hardware-accelerated rendering of HTML content. Previously, some accessibility tools depended on IE’s GDI-based rendering to scrape information from Web sites. Since IE9 has no GDI-based rendering, these features do not work with IE9. Anticipating this problem, Microsoft has worked closely with assistive technology vendors (ATVs) throughout the IE9 development cycle to devise a solution. This blog post describes the changes made in IE9 to enable assistive technologies—particularly screen readers—to work well with its hardware-accelerated architecture.

How Screen Readers Work

Screen readers read web pages as if they were straightforward text documents: users can use simple keyboard shortcuts to read the characters, words, lines, and paragraphs of a Web page. Users access other parts of a web page such as hyperlinks and images in a similar fashion. Additional shortcuts allow the user to interact efficiently with the input controls of a form and other elements of rich Web pages. In general, screen readers enable the user to move seamlessly through a web page, just as they would move through a text document with a cursor.

However, moving though a Web page becomes more difficult when the page employs modern Web techniques such as dynamic content, complex layout, and AJAX technologies. For this reason, screen readers often build an off-screen model of the Web page. The purpose of the off-screen model (OSM) is to keep an updated version of the Web page and the spatial relationships of its text, images, edit boxes, and form controls. The OSM essentially maintains a snapshot of the page, making it easier for the user to navigate around and hear the content on the page.

Efficiently building and maintaining an off-screen model is a tricky task employing the use of many technologies including: the HTML DOM, W3C Accessible Rich Internet Applications (WAI-ARIA) markup, the Microsoft Active Accessibility (MSAA) interfaces, the Microsoft UI Automation (UIA) APIs, and GDI-based mirror drivers.

It is this last technology, GDI-based mirror drivers, which does not work with IE9’s DirectX-based hardware-accelerated rendering. In its place, IE9 expands its UIA interfaces to provide the information ATs previously gathered with a mirror driver.

IE9’s Expanded UI Automation Support

In order to provide the information previously inferred from mirror driver output and to provide a robust, forward-looking solution for building off-screen models, IE9 adds support for the UIA Text Pattern (ITextProvider interface).

Using the methods on this interface and the objects it returns, the assistive technology can gather the semantic unit (character, word, line, paragraph, etc.), attributes (size, style, color, etc.), and layout (bounding rectangles) of all the text on a page or just the text in a specific range (visible, selected, etc.). The methods return text ranges in screen-render-order, allowing the AT to keep track of the z-order of ranges that may overlap.

Using UIA’s rich interfaces and caching abilities, coupled with the other technologies listed previously, we believe it is possible for assistive technology vendors to achieve performance equal to the old mirror driver approach.

Working with ATVs

Microsoft is working closely with assistive technology vendors as they update their products for IE9’s release candidate and general release. Hardware acceleration provides significant performance advantages for all users and with support for UIA Text Pattern, IE9 continues to be accessible to our customers who rely on screen readers.

—Sharon Newman, Program Manager

Comments

• Anonymous
  February 15, 2011
  I am so happy the screen reader technology will get better. My eyes are fine (for now) but I will need a screen reader to read the blurry text in IE9.

• Anonymous
  February 15, 2011
  Could the interface to the list of subscribed feeds be improved for screenreaders? Currently, to find out if their are any feeds with new content, you have to down arrow through all the feeds to find the ones with new appended in parenthesis. If there are alot of feeds, this is extremely tedious. I have suggested previously, that one solution would be to have a keystroke to move to the next feed with new content, for example ctrl+u, in a similar manner to moving to the next unread email in an email client. For the most part, for users of screen readers, the feeds in IE are easy to use, but because of the above problem they aren't usable if you have a large number of feeds.

• Anonymous
  February 15, 2011
  Aren't there CSS specifications for specifying how to make your page accessible to the deaf?  Its seems it would be more inline with IE9's goal of "letting the site shine through" to leave the accessibility to the designer.  (Expression Web already has these built in)

• Anonymous
  February 15, 2011
  The comment has been removed

• Anonymous
  February 15, 2011
  The "working closely with assistive technology vendors" must only refer to some sort of private back-channel communication with existing large players.   I've been trying to get an answer about IE9 UIAutomation support from the official forum and had no response from the various MSFT folk there at all. social.msdn.microsoft.com/.../3c23cc42-2ef0-4590-8d99-e2f4ae5bceab Is there some magic to get a ITextProvider with the current RC?   The standard SDK Inspect tool can't see that interface.

• Anonymous
  February 18, 2011
  Better control over colors, flashing objects and low contract situations will greatly help us.   Many sites use overly low contrast text to background colors which negatively affects persons with reduced color differentiation ability (e.g., yellowish text on a white background for those over 55 years of age).