Share via

Automating Flutter Web App via WebView2 - Unable to Access Dynamic Elements (Popups/Submenus) Even with Accessibility Enabled

test code 21 Reputation points
2026-05-01T07:31:41.79+00:00

We are working on automating a web application built with Flutter, using WebView2 in a desktop application.

Since Flutter Web renders the UI using a canvas-based approach, standard DOM elements are not accessible. This makes traditional automation methods (e.g., locating elements by ID, class, or XPath) ineffective.

We do not have the ability to modify the application to add test hooks or identifiers.

Looking for suggestions or approaches to handle automation in such scenarios.

Our Workaround

To overcome this limitation, we found a workaround by enabling accessibility features in the Flutter app. This exposes aria-label attributes, allowing partial interaction with elements.

We are triggering accessibility mode using the following script:

document.querySelector('[aria-label="Enable accessibility"]').click();

Once accessibility is enabled, we attempt to interact with elements like this:

let button = document.querySelector('[aria-label=""Login""]');

if (button) { button.click(); }

This works for some visible elements such as buttons and basic controls.

Problem

While the accessibility workaround partially works, we are facing several limitations:

  1. Not all elements expose aria-labels, especially:Popups, Dropdowns / submenus, Dynamic overlays
  2. No access to unique identifiers like id or stable selectors for many elements.
  3. Elements rendered after interactions (like opening menus) are often: Not accessible via querySelector
  4. The DOM structure remains minimal due to Flutter’s canvas rendering, limiting automation capabilities

Questions

We are looking for guidance or best practices on the following:

  1. Is there a reliable way to automate Flutter Web apps using WebView2 or similar tools?
  2. How can we interact with dynamically rendered elements like popups and submenus?
  3. Are there any alternative approaches (e.g., DevTools protocol, accessibility tree inspection, or Flutter-specific hooks)?
  4. Would using tools like Playwright, Puppeteer, or WinAppDriver provide better results in this scenario?

Any suggestions, workarounds, or insights would be greatly appreciated.

Community Center | Not monitored

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Q&A Assist
    2026-05-01T07:32:19.51+00:00
    AI generated content. This question contains an answer created with AI from Q&A Assist. Learn more

    Flutter Web’s canvas-based rendering means the DOM is not a faithful representation of the UI, so any tool that relies on DOM structure or standard selectors will be fundamentally limited. The only reliable automation surface in this scenario is the accessibility/automation layer that the browser and WebView2 expose.

    Key points and options based on the available information:

    1. WebView2 and UI Automation
    • On Windows, WebView2 content is exposed through Microsoft UI Automation (UIA) when the browser (Edge/WebView2) has accessibility enabled. Assistive technologies and automated test scripts can then interact with the UI via the UIA tree rather than the DOM.
    • Microsoft Edge’s accessibility system maps HTML and ARIA to UIA objects. When Flutter exposes ARIA attributes (as in the current workaround), those are projected into the UIA tree, which can then be consumed by automation tools.
    • Automated test scripts can use the Microsoft UI Automation framework to locate and interact with elements by their accessible properties (Name, ControlType, etc.), instead of DOM selectors.

    This suggests a shift from DOM-based automation to UIA-based automation for WebView2-hosted Flutter Web content.

    1. Tools that can use the UIA/accessibility layer

    Several tools in the Microsoft ecosystem can help inspect and automate via accessibility rather than DOM:

    • Accessibility Insights for Windows
      • Can inspect UI Automation elements, properties, control patterns, and events for Windows apps, including WebView2 content when exposed via Edge’s accessibility pipeline.
      • Use it to verify which Flutter elements actually surface as accessible objects (including popups, menus, overlays) and what properties they expose.
    • Microsoft UI Automation API
      • Provides programmatic access to most UI elements on the desktop and is explicitly designed for both assistive technologies and automated test scripts.
      • For dynamic elements (popups, submenus, overlays), use UIA to:
        • Listen for UIA events (e.g., structure changed, menu opened) to detect when new elements appear.
        • Query the UIA tree from the WebView2 root to find children by Name, ControlType, or other accessible properties.

    If the Flutter app does not expose certain popups or overlays to the accessibility layer at all, no automation tool that depends on UIA or ARIA will be able to interact with them reliably.

    1. When frameworks don’t fully support UI Automation

    If the framework or rendering approach does not implement UI Automation for some parts of the UI, options are limited:

    • Some frameworks simply do not surface all UI elements to UIA. In such cases, system-provided tools like Narrator or Magnifier (and any UIA-based automation) cannot see those elements.
    • Possible workarounds in such situations are:
      • Relying on third-party accessibility tools that might use different techniques.
      • Incorporating a UI Automation provider library into the app.
      • Updating to a framework version that has better accessibility support.
      • Porting to a framework that natively supports UI Automation.

    In this scenario, without the ability to modify the Flutter app, these options are mostly under the control of the app’s developers, not the automation team.

    1. DevTools protocol, Playwright, Puppeteer, WinAppDriver

    From the available information:

    • Playwright
      • Can automate and test webpages in a WebView2 control, but still relies on what the browser exposes (DOM and accessibility). For a canvas-heavy Flutter app, Playwright will face the same DOM limitations unless it can leverage the accessibility tree.
    • WebDriver / DevTools
      • Microsoft Edge WebDriver can attach to a running WebView2 instance and automate the web content, but again, this is primarily DOM-centric. For native UI around the WebView2, tools like WinAppDriver or UIA can be used.
    • WinAppDriver
      • Useful for automating the native Windows shell around WebView2 (buttons, dialogs, etc.), not the canvas-rendered Flutter content inside the WebView2, unless that content is exposed via UIA.

    Given the canvas-based rendering, switching to Playwright, Puppeteer, or WebDriver alone will not solve the fundamental issue of missing DOM elements. The decisive factor is how much of the Flutter UI is exposed via ARIA/UIA.

    1. Practical guidance for the described problems

    Within the constraints described:

    • Use Accessibility Insights for Windows to:
      • Inspect the WebView2 content and confirm which elements (including popups and submenus) appear in the UIA tree.
      • Identify accessible names, roles, and patterns that can be used as selectors in UIA-based automation.
    • Use the Microsoft UI Automation API (or a UIA-based test framework) to:
      • Listen for UIA events when menus or overlays open.
      • Traverse the UIA tree to locate dynamic elements by accessible properties rather than DOM selectors.
    • If specific popups or overlays never appear in the accessibility tree, then:
      • Those elements are effectively not automatable via supported Microsoft accessibility/automation mechanisms.
      • The only durable fix is for the Flutter app to improve its accessibility implementation (e.g., exposing those elements with proper ARIA roles/names so Edge can map them into UIA).

    Without changes to the Flutter app’s accessibility, there is no fully reliable, general-purpose way—via WebView2, Playwright, Puppeteer, or WinAppDriver—to automate all dynamic elements rendered purely on a canvas.

    References:

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.