Edit

UI automation fails and element picker or recorder can't see UI elements

This article provides troubleshooting guidance for issues where Power Automate for desktop can't view or interact with specific UI elements in a desktop application.

Symptoms

When you try to use UI automation in Power Automate for desktop, you might experience the following issues:

  • During design time: The element picker or recorder doesn't show any UI elements when you try to capture elements for automation.
  • During runtime: UI automation actions fail to execute correctly in both attended and unattended desktop flow runs.

Cause 1: Interfering background processes

Some background processes interfere with UI automation functionality in Power Automate for desktop. Specifically, the Microsoft Accessory Center or DesktopBridge processes prevent UI automation from working correctly.

Solution: Terminate interfering processes

To resolve this issue, terminate the interfering processes and disable them from running in the future.

Cause 2: Incompatible UI frameworks

Some desktop applications use custom UI frameworks or rendering methods that aren't compatible with the standard accessibility APIs that Power Automate for desktop relies on. This incompatibility makes certain elements invisible to the element picker.

Important

For coordinate-based solutions (Solutions 3 and 4) to work reliably:

  • The screen configuration (resolution, DPI settings, scaling) must remain static at runtime and match the configuration used during development.
  • Changes to the application's user interface structure might affect the effectiveness of these workarounds.
  • Test your flows thoroughly when deploying to different machines or environments.

Solution 1: Use Move mouse to image action

  1. Add a Move mouse to image action to your flow.
  2. Capture an image of the UI element you want to interact with.
  3. Enable the Send a click after moving mouse option.
  4. Configure any additional settings as needed.

This action moves the mouse to the first occurrence of the specified image and sends a left click.

Solution 2: Use Move mouse to text on screen (OCR) action

  1. Add a Move mouse to text on screen (OCR) action to your flow.
  2. Specify the text that appears on or near the UI element you want to interact with.
  3. Enable the Send a click after moving mouse option.
  4. Configure OCR settings and click options as needed.

This action uses Optical Character Recognition (OCR) to locate text on the screen or foreground window and sends a mouse click to that location.

Solution 3: Use the recorder with coordinates

  1. Open Power Automate for desktop and create a new flow.
  2. Select Recorder from the toolbar.
  3. Start recording and perform the click action on the desired UI element in your desktop application.
  4. Stop the recording.

The recorder creates a Click UI element in window action with coordinate-based input parameters populated by using the respective values from your click action.

Solution 4: Use Move mouse and Send mouse click actions

  1. Add a Move mouse action to your flow.
  2. Configure the action to move the mouse to the specific coordinates of the target element.
  3. Add a Send mouse click action immediately after the Move mouse action.
  4. Configure the Send mouse click action to perform the desired click (left, right, or double-click).

Related content

  • Last updated on