Training
Module
Use Power Automate for desktop to interact with windows and applications - Training
Learn how Power Automate for desktop identifies and interacts with UI elements and windows.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Note
This documentation is intended for .NET Framework developers who want to use the managed UI Automation classes defined in the System.Windows.Automation namespace. For the latest information about UI Automation, see Windows Automation API: UI Automation.
This topic contains scenarios and sample code that show how and when the AutomationIdProperty can be used to locate an element within the UI Automation tree.
AutomationIdProperty uniquely identifies a UI Automation element from its siblings. For more information on property identifiers related to control identification, see UI Automation Properties Overview.
Note
AutomationIdProperty does not guarantee a unique identity throughout the tree; it typically needs container and scope information to be useful. For example, an application may contain a menu control with multiple top-level menu items that, in turn, have multiple child menu items. These secondary menu items may be identified by a generic scheme such as "Item1", "Item 2", and so on, allowing duplicate identifiers for children across top-level menu items.
Three primary UI Automation client application scenarios have been identified that require the use of AutomationIdProperty to achieve accurate and consistent results when searching for elements.
Note
AutomationIdProperty is supported by all UI Automation elements in the control view except top-level application windows, UI Automation elements derived from Windows Presentation Foundation (WPF) controls that do not have an ID or x:Uid, and UI Automation elements derived from Win32 controls that do not have a control ID.
Caution
In general, you should try to obtain only direct children of the RootElement. A search for descendants may iterate through hundreds or even thousands of elements, possibly resulting in a stack overflow. If you are attempting to obtain a specific element at a lower level, you should start your search from the application window or from a container at a lower level.
///--------------------------------------------------------------------
/// <summary>
/// Finds all elements in the UI Automation tree that have a specified
/// AutomationID.
/// </summary>
/// <param name="targetApp">
/// The root element from which to start searching.
/// </param>
/// <param name="automationID">
/// The AutomationID value of interest.
/// </param>
/// <returns>
/// The collection of UI Automation elements that have the specified
/// AutomationID value.
/// </returns>
///--------------------------------------------------------------------
private AutomationElementCollection FindElementFromAutomationID(AutomationElement targetApp,
string automationID)
{
return targetApp.FindAll(
TreeScope.Descendants,
new PropertyCondition(AutomationElement.AutomationIdProperty, automationID));
}
'''--------------------------------------------------------------------
''' <summary>
''' Finds all elements in the UI Automation tree that have a specified
''' AutomationID.
''' </summary>
''' <param name="targetApp">
''' The root element from which to start searching.
''' </param>
''' <param name="automationID">
''' The AutomationID value of interest.
''' </param>
''' <returns>
''' The collection of automation elements that have the specified
''' AutomationID value.
''' </returns>
'''--------------------------------------------------------------------
Private Function FindElementFromAutomationID( _
ByVal targetApp As AutomationElement, _
ByVal automationID As String) As AutomationElementCollection
Return targetApp.FindAll( _
TreeScope.Descendants, _
New PropertyCondition( _
AutomationElement.AutomationIdProperty, automationID))
End Function 'FindElementFromAutomationID
///--------------------------------------------------------------------
/// <summary>
/// Creates a UI Automation thread.
/// </summary>
/// <param name="sender">Object that raised the event.</param>
/// <param name="e">Event arguments.</param>
/// <remarks>
/// UI Automation must be called on a separate thread if the client
/// application itself could become a target for event handling.
/// For example, focus tracking is a desktop event that could involve
/// the client application.
/// </remarks>
///--------------------------------------------------------------------
private void CreateUIAThread(object sender, EventArgs e)
{
// Start another thread to do the UI Automation work.
ThreadStart threadDelegate = new ThreadStart(CreateUIAWorker);
Thread workerThread = new Thread(threadDelegate);
workerThread.Start();
}
///--------------------------------------------------------------------
/// <summary>
/// Delegated method for ThreadStart. Creates a UI Automation worker
/// class that does all UI Automation related work.
/// </summary>
///--------------------------------------------------------------------
public void CreateUIAWorker()
{
uiautoWorker = new FindByAutomationID(targetApp);
}
private FindByAutomationID uiautoWorker;
'''--------------------------------------------------------------------
''' <summary>
''' Creates a UI Automation thread.
''' </summary>
''' <param name="sender">Object that raised the event.</param>
''' <param name="e">Event arguments.</param>
''' <remarks>
''' UI Automation must be called on a separate thread if the client
''' application itself could become a target for event handling.
''' For example, focus tracking is a desktop event that could involve
''' the client application.
''' </remarks>
'''--------------------------------------------------------------------
Private Sub CreateUIAThread(ByVal sender As Object, ByVal e As EventArgs)
' Start another thread to do the UI Automation work.
Dim threadDelegate As New ThreadStart(AddressOf CreateUIAWorker)
Dim workerThread As New Thread(threadDelegate)
workerThread.Start()
End Sub
'''--------------------------------------------------------------------
''' <summary>
''' Delegated method for ThreadStart. Creates a UI Automation worker
''' class that does all UI Automation related work.
''' </summary>
'''--------------------------------------------------------------------
Public Sub CreateUIAWorker()
uiautoWorker = New UIAWorker(targetApp)
End Sub
Private uiautoWorker As UIAWorker
///--------------------------------------------------------------------
/// <summary>
/// Function to playback through a series of recorded events calling
/// a WriteToScript function for each event of interest.
/// </summary>
/// <remarks>
/// A major drawback to using AutomationID for recording user
/// interactions in a volatile UI is the probability of catastrophic
/// change in the UI. For example, the //Processes// dialog where items
/// in the listbox container can change with no input from the user.
/// This mandates that a record and playback application must be
/// reliant on the tester owning the UI being tested. In other words,
/// there has to be a contract between the provider and client that
/// excludes uncontrolled, external applications. The added benefit
/// is the guarantee that each control in the UI should have an
/// AutomationID assigned to it.
///
/// This function relies on a UI Automation worker class to create
/// the System.Collections.Generic.Queue object that stores the
/// information for the recorded user interactions. This
/// allows post-processing of the recorded items prior to actually
/// writing them to a script. If this is not necessary the interaction
/// could be written to the script immediately.
/// </remarks>
///--------------------------------------------------------------------
private void Playback(AutomationElement targetApp)
{
AutomationElement element;
foreach(ElementStore storedItem in uiautoWorker.elementQueue)
{
PropertyCondition propertyCondition =
new PropertyCondition(
AutomationElement.AutomationIdProperty, storedItem.AutomationID);
// Confirm the existence of a control.
// Depending on the controls and complexity of interaction
// this step may not be necessary or may require additional
// functionality. For example, to confirm the existence of a
// child menu item that had been invoked the parent menu item
// would have to be expanded.
element = targetApp.FindFirst(TreeScope.Descendants, propertyCondition);
if(element == null)
{
// Control not available, unable to continue.
// TODO: Handle error condition.
return;
}
WriteToScript(storedItem.AutomationID, storedItem.EventID);
}
}
///--------------------------------------------------------------------
/// <summary>
/// Generates script code and outputs the code to a text control in
/// the client.
/// </summary>
/// <param name="automationID">
/// The AutomationID of the current control.
/// </param>
/// <param name="eventID">
/// The event recorded on that control.
/// </param>
///--------------------------------------------------------------------
private void WriteToScript(string automationID, string eventID)
{
// Script code would be generated and written to an output file
// as plain text at this point, but for the
// purposes of this example we just write to the console.
Console.WriteLine(automationID + " - " + eventID);
}
'''--------------------------------------------------------------------
''' <summary>
''' Function to playback through a series of recorded events calling
''' a WriteToScript function for each event of interest.
''' </summary>
''' <remarks>
''' A major drawback to using AutomationID for recording user
''' interactions in a volatile UI is the probability of catastrophic
''' change in the UI. For example, the 'Processes' dialog where items
''' in the listbox container can change with no input from the user.
''' This mandates that a record and playback application must be
''' reliant on the tester owning the UI being tested. In other words,
''' there has to be a contract between the provider and client that
''' excludes uncontrolled, external applications. The added benefit
''' is the guarantee that each control in the UI should have an
''' AutomationID assigned to it.
'''
''' This function relies on a UI Automation worker class to create
''' the System.Collections.Generic.Queue object that stores the
''' information for the recorded user interactions. This
''' allows post-processing of the recorded items prior to actually
''' writing them to a script. If this is not necessary the interaction
''' could be written to the script immediately.
''' </remarks>
'''--------------------------------------------------------------------
Private Sub Playback(ByVal targetApp As AutomationElement)
Dim element As AutomationElement
Dim storedItem As ElementStore
For Each storedItem In uiautoWorker.elementQueue
Dim propertyCondition As New PropertyCondition( _
AutomationElement.AutomationIdProperty, storedItem.AutomationID)
' Confirm the existence of a control.
' Depending on the controls and complexity of interaction
' this step may not be necessary or may require additional
' functionality. For example, to confirm the existence of a
' child menu item that had been invoked the parent menu item
' would have to be expanded.
element = targetApp.FindFirst( _
TreeScope.Descendants, propertyCondition)
If element Is Nothing Then
' Control not available, unable to continue.
' TODO: Handle error condition.
Return
End If
WriteToScript(storedItem.AutomationID, storedItem.EventID)
Next storedItem
End Sub
'''--------------------------------------------------------------------
''' <summary>
''' Generates script code and outputs the code to a text control in
''' the client.
''' </summary>
''' <param name="automationID">
''' The AutomationID of the current control.
''' </param>
''' <param name="eventID">
''' The event recorded on that control.
''' </param>
'''--------------------------------------------------------------------
Private Sub WriteToScript( _
ByVal automationID As String, ByVal eventID As String)
' Script code would be generated and written to an output file
' as plain text at this point, but for the
' purposes of this example we just write to the console.
Console.WriteLine(automationID + " - " + eventID)
End Sub
Training
Module
Use Power Automate for desktop to interact with windows and applications - Training
Learn how Power Automate for desktop identifies and interacts with UI elements and windows.
Documentation
Obtaining UI Automation Elements - .NET Framework
Review various ways to obtain UI Automation element (AutomationElement) objects for user interface (UI) elements.
Find a UI Automation Element Based on a Property Condition - .NET Framework
Find a UI Automation element based on a property condition. Locate an element within the UI Automation tree based on a specific property or properties.
Get UI Automation Element Properties - .NET Framework
See instructions and an example that shows how to retrieve current or cached properties of a UI Automation element.
UI Automation Properties for Clients - .NET Framework
Read an overview of UI Automation properties as they are exposed to UI Automation client applications.