Using Inspect.exe to access UI elements that WinAutomation is not able to see

  • Article

There are some desktop applications (let's take Spotify for example), which are designed in such a way that prevents WinAutomation to have access to their UI.

A possible workaround could be using the Inspect.exe (a free tool provided with the Microsoft Windows SDK).

In this example, we are trying to automate the login process in the Spotify client by following these steps:

  1. Have the Spotify client up and running and then run Inspect.exe as administrator.

  2. Use the folder tree in Inspect.exe to navigate to the element that we wish to capture. In this step, we will have to identify one or more properties of the elements, which will be used later on to create the selectors in WinAutomation.

    Screenshot of Spotify login element properties.

  3. In the Process Designer, click on "Add Control" in the Controls Repository pane and then on "Add Empty Root Control". Afterwards, right click on the newly added "Empty Root Control" and select "Add Empty Child Control".

    Screenshot shows how to add empty root control.

  4. Rename the Empty Root Control accordingly and then double click on it, click on "Edit" and replace the "Custom Selector" with the appropriate control.

    In our case, the control for the "Spotify Pane" would be: :desktop > pane[class="Chrome_WidgetWin_0"][process="Spotify"]

  5. Also, double click on the newly added "Empty Control", click on "Edit" and replace the "Custom Selector" with the appropriate control for the element we wish to capture.

    In our case, the control for the "Email" field would be: document edit[name="Email"]:eq(0)*

    We are using the "name" property of the element and we are using the notation '*=', which means that the name contains the word "Email". We are also using the ordinal attribute for the element, which is eq(0), to separate it from the "Password" field.

    Screenshot of Spotify email or username edit properties.

    Accordingly, the control for the "Password" field would be: document edit[name="Password"]:eq(1)

    Screenshot of Spotify password edit properties.

    Lastly, the control for the "LOG IN" button would be: document button[name="LOG IN"]

    Screenshot of Spotify log in button edit properties.

  6. After creating the selectors for the elements, we can use the appropriate actions, such as "Populate Text Field in Window" or "Click Button in Window" to interact with those elements as we wish.