UIAutomation not catching all elements

Eliza 21 Reputation points
2021-09-09T04:33:49.477+00:00

Im using the code above to walk through the descendants of an element, however its not catching everything like i can see using inspect.exe.
inspect-objects

I'm testing in the chrome browser, in this current webpage, its catching just current opened pages, bookmarks, and browser buttons, its missing to catch the page content.

While testing in others windows like a folder for example, it does catch everything correctly, but its taking too much time, in a folder with 120 elements it takes up to 1.4 seconds, i wonder if the code could be improved in any manner?

WCHAR window[250] = L"Ask a question - Microsoft Q&A";  
IUIAutomationElement *element = GetTopLevelWindowByName(window);  
ListDescendants(element, 2);  
  
IUIAutomationElement* GetTopLevelWindowByName(LPWSTR windowName)  
{  
    if (windowName == NULL)  
        return NULL;  
  
	CComPtr<IUIAutomation> g_pAutomation;  
	if (SUCCEEDED(CoInitialize(NULL)))  
	{  
		if (!SUCCEEDED(g_pAutomation.CoCreateInstance(CLSID_CUIAutomation8))) // or CLSID_CUIAutomation  
			return NULL;  
	}  
  
    VARIANT varProp;  
    varProp.vt = VT_BSTR;  
    varProp.bstrVal = SysAllocString(windowName);  
    if (varProp.bstrVal == NULL)  
        return NULL;  
  
    IUIAutomationElement* pRoot        = NULL;  
    IUIAutomationElement* pFound       = NULL;  
    IUIAutomationCondition* pCondition = NULL;  
  
    // Get the desktop element.   
    HRESULT hr = g_pAutomation->GetRootElement(&pRoot);  
    if (FAILED(hr) || pRoot == NULL)  
        goto cleanup;  
  
    // Get a top-level element by name, such as "Program Manager"  
    hr = g_pAutomation->CreatePropertyCondition(UIA_NamePropertyId, varProp, &pCondition);  
    if (FAILED(hr))  
        goto cleanup;  
  
    pRoot->FindFirst(TreeScope_Children, pCondition, &pFound);  
  
cleanup:  
    if (pRoot != NULL)  
        pRoot->Release();  
  
    if (pCondition != NULL)  
        pCondition->Release();  
  
    VariantClear(&varProp);  
    return pFound;  
}  
  
  
  
void ListDescendants(IUIAutomationElement* pParent, int indent)  
{  
	static CComPtr<IUIAutomation> g_pAutomation;  
  
	if (!g_pAutomation) {  
		if (SUCCEEDED(CoInitialize(NULL)))  
		{  
			if (!SUCCEEDED(g_pAutomation.CoCreateInstance(CLSID_CUIAutomation8))) // or CLSID_CUIAutomation  
				return;  
		}  
	}  
  
  
	if (pParent == NULL)  
		return;  
  
	IUIAutomationTreeWalker* pControlWalker = NULL;  
	IUIAutomationElement* pNode = NULL;  
  
	g_pAutomation->get_ControlViewWalker(&pControlWalker);  
	if (pControlWalker == NULL)  
		goto cleanup;  
  
	pControlWalker->GetFirstChildElement(pParent, &pNode);  
	if (pNode == NULL)  
		goto cleanup;  
  
	while (pNode)  
	{  
		BSTR sName;  
		pNode->get_CurrentName(&sName);  
  
		UIA_HWND uia_hwnd;  
		pNode->get_CurrentNativeWindowHandle(&uia_hwnd);  
  
		RECT rect;  
		pNode->get_CurrentBoundingRectangle(&rect);  
  
		ListDescendants(pNode, indent + 1);  
		IUIAutomationElement* pNext;  
		pControlWalker->GetNextSiblingElement(pNode, &pNext);  
		pNode->Release();  
		pNode = pNext;  
	}  
  
cleanup:  
	if (pControlWalker != NULL)  
		pControlWalker->Release();  
  
	if (pNode != NULL)  
		pNode->Release();  
  
	return;  
}  
Windows development | Windows API - Win32
Developer technologies | C++
0 comments No comments
{count} votes

3 answers

Sort by: Most helpful
  1. Xiaopo Yang - MSFT 12,731 Reputation points Microsoft External Staff
    2021-09-10T06:49:53.487+00:00

    Perhaps page contents are not IUIAutomationElement nodes. It is a UI Automation document content client sample which can access the content of a document that is being displayed in another application's window.

    0 comments No comments

  2. Eliza 21 Reputation points
    2021-09-10T06:53:17.86+00:00

    Hello XiaopoYang, the source code compiles only in vs2012? no chance to compile in vs19?


  3. Castorix31 90,686 Reputation points
    2021-09-10T17:07:14.903+00:00

    You can get all the elements in the hierarchy with IUIAutomationElement::FindAll or IUIAutomationElement::FindAllBuildCache and TreeScope_Descendants
    There are returned in an IUIAutomationElementArray array, so it is faster than IUIAutomationTreeWalker
    If I test with Edge Chromium on this page, I get all the elements inside the page, like all your code


Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.