Get the ElementFromPoint (IUIAutomation) from the 2nd scaled monitor

Question

Hello,
I'm getting the wrong IUIAutomation element from the 2nd monitor, if use it in a browser window like edge (which is not the main monitor and has a dpi for i.e. 125% in the windows settings.)
I tried a few things and can't find a working solution.

The following is not working (they are all far above the element under the mouse cursor):

• GetCursorPos
• GetPhysicalCursorPos
• Mousehook (hookStruct.pt.X + " "+ hookStruct.pt.Y)
• PhysicalToLogicalPointForPerMonitorDPI (don't understand how i could get the cursor position since i'm getting the window's data?!)

I've read:
https://learn.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-and-screen-scaling
https://support.microsoft.com/en-us/topic/windows-scaling-issues-for-high-dpi-devices-508483cd-7c59-0d08-12b0-960b99aa347d

It's not supported by a method, so i have to calculate it by myself after getting GetCursorPos?! (Getting the dpi, the monitor bounds to offset it against the GetCursorPos.)

[...and while i'm asking, i'm reading that it could be working by SetThreadDpiAwarenessContext function? (because it looks really weird, if my form's dpiAwareness is true). ------ FAILED]

EDIT: I also tried to get the dpi and monitor bounds for calculating it by the following example (gives me really weird data) :

https://stackoverflow.com/questions/29438430/how-to-get-dpi-scale-for-all-screens

Isn't there a solution for just getting the set dpi from the windows settings for the specific monitor and simply calculating the right cursor position for finding the element?!

Regards

Answer 1

Hello?!
I've discovered that i get the right ElementFromPoint on my 2nd monitor, if i set the DPI Awareness of my application to "Per-Monitor" and the target application's DPI Awareness is Per-Monitor like Firefox.
I discovered this in the taskmanager, column "DPI Awareness" and tested it only in Chrome and Edge before (both are Unaware), hence the confusion.
How can i solve this and get the right ElementFromPoint in unaware applications like Chrome and Edge?

(Will try this as suggested: https://learn.microsoft.com/en-us/windows/win32/gdi/positioning-objects-on-a-multiple-display-setup)

https://learn.microsoft.com/en-us/dotnet/framework/ui-automation/ui-automation-and-screen-scaling

Regards

Answer 2

Hey,
Don't know what i did but it's working also in Chrome now (at the end in Firefox and Chrome. I've changed the DPI Awareness in the manifest to unaware.
I wanted it to work also in other browsers like Edge (Opera...etc.) since i started to use Edge myself.
I've used the code of Cody Gray to convert the logical to physical cursor position within a dedicated thread with SetThreadDpiAwarenessContext:
https://stackoverflow.com/questions/33507031/detect-if-non-dpi-aware-application-has-been-scaled-virtualized
(That doesn't work exactly for the mouse position and it's really fast out of the physical window when you are still within the window with the cursor on the screen. Sorry, if i mess things up here!)

The issue in for i.e. in Edge is, that if a website has a side menu on the left (maybe 1/5 of the width), it takes more than the half of the physical width (not exact values). That's what i've discovered by manual testing, it seems that it's structured totally different as what you see on the screen. I think there are several things that I can't influence and it's by implementation of the manufacturer or conditions in general (zoom factor...or so).
I will ignore the side menu as a workaround in any other browser for my case (i could imagine a workaround by other properties like ControlType).

Thanks for the support.

Interesting:
https://blogs.windows.com/windowsdeveloper/2016/10/24/high-dpi-scaling-improvements-for-desktop-applications-and-mixed-mode-dpi-scaling-in-the-windows-10-anniversary-update/