[UWP] Bugreport: InputInjector injects touches in wrong position in case display is scaled

Question

We are evaluating using the Windows.UI.Input.Preview.Injection namespace to automate testing our UWP (WinUI 2.8) application.
We would like to simulate inputs using InputInjector. We tried the sample app available at:
https://learn.microsoft.com/en-us/windows/apps/design/input/input-injection . Using a multi-monitor setup (which most of our developers have), we determined that inputs are generated in the wrong positions. We tried using 100% scale on all monitors, switching primary displays etc., but nothing helped.

Answer 1

Hello,

Welcome to Microsoft Q&A!

I could reproduce the behavior you mentioned now with a monitor which scale value is not 100%. The injected pointer position is using a wrong value. The value is not scaled. Yes, we could write code to solve this. What we need to do is just handle the DPI change event in your app and then get the DisplayInformation.RawPixelsPerViewPixel Property. It represents the number of raw (physical) pixels for each view (layout) pixel. After that, we could calculate the correct position.

You will need to create a rawPixelsPerViewPixel value in the MainPage first.

 public double rawPixelsPerViewPixel { get; set; }

Then please handle the DisplayInformation.DpiChanged Event in the MainPage and get the rawPixelsPerViewPixel value

  public MainPage()
        {
            this.InitializeComponent();

            ApplicationView.PreferredLaunchViewSize =
                new Size(600, 200);
            ApplicationView.PreferredLaunchWindowingMode =
                ApplicationViewWindowingMode.PreferredLaunchViewSize;


            DisplayInformation displayInformation = DisplayInformation.GetForCurrentView();
            displayInformation.DpiChanged += DisplayInformation_DpiChanged;
            // Give a default value for scale value 100%
            rawPixelsPerViewPixel = 1.00;

            // Button handles PointerPressed/PointerReleased in 
            // the Tapped routed event, but we need the container Grid 
            // to handle them also. Add a handler for both 
            // PointerPressedEvent and PointerReleasedEvent on the input Grid 
            // and set handledEventsToo to true.
            ContainerInput.AddHandler(PointerPressedEvent,
                new PointerEventHandler(ContainerInput_PointerPressed), true);
            ContainerInput.AddHandler(PointerReleasedEvent,
                new PointerEventHandler(ContainerInput_PointerReleased), true);
        }


       private void DisplayInformation_DpiChanged(DisplayInformation sender, object args)
        {
            Debug.WriteLine("DPI change");

            DisplayInformation displayInformation = DisplayInformation.GetForCurrentView();
            rawPixelsPerViewPixel = displayInformation.RawPixelsPerViewPixel;
        }

After that, what you need to do is just to give the inject pointer a correct position. In the InjectTouchForMouse method, modify the injectionPoint.

 // Get the screen coordinates (relative to the input area) 
                // of the input pointer.
                int pointerPointX = (int)pointerPoint.Position.X;
                int pointerPointY = (int)pointerPoint.Position.Y;

                // Create the point for input injection and calculate its screen location.
                Point injectionPoint =
                    new Point(
                        (appBoundsTopLeft.X + injectAreaTopLeft.X + pointerPointX) * rawPixelsPerViewPixel,
                        (appBoundsTopLeft.Y + injectAreaTopLeft.Y + pointerPointY) * rawPixelsPerViewPixel);

After that, the app should be able to work correctly in monitors with different scale value.

Thank you.

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.