Bagikan melalui

Find a bitmap on the VM screen

  • Artikel

Continuing with my series on how to do interesting automation stuff with Hyper-V – today I want to show you how to use PowerShell and the AForge.NET library to locate a bitmap on a virtual machine screen.

For example – imagine that I had a test running in a Windows Server Core 2016 virtual machine:

source

And I wanted to be able to tell if PowerShell was running interactively – but I did not want to interfere with the guest operating system in anyway.  What could I do?  Well – I could get the screen and look for a bitmap that told me that PowerShell was running.  Something like this (enlarged to make it easier to view):

PShell

Fortunately – this is quite easy to do thanks to the handy AForge.NET libraries.  The result looks something like this:

If I run this script with a single PowerShell Window open I get this result:
Rectangle                        Similarity ---------                        ---------- {X=381,Y=371,Width=13,Height=12}          1
One perfect match at X=381, Y=371

If I open up two more PowerShell Windows and run this script I get this result:
Rectangle                        Similarity ---------                        ---------- {X=303,Y=522,Width=13,Height=12}          1 {X=535,Y=355,Width=13,Height=12}   0.993087 {X=74,Y=700,Width=13,Height=12}    0.993087
One perfect match – and two close matches.  The reason for the difference here is that the close matches are PowerShell Windows that do not have focus – so the graphic is slightly different.

One final point to make about this code sample: Hyper-V produces a 16bit RGB565 image for a screen capture.  Unfortunately the AForge.NET libraries do not accept this format.  So in my sample code you will see that I upsample into 24bit RGB in order to make everything work.

Cheers,
Ben

Comments

  • Anonymous
    August 16, 2016
    Thanks this is a very useful technique that I can see using for all kinds of tests but I've had trouble implementing this myself. When loading the AForge.dll I get a "Could not log file or assembly... or one of its dependencies". Is there a dependency other than System.Drawing I need to make this work? Thanks!-Johann
    • Anonymous
      August 16, 2016
      I was able to resolve this problem by extracting the libraries from the AForge.NET download using WinRar instead of what I was using before. Works exactly as described in the post.