How to get specific image coordinates from within bitmap

Mansour_Dalir 1,676

User's image

Is there a special (Library/Class) for such an achievement? which can find and return the coordinates by giving a symbol of a specific image from within the larger image

Xiaopo Yang - MSFT 11,661 Reputation points Microsoft Vendor

2024-05-21T05:52:21.8966667+00:00

Hello @Mansour_Dalir, what project type do you use? Windows Forms App, Windows Desktop Application or WinUI3?
Mansour_Dalir 1,676 Reputation points

2024-05-21T06:15:47.39+00:00

hi @Xiaopo Yang - MSFT I don't understand much about the difference you mentioned.(I would be grateful if you could guide me on the difference between them. Anyway) But this is what I have VS 2019 - Windows Forms App(.NET Framework) . I just wanted to know if there is a specific class for this that can be processed on bitmap images

1 answer

Jiachen Li-MSFT 27,411 Microsoft Vendor

Hi @Mansour_Dalir ,

There is no such basic .NET class that can achieve.

You can refer to the following code, this method is straightforward but may be slow for large images or complex comparisons.

If you want more advanced and efficient image processing, you can find some third-party libraries.

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Dim mainImage As Bitmap = Bitmap.FromFile("path_to_main_image.jpg")
        Dim template As Bitmap = Bitmap.FromFile("path_to_template_image.jpg")

        Dim mainWidth As Integer = mainImage.Width
        Dim mainHeight As Integer = mainImage.Height
        Dim templateWidth As Integer = template.Width
        Dim templateHeight As Integer = template.Height

        ' Loop through the main image to find the template
        For x As Integer = 0 To mainWidth - templateWidth
            For y As Integer = 0 To mainHeight - templateHeight
                If CompareImages(mainImage, template, x, y) Then
                    ' Match found
                    MessageBox.Show("Match found at X: " & x & ", Y: " & y)
                    ' Draw a rectangle around the matching area
                    Dim g As Graphics = Graphics.FromImage(mainImage)
                    g.DrawRectangle(New Pen(Color.Red, 3), New Rectangle(x, y, templateWidth, templateHeight))
                    PictureBox1.Image = mainImage
                    g.Dispose()
                    Return
                End If
            Next
        Next

        MessageBox.Show("No match found")
    End Sub

    Private Function CompareImages(mainImage As Bitmap, template As Bitmap, startX As Integer, startY As Integer) As Boolean
        For x As Integer = 0 To template.Width - 1
            For y As Integer = 0 To template.Height - 1
                Dim mainPixel As Color = mainImage.GetPixel(startX + x, startY + y)
                Dim templatePixel As Color = template.GetPixel(x, y)
                ' Check if pixels are similar (you can adjust the tolerance as needed)
                If Not PixelsAreSimilar(mainPixel, templatePixel) Then
                    Return False
                End If
            Next
        Next
        Return True
    End Function

    Private Function PixelsAreSimilar(pixel1 As Color, pixel2 As Color, Optional tolerance As Integer = 50) As Boolean
        ' Compare the RGB values of the pixels within a tolerance
        Return Math.Abs(CInt(pixel1.R) - CInt(pixel2.R)) <= tolerance AndAlso
               Math.Abs(CInt(pixel1.G) - CInt(pixel2.G)) <= tolerance AndAlso
               Math.Abs(CInt(pixel1.B) - CInt(pixel2.B)) <= tolerance
    End Function

Best Regards.

Jiachen Li

If the answer is helpful, please click "Accept Answer" and upvote it.

Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Mansour_Dalir 1,676 Reputation points

2024-05-21T07:16:50.1033333+00:00
hi @Jiachen Li-MSFT Error :System.OverflowException: 'Arithmetic operation resulted in an overflow.' on this Part:

Return Math.Abs(pixel1.R - pixel2.R) <= tolerance AndAlso Math.Abs(pixel1.G - pixel2.G) <= tolerance AndAlso Math.Abs(pixel1.B - pixel2.B) <= tolerance

But I wish there was an answer with high speed and performance. I think it will take some time for large images like the desktop screen image
Mansour_Dalir 1,676 Reputation points

2024-05-21T07:30:44.7833333+00:00

In order for the answer to reach its highest speed, I know the idea and algorithm of this work. If I explain my algorithm, can you translate it into programming?
Jiachen Li-MSFT 27,411 Reputation points Microsoft Vendor

2024-05-21T07:40:07.4166667+00:00

I have edited my code to fix the error.

If you're looking for more efficient matching, then finding a full-fledged third-party library is obviously the easier option.

Mansour_Dalir 1,676

hi @Jiachen Li-MSFT

So far I was able to write a few lines of code to find the image this way. But the rest needs to be checked. Thank you if you check the code and bring this to a conclusion.

User's image

    Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
        Dim intMain() As Integer = GetRgbValues(picMain.Image)
        Dim intTemp() As Integer = GetRgbValues(picTemp.Image)
        Dim ArrTemp = SplitArrayBySpecifiedStep(intTemp, picTemp.Image.Width)
        Dim FirstLine As Integer() = ArrTemp(0)
        Dim IndexOfFirstRow As Integer() = GetIndexsBySpecifiedOrder(intMain, FirstLine) '
        'So far it is going well, but the rest needs to be checked
        Dim ArrMain = SplitArrayBySpecifiedStep(intMain, picMain.Image.Width)
        Dim Coordinates_X_Y As New List(Of Rectangle)
        For Each StartFind In IndexOfFirstRow
            Dim i = StartFind Mod picMain.Image.Width
            Dim t = StartFind \ picMain.Image.Width
            Dim isY As Integer = t
            Dim IsX As Integer = i
            Dim thisIsIt As Integer = 0
            For startCheck = 1 To ArrTemp.Length - 1
                t += 1
                Dim gg = ArrMain(t).Skip(i).Take(ArrTemp(startCheck).Length).Select(Function(h) h).ToArray
                If ArraysSimilarity(gg, ArrTemp(startCheck)) Then
                    thisIsIt += 1
                End If
            Next
            If thisIsIt = ArrTemp.Length - 1 Then
                Dim rec As New Rectangle(IsX, isY, picTemp.Image.Width, picTemp.Image.Height)
                Coordinates_X_Y.Add(rec)
            End If
        Next
        For Each Rec In Coordinates_X_Y
            Dim bt As New Bitmap(picMain.Image)
            Dim g As Graphics = Graphics.FromImage(bt)
            g.DrawRectangle(New Pen(Color.Red, 3), New Rectangle(Rec.X, Rec.Y, Rec.Width, Rec.Height))
            picMain.Image = bt
            g.Dispose()
            ListBox1.Items.Add(Rec.X & "," & Rec.Y)
        Next

    End Sub
    Function ArraysSimilarity(InArray As Integer(), Simiilar As Integer(), Optional tolerance As Integer = 10) As Boolean
        ArraysSimilarity = InArray.SequenceEqual(Simiilar)
    End Function
    Function SplitArrayBySpecifiedStep(InArray As Integer(), ByStep As Integer) As Integer()()
        SplitArrayBySpecifiedStep = InArray.Select(Function(num, index) New With {.num = num, .index = index}) _
             .GroupBy(Function(x) x.index \ ByStep) _
             .Select(Function(g) g.Select(Function(x) x.num).ToArray).ToArray
    End Function
    Function GetIndexsBySpecifiedOrder(inArray As Integer(), SpecifiedOrder As Integer()) As Integer()
        GetIndexsBySpecifiedOrder = inArray.Select(Function(value, index) index) _
                    .Where(Function(index) inArray(index) = SpecifiedOrder(0) AndAlso
                                          If(index + SpecifiedOrder.Length <= inArray.Length,
                                             Enumerable.SequenceEqual(inArray.Skip(index).Take(SpecifiedOrder.Length), SpecifiedOrder),
                                             False)) _
                    .ToArray()
    End Function
    Private Function GetRgbValues(inBitmap As Bitmap) As Integer()
        Dim recBitmap As New Rectangle(0, 0, inBitmap.Width, inBitmap.Height)
        Dim DataMain As System.Drawing.Imaging.BitmapData = inBitmap.LockBits(recBitmap, Drawing.Imaging.ImageLockMode.ReadWrite, Imaging.PixelFormat.Format32bppArgb)
        Dim ptr As IntPtr = DataMain.Scan0
        Dim bytes As Integer = Math.Abs(DataMain.Stride) * recBitmap.Height

        Dim rgbValues(bytes / 4 - 1) As Integer
        System.Runtime.InteropServices.Marshal.Copy(ptr, rgbValues, 0, rgbValues.Length)
        inBitmap.UnlockBits(DataMain)
        Return rgbValues
    End Function

I could not get color values from byte type. To simplify the task of receiving from integer. You receive from the byte. You receive from the byte so that the tolerance amount can be put in (the following function).

 Function ArraysSimilarity(InArray As Integer(), Simiilar As Integer(), Optional tolerance As Integer = 10) As Boolean
        ArraysSimilarity = InArray.SequenceEqual(Simiilar)
    End Function

Mansour_Dalir 1,676 Reputation points

2024-05-23T15:41:38.3333333+00:00

result
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Jiachen Li-MSFT 27,411 Reputation points Microsoft Vendor

2024-05-24T08:12:20.53+00:00

Tolerance is not required, if the specific image you provide is precise.
Mansour_Dalir 1,676 Reputation points

2024-05-24T09:46:18.41+00:00

hi @Jiachen Li-MSFT

Why is the last command deleted? I had provided a new code. that you check. We will fix the code problems.
Jiachen Li-MSFT 27,411 Reputation points Microsoft Vendor

2024-05-27T03:36:02.4133333+00:00

Hi @Mansour_Dalir ,

Consider post the new code in a new problem with your new issue description.

Share via

How to get specific image coordinates from within bitmap

1 answer