I have located and fixed my problem. The scalex and scaley functions do not function properly when changing coordinate systems. They only handle simple scaling., Thus, when floodfill and scale(-2.5,-2.5)-(2.5,2.5) come together and you use
Sub ScaleToPixels(Pic As PictureBox, inp As ObjPt, outv As ObjPt)
With inp
outv.x = Pic.ScaleX(.x, Pic.ScaleMode, vbPixels)
outv.y = Pic.ScaleY(-.y, Pic.ScaleMode, vbPixels)
End With
End Sub
to give floodfill its starting point in pixels, it gives you the scale(-2.5,-2.5)-(2.5,2.5) coordinate system you specified instead of scale(0,0)-(maxx,maxy) that floodfill requires. So you have to something like this.
Sub ScaleToPixels(Pic As PictureBox, inp As ObjPt, outv As ObjPt)
With inp
outv.x = 238 + Pic.ScaleX(.x, Pic.ScaleMode, vbPixels)
outv.y = 238 - Pic.ScaleY(-.y, Pic.ScaleMode, vbPixels)
End With
End Sub
238 is the mid-point pixel for my case.