Hi Greg_Oz,
I'm Rodrigo and I will help you.
There isn't an option for the website owner to disable the visual search for Edge users, only for the users to disable it locally.
You can leave your suggestion to Microsoft on the Feedback Hub to add a string that allows the website owners to disable the Visual Search feature on their websites.On Edge can select the three dots on the top right corner > Help & Feedback > Send Feedback or open the Feedback Hub on the Start Menu or use the shortcut WinKey + F.
They care about this kind of feedback and there are other users requesting the same as you, let them know.