Hello,
You can reference this tutorial: https://www.elevenforum.com/t/remove-shortcut-a...
In the tutorial, OP was asking how to remove the arrow icon, and the concept behind the solution is to replace it with a blank icon file, so it appears the arrow is gone. So you can use the same concept, except you will replace it with your own icon file that has the design you want. I hope this makes sense.
Note: This is a non-Microsoft website. The page appears to be providing accurate, safe information. Watch out for ads on the site that may advertise products frequently classified as a PUP (Potentially Unwanted Products). Thoroughly research any product advertised on the site before you decide to download and install it.