Thank you Dave. I installed Revo Uninstaller, and it does not show it as an installed program either!
So after that didn't work, I went digging through the C drive and specifically looked in this folder:
C:\Program Files\Google\Drive File Stream\91.0.2.0
Where 91.0.2.0 is the current version of Google Drive.
And inside that folder is a program called "Uninstall". So I ran it and a pop up asked me if I want to uninstall Google Drive, to which I said yes, and it did indeed uninstall it. I haven't looked very deeply into it but it does look like it cleaned up after itself pretty nicely.