There were several things that needed to happen in order to fix this:
Configure Loopback settings correctly: https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/accessing-server-locally-with-fqdn-cname-alias-denied
Disable "Enhanced Protection Mode" on Internet Explorer.
Configure Group Policy to allow users to bypass bad certificates.
In my case, #3 was required because the machine cert has a Principal Name that matches the machine's hostname, but the website name is slightly different and thus it thinks it's wrong.
Additionally, Microsoft Guidance claims you can access the site with localhost as the hostname, but this never worked - I was only able to get this to work when using the website's actual name.