It was really weird. I was having all sorts of issues, both directions. RDP worked fine all the time. But my SSL certificate renewals were failing.
In frustration, I did what I should have done 3 hours earlier and rebooted the server, and that resolved everything. Have no idea why though.