Remote Desktop Error Code 0x204 BUT once logged with Teamviewer it then works.

Question

I'm getting this notorious error "Remote Desktop Error Code 0x204" when trying to connect from Mac to Windows 10 (same Win to Win)
But the interesting thing is that if I connect to the PC with Teamveiwer before, then I can easily connect with Remote Desktop with no issue straightaway. Seems like Windows blocks something when I use Remote Desktop first but somehow Teamviewer unblocks it.

I've tried all this sort of fixes out there. Nothing has helped. It still won't connect. So before connecting with Remote Desktop App, I use Teamviewer.

P.S. I don't just use Teamviewer bc it doesn't work properly with headless PCs. The screen is just black. I know about HDMI dummy things to emulate a display, but it didn't work on me.

Any ideas?
Thank you!