RDP slow internally when deny access to internet. But fast when we allow traffic to the internet via our firewall.

Question

The condition is as following:

If we have DENY any any on outgoing traffic to WAN from all mashines, our RDP is very slow connecting. It is hanging at "securing remote connection" for around a minutes or so, and then it connects.

If we have Allow any any on outgoing traffic to WAN from all mashines, out RDP is back to normal and instantly connect and ask for password.

Why is this? We of course do not want internet access at all, but if needed we can allow specific ports for specific locations on WAN.

PS. It is a fully on premise setup