Have you checked the event log if some related event error could be found?
First check if VPN client can ping the remote computer with source VPN address when VPN connected.
Check whether to configure any policy on VPN Server to deny the remote desktop.
To eliminate potential influence against VPN connection, please turn off windows firewall of VPN server and client and try it again.
Hope this helps and please help to accept as Answer if the response is useful.