You may consider disabling other Session Hosts during scheduled times to access a specific SH and observe the login process. Alternatively, you can remove it from your collection and connect directly via MSTSC. If load balancing directs users to a different session host, there may be an issue with the specific SH.
Should you manage to connect after removing it from your collection, it likely indicates a load balancing problem; check your connection broker.
If the connection still fails, further investigation of your session host is required.
Please open your Event Viewer and review the System/Application logs, as well as all Remote Desktop/Terminal Services logs.