Hello,
For your description, sorry that I have no way to implement it.
However, you can create an Active Directory group for general users and add all users who should be able to log on to either server to this group. Then, create another Active Directory group for the specific users that will always connect to a single server and add those users to this group. This way, you can ensure that all users of the collection can log on to either one of the servers, while a specific group of users always logs on to a single specific server.
I hope the above information is helpful.
Regards,
Jacen Wang