Random RDS Server Disconnects

Question

I am currently looking into an issue with RDS severs randomly disconnecting our Users. We have 6 RDS servers (Windows Server 2012 R2) connecting via our Connection Broker (Windows Server 2012 R2).

From our connection broker when viewing our RDS collection pool, we can see our Users being disconnected from the RDS servers at random times throughout the day. Below is a quick rundown of our RDS environment.

1 Connection Broker (Windows Server 2012 R2)
6 Terminal Servers (Windows Sever 2012 R2)
Approx.140 Terminal Server Users (using various thin client models to connect via RDP)

The sessions will reconnect the User back to the exact point in their remote session as if they never disconnected. We do have some Users who will connect from their homes via their personal computer to a RDS server via our remote desktop gateway who are getting disconnected, but they too will get disconnected. From our connection broker server, we can see Windows logs of the User's & their disconnections from remote computer (RDS server), but no further information other than the domain User & their remote computer they disconnected from is provided.

We've tried rolling back our updates which we installed 3 days prior to our 1st reported issue, doing packet captures from our network switches, & number of other of fixes found off the web. Just wanted to check the Microsoft community to see there's anyone who Manages terminal servers or RDS servers or any Microsoft Employee who post here that could provide some input.