We get that exact error from time to time for hours on end, during which we can't connect for more than 2 seconds until the connection is dropped. And this is on two different AVD sessions in two different regions. Restarts and deallocations of the VM's don't help.
Then, miraculously, hours later or the next morning, everything is fine again for 2-3 weeks. Until it all goes to pieces again, and we have another 2-8 hours of connectivity issues. I think this can also be caused by performance issues on Microsoft's end.
We have three users from three different US time zones having the same issue at the same time each time this happens. Then all of a sudden, we can all get on again.