Since you mentioned that it works fine initially with low load, I suspect this to be a bandwidth related issue. To troubleshoot further-
- Please check to see if your express route bandwidth threshold is being reached. You can use Express Route monitoring tools available on Azure to check the same. If the bandwidth threshold is being reached, please see if you can upgrade the bandwidth speed and see if that helps with the timeout errors.
- You could further do an Iperf test to determine the bandwidth being achieved over this Express route connection to further understand if you are getting the speed as expected.
- You can also try connecting via internet or S2S VPN if there is a way to do this and see if that makes any difference to see if the network is the bottleneck or the endpoints.
Please let me know further if you need any more assistance and we will be glad to help. Thank you!