Hi, Paul
You are bound by the internet link from Azure to the remote service.
10ms to 20ms is pretty tight for access to internet-based services. These are express route speeds over the backend Microsoft backbone. Even if you were to set up a site-to-site VPN between Azure and AWS, the speed would be bound by that link.
Is the AWS service within your control?
If it is, set up a VM in AWS and Azure and run the Azure Connectivity Toolkit (Troubleshooting network performance) to get a valid speed test; then you could TEST a Virtual WAN/Site to Site connection to AWS as a test as well.