ASR gateway in East US times out on half of all requests, failing app consistent recovery points

Question

Three Linux VMs replicating with Azure Site Recovery A2A in East US fail app consistent recovery points several times a day. Two identical VMs in other regions never fail.

The agent logs show repeated curl error 56, failure when receiving data from the peer, when calling the ASR gateway. Counted over seven days:

East US VM 1: 457 failed calls, 429 succeeded.

East US VM 2: 318 failed calls, 313 succeeded.

East US VM 3: 260 failed calls, 271 succeeded.

East US 2 VM: 0 failed calls, 872 succeeded.

West US 2 VM: 0 failed calls, 221 succeeded.

Running curl by hand against the gateway endpoint from inside the VMs reproduces it: 40 to 60 percent of requests time out in East US, and none time out from the other two regions. The connection is established and the request is sent, then the peer stops responding until the client gives up.

All five VMs run the same kernel, the same agent version and the same replication model. Disk write rate peaks at 0.29 MB per second against a 20 MB per second limit. The portal reports the agent as healthy with no configuration issues.

Has anyone seen an ASR gateway behave like this, and is there any supported way to move a protected item to a different gateway in the same source region? Disabling and re-enabling replication, using a new vault, and changing the replication model all kept the same gateway.

Versions, identical across all five VMs:

Mobility Service 9.67.0.0, build RELEASE_9.67.0.0_GA_7789_Apr_30_2026_INMAGE.

Kernel 6.11.0-1018-azure.

Ubuntu 24.04 LTS server, Standard_B2ms, Trusted Launch with vTPM and Secure Boot.

The only guest difference I found is the image build date, 24.04.202603120 on one failing VM against 24.04.202510010 on a working one, and that does not track the failures in either direction.

Error codes, as they appear in the portal and in the agent log:

152007, app-consistent recovery tag generation failed.

152006, recovery tag generation has failed 3 times consecutively.

153006, no app-consistent recovery point available in the last 240 minutes.

vacpError 10018 in vacp.log, VM change tracking details post validation failed.

curl error 56 in vacp.log, failure when receiving data from the peer.

Failing snapshot runs exit with status 155 or 4, and the portal alert text quotes those same values.

Infrastructure comparison: no NSG has a custom outbound rule, no subnet has a route table, NAT gateway, service endpoint or custom DNS, accelerated networking is off everywhere, MTU is 1500.

The gateway resolves to a single IPv4 address per region, so this is not DNS or a multi-endpoint issue. Five consecutive calls to that one IP from the failing East US VM: timeout at 10.0s, timeout at 10.0s, HTTP 401 in 0.107s, timeout at 10.0s, HTTP 401 in 0.097s. The same five calls from the East US 2 VM to its own gateway IP: 401 in 0.089s, 0.091s, 0.089s, 0.097s, 0.089s.

The 401 is expected without agent credentials and confirms the endpoint answered. Same client, same destination IP, alternating between answering in under a tenth of a second and never answering at all. When it does answer, TCP connect takes 0.008s and the TLS handshake 0.057s, which rules out a slow path or an MTU problem.

Answer 1

Hi @RuiMoreira-7513

That's a really well-built case, and it pretty much rules out your side. Same client and same IP, with a clean 0.1s answer on one call and complete silence on the next, after TCP and TLS both succeed. That pattern usually means one unhealthy backend instance behind the regional gateway's load balancer, and your calls only fail when they land on it. Nothing in your VMs or network can fix that, so it needs Microsoft to pull the bad node.

1. There's no supported way to pin a protected item to a different gateway. As you found, new vaults and re-enabling replication stay on the same regional endpoint.
2. The closest supported workaround is private endpoints for the vault. With them, the agent talks to the vault over private IPs instead of the public gateway address, which changes the ingress path. Private endpoints can only be added to a vault with no protected items, so it means a new vault plus private DNS zones. It isn't guaranteed to avoid the faulty node, but it's worth a test on one VM.
3. Meanwhile, crash-consistent points should keep coming, so your RPO is still covered. If the alerts are noisy, you can lengthen the app-consistent snapshot frequency in the replication policy for now.
4. Open a technical support case for Site Recovery and attach exactly what you posted here: the per-VM counts, the five timed calls to the same IP, the UTC timestamps, the vault name, and ideally a tcpdump from one failing call showing the request leaving with no response. Ask for it to go to the ASR service team as a suspected unhealthy regional gateway node in East US. Evidence this clean usually gets routed fast.

Moderators here can also escalate it and will ask for your vault details by private message.

If this helped, please click Accept Answer so others seeing the same timeouts can find it.

References: https://learn.microsoft.com/en-us/azure/site-recovery/azure-to-azure-how-to-enable-replication-private-endpoints https://learn.microsoft.com/en-us/azure/site-recovery/azure-to-azure-troubleshoot-network-connectivity