With two S2S VPN connections, how should traffic be routed to a certain S2S connection in the event of ER downtime?

Question

With two S2S VPN connections, how should traffic be routed to a certain S2S connection in the event of ER downtime?

veerabose chandran 260

Please check the attached diagram for network topology & traffic flow.

ENV

A) On-perm

B) Azure

C) 3rd Party vendor

Currently traffic flows from A(on-perm) -> Express Route -> B(Azure) -> S2S conn 2 -> C (Vendor) and return traffic flow via C -> B -> A using the same path --> its working fine.
Client want to test the ER failure situation, and they did some network level modification on on-perm and routed all the traffic from A(on-perm) -> Privat connection -> C (Vendor) -> S2S Conn2 -> B(Azure), traffic reached B(Azure), however return traffic failed to use the same path, we believe since ER circuit is in active state and ERGW take precede than VPN gateway. --> Failed.

we have query regarding point 2)

Query1) Traffic from A(on-perm) -> Privat connection -> C (Vendor) -> S2S Conn2 -> B(Azure S2S conn2), since already there is B(Azure S2S conn1) connection to A(on-perm), will traffic goes to source A(on-perm) using the B(Azure S2S conn1) or B(Azure S2S conn2), is there a option to set routing priority to use the B(Azure S2S conn2)?

Thanks

Veera.

Accepted answer

0 additional answers

Your answer

Answer 1

Luis Arias 8,611 Moderator

Hi Veera,

Based on the information you’ve provided, it seems that you’re trying to test a failover scenario where traffic is rerouted from your on-premises network (A) to a third-party vendor © and then to Azure (B) via a private connection and Site-to-Site (S2S) connection. However, the return traffic is not following the same path.

This could be due to the active state of the ExpressRoute (ER) circuit and the precedence of the ExpressRoute Gateway (ERGW) over the VPN gateway. The implications:

In Azure, the routing of traffic is determined by system routes, user-defined routes, and Border Gateway Protocol (BGP) routes, in that order so when there are multiple routes to the same destination, Azure selects the route type based on the priority order: User-defined routes > BGP routes > System routes

https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview

If there are two S2S connections from Azure (B) to your on-premises network (A), the traffic will follow the route determined by Azure’s routing mechanisms. If both S2S connections are advertised with BGP, Azure VPN gateway will honor AS Path prepending to help make routing decisions. That's mean A shorter AS Path will be preferred in BGP path selection

https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-vpn-faq#does-azure-vpn-gateway-honor-as-path-prepending-to-influence-routing-decisions-between-multiple-connections-to-my-on-premises-sites

If you want to change the routing behaviour to prefer S2S connection 2, you might need to modify the BGP advertisements or use user-defined routes. However, manipulating routing might have unintended side effects. Therefore, be carefull with any change on BGP could impact your entire network.

Here another question than you can check with information about active/active azure routing:

https://learn.microsoft.com/en-us/answers/questions/487275/active-active-azure-s2s-vpn-and-asymmetric-routing

Let me know if this information is helpful.

Cheers,

Luis

veerabose chandran 260 Reputation points

2023-12-01T03:52:42.76+00:00

Hi Luis Arias,

Good day.

Thank you for your detailed information and time; it was really helpful. Could you please assist me through the process of creating the UDR to route traffic to S2S connection 2?

I attempted to create a UDR after selecting the next hop as the virtual network gateway. There is no option to choose the preferred S2S.

Thanks

Veera.
KapilAnanth-MSFT 49,421 Reputation points Microsoft Employee Moderator

2023-12-01T05:17:17.7166667+00:00

Hi Veera,

You cannot set UDR to prefer one S2S Connection over other.

You can only set the next Hop as VNET Gateway.

As mentioned by Luis Arias,

If there are two S2S connections from Azure (B) to your on-premises network (A), the traffic will follow the route determined by Azure’s routing mechanisms. If both S2S connections are advertised with BGP, Azure VPN gateway will honor AS Path prepending to help make routing decisions. That's mean A shorter AS Path will be preferred in BGP path selection https://learn.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-vpn-faq#does-azure-vpn-gateway-honor-as-path-prepending-to-influence-routing-decisions-between-multiple-connections-to-my-on-premises-sites

i.e., the traffic will go to the S2S Connection from which you are advertising the route.

If both the S2S Connections are advertising the same route, BGP AS Path prepending is honored.

Cheers,

Kapil.
veerabose chandran 260 Reputation points

2023-12-01T05:42:17.4166667+00:00

Hi Kapil Anand,

Good day,

Thank for you response and time. I have one more question. I currently have ERGW and VPGW. Without disabling the ERGW, can I route traffic over VPNGW using UDR, in UDR if I select nexthop as the appliance and provide the VPNGW public IP? Will it be successful?

Thanks

Veera.
KapilAnanth-MSFT 49,421 Reputation points Microsoft Employee Moderator

2023-12-01T06:23:31.0433333+00:00

Hi Veera,

Good day!

Unfortunately, you cannot.

The next Hop is "Virtual network Gateway" and not VPN Gateway.

A subtle, but important point to note.

The only way to achieve VPN gateway priority over ExR Gateway is to advertise a shorter prefix via the VPN Gateway.

See: How Azure selects a route

Cheers,

Kapil
veerabose chandran 260 Reputation points

2023-12-04T03:28:40.8666667+00:00

Hi Kapil Aanad,

Thank for you response and time. I have rephrased my earlier question, I currently have ERGW and VNGW(S2S TUNNEL). Without disabling the ERGW, we have BGP peering configured both ERGW & VNGW. can I route traffic over VNGW(S2S TUNNEL) using UDR, in UDR if I select nexthop as the Virtual appliance and provide the VNGW(S2S TUNNEL) public IP? Will be traffic go via S2S tunnel than ERGW?

Thanks

Veera.
GitaraniSharma-MSFT 49,911 Reputation points Microsoft Employee Moderator

2023-12-05T14:27:33.11+00:00

Hello @Anonymous ,

I currently have ERGW and VNGW (S2S TUNNEL). Without disabling the ERGW, we have BGP peering configured both ERGW & VNGW. Can I route traffic over VNGW (S2S TUNNEL) using UDR, in UDR if I select nexthop as the Virtual appliance and provide the VNGW (S2S TUNNEL) public IP? Will be traffic go via S2S tunnel than ERGW?

No, it will not work. The UDR to route traffic via a VPN gateway should be created with a next hop as "Virtual network gateway".

But you can't specify Virtual Network Gateways as next hop in UDR, if you have VPN and ExpressRoute coexisting connections.

Refer: https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview#user-defined

When you have an ExpressRoute and Site-to-Site coexistence setup and both routes are the same, ExpressRoute circuit is preferred over Site-to-Site VPN. The ExpressRoute circuit is always the primary link. Data flows through the Site-to-Site VPN path only if the ExpressRoute circuit fails.

While ExpressRoute circuit is preferred over Site-to-Site VPN when both routes are the same, Azure will use the longest prefix match to choose the route towards the packet's destination.

Refer: https://learn.microsoft.com/en-us/azure/expressroute/how-to-configure-coexisting-gateway-portal#configure-a-site-to-site-vpn-as-a-failover-path-for-expressroute

The only way to prefer VPN gateway over ExR Gateway is to advertise a shorter/more specific prefix via the VPN Gateway.

Advertise more specific prefixes on the VPN BGP session: You can advertise a larger range over ExpressRoute private peering and more specific ranges in the VPN BGP session. For example, advertise 10.0.0.0/16 over ExpressRoute, and 10.0.1.0/24 over VPN.

This way Azure will use the longest prefix match and send traffic to 10.0.1.0/24 over the VPN connection rather than directly over ExpressRoute.

Refer: https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview#how-azure-selects-a-route

Regards,

Gita
veerabose chandran 260 Reputation points

2023-12-13T04:29:13.65+00:00

Hi Gita,

Good day.

Thank you for your detailed information and time; it was really helpful. I have one more question; could you please check and respond?

If I need to modify the route in an existing ER circuit private peering? Is it better to perform it on Azure end or at the Express Route PE or CE router?

Thanks

Veera.
KapilAnanth-MSFT 49,421 Reputation points Microsoft Employee Moderator

2023-12-13T04:33:29.02+00:00
Hello @Veera ,

For a Private Peering,

You cannot influence the routes from Azure End

The Circuit will advertise to OnPrem, all the VNET Address spaces this is linked to via BGP.

If your goal is to change the OnPrem Routes advertised to Azure , you must make the changes in the CE Router.

Specify the OnPREM address range that needs to be exchanged via BGP.

Cheers,

Kapil

Share via

With two S2S VPN connections, how should traffic be routed to a certain S2S connection in the event of ER downtime?

0 additional answers

Your answer