Azure Route Server - ExpressRoute - NVA

Question

Azure Route Server - ExpressRoute - NVA

Nathan Farrar 67

Looking to develop a dynamic routing architecture. I'd like to remove the need to use UDRs if possible.

The goal is to have an on-premises data center connect to an Azure spoke VNet via an ExpressRoute and through an NVA in a hub VNet. I've used Azure Route Server to peer with the NVA so it can learn all VNets and provide a default route without using UDRs. The issue I'm expecting is asynchronous routing.

I know that we can use a UDR on the GatewaySubnet to direct incoming traffic to the NVA first, then the traffic would be forwarded to the spoke subnet.

But return traffic is the issue, since I want the NVAs to learn new VNets and ExpressRoute prefixes:

Here is where I think the issue will be:

The spoke VNets will learn ExpressRoute prefixes via the VNet peering (Route Server can't control this)
A 0.0.0.0/0 will also be learned from the NVAs (Via the Route Server)
The ExpressRoute prefixes will always take precedence
Return traffic will bypass the NVA and go directly to the GatewaySubnet and out the ExpressRoute

Without additional control, I don't see that it is possible to control the return routes without placing static UDR routes at each of the spokes to override the ExpressRoute learned routes. Using the UDRs make the Route Server less useful. It'll help the NVAs learn the VNets and ExpressRoute prefixes but we can't prevent ExpressRoute prefixes from being advertised directly to the spoke VNets via the VNet peering, so we have to use UDRs and always make sure we have any new routes for the on-premises data center added manually.

Thoughts?

Nathan Farrar 67 Reputation points

2022-07-29T12:54:22.657+00:00

HI Gita,

This is in the design phase, so it hasn't been built yet.. BUT I would intend to enable "Use the remote virtual network's gateway or Route Server"
on the peering with the hub.

Thanks for the link!

-Nathan
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2022-07-29T13:38:10.573+00:00

Hello @Anonymous ,

Thank you for the update.

Since your setup is in design phase, I have converted my comment to answer for better visibility.

Kindly let us know if the below helps or you need further assistance on this issue.

----------------------------------------------------------------------------------------------------------------

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.

1 answer

Your answer

Nathan Farrar 67 Reputation points

2022-07-29T12:54:22.657+00:00

HI Gita,

This is in the design phase, so it hasn't been built yet.. BUT I would intend to enable "Use the remote virtual network's gateway or Route Server"
on the peering with the hub.

Thanks for the link!

-Nathan
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2022-07-29T13:38:10.573+00:00

Hello @Anonymous ,

Thank you for the update.

Since your setup is in design phase, I have converted my comment to answer for better visibility.

Kindly let us know if the below helps or you need further assistance on this issue.

----------------------------------------------------------------------------------------------------------------

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.

Answer 1

GitaraniSharma-MSFT 50,096 Microsoft Employee Moderator

Hello @Anonymous ,

Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

I understand that you have a hub-spoke model on Azure and it is connected to your on-premises via ExpressRoute. You have a NVA in your hub Vnet for routing the on-prem traffic via the NVA and used Azure Route Server to peer with the NVA so it can learn all VNets and provide a default route without using UDRs but you are facing asymmetric routing issue.

Could you please confirm that the Vnet peering between hub and spoke Vnets has the "Use Remote Gateway" option enabled on the spoke Vnet peering?
Refer : https://learn.microsoft.com/en-us/azure/route-server/route-server-faq#does-azure-route-server-support-virtual-network-peering

I also checked internally and found that there was a known issue where - when NVA advertises default route to Route server, this route is propagated to hub vnet but not spoke vnets. But looks like a fix was already rolled out by the Product Group team.

If your Vnet peering is configured correctly with the "Use Remote Gateway" option enabled and you are still facing this issue, it may require a deeper investigation, so if you have a support plan, I request you file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

Kindly let us know if the above helps or you need further assistance on this issue.

----------------------------------------------------------------------------------------------------------------

Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

Nathan Farrar 67 Reputation points

2022-07-29T12:56:12.91+00:00

Does the use of the Route Server prevent the automatic propagation of ExpressRoute routes directly to the peered VNet?
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2022-07-29T13:36:18.95+00:00

Hello @Anonymous ,

When you deploy an Azure Route Server along with an ExpressRoute gateway and an NVA in a virtual network, by default Azure Route Server doesn’t propagate the routes it receives from the NVA and ExpressRoute gateway between each other. Once you enable the route exchange, ExpressRoute and the NVA will learn each other’s routes. When the same route is learned over ExpressRoute and NVA, the ExpressRoute network will be preferred.
Refer : https://learn.microsoft.com/en-us/azure/route-server/expressroute-vpn-support

Regards,
Gita
Nathan Farrar 67 Reputation points

2022-07-29T18:21:47.357+00:00

Thank you for the information, but I was referring to the peering relationship between two spoke VNets. When a peering exists without Azure Route Server, and an ExpressRoute connection exists, routes will be propagated to the spoke VNet if the option in the peering setup is selected.

My question is; when a Route Server exists in a hub VNet, does the ExpressRoute route propagation still happen behind the scenes to the peered VNet or does the presence of the route server prevent the propagation of ExpressRoute learned routes?

Issue is;

Inbound traffic:
On-premises -> ExpressRoute -> NVA -> Peered VNet

Return traffic:
Peered VNet -> ExpressRoute -> On-Premises

If the peered spoke VNet knows about the more specific express route prefixes, the default route advertised from the NVA will be ignored. My experience is that ExpressRoute prefixes are shared to peered VNets, as would be expected but this gets in the way of the operation of a Route Server.
GitaraniSharma-MSFT 50,096 Reputation points Microsoft Employee Moderator

2022-08-04T17:56:23.973+00:00
Hello @Anonymous ,

Apologies for the delay in response.

When a Route Server exists in a hub VNet and it is peered with the spoke Vnet with the "use remote gateway" option, the ExpressRoute route propagation still happen behind the scenes to the peered VNet along with the ARS routes.

If the Route Server is deployed in the hub VNet with an NVA advertising a 0.0.0.0/0 route and an ExpressRoute gateway advertising prefixes from on-premises, it will configure all routes (both the 0.0.0.0/0 from the NVA and the on-premises prefixes) on the virtual machines in its VNet and directly peered VNets. As a consequence, since the on-premises prefixes will be more specific than 0.0.0.0/0, traffic between on-premises and Azure will bypass the NVA.

To avoid this, you could do one of the following:

Disable BGP propagation in the VM subnets and configure UDRs to next hop NVA. --> Since, you would like to avoid this, you can go with option 2.

Another alternative is a more dynamic approach. It is possible using different Azure Route Servers for different functionality: one of them will be responsible for interacting with the Virtual Network Gateways, and the other one for interacting with the Virtual Network routing.

Refer : https://learn.microsoft.com/en-us/azure/route-server/route-injection-in-spokes#different-route-servers-to-advertise-routes-to-virtual-network-gateways-and-to-vnets

Kindly let us know if the above helps or you need further assistance on this issue.

Regards,
Gita

Share via

Azure Route Server - ExpressRoute - NVA

1 answer

Your answer