Connectivity issues on newly provisioned ExpressRoute circuits

Chris Liesfield 20 Reputation points
2023-05-18T03:27:46.8866667+00:00

Hi All. We've recently procured some ExpressRoute circuits with conventional dot1q tagging - not q-in-q.

We have been provided with VLAN IDs to use on the physical router interfaces. We have also been provided with VLAN IDs to use on the Microsoft side.

Having configured both sides, we are not able to ping the Microsoft end (10.7.12.42 on this particular circuit) using the primary interface address or secondary interface address and ARP requests on the routers remain incomplete, so it appears layer two connectivity is failing.

Configuration from router provided below:-

interface TenGigabitEthernet0/1/4.1104

description Axon-AZURE-Private-Access-One

encapsulation dot1Q 1104

vrf forwarding azure

ip flow monitor Stealthwatch unicast input

ip flow monitor Stealthwatch unicast output

ip address 10.7.12.49 255.255.255.252 secondary

ip address 10.7.12.41 255.255.255.252

For the partner provider, they have the following relevant configuration:-

Local VLAN: 1104

Remote VLAN: 32

Link state: Up (remote MAC address(es) learned)

Is there anything I am missing here? Also, is this the correct way to configure the secondary IP address, because I cannot think of an alternative method? I also remain confused as to how BGP sessions will be established using the secondary IP address, to the secondary ExpressRoute interface on the Microsoft side, as I recall secondary addresses are not supported for eBGP.

Thanks in advance for your replies.

Azure ExpressRoute
Azure ExpressRoute
An Azure service that provides private connections between Azure datacenters and infrastructure, either on premises or in a colocation environment.
342 questions
{count} votes

Accepted answer
  1. GitaraniSharma-MSFT 49,391 Reputation points Microsoft Employee
    2023-05-18T10:52:35.26+00:00

    Hello @Chris Liesfield ,

    Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

    I understand that you are facing connectivity issues on your newly provisioned ExpressRoute circuit and the ARP is failing.

    1# Let's start with ARP failure:

    "Having configured both sides, we are not able to ping the Microsoft end (10.7.12.42 on this particular circuit) using the primary interface address or secondary interface address and ARP requests on the routers remain incomplete, so it appears layer two connectivity is failing."

    Could you please try to get the ARP table and check the results following the below doc?

    Refer: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-troubleshooting-arp-resource-manager#getting-the-arp-tables-for-your-expressroute-circuit

    As mentioned, if a problem with the on-premises or connectivity provider occurs, the ARP table will show one of two things. You'll either see the on-premises MAC address show incomplete or only see the Microsoft entry in the ARP table.

    If this is your case, open a support request with your connectivity provider to debug such issues. If the ARP table does not have IP addresses of the interfaces mapped to MAC addresses, review the following information:

    • If the first IP address of the /30 subnet assigned for the link between the MSEE-PR and MSEE is used on the interface of MSEE-PR. Azure always uses the second IP address for MSEEs.
    • Verify if the customer (C-Tag) and service (S-Tag) VLAN tags match both on MSEE-PR and MSEE pair.

    2# Now, coming to the router configuration using dot1Q encapsulation, you could refer the below doc which provides interface and routing configuration samples for Cisco IOS-XE and Juniper MX series routers when you're working with Azure ExpressRoute.

    Refer: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-config-samples-routing

    NOTE: As mentioned in the doc, samples on this page are purely for guidance. You must work with your vendor's sales/technical team and your networking team to find appropriate configurations to meet your needs. Microsoft won't support issues related to configurations listed in this page. Contact your device vendor for support issues.

    To further validate your ExpressRoute connectivity, you could refer the below doc:

    https://learn.microsoft.com/en-us/azure/expressroute/expressroute-troubleshooting-expressroute-overview

    3# Lastly, I would like to address your final question regarding secondary IP address: "I also remain confused as to how BGP sessions will be established using the secondary IP address, to the secondary ExpressRoute interface on the Microsoft side".

    Azure ExpressRoute uses BGP, an industry standard dynamic routing protocol, to exchange routes between your on-premises network, your instances in Azure, and Microsoft public addresses. We establish multiple BGP sessions with your network for different traffic profiles. An ExpressRoute circuit has multiple routing domains/peerings associated with it: Azure public, Azure private, and Microsoft. Each peering is configured identically on a pair of routers (in active-active or load sharing configuration) for high availability.

    Each peering requires separate BGP sessions (one pair for each peering type). The BGP session pairs provide a highly available link.

    Your service provider can establish two Ethernet virtual circuits over the physical connection. The physical connection (for example, an optical fiber) is terminated on a layer 1 (L1) device. The two Ethernet virtual circuits are tagged with different VLAN IDs, one for the primary circuit, and one for the secondary.

    So basically, we are using a pair of VLAN subinterfaces with the same VLAN ID and different /30 subnets for primary and secondary BGP sessions/peerings.

    To establish a redundant pair of BGP sessions per peering, we require the below:

    • A pair of subnets that aren't part of any address space reserved for virtual networks. One subnet will be used for the primary link, while the other will be used for the secondary link. From each of these subnets, you'll assign the first usable IP address to your router as Microsoft uses the second usable IP for its router. You have three options for this pair of subnets:
    • IPv4: Two /30 subnets.
    • IPv6: Two /126 subnets.
    • Both: Two /30 subnets and two /126 subnets.
    • A valid VLAN ID to establish this peering on. Ensure that no other peering in the circuit uses the same VLAN ID. For both Primary and Secondary links you must use the same VLAN ID.
    • AS number for peering.

    Refer: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-routing#ip-addresses-used-for-peerings

    https://learn.microsoft.com/en-us/azure/expressroute/expressroute-howto-routing-portal-resource-manager#private

    Kindly let us know if the above helps or you need further assistance on this issue.


    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.

    0 comments No comments

0 additional answers

Sort by: Most helpful