Configure multiple network interfaces in Azure Linux virtual machines
Article
Note
CentOS referenced in this article is a Linux distribution and will reach End Of Life (EOL). Consider your use and plan accordingly. For more information, see CentOS End Of Life guidance.
This article discusses how to configure multiple virtual network interfaces in Azure Linux virtual machines (VMs) that run the most common Linux distributions.
Summary
You can create an Azure VM that has multiple network interfaces attached to it. A common scenario is to have different subnets for front-end and back-end connectivity, or a network that's dedicated to a monitoring or backup solution.
This article provides the required configuration for multiple network interfaces to work in an Azure Linux VM that's based on the following sample scenario:
The VM has two or more network interfaces in the same subnet.
The VM has two or more network interfaces in different subnets but in the same Virtual Network (VNET).
In each scenario, the connectivity can be tested from any VM in the same VNET.
Important
This same process could also be followed in VMs with more than two NICs.
Configure guest OS for multiple network interfaces
When you add multiple network interfaces to a Linux VM, you have to create routing rules. These rules enable the VM to send and receive traffic that belongs to a specific network interface. Otherwise, traffic can't be processed correctly. For example, traffic that belongs to eth1 can't be processed correctly by the defined default route.
The following sections provides the required configuration for two network interfaces to work in Linux VMs that run the most common Linux distributions.
Note
Run all the commands in the following sections by using root privileges (by switching to the root or by using the sudo command utility).
In each section, assume that the VM has two network interfaces that have any of the following settings:
Routing (the output of the sudo ip route show command):
Two NICs in the same subnet:
default via 10.0.1.1 dev eth0 proto static metric 100
10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.4 metric 100
10.0.1.0/24 dev eth1 proto kernel scope link src 10.0.1.5 metric 101
168.63.129.16 via 10.0.1.1 dev eth0 proto dhcp metric 100
169.254.169.254 via 10.0.1.1 dev eth0 proto dhcp metric 100
Two NICs in different subnets but in the same VNET:
default via 10.0.1.1 dev eth0 proto static metric 100
10.0.1.0/24 dev eth0 proto kernel scope link src 10.0.1.4 metric 100
10.0.2.0/24 dev eth1 proto kernel scope link src 10.0.2.5 metric 101
168.63.129.16 via 10.0.1.1 dev eth0 proto dhcp metric 100
169.254.169.254 via 10.0.1.1 dev eth0 proto dhcp metric 100
Interfaces (the output of the sudo ip address show command):
Two NICs in the same subnet:
lo: inet 127.0.0.1/8 scope host lo
eth0: inet 10.0.1.4/24 brd 10.0.1.255 scope global eth0
eth1: inet 10.0.1.5/24 brd 10.0.1.255 scope global eth1
Two NICs in different subnets but in the same VNET:
lo: inet 127.0.0.1/8 scope host lo
eth0: inet 10.0.1.4/24 brd 10.0.1.255 scope global eth0
eth1: inet 10.0.2.5/24 brd 10.0.2.255 scope global eth1
If more network interfaces are attached to the VM, add extra routing tables (for example, 202 eth2-rt, 203 eth3-rt, and so on).
Make sure a configuration file exists for each network interface in the /etc/sysconfig/network-scripts/ directory. You can create new network interface configuration files based on the ifcfg-eth0 configuration file (modify the DEVICE line and remove the DHCP_HOSTNAME and HWADDR lines from the new file). To do this, run the following commands:
sudo cat /etc/sysconfig/network-scripts/ifcfg-eth0 > /etc/sysconfig/network-scripts/ifcfg-eth1
sudo sed -i 's/DEVICE=eth0/DEVICE=eth1/' /etc/sysconfig/network-scripts/ifcfg-eth1
sudo sed -i '/DHCP_HOSTNAME/d' /etc/sysconfig/network-scripts/ifcfg-eth1
sudo sed -i '/HWADDR/d' /etc/sysconfig/network-scripts/ifcfg-eth1
To make the change persistent and applied during network stack activation, edit the files /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1 (ifcfg-eth2, ifcfg-eth3, and so on, if the VM has more than two network interfaces) and change the value of NM_CONTROLLED from yes to no. To do this, run the following commands:
sudo cp -rp /etc/sysconfig/network-scripts/ifcfg-eth0 /tmp/ifcfg-eth0.bkp
sudo cp -rp /etc/sysconfig/network-scripts/ifcfg-eth1 /tmp/ifcfg-eth1.bkp
sudo sed -i 's/NM_CONTROLLED=yes/NM_CONTROLLED=no/' /etc/sysconfig/network-scripts/ifcfg-eth0
sudo sed -i 's/NM_CONTROLLED=yes/NM_CONTROLLED=no/' /etc/sysconfig/network-scripts/ifcfg-eth1
Note
Verify that the NM_CONTROLLED=no line is added to both the /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1 files by using the cat /etc/sysconfig/network-scripts/ifcfg-eth* command. If the line isn't in the files, add it manually by using the sudo echo "NM_CONTROLLED=no" >> /etc/sysconfig/network-scripts/ifcfg-eth0 and sudo echo "NM_CONTROLLED=no" >> /etc/sysconfig/network-scripts/ifcfg-eth1 commands.
After modifying this configuration, restart network services to apply the changes by running the following command:
sudo systemctl restart network
Create corresponding rule and route files by using any text editor (in the following examples, the vi editor is used), and add appropriate rules and routes to each file. Use the following steps to create one set of rule-eth# and route-eth# files per network interface. Replace the IP address and subnet information accordingly in every step. If you have more network interfaces, create the same set of rule-eth# and route-eth# files for each interface by using the corresponding IP address, network, and gateway details.
Create rules and routes for eth0:
To create the rule file for eth0, open the file /etc/sysconfig/network-scripts/rule-eth0.
sudo vi /etc/sysconfig/network-scripts/rule-eth0
Add the following content to the rule file. Replace the IP address accordingly, make sure you specify the IPv4 address in the configuration, and preserve the 32-bit value:
from 10.0.1.4/32 table eth0-rt
to 10.0.1.4/32 table eth0-rt
To create the route file for eth0, open the file /etc/sysconfig/network-scripts/route-eth0.
sudo vi /etc/sysconfig/network-scripts/route-eth0
Add the following content to the route file. Replace the network and gateway values accordingly.
10.0.1.0/24 dev eth0 table eth0-rt
default via 10.0.1.1 dev eth0 table eth0-rt
Create rules and routes for eth1:
To create the rule file for eth1, open the file /etc/sysconfig/network-scripts/rule-eth1.
sudo vi /etc/sysconfig/network-scripts/rule-eth1
Add the following content to the rule file. Replace the IP address accordingly, make sure that you specify the IPv4 address in the command, and preserve the 32-bit value.
Two NICs in the same subnet:
from 10.0.1.5/32 table eth1-rt
to 10.0.1.5/32 table eth1-rt
Two NICs in different subnets but in the same VNET:
from 10.0.2.5/32 table eth1-rt
to 10.0.2.5/32 table eth1-rt
To create the route file for eth1, open the file /etc/sysconfig/network-scripts/route-eth1.
sudo vi /etc/sysconfig/network-scripts/route-eth1
Add the following content to the route file. Replace the network and gateway values accordingly.
Two NICs in the same subnet:
10.0.1.0/24 dev eth1 table eth1-rt
default via 10.0.1.1 dev eth1 table eth1-rt
Two NICs in different subnets but in the same VNET:
10.0.2.0/24 dev eth1 table eth1-rt
default via 10.0.2.1 dev eth1 table eth1-rt
To apply the changes, run the following command to restart the network service:
sudo systemctl restart network
The routing rules are now correctly set, and connectivity should work from any network interface. You can test the connectivity by using Secure Shell (SSH) or pinging both IPs from a VM in the same VNET.
Verify that the current routes and rules are loaded by using the following commands:
sudo ip route show
sudo ip rule show
Important
If you still have issues communicating with the second NIC, restart the VM by using the sudo reboot command, repeat step 7, and test the connectivity again.
By default, policy routing isn't installed in Red Hat Enterprise Linux (RHEL)/CentOS 8.x. To configure multiple network interfaces, install and enable the policy routing. To do this, run the following commands:
In RHEL/CentOS 8.x, the NM_CONTROLLED setting is set to yes in each network configuration file (/etc/sysconfig/network-scripts/ifcfg-eth#) by default. To avoid issues, make sure this setting isn't set to no in your specific configuration file. Also, make sure that a corresponding network configuration file is created per each network interface. The following is the sample configuration for network configuration files:
If more network interfaces are attached to the VM, add extra routing tables (for example, 202 eth2-rt, 203 eth3-rt, and so on).
Create the corresponding rules and routes configuration files for each of your NICs, and add the appropriate rules and routes to each. To create one set of rule-eth# and route-eth# files per network interface, follow the steps below. Replace the IP address and subnet information accordingly in every step. If you have more network interfaces, create the same set of rule-eth# and route-eth# files for each interface by using the corresponding IP and subnet details.
Create rules and routes for eth0:
Create the rule file for eth0 by running the following command:
sudo vi /etc/sysconfig/network-scripts/rule-eth0
Add the following content to the rule file. Replace the IP address accordingly, and make sure that you specify the IPv4 address in the command.
from 10.0.1.4/32 table eth0-rt
to 10.0.1.4/32 table eth0-rt
Create the route file for eth0 by running the following command:
sudo vi /etc/sysconfig/network-scripts/route-eth0
Add the following content to the route file:
10.0.1.0/24 dev eth0 table eth0-rt
default via 10.0.1.1 dev eth0 table eth0-rt
Create rules and routes for eth1:
Create the rule file for eth1 by running the following command:
sudo vi /etc/sysconfig/network-scripts/rule-eth1
Add the following content to the rule file. Replace the IP address accordingly, make sure that you specify the IPv4 address in the command, and preserve the 32-bit value.
Two NICs in the same subnet:
from 10.0.1.5/32 table eth1-rt
to 10.0.1.5/32 table eth1-rt
Two NICs in different subnets but in the same VNET:
from 10.0.2.5/32 table eth1-rt
to 10.0.2.5/32 table eth1-rt
Create the route file for eth1 by running the following command:
sudo vi /etc/sysconfig/network-scripts/route-eth1
Add the following content to the route file:
Two NICs in the same subnet:
10.0.1.0/24 dev eth1 table eth1-rt
default via 10.0.1.1 dev eth1 table eth1-rt
Two NICs in different subnets but in the same VNET:
10.0.2.0/24 dev eth1 table eth1-rt
default via 10.0.2.1 dev eth1 table eth1-rt
To apply the changes, restart the network service by running the following command:
sudo systemctl restart NetworkManager
The routing rules are now correctly set, and connectivity should work from any network interface. You can test the connectivity by using Secure Shell (SSH) or pinging both IPs from a VM in the same VNET.
Verify that the current routes and rules are loaded by using the following commands:
sudo ip route show
sudo ip rule show
Important
If you still don't have connectivity to the second NIC, restart the VM by using the sudo reboot command, repeat step 6, and test the connectivity again.
Add two routing tables to the /etc/iproute2/rt_tables file by running the following commands:
If more network interfaces are attached to the VM, add extra routing tables (for example, 202 eth2-rt, 203 eth3-rt, and so on).
If Cloud-Init automation is set (by default, it's set in the Azure Ubuntu images), make sure that the network CI automation is disabled so that the /etc/netplan/50-cloud-init.yaml file doesn't get overwritten every time the system is restarted.
Create the /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg file by using the following command and content:
sudo vi /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
network:
config: disabled
Modify the netplan configuration file /etc/netplan/50-cloud-init.yaml by using any text editor, and include the following routes and policy-routing blocks for each network interface section:
routes:
- to: <subnet>/24
via: <gatewayIP>
metric: 200
table: 201
- to: 0.0.0.0/0
via: <gatewayIP>
table: <routingTableID>
routing-policy:
- from: <NICIP>/32
table: <routingTableID>
- to: <NICIP>/32
table: <routingTableID>
Replace the subnet, MAC address, and IP address information for each network interface (eth0 and eth1) accordingly. Make sure the 32-bit value is preserved in the routing-policy block.
Here's the sample configuration file that uses the given sample details:
If you use any of the previous configuration file samples, make sure the MAC address value is replaced accordingly. You can get the corresponding NIC MAC address from the output of the ip a | grep ether | awk '{print $2}' command.
Apply the changes by running the following command:
sudo netplan apply
To test the connectivity, ping or SSH both IPs from another VM in the same VNET:
ping 10.0.1.4
ping 10.0.1.5
Add two routing tables to the /etc/iproute2/rt_tables file by running the following commands:
If more network interfaces are attached to the VM, add extra routing tables (for example, 202 eth2-rt, 203 eth3-rt, and so on).
Create the scripts that have the routes and rules for each network interface in the /etc/sysconfig/network/scripts/ directory by using any text editor (in the following commands, the vi text editor is used).
/etc/sysconfig/network/scripts/ifup-route.eth0
sudo vi /etc/sysconfig/network/scripts/ifup-route.eth0
#!/bin/bash
/sbin/ip route add default via 10.0.1.1 dev eth0 table eth0-rt
/sbin/ip rule add from 10.0.1.4/32 table eth0-rt
/sbin/ip rule add to 10.0.1.4/32 table eth0-rt
/etc/sysconfig/network/scripts/ifup-route.eth1
sudo vi /etc/sysconfig/network/scripts/ifup-route.eth1
Two NICs in the same subnet:
#!/bin/bash
/sbin/ip route add 10.0.1.0/24 dev eth1 table eth1-rt
/sbin/ip route add default via 10.0.1.1 dev eth1 table eth1-rt
/sbin/ip rule add from 10.0.1.5/32 table eth1-rt
/sbin/ip rule add to 10.0.1.5/32 table eth1-rt
Two NICs in different subnets but in the same VNET:
#!/bin/bash
/sbin/ip route add 10.0.2.0/24 dev eth1 table eth1-rt
/sbin/ip route add default via 10.0.2.1 dev eth1 table eth1-rt
/sbin/ip rule add from 10.0.2.5/32 table eth1-rt
/sbin/ip rule add to 10.0.2.5/32 table eth1-rt
Adjust the network and IP address information accordingly, and preserve the 32-bit value. If there are more than two NICs, make sure that the corresponding IP rules and IP routes are included for each one.
Provide execution permissions for both scripts by using the following commands:
Modify the network configuration files for both eth0 and eth1 (/etc/sysconfig/network/ifcfg-eth#) by using any text editor, and include the following line in both files to point to the corresponding script:
In the network configuration file /etc/sysconfig/network/ifcfg-eth0:
If the /etc/sysconfig/network/ifcfg-eth1 file doesn't exist, create it by using the content from the /etc/sysconfig/network/ifcfg-eth0 file. Make sure the POST_UP_SCRIPT is adjusted to use the corresponding script. To do this, execute the following command:
To apply the changes, restart the network service:
sudo systemctl restart network
The routing rules are now correctly set, and connectivity should work from any network interface. You can test the connectivity by using Secure Shell (SSH) or pinging both IPs from a VM in the same VNET.
Verify that the current routes and rules are loaded by using the following commands:
sudo ip route show
sudo ip rule show
Important
If you still don't have connectivity to the second NIC, restart the VM by using the sudo reboot command, repeat step 6, and test the connectivity again.
Add two routing tables to the /etc/iproute2/rt_tables file by running the following commands:
If more network interfaces are attached to the VM, add extra routing tables (for example, 202 eth2-rt, 203 eth3-rt, and so on).
Create or modify the configuration file /etc/network/interfaces.d/50-cloud-init with any text editor.
sudo vi /etc/network/interfaces.d/50-cloud-init
Use the following configuration:
Two NICs in the same subnet:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
metric 100
up /usr/sbin/ip route add default via 10.0.1.1 dev eth0 table eth0-rt
up /usr/sbin/ip rule add from 10.0.1.4/32 table eth0-rt
up /usr/sbin/ip rule add to 10.0.1.4/32 table eth0-rt
auto eth1
iface eth1 inet dhcp
metric 200
up /usr/sbin/ip route add 10.0.1.0/24 dev eth1 table eth1-rt
up /usr/sbin/ip route add default via 10.0.1.1 dev eth1 table eth1-rt
up /usr/sbin/ip rule add from 10.0.1.5/32 table eth1-rt
up /usr/sbin/ip rule add to 10.0.1.5/32 table eth1-rt
Two NICs in different subnets but in the same VNET:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
metric 100
up /usr/sbin/ip route add default via 10.0.1.1 dev eth0 table eth0-rt
up /usr/sbin/ip rule add from 10.0.1.4/32 table eth0-rt
up /usr/sbin/ip rule add to 10.0.1.4/32 table eth0-rt
auto eth1
iface eth1 inet dhcp
metric 200
up /usr/sbin/ip route add 10.0.2.0/24 dev eth1 table eth1-rt
up /usr/sbin/ip route add default via 10.0.2.1 dev eth1 table eth1-rt
up /usr/sbin/ip rule add from 10.0.2.5/32 table eth1-rt
up /usr/sbin/ip rule add to 10.0.2.5/32 table eth1-rt
Adjust the network and IP address information accordingly, and preserve the 32-bit value. If there are more than two NICs, make sure that the corresponding IP rules and IP routes are included for each one.
Activate the new configuration by using the following command:
sudo systemctl restart networking
The routing rules are now correctly set, and connectivity should work from any network interface. You can test the connectivity by using Secure Shell (SSH) or pinging both IPs from a VM in the same VNET.
Verify that the current routes and rules are loaded by using the following commands:
sudo ip route show
sudo ip rule show
Important
If you still don't have connectivity to the second NIC, restart the VM by using the sudo reboot command, repeat step 4, and test the connectivity again.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see: https://aka.ms/ContentUserFeedback.
