Adding secondary ip to linux vm in Azure portal gives it as primary ip in Ubuntu 18.04

Question

I have a Ubuntu 18.04 vm running in Azure. I have added a secondary public ip to this vm, because I need to serve a webservice from a specific ip with different ssl certificates.

10.0.3.4 with public ip ... is configured primary in Azure portal.
10.0.3.5 with public ip ... is configured secondary in Azure portal.

If I login to the server and i do ip addr show eth0. I see the following

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000  
    link/ether 00:0d:3a:ac:e5:aa brd ff:ff:ff:ff:ff:ff  

    inet 10.0.3.5/24 brd 10.0.3.255 scope global eth0  
       valid_lft forever preferred_lft forever  
    inet 10.0.3.4/24 brd 10.0.3.255 scope global secondary eth0  
       valid_lft forever preferred_lft forever  

The weird part is that 10.0.3.4 is configured as secondary while in Azure portal it's configured as primary.
I need 10.0.3.4 to be configured in Ubuntu as primary, because docker uses the primary ip for outgoing traffic.

My /etc/netplan/50-cloud-init.yaml looks like:

network:  
    ethernets:  
        eth0:  
            addresses:  
            - 10.0.3.5/24  
            dhcp4: true  
            dhcp4-overrides:  
                route-metric: 100  
            dhcp6: false  
            match:  
                driver: hv_netvsc  
                macaddress: 00:0d:3a:ac:e5:aa  
            set-name: eth0  
    version: 2  

I already tried to add 10.0.3.4 to the addresses option, but this also doesn't seem to work. Does anyone know how to accomplish this?

Answer 1

Hi @docter
I see 3 possibilities

1 - Invert the public addresses in the portal, since linux is inverting the addresses, treat it as inverted in the portal.

2 - Change the default gateway to use the interface:

3 - use two interfaces instead of two ips on the same interface

In the example below, on a Azure linux machine I did the same test:

In your case, the command just like:

ip route replace default via 10..0.3.1 src 10.0.3.4

Answer 2

The strange part is that if I do a wget https://host2 from the host and I look in the logs from host2 I see the primary ip in the logs. When I log into docker and do the same wget I see the secondary ip in the logs. So on the host the ip marked as secondary is used as primary for outgoing requests. However docker uses the ip marked as primary for its outgoing requests.

If I add a second nic, with the new ip, i can't connect to it.

Answer 3

This is my route list:

default via 10.0.3.1 dev eth0 proto dhcp src 10.0.3.4 metric 100
10. 0.3.0/24 dev eth0 proto kernel scope link src 10.0.3.5
10. 0.3.0/24 dev docker0 proto kernel scope link src 10.0.3.4 linkdown
168. 63.129.16 via 10.0.3.1 dev eth0 proto dhcp src 10.0.3.4 metric 100
169. 254.169.254 via 10.0.3.1 dev eth0 proto dhcp src 10.0.3.4 metric 100
172. 17.0.0/16 dev br-dca977dfed88 proto kernel scope link src 172.17.0.1
172. 18.0.0/16 dev br-f9422cefa4d7 proto kernel scope link src 172.18.0.1

and this is my ifconfig

docker0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 10.0.3.4 netmask 255.255.255.0 broadcast 10.0.3.255
ether 02:42:1b:aa:34:e0 txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.3.5 netmask 255.255.255.0 broadcast 10.0.3.255
inet6 fe80::20d:3aff:feac:e5aa prefixlen 64 scopeid 0x20<link>
ether 00:0d:3a:ac:e5:aa txqueuelen 1000 (Ethernet)
RX packets 1347 bytes 553000 (553.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1513 bytes 232050 (232.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

the weird part is that eth0 has 10.0.3.5 and docker0 has 10.0.3.4