Started netserver on remote machine, but netperf cannot make connection

Hinata Hajime 1 Reputation point
2021-12-21T19:08:17.49+00:00

I'm using two azure machines to run a TCP_STREAM test from netperf between them.

The remote machine is called mymachine-Standard-D8as-v4-premium-disk-west and the local one is called mymachine-Standard-D8as-v4-premium-disk.

I follow these steps:

  1. Start the netserver in the remote server at port 2333: ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$ sudo netserver -p 2333
    Starting netserver with host 'IN(6)ADDR_ANY' port '2333' and family AF_UNSPEC make sure it is started: ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$ ps -ef | grep netserver
    root 4139 1 0 16:16 ? 00:00:00 /usr/bin/netserver
    root 26990 1 0 16:21 ? 00:00:00 netserver -p 2333
    ubuntu 27011 1906 0 16:25 pts/0 00:00:00 grep --color=auto netserver ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$sudo lsof -i tcp:2333
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    netserver 26990 root 3u IPv6 66361 0t0 TCP *:2333 (LISTEN)
  2. Get the IP address of the remote machine: enP6064s1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
    ether 00:0d:3a:f6:09:99 txqueuelen 1000 (Ethernet)
    RX packets 53433 bytes 76754512 (76.7 MB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 8200 bytes 1287344 (1.2 MB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet 10.2.0.27 netmask 255.255.192.0 broadcast 10.2.63.255
    inet6 fe80::20d:3aff:fef6:999 prefixlen 64 scopeid 0x20<link>
    ether 00:0d:3a:f6:09:99 txqueuelen 1000 (Ethernet)
    RX packets 12508 bytes 80101094 (80.1 MB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 8136 bytes 1286894 (1.2 MB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
    inet 127.0.0.1 netmask 255.0.0.0
    inet6 ::1 prefixlen 128 scopeid 0x10<host>
    loop txqueuelen 1000 (Local Loopback)
    RX packets 190 bytes 25942 (25.9 KB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 190 bytes 25942 (25.9 KB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Which shows the IP address for the west machine is 10.2.0.27!

  1. Run netperf from the local machine netperf -H 10.2.0.27 -p 2333 -D 0.5 -c -C -l 20 -t TCP_STREAM -P 0 -f g -- -b 2 -D -o throughput,throughput_units

and now it gives me:

establish control: are you sure there is a netserver listening on 10.2.0.27 at port 2333?
establish_control could not establish the control connection from 0.0.0.0 port 0 address family AF_UNSPEC to 10.2.0.27 port 2333 address family AF_INET
This is so confusing! Because I have verified that the netserver is listening on 10.2.0.27 at port 2333. Actually, the same procedure works fine on GCE's machines.

I wonder what may cause this and how to fix it.

Per comments from @waltinator, I looked at the ip route, ufw status, sudo traceroute the-other-system on both machine.

On the west machine:

ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$ ip route
default via 10.2.0.1 dev eth0 proto dhcp src 10.2.0.27 metric 100 
10.2.0.0/18 dev eth0 proto kernel scope link src 10.2.0.27 
168.63.129.16 via 10.2.0.1 dev eth0 proto dhcp src 10.2.0.27 metric 100 
169.254.169.254 via 10.2.0.1 dev eth0 proto dhcp src 10.2.0.27 metric 100 
ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$ ufw status
ERROR: You need to be root to run this script
ubuntu@mymachine-Standard-D8as-v4-premium-disk-west-0001:~$ sudo traceroute 10.1.0.62
traceroute to 10.1.0.62 (10.1.0.62), 30 hops max, 60 byte packets
 1  * * *
...
30  * * *

On the local machine:

ubuntu@jane-cr22-Standard-D8as-v4-premium-disk-0001:~$ ip route
default via 10.1.0.1 dev eth0 proto dhcp src 10.1.0.62 metric 100 
10.1.0.0/18 dev eth0 proto kernel scope link src 10.1.0.62 
168.63.129.16 via 10.1.0.1 dev eth0 proto dhcp src 10.1.0.62 metric 100 
169.254.169.254 via 10.1.0.1 dev eth0 proto dhcp src 10.1.0.62 metric 100 
ubuntu@mymachine-Standard-D8as-v4-premium-disk-0001:~$ ufw status
ERROR: You need to be root to run this script
ubuntu@mymachine-Standard-D8as-v4-premium-disk-0001:~$ sudo ufw status
Status: inactive
ubuntu@mymachine-Standard-D8as-v4-premium-disk-0001:~$ sudo traceroute  10.2.0.27
traceroute to 10.2.0.27 (10.2.0.27), 30 hops max, 60 byte packets
 1  * * *
...
30  * * *

Besides, for the west machine, I've added an inbound port rule that has priority 100, allowing any source and destination, with port range 2333. But still, the local machine cannot connect with the remote one.

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,157 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,159 questions
Azure Network Watcher
Azure Network Watcher
An Azure service that is used to monitor, diagnose, and gain insights into network performance and health.
159 questions
{count} votes