Variable network performance

Question

When running performance benchmark tests for Redpanda on Azure I'm experiencing variable network performance. Redpanda is a high throughput, low latency messaging system that performs best on modern hardware with NVMe disks, which is why I'm using L16s_v2 virtual machines. Data sent to Redpanda is written to disk and also replicated to other machines (brokers) over the network for durability using the Raft consensus algorithm.

L16s_v2 have an expected network bandwidth of 6.4 Gbps, but using iperf3 to test the network speed between VMs, I've measured as low as 808 Mbps. Having a cluster of 3 VMs with a connection variability of up to 4 Gbps is causing lower than expected benchmark results when compared to other environments. The VMs have been added to a Proximity Placement Group, and accelerated networking is enabled on the interfaces, but is there anything else I can do to get stable, and expected network bandwidth?

Answer 1

Hello @JRK ,

Welcome to the Microsoft Q&A platform. Happy to answer your questions.

Expected network bandwidth is the maximum aggregated bandwidth allocated per VM type across all NICs, for all destinations. Upper limits are not guaranteed, but are intended to provide guidance for selecting the right VM type for the intended application. Actual network performance will depend on a variety of factors including network congestion, application loads, and network settings.

For information on optimizing network throughput, see virtual-network-optimize-network-bandwidth To achieve the expected network performance on Linux or Windows, it may be necessary to select a specific version or optimize your VM. For more information, see virtual-network-bandwidth-testing

Additionally, you can refer to the following document to test the network latency between Azure VMs virtual-network-test-latency

Reference : https://learn.microsoft.com/en-us/azure/virtual-network/virtual-machine-network-throughput
https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-test-latency#tips-for-creating-an-optimal-vm-configuration