NIC models used in Linux VM change randomly

flexiWAN Networks 21 Reputation points
2022-07-12T07:56:46.79+00:00

Hello,

We notice that, from time to time, when rebooting our Azure Linux VM with multiple NICs attached, the NIC model detected on the guest OS for all of them change. This is our scenario:

  • Azure VM type Standard DS3 v2
  • Running Linux (Ubuntu 18.04 LTS)
  • 4 NICs attached to the VM with accelerated network enabled on all of them

According to the information provided by lspci tool, we have identified the following two NIC models:

  • Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80) --> The one that is used most of the times, and the one we need for our use case.
  • Ethernet controller: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function] --> The one that, from time to time, Azure decides to use within the VM.

We have not modified any setting on the above scenario, however, this change happens without any apparent reaseon. Our qestions are the following:

  1. Is this behavior expected? If so, can you please explain the reason behind doing this change in order to better understand what's happening under the hood?
  2. Is there any method provided by Azure that we can use on our VM to stick to the same NIC model all the time? If not, would using a dedicated host solve the problem?

We are asking these questions because our use case relies on using the same NIC model inside the VM, even if it needs to be rebooted from time to time. So, changing dynamically the NIC model prevents us to progress our work on Azure, unfortunately.

Thank you so much in advance.

Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
6,843 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,035 questions
{count} votes

Accepted answer
  1. KapilAnanth-MSFT 32,351 Reputation points Microsoft Employee
    2022-07-14T17:57:03.18+00:00

    Hi @flexiWAN Networks ,

    I am summarizing our discussion and posting it as a single answer as it might be beneficial to other community members reading this thread.

    You had a specific requirement that you wanted your Accelerated Networking VM to get hosted in a specific Physical server with Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] NIC.

    **To Explain why this is happening, **

    • When you stop the virtual machine, it will be deallocated from the current physical server and when you start the machine again it will be started/allocated on the new physical server in the data center and that particular machine will have a new hardware MAC address.
    • So, for every reboot, the VM may land in any Host and there is no guarantee that it would land on X-4 family
    • Currently, there is no mechanism to specify the host where a VM would land in Azure
    • Refer: https://learn.microsoft.com/en-us/azure/virtual-network/accelerated-networking-how-it-works
    • List item
    • You wanted to know if the same can be accomplished via Azure Dedicated Host

    Now, with respect to Azure Dedicated Host,

    I hope this provides you with more clarity on your question and how you may achieve this.

    Thanks,
    Kapil


    Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.

    1 person found this answer helpful.

2 additional answers

Sort by: Most helpful
  1. KapilAnanth-MSFT 32,351 Reputation points Microsoft Employee
    2022-07-12T11:42:10.51+00:00

    Hi @flexiWAN Networks ,

    Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

    I understand you are leveraging accelerated networking and you notice that the NIC models are switching between,

    • Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function] (rev 80) ----------> preferred.
    • Ethernet controller: Mellanox Technologies MT27500/MT27520 Family

    This is by design.

    To explain why this is happening,

    • When you stop the virtual machine it will be deallocated from the current physical server and when you start the machine again it will be started/allocated on the new physical server in the data center and that particular machine will have a new hardware MAC address.
    • If you shut down the VM through the OS, the VM is stopped and not deallocated. In that case, there would be no ghosted NIC because the lease is never lost for the existing hardware. The VM would start on the same hardware/physical server in the data center, have the same resources, and no ghosted NIC would appear.
      -Refer: https://learn.microsoft.com/en-us/azure/virtual-network/accelerated-networking-overview, for the difference in the design for accelerated and normal networking.
    • List item

    Refer : https://learn.microsoft.com/en-us/azure/virtual-network/accelerated-networking-how-it-works

    • List item
    • List item
    3 people found this answer helpful.
    0 comments No comments

  2. Andreas Baumgarten 92,481 Reputation points MVP
    2022-07-12T09:46:01.51+00:00

    Hi @flexiWAN Networks ,

    my guess, I am not 100% sure, the reason for seeing different types of NICs is related to different NIC types of the Azure hosts the VM is running on.

    This might be the same "issue" if you move the VMs on a dedicated host. The main benefit of running VMs on a dedicated host in Azure is that the host is "reserved" for your organization. As far as I know, a dedicated host does not guarantee any specific hardware setup or device type.

    As far as I know there is no method/option available by default to stick with the same NIC model of a VM on an Azure host.

    ----------

    (If the reply was helpful please don't forget to upvote and/or accept as answer, thank you)

    Regards
    Andreas Baumgarten