Software Defined Networking

Why - The demand for cloud computing is increasing due to number connected devices and applications. Each XBOX One has 3 virtual servers in the cloud. There is tons of data stored in the cloud due to applications such as OneDrive, Facebook and others. This increased demand is putting a lot of pressure on the cloud datacenters to provide secure and reliable service while reducing their cost. The cost includes both capital expenses (CAPEX) and operational expenses (OPEX). Networking is a major contributor to the overall cost of the datacenter. This is due to many factors such as specialized hardware, vendor specific tools and slow pace of innovation. SDN brings the promise to solve this. SDN vision is to provide a software defined network that is cost effective and provides rapid innovation and operational flexibility. The larger vision is to have a software defined datacenter (SDD), however, the focus of this post is SDN and OpenFlow only.

What - There is no single definition for SDN. In fact, at times it seems that SDN is something that would solve world hunger (wishful thinking, but it won't). SDN has become an umbrella term that is used for anything and everything networking related. However, if you peel away the hype and marketing surrounding SDN (including my own above ), you would see that SDN is a design concept or philosophy for building software for network management. And even though there is no single definition for SDN, there are a set of guiding principles in this design philosophy that most people seem to agree to. These principles are the key to understanding SDN.

1. Software

The first principle in the SDN is software i.e. doing things in software (as possible). This is an important principle because doing things in software bring tremendous flexibility. You can innovate and update it faster, you can run on different hardware etc. This is no different than what has happened to compute with server virtualization.

Virtualization has brought tremendous flexibility in running a large scale data center. Virtual Machines provides unprecedented flexibility such as ability to run multiple operating systems on a single physical machine, live migration from one physical server to another without dropping active TCP connections, being able to snapshot a running virtual machine, ability to dynamically adjust underlying resources available to the VM etc. Using software in networking can bring the same flexibility to networking. This includes things like network virtualization and network function virtualization, which would be topic of another post in the future.

2. Decoupling

The second principle in SDN is of decoupling or separation of functionality. This is not a new concept in itself, every software engineer know about modularity and decoupled functionality. Even in networking, the network stack is built in a modular fashion where there are different layers. In SDN, this decoupling is about dividing the overall network management into three distinct planes called management, control and data plane.

Management plane is responsible for coordinating among various subsystems of the datacenter like compute, storage and network, as well as taking user input and converting it to control plane constructs.

Control plane is the brain of the network and has the intelligence to make overall network function. It is responsible for configuration of data plane devices and ensuring overall functioning of the network.

Data plane represents networking devices (physical or virtual) that do the actual network packet processing. The data plane should provide high performance packet processing but should be relatively simpler (or have lesser intelligence) and do what is programmed by the control plane.

3. Standardization

The next principle of SDN is standardization. Once you decouple the network management into multiple layers, you would want a standard interface between these layers such that it is possible to replace one data plane device with another one or use different management applications such as OpenStack working with different network controllers etc. The interface exposed by the control plane is referred as northbound and the one exposed by data plane is referred as southbound interface.

While standards based interfaces are better, in many cases, an abstraction based approach is needed due to ecosystem diversity and differentiation. For example, OpenDaylight uses something called a SAL (service abstraction layer) to allow the control plane services to talk to different data plane devices that may support different feature sets or different protocols for configuration. This allows control plane services to be independent of diversity in underlying data plane interfaces. Now, as we know, maintaining such abstractions are hard long term, so standards based interfaces are better, but from practical standpoint that is hard to achieve given the realities of the market.

4. Centralization

The last principle of SDN is centralization of policy. This is one of the key principle of SDN because doing centralization of networking policy, you can treat the whole network as a single unit and allow control plane to program various devices in a coherent fashion. It removes the model of networking policy being scattered around various devices making the diagnostics of the network a nightmare and incurring significant operational overhead. Please note that the centralization is logical centralization i.e. even though control plane can be running on multiple systems in a distributed fashion, it is acting as a single unit. This logically centralized implementation of a control plane is typically called a network controller. One can implement network controller where it runs in a single server or it can run in a distributed fashion on multiple servers. However, it is one entity that manages the overall network and programming of networking devices. The centralization is not new, e.g. in an operating system, a kernel is a logically centralized entity comprising of various kernel modules. In fact, we can map an OS stack to SDN stack and see the similarities between the two.

From this picture, we can see that how in an operating system, the kernel is the brain that manages various devices, in an SDN stack, network controller is the brain and controls the overall network.

PS: Few months back a picture in an article motivated me to draw this picture above. I am not able to find the original article that had a similar picture in it. If anyone has seen that, I would love to put a reference to that article in this post.

OpenFlow

OpenFlow has its roots in project Ethane by Martin Casado that presents the idea of a logically centralized controller, which contains the network policy for all packets and switches in the network process packets based on this policy. Building upon this idea, OpenFlow specifications were developed. There are few key things that OpenFlow provides as described below.

1. Flow based Data Plane

OpenFlow provides policy configuration in the switches using a flow based rules. The advantage of this approach is that the policy application is consistent. It also allows an application to innovate in packet processing by chaining various flow rules based on their specific requirements. However, a flow based data plane is not necessary for SDN, it is just one more way in which data plane devices can configured. One can argue that flow based data plane approach is better, and while it is generally true, in specific cases, even that can be debated. For example, is flow based policy the most efficient representation? Does it provide the most efficient packet processing (at least in hardware switches)?

In many cases, flow based policies may not be best way to represent your packet processing rules, e.g. earlier versions of OpenFlow didn't have great story for ECMP and when this support was added, it was added via group types and not as flow rules or actions.

In the end, irrespective of whether flow based data plane is better or not, is orthogonal to SDN. SDN is about programming devices and whether a device supports OpenFlow or not, is immaterial.

2. Standards based Policy

OpenFlow provided a standard way to program switches, which is very nice. However, as I mentioned earlier, given the heterogeneous nature of the policy configuration and diverse ecosystem, an SDN based solution needs to support an abstraction layer (at least for foreseeable future). So while OpenFlow did push for standards based policy configuration, it is not mandatory for SDN and SDN is not tied to OpenFlow.

3. Controller based Design Concept

Ethane (and in turn OpenFlow) provides a conceptual model where the network state is kept in a logically centralized controller and various network devices obtain state from this controller for their packet processing. This I believe is the key concept that created the whole notion of SDN. However, again while Ethane created this design concept (and I think everyone would agree that its an awesome concept), it is a design concept, and like object oriented programming or other design concepts and is not tied to to a particular technology such as OpenFlow. In fact, most applications of this concept you see are not tied to OpenFlow, e.g. be it OpenDaylight Controller, Juniper Contrail, Cisco ACI and to some extent even NSX.

Closing Thoughts

Hopefully this article provided some clarity around SDN and its relation to OpenFlow. I would like to say though, that the role of SDN in simplifying network management is tremendous. With SDN, software becomes the entity where intelligence of the network lives. The policy application moves to virtual switches and networks get virtualized using network virtualization. An SDN controller ties all these together to manage network as a whole and provides autonomous network management where human intervention is needed only upon hardware failures.  It can program 1000s of virtual switches, and keep their configuration consistent, which would be impractical to do manually. It can reduce the overall cost of networks (especially OPEX) significantly.

In such a world, where intelligence live in the virtual switches (or network edge), the role of physical network diminishes as well. I am sure, my hardware friends would disagree with me on diminishing role of hardware. They are right, this is not black and white. You still need hardware to carry your packets from one place to another, you may want to offload processing to hardware to meet the performance requirements, you may need hardware to bridge between physical and virtual world etc. However, the key change with SDN is that intelligence lies in the controller and hardware competes in terms of its processing capabilities. It is a step towards commoditization of networking hardware.

Future

We have been building an SDN solution for Windows and it is available in the Windows 10 Technical Preview, look for a new server role called Network Controller. Windows Network Controller (WNC) is designed as a scalable and highly reliable application that would let you manage physical and virtual networking elements. In a future post, I would describe the overall architecture of WNC and how it meets the reliability and scalability requirements using technologies such as Windows Fabric and techniques such as goal state (or desired state) based programming.

References

• OpenFlow and SDN
• Ethane Paper
• OpenFlow Specifications
• SDNCentral Article on SDN
• PacketLife Article on SDN
• Network Controllers
• OpenDaylight Technical Overview
• OpenContrail Architecture
• Cisco ACI Papers
• VMWare NSX Design Guide

The series is mostly written from software perspective and hardware details are only provided where necessary for understanding of the concepts. The series is focused on x86/x64 based architectures though the concepts described here can be applied to other architectures.

Introduction

Interrupts are signals sent to the processor to indicate a condition that requires immediate attention. An interrupt may cause processor to interrupt its current instruction stream and switch to a different instruction stream. Interrupts typically have priority associated with them and only an interrupt with priority higher than the currently executing instruction stream can interrupt the processor. A lower priority interrupt may remain pending until the processor is done executing higher priority code. An interrupt is typically handled via special subroutines called interrupt handlers or interrupt service routines (ISRs). ISRs typically run for a short duration, doing the minimal work required handle the interrupting condition and return to the next pending instruction stream. In many cases, ISRs queue work to be executed at a later point of time at a lower priority level to ensure that the lower priority interrupts are not disabled for a long duration.

8086 Microprocessors and Programmable Interrupt Controller

Intel 8086 based microprocessors up to 80386 supported only a uni-processor design and processor supported 256 interrupts. PC based on this architecture used an interrupt controller known as programmable interrupt controller (PIC), also referred as Intel 8259A, to support hardware interrupts from external sources. One PIC had 8 interrupt inputs and various devices were connected to it. In the PC/AT based architectures, two PICs were connected in a cascaded fashion providing 15 hardware interrupts. There could be up to 15 devices connected via this PIC configuration to the PC. These devices include devices such as System Timer, Keyboard, Hard Disk Controller etc. The remaining interrupts (out of 256) are used for non-maskable interrupt, exceptions and software interrupts. Each of these type of interrupts are described below.

Hardware Interrupts: These are the interrupts that are sent to processor by external devices that are connected to PIC. Multiple devices can also share an interrupt, in which case, multiple ISRs from each device need to be chained together, as without running device's ISR, it cannot be determined as to which device is issuing the interrupt. There can be two types of hardware interrupts, edge triggered and level triggered. Edge triggered interrupts are asserted when the interrupt line transitions between high and low state. These interrupts are issued per transition and not repeated. Many hardware devices issue edge triggered interrupt when a transition of a state occurs, e.g. in networking when the packet queue goes from empty to non-empty. This makes it critical to never miss an edge triggered interrupt, because failure to handle one interrupt may result in no further interrupts from happening (e.g. because the packet queue never goes from empty to non-empty again). Level triggered interrupts happen as long as the interrupt line is held at active level (active level can be high or low for a device). These interrupts require acknowledgement at device to indicate that the interrupt is services so that the interrupt line is moved to inactive state. This is typically done by programming the device and then issuing an EOI (end of interrupt) request.

Non Maskable Interrupts: These are hardware interrupts that cannot be masked. Interrupt 2 is reserved for non-maskable interrupts. These are usually sent due to unrecoverable errors such as non-correctable bit errors in RAM etc.

Exceptions: The difference between exception and interrupts are that exceptions are raised due to certain abnormal conditions. The exceptions include things such as divide by zero exception, invalid opcode exception, breakpoint or single step exception etc. These conditions are raised due to processor exceuting an instruction that causes processor to raise an exception. This can be an arithmetic instruction resulting in divide by zero exception or an INT 3 instruction causing processor to raise a breakpoint exception. The exceptions can be further categorized into faults, traps, aborts etc. However, from software handling perspective, the ISR for all types of exception and interrupts is written in a similar fashion. It is up to system software to decide specific handling for certain exception types as needed. 

Software Interrupts: These are interrupts that are generated by software. On 8086, using instruction INT, software can cause an interrupt to be generated. The usage of software interrupts depends on the design of system software. They may be used by system software to raise exceptions, cause tasks to be paused or switched or any other purpose suitable for the system software to function. A common usage for this is INT 3, which is an instruction put in by the software to cause a breakpoint exception, useful during debugging programs.

Multiprocessor Architecture and Advanced Programmable Interrupt Controller

Intel multiprocessor architecture introduced an enhanced interrupt controller, referred as Advanced Programmable Interrupt Controller (or APIC). APIC is an umbrella term that refers to two separate components, known as local APIC (LAPIC) and I/O APIC (IOAPIC) as described below.

Local APIC (LAPIC): Each processor in a multiprocessor system consists of one LAPIC. LAPIC is responsible for receiving various interrupt requests and delivering them to the processor, handling prioritization of interrupts, sending interrupts to other processors (known as inter processor interrupts or IPIs). IPIs are important part of a multiprocessor operating systems as they are used for synchronization across different processors. LAPIC can be connected directly to I/O devices via local interrupt inputs or through IOAPIC via external interrupt inputs. LAPIC can generate interrupts due to interrupt requests received from various sources such as IPIs received from other processors or itself (known as self-IPI), interrupts coming from LINT or EXTINT, thermal and performance interrupts or APIC timer interrupt etc.

I/O APIC (IOAPIC):  IOAPIC connects to the devices to allow device interrupt requests to be routed to LAPIC(s). There can be one or more IOAPIC in the system. IOAPIC receives interrupt requests from the devices and sends them to LAPIC(s) based upon the redirection table entries (RTE) programmed in the IOAPIC.

Message Signalled Interrupts (MSI)

PCI introduced a concept of message signalled interrupts. MSI worked by doing a DMA of 64-bit value (MSI data) to a 64-bit physical memory location (MSI address). This memory location is a set of special addresses that are intercepted by the chipset and appropriate interrupt request is sent to the LAPIC. Please note that PCI devices can also be connected via IOAPIC, in which case, the interrupt request to LAPIC is generated based on IOAPIC RTE. However, for PCI devices, not connected via IOAPIC, interrupt requests are generated by doing DMA of MSI data to MSI address. Using MSI, removes the dependency on IOAPIC and provides a more flexible way to generate interrupts as the interrupt requests are not limited by number of interrupt inputs on the IOAPIC.

Summary

I have very briefly touched upon the topic of interrupts in this post. It is a complex topic with lots of details, so I have provided a list of references for the folks who would like to learn more about it. In future posts in this series, I would talk about, how interrupts are handled in Hyper-V environment and then moving on to topic of interrupt remapping and how interrupt requests from physical devices are routed to virtual machines.

References

• Programmable Interrupt Controller
• Intel 8259A Datasheet
• Interrupts and Handlers
• Using Interrupts
• Advanced Programmable Interrupt Controller
• Intel System Programming Guide - Chapter 10
• Intel IOAPIC Datasheet
• Multiprocessor Specification

In this post I would talk about a feature we built called dynamic VMQ, a feature designed to provide optimal processor utilization across changing workload that was not possible with static processor allocation for VMQ as done in Windows 7 (or Windows Server 2008 R2 release). However, before we dig deeper into dynamic VMQ, let me recap the processor utilization for no VMQ and static VMQ cases.

No VMQ

With no VMQ, all networking packets gets processed on a single processor, resulting in a single processor becoming the bottleneck for the overall network throughput.

Static VMQ

With static VMQ, queues for different VMs gets affinitized to processors in a round-robin fashion. This happens during VM startup and stays until the VM is restarted. If this assignment is not optimal as in either multiple network heavy VMQs are affinitized to the same processor (either permanently or due to changing workload) or multiple smaller VMQs are affinitized to different processors, it results in either processor bottleneck and reduced throughput or extraneous processor utilization and power consumption.

Dynamic VMQ

The basic premise of dynamic VMQ was to use as many processors as needed for network packet processing. Under low network throughput, fewer processors are used, thus resulting in lower power consumption. As power is one of the major cost contributor to the data center, this helps reduce the overall power footprint of the data center. Under high network throughput, more processors are used, thus allowing best utilization of the processing power of the data center and meet the needs of most demanding workloads.

The key requirement to support dynamic VMQ from NICs is to allow dynamic change in  VMQ to processor affinity. A NIC driver can advertise support for dynamic affinity during its initialization and VMSWITCH can leverage that to optimize the processor utilization.

VMSWITCH uses a set of data and heuristics to determine the optimal VMQ to processor affinity. It tracks the overall processor utilization of each processor. It also calculates the total time spent in network processing for each VMQ as well as the total number of network packets processed for each VMQ. 

Using these numbers, VMSWITCH detects, whether a processor has exceeded its threshold capacity or not. If the processor has exceeded the threshold capacity, it is treated as bottleneck and a VMQ spreading selection algorithm is run to see if there is a candidate VMQ to move "away" from the processor. If a candidate VMQ is found, another selection algorithm is run to find the candidate destination processor. A successful selection results in target VMQ moving to destination processor.

Conversely, if a processor is associated with a VMQ and the processor usage goes below a minimum threshold, a coalescing algorithm runs to coalesce the VMQs to fewer processors. This results in using more processors when the network processing demand is high and fewer processors usage when the demand is low.

Please do note that, even though dynamic VMQ allows a more optimal usage of multiple processors in the system and fixes the problems with static VMQ, a single VMQ is still limited to a single processor. In Windows server 2012 R2, we have built a feature called virtual RSS to allow a single VMQ to use more than one processor thus allowing a single VM to achieve much higher throughput.

Ping Pong Problem

One problem we encountered with the dynamic VMQ was the ping pong effect. Under certain conditions, the processor and VMQ usage resulted in a situation, where one VMQ would continuously move from one processor to another. This happened mainly when the VMQ move resulted in the destination processor becoming the bottleneck and source processor becoming a good candidate for moving a VMQ. The VMQ would jump back and forth between source and destination processor. In the performance analysis, this turned out to not be a deal breaker as the effect of change in VMQ processor affinity was minimal. However, in Windows Server 2012 R2, with the work on virtual RSS, a temporal affinity using credit based system was added to take care of this problem.

Summary

In this post I described dynamic VMQ, how it works to provide optimal processor utilization under dynamic workload. In a future post, I would describe virtual RSS, how it solves the single processor bottleneck for each VMQ and allows a single VM to achieve more than 30Gbps throughput. In that post, I would also describe the fix for the ping pong problem mentioned above.

Background

In Windows networking stack, to utilize multiple processors in a machine, a feature called RSS (or receive side scaling) is used. This feature was co-developed by Microsoft working with hardware partners. It provides two main features:

• It allows incoming traffic to be put on different queues that get processed on different processors based on TCP/UDP stream information i.e. source and destination IP and ports.
• It allows sent traffic to be put on specific queues and completion for sent traffic to be handled on a specific processor based on the TCP/UDP stream information.

RSS allows different TCP/UDP streams to be handled on different processors without causing out of order packets thus allowing multiple processors to be used for networking.

Before RSS, NIC used to interrupt a single processor to indicate reveived packets or send completions and software distribution was needed to distribute the processing to different processors, resulting in expensive inter-processor signaling etc. With RSS, NIC sends interrupt to a specific processor, as specified in the RSS indirection table, to indicate received packets or send completions, thereby allowing multiple processors to be used concurrently without any additional software signalling.

You can read more about RSS here and here.

Using RSS in VMSWITCH, however, was not practical since RSS in the physical NIC supports a single indirection table and single key to hash the incoming packets. However, in case of virtualization, there are multiple VMs running on a single host and thus you end up with multiple indirection tables, that is not supported in hardware. While it was possible to create new indirection table in VMSWITCH and then use that for all the VMs, the performance testing showed that to cause regressions under certain workloads.

This caused VMSWITCH to always use the default processor that was associated with the physical NIC for both receive and send-complete handling. This limited the networking bandwidth in Hyper-V to the processing capacity of a single processor which was around 5-6Gbps on a high end processor with packet size varying from 1KB-4KB.

A better solution was needed to allow VMSWITCH to leverage multiple processors in a deterministic fashion.

Virtual Machine Queues (VMQ)

After building the first version of VMSWITCH, I moved to the hypervisor team, as I wanted to work on the hypervisor kernel which is a bare metal OS that runs directly on physical hardware. VMSWITCH ownership moved from Hyper-V team to networking team in Windows to better align Hyper-V networking with overall Windows networking stack. To solve the problem of single processor bottleneck in VMSWITCH, Windows networking team designed a feature called virtual machine queues that allowed VMSWITCH to use multiple processors for networking traffic. This feature allowed a NIC to have separate queues that gets associated with one or more destination MAC address, instead of an RSS style indirection table as shown in the picture below.

With VMQ, it was possible to allocate separate queues for separate VMs and put the incoming traffic for the specific VM (as identified based on destination MAC address in the packet) into that queue. Each queue could be associated with its own interrupt that goes to a separate  processor or multiple queues interrupt could be associated to a single processor, providing flexibility in processor utilization as needed. With VMQ, it was possible to get to line rate performance on 10Gbps NICs and use multiple processors solving the single processor bottleneck issue completely.

The first iteration of VMQ allowed static allocation of number of processors on a per physical NIC basis and various VMQ were associated with these processors on a round-robin basis. For example, as shown in picture above, two processors are used for VMQ and VMQ1 and VMQ2 are associated with processor0 and VMQ3 is associated with processor1. This static allocation could not optimize the overall processor allocation and utilization for dynamic workloads. For example, in a round-robin fashion, you could end up with VM1 and VM9 on a same processor and if both became networking heavy, it wasn't possible to use other processors for their traffic. We solved this problem by building a feature called dynamic VMQ and that would be the topic of my next post.

VMSWITCH provides an extremely high performance packet processing pipeline by using various techniques such as lock free data path, using pre-allocated memory buffers, batch packet processing etc. In addition, it leverages the packet processing offloads provided by underlying physical NIC hardware. These offloads do some of the packet  processing in NIC hardware, thereby reducing the overall CPU usage and providing a high performance networking. If you are unfamiliar with NIC offloads, you may want to first read about them here and here.

VMSWITCH supports various NIC offloads such as Checksum, Large Send (LSO) and IPsec task offload. In rest of this post, I would talk about the high level architecture for supporting these offloads.

There are multiple types of NIC objects in VMSWITCH, to support legacy operating systems as well as high performance enlightened operating systems. The legacy NIC is used only for scenarios such as PXE boot and older operating systems. The legacy NIC is not capable of supporting NIC offloads because we are emulating a hardware that did not support such offloads. The other three types of NIC support NIC offload as shown in picture below.

VMSWITCH advertises offload support to the network stack for the host and VM virtual NICs. For Windows OS, this is done by advertising offload support using NDIS primitives. In case of VM virtual NIC, the appropriate offload capability information is sent via VMBUS messages and in the VM it is converted to NDIS objects and advertised to the VM network stack by the virtual NIC driver. For Linux OS, the enlightened virtual NIC driver in VM (referred as NetVSC) converts the offload capability information in Linux specific format and advertises it to Linux network stack.

VMSWITCH is a consumer of offload capability of the physical NIC. When a vSwitch is created and it is attached to a physical NIC, VMSWITCH queries the offload capabilities of the NIC and stores this information in the respective NIC object.

This way, VMSWITCH builds the offload capability map of each NIC that is connected to a particular vSwitch. There are some offloads, that VMSWITCH always advertises to host and VM virtual NIC, these are mainly checksum and large send offload, while there are other offloads such as IPsec task offload, that is only advertised if physical NIC supports that offload. This is described below in detail.

Send Checksum and Large Send Offload Handling

These offloads are always advertised as supported by VMSWITCH to Host and VM Virtual NIC. The reason is that before NDIS 6.0, there was no way to dynamically advertise support for these. The other reason was that these are required to be supported by physical NICs in Windows, hence it was guaranteed that underlying physical NIC would always support these. The handling of these is done in a way that if the offload is not available, for any reason, such as a user disabled these on the physical NIC (e.g. by using NetAdapter powershell cmdlets or going to advanced configuration of the physical NIC in Windows) or the packet is going from one VM to another VM etc., then VMSWITCH emulated the offload in software.

When network stack in the host or VM sends a packet, they put a metadata in the packet indicating whether checksum calculation or segmentation (for large send offload) is needed on the packet. This information is sent to VMSWITCH either via NDIS or VMBUS (based on host or VM virtual NIC). VMSWITCH, first processes the packet, as it normally does, i.e. apply ingress, forwarding and egress policy steps. Once it calculates the destination vPort list for the packet, it checks whether the destination NIC has capability to carry out the requested offload or not. Now only packets going to physical network via physical NIC can leverage hardware offload. So if the packet is not going to physical NIC or going to physical NIC that is not capable of doing this offload, VMSWITCH carries out the offload in software and sends the packet to the NIC.

Since the primary scenario for I/O is on the physical network, this works well, where hardware NIC offload is leveraged and for packets going to host or VMs, the software offload is performed. The software offload performing in VMSWITCH is no worse than doing it in the host or VM network stack, since anyways this work had to be done by the CPU. In fact, it is a little better to do in VMSWITCH because in case of LSO, it allows VM to send more than MTU size frames to VMSWITCH, resulting in fewer packet transfers between VM and VMSWITCH.

Receive Checksum Offload

This offload is always advertised as supported by VMSWITCH to host and VM virtual NIC. This offload allows a physical NIC to verify IP, TCP, UDP checksum in the packet and indicate (via packet metadata) whether the checksum in the packet is valid or not. VMSWITCH leverages this data for packets coming from physical NIC and transfers it to host or VM virtual NIC. Also note that, if VMSWITCH does a software offload of a send checksum request such as case where a packet from VM is going to another VM or to host virtual NIC or vice-versa etc., it marks the receive checksum bits as verified. This avoids unnecessary checksum validation in host or VM network stack and reducing the overall CPU usage.

IPsec Task Offload

IPsec task offload is more complicated, mainly because its computation in the host can result in host partition consuming a lot of CPU on behalf of the VM. This can allow a VM to cause a lot of CPU consumption in the host that is not attributed to the VM and result in kind of a DOS attack on the host. IPsec task offload support in Windows allows the offload capability to be dynamic and it also allows a per destination based security association (or SA) thereby allowing VMSWITCH to only allow offload of IPsec encryption to destinations that are reachable via a physical NIC that has IPsec task offload support.

The IPsec task offload support is dynamically calculated and advertised to the virtual NICs based upon the capability of the underlying physical NIC. Also when a VM requests a new security association for offload, it is validated and only allowed, if the destination would be reachable via physical NIC. If for any reason, the destination changes, e.g. a remote VM that was accessible via physical NIC, live migrates to the same host as the sending VM, VMSWITCH uploads the security associations from the sending VM, so that the sending VM starts doing IPsec encryption itself and not rely upon offload of the same.

Summary

In this post, I gave a high level description of how offloads are used in VMSWITCH to provide high performance network device virtualization. In future posts, I would talk about other performance techniques such as VMQ, mostly lock free and allocation free data path etc.

Standards Compliant Virtual Switch

As I mentioned in my previous post, the main goal in building VMSWITCH was to build a standards compliant, high performance virtual switch. There are many ways to interpret standards compliant because there are many standards. So to be specific, our goal was to build packet forwarding based on 802.1q and mimic physical network semantics (such as link up/down) as much as possible.  This was to make sure that whatever works in a physical network, works in the virtual environment as well. We defined three main objects in VMSWITCH, vSwitch, vPort and NIC. A vSwitch is an instance of a virtual switch that provides packet forwarding and various other features provided by a switch such as QoS, ACL etc. A vPort is analogous to a physical switch port and has configuration associated with it for various features. And finally, a NIC objects, that acts as the endpoint connecting to a vPort. This is similar to the physical network, where a host has a physical NIC that connects to a physical port on a physical switch.

The clear separation of these objects, allowed us to model our switch design and code independent of the type of network device we virtualize e.g. legacy NIC versus virtual NIC as well as support network semantics similar to physical network. For example, by disconnecting a NIC from a vPort, we could generate link down notification. It also gave us the flexibility to support various types of vSwitch (discussed later) without requiring any changes in our core VMSWITCH implementation.

The picture below shows an example setup, multiple vSwitch are created with different number of vPort on each one of them and vPort(s) connecting to different types of NIC(s).

This flexibility allowed creation of different types of vSwitch in the Hyper-V UI. If you use Hyper-V UI, you would notice that vSwitch creation page allows three different types of vSwitches, External, Internal and Private vSwitch. The external switch provides connectivity via a physical NIC, internal switch only provides connectivity to root partition via host virtual NIC and private switch only provides inter-VM connectivity. The key here is that, in the VMSWITCH implementation, there is no vSwitch type, each vSwitch is the same. The UI classifies the vSwitch, based on what types of NICs are connected to the vPort(s) of that vSwitch. This kept VMSWITCH design simpler but allowed us to provide a friendlier UX for our customers.

Type of NIC Objects

There are four types of NIC objects supported in VMSWITCH and these are described below.

Physical NIC: It is also referred as Protocol NIC, is the object that represents a physical NIC in VMSWITCH. This type of NIC is used by VMSWITCH to send packets in/out to the physical network. Please note that the code that handles the I/O on a NIC abstracts the NIC object and hides the NIC specific behavior. All other code in VMSWITCH is agnostic to the type of NIC and operates on NIC as a generic object.

Host Virtual NIC: This is a virtual NIC that is created in the host (or root partition) of Hyper-V. This NIC is created to allow host to communicate with virtual machines or physical network. This type of NIC is especially useful in a single NIC setup, but even used in cases where multiple NICs are teamed together and all I/O flows via VMSWITCH, such as a 2x10Gbps NICs teamed together in a server.

VM Virtual NIC: This type of NIC is created in the VM over a virtual bus (called VMBUS). This NIC is only available on operating system, that have support for VMBUS. Such operating systems are also referred as enlightened operating systems. This NIC provides high performance networking to VMs.

VM Legacy NIC: This type of NIC is created in a VM by emulating a real hardware NIC (Intel DEC 21140A adapter). This type of NIC is provided to support enlightened OS as well as supporting PXE boot. The PXE boot issue is solved in recent Hyper-V release by having an enlightened UEFI, however this NIC is still supported for unenlightened operating systems.

Control and Data Plane

The control interface to VMSWITCH is via IOCTL, which is standard mechanism in Windows to interface user mode applications with kernel mode. The management application invokes various IOCTLs to carry out operations such as creation of a vSwitch, connecting a vPort to a NIC etc.

The data plane interface is specific to each type of NIC that connects to a vPort. The overall control and data plane interactions are shown in the picture below.

This picture shows a vSwitch with 4 vPort, and each vPort connects to a different type of NIC. The control plane interaction happens via IOCTL. The picture also shows the internal representation of various NICs in VMSWITCH.  The data transfer happens via interface specific to the type of NIC. 

A physical NIC connected to VMSWITCH is represented as NDIS protocol and does packet I/O via NDIS interfaces defined for a protocol driver. The host vNIC is represented by an NDIS Miniport in VMSWITCH and does packet I/O via NDIS interfaces defined for a miniport driver. The legacy NIC is supported by emulation running in VMWP process and is represented in VMSWITCH as direct I/O NIC. It uses IRP_MJ_READ and IRP_MJ_WRITE to do packet I/O with VMSWITCH. Lastly, a virtual NIC in the VM is represented as VM NIC in VMSWITCH and uses VMBUS interfaces for packet I/O.

Core Packet Processing

The core VMSWITCH packet processing logic has three layers, Ingress, Forwarding and Egress. In the initial version, we called it Source, Route and Destination. In the source processing step, we applied various source vPort specific policies on the packet and if the packet was not dropped, then we did the packet forwarding processing and once we computed the destination vPort list of the packet, for each destination, we applied destination vPort specific policies. Packets that did not get dropped based on these policies were delivered to their destination vPort(s).

All NIC entry points to VMSWITCH do the interface specific processing on the packets and convert the packets from interface specific format to a common format that is fed to VMSWITCH packet processing pipeline. The VMSWITCH packet processing code is largely independent of the type of NIC, except for cases where offload handling is needed, since offloads are specific to the type of NIC. The modularity of this architecture made is easy for us to move VMSWITCH to Extensible VMSWITCH during Windows Server 2012 release with minimal changes to our core packet processing logic.

Performance

The packet processing pipeline of VMSWITCH is largely lock free, allowing for parallel processing of packets. By using features such as VMQ, VMSWITCH is able to utilize any available CPU for packet processing and carry those out without requiring serialization as much as possible. Most of the operations are lock free and where lock is needed, they are either held for short duration or interlocked operations are used. As an example of lock free operation, VMSWITCH maintains various packet counters on vSwitch, vPort and NIC objects. Updating to these counters atomically can add significant CPU cycles to the overall packet processing cost. To reduce (or almost eliminate) this overhead, per processor packet counters are used that are updated at DISPATCH_LEVEL thus providing lock free updates to these. A per processor counter design makes querying of packet counters less efficient, because now we have to add all per-processor counters. However, since querying of counters is not very frequent as compared to updating the counters on data path, the optimization helps in the critical path i.e. the core packet processing. Similar techniques are used to make sure that overall packet processing is highly parallel and efficient. This coupled with offload support is able to provide performance that is needed by some of the most demanding workloads running in VMs.

This is all I have for today, in future posts I would talk about VMSWITCH offload support, Extensibility model and more.

The goal was simple (well it sounded simple), build a compliant, best in class, high performance network device virtualization. It was my first project to build something in kernel mode, it was exciting, and challenging. We spent countless hours on the whiteboard debating how the overall solution should look like, what should each component do, what would be the right layering, right modules and all those fun parts of the project. And this is what we built eventually.

First and foremost, we decided that we would build a switch that is 802.1Q compliant and behaves like a physical switch so that the VMs (or child partitions) running on Hyper-V would have exactly same network semantics as a physical machine. This makes sure that virtualization doesn't create a bar to adoption and what works in physical environment, would work in a virtual environment. This is the component VMSWITCH shown above. I will talk more about it in a future post.

Next thing we decided was to create two separate device types, one that emulated a real hardware, in our case Intel DEC 21140A, shown as "Legacy NIC" above and other than ran over a virtual bus (VMBUS), shown as "Virtual NIC" above. Legacy NIC was to provide compatibility with existing operating systems since most of them had a device driver for it already, whereas Virtual NIC was to provide high speed networking for "enlightened" operating systems for which we decided to build our virtual bus.

Legacy NIC is built using device emulation, such that to the operating system running in the VM, it seems like a real physical device and accessed via the bus and device registers. The access to these registers is intercepted by the hypervisor, that are then redirected to a user mode process VMWP in the root partition, which carries over the emulation. It then sends the network packet to VMSWITCH, which does the forwarding. Similarly, when VMSWITCH receives a packet destined for the legacy NIC, it is given to VMWP, which indicates it to the child partition by issuing an interrupt via the legacy NIC device emulation. This, is a slow process due to multiple context switches between, child partition, hypervisor and root partition. However this allows networking to work on "unenlightend" or legacy operating system on Hyper-V.

Virtual NIC is built over the virtual bus and is virtualization aware. Being virtualization aware, it doesn't go through traditional device access mechanisms and uses shared memory to transfer data between child partition and root partition. Virtual NIC makes it easier to provide various NIC offloads to the child partition and since we don't have to fiddle around with emulating device registers for every new offload, we can enable new offloads easily. I would talk about, how some of the offloads work on Hyper-V in a future post.

The picture below (courtesy: http://msdn.microsoft.com/en-us/library/dd722833(v=bts.10).aspx) does a good job in showing the difference between enlightened and unenlightend operating system on Hyper-V. 

So this is how network device virtualization works from a 1000 ft level on Hyper-V. In future post, I would go in the details of how VMSWITCH works, how offload support and various other features work. My plan is to finish the basic device virtualization series and then switch to network virtualization and SDN, and describe the internals of some of the cool stuff we are doing in Hyper-V.

Thanks for reading, I hope you find it useful. Please feel free to let me know if you would like me to talk about a particular area or specific topic.