NIC Teaming in Windows Server 2012 - Do I Need to Configure My Switch?
Windows Server 2012 and our FREE Hyper-V Server 2012 include native teaming of network adapters to provide load balancing, bandwidth aggregation and failover capabilities using standard Ethernet NICs. IT Pros have been really interested in NIC teaming, because it can provide the extra bandwidth and fault tolerance needed for business critical networks segments supporting virtual machines, clusters, live migrations and storage - all over commodity network adapters!
My friend and colleague, Kevin Remde, has written a great article on his "Full of I.T." blog that provides a detailed technical overview of NIC teaming features. Be sure to check it out!
NIC Teaming is GREAT! But ... Do I need to configure my network switches?
One of the common questions on NIC teaming is whether special network switches and/or customized switch configurations are needed to fully support NIC Teaming in Windows Server 2012. And, the answer is ... it depends! You see, NIC Teaming can be configured in either a "Switch Independent" mode or a "Switch Dependent" mode.
When configuring NIC Teaming in "Switch Independent" mode, the teaming configuration will work with any Ethernet switches - even non-intelligent switches - because Windows Server 2012 handles all of the intelligence with this NIC teaming mode. No special configuration changes are necessary. You can even use this mode to team across separate switches in scenarios that require switch diversity for extra network fault tolerance. See below for the configuration setting for "Switch Independent" mode ...
Are there any special considerations when using "Switch Independent" mode?
When using Switch Independent mode, there are a few considerations to keep in mind:
- If you are using NIC teaming for Failover, rather than load balancing across multiple active NICs, switch independent mode works great. Just set your second NIC in the team as the "Standby adapter" and you're all set. When a failover occurs, the Standby NIC will become active and the server will notify the network switch of this change via standard network protocols.
- When using NIC teaming for Load Balancing across multiple active NICs, there's two load balancing modes you can use with "Switch Independent" mode:
- Address Hash - Load balances outbound network traffic across all active NICs, but only receives inbound traffic via one of the NICs in the team. Since the switch isn't actively involved in the "Switch Independent" teaming mode, it can't load balance inbound traffic across all active NICs. This mode can work well if your server has little traffic inbound and lots of traffic outbound that you are trying to load balance - Web servers and FTP servers are examples of typical server roles that work well in this scenario.
- Hyper-V Port - If your server is a Hyper-V host with multiple running VMs, this load balancing mode is normally preferred in most situations. When using "Hyper-V Port" load balancing, VM's will be distributed across the network team and each VM's outbound and inbound traffic will be handled by a specific active NIC. This mode works really well in scenarios where you are consolidating many VM's on a physical Hyper-V host, but where none of the VM's are generating a network load that exceeds the bandwidth of one NIC in the team. In this use case, NIC teaming provides a very cost-effective way of load balancing the aggregate traffic from all VMs across the active team members, but remember ... each VM is assigned to a specific NIC in the team, so none of the VM's will be able to access more bandwidth than what one NIC provides.
- Address Hash - Load balances outbound network traffic across all active NICs, but only receives inbound traffic via one of the NICs in the team. Since the switch isn't actively involved in the "Switch Independent" teaming mode, it can't load balance inbound traffic across all active NICs. This mode can work well if your server has little traffic inbound and lots of traffic outbound that you are trying to load balance - Web servers and FTP servers are examples of typical server roles that work well in this scenario.
Here's where you set the "Load Balancing Mode" when configuring NIC teaming:
What about other NIC teaming scenarios?
If you have servers that align with other usage scenarios, you'll likely need to get your network switches involved in the NIC teaming configuration by using one of the "Switch Dependent" NIC teaming modes with "Address Hash" load balancing. Specifically, I've seen the following usage scenarios arise where a switch dependent team mode is needed:
- Physical servers with heavy inbound AND outbound network load
- Virtual Machines that have a network load greater than the bandwidth capacity of one NIC in the team
In these scenarios, you can configure the NIC team to use one of the "Switch Dependent" team modes by configuring the "Teaming Mode" to either of the following values:
- LACP - if your network switch supports the Link Aggregation Control Protocol ( aka., 802.1ax ) for dynamic link aggregation.
- Static - if your network switch doesn't support LACP and requires a static link aggregation configuration
Note that when configuring one of these "Switch Dependent" modes, you'll also need to configure your network switch to appropriately handle traffic the network team. On the Cisco switch in my lab, here's the configuration I used with the "Static" team mode:
kemlabsw01(config)# int port-channel1
kemlabsw01(config-if)# description NIC team for Windows Server 2012
kemlabsw01(config-if)# int gi0/23
kemlabsw01(config-if)# channel-group 1 mode on
kemlabsw01(config-if)# int gi0/24
kemlabsw01(config-if)# channel-group 1 mode on
kemlabsw01(config)# port-channel load-balance src-dst-ip
Your Turn ... Now You Try It!
As part of our Windows Server 2012 "Early Experts" Installer Quest, we have additional study resources and a hands-on lab available where you can practice NIC teaming for FREE. Join us and begin the Installer Quest today!
Share Your Experiences!
Are you using NIC Teaming in your environment? Share your experiences in the comments below!
Hope this helps,
Keith
Comments
Anonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
Hi Sig, Yes, the Etherchannel is created to distribute traffic across a set of aggregated switch ports into the server, and then VLAN trunking is applied to this configuration to permit passed traffic to include VLAN tags. Because Etherchannel's are configured on a switch-by-switch basis, stacking the 2960s is needed for the switches to be seen as a single unit. I don't have your specific equipment in my lab, so I won't be able to prepare a specific config for your environment. I'd suggest referencing the 2960-specific config pages on cisco.com at www.cisco.com/.../tsd_products_support_reference_guides.html. Hope this helps! KeithAnonymous
January 01, 2003
Hi Ray, NIC Teaming can be used to increase aggregate throughput as well. Inbound traffic distribution to a server with teamed interfaces is controlled by the switch configuration, so double-check your switch documentation to make sure it's setup correctly and that it supports teaming/etherchannels and load distribution. Also check that your switch isn't throttling traffic and that your switch backplane can handle the load you are wanting. Hope this helps! KeithAnonymous
January 01, 2003
Hi Jordan, Each NIC team includes automatic failover capabilities, so in the event of a NIC failure, the VMs that were previously communicating via the failed physical NIC will each failover to one of the remaining NICs. Hope this helps! KeithAnonymous
January 01, 2003
Hi Sig, You've got a couple choices - you could configure the switch ports as access ports in a VLAN, which would permit only 1 VLAN per switch port and team. Or, you could configure the switch ports as trunk ports for several VLANs and then define separate Team Interfaces on the NIC Team - one Team Interface per VLAN. This second configuration option may provide you with more flexibility, as it will allow you to run multiple VLANs over a single NIC team with each Team Interface configured to split out a single VLAN to the server. However, you'll need to use a switch dependent configuration on a single switch to make this work properly. I'm planning a separate NIC Teaming VLAN post this month with a detailed step-by-step, so be sure to check back for it. Hope this helps! KeithAnonymous
January 01, 2003
Hi Shandilye, Standby adapters are not supported by Switch-dependent NIC teaming to avoid MAC flapping issues that could otherwise occur with the network switch. Hope this helps! KeithAnonymous
January 01, 2003
Hello Keith Great blog post, could I ask a question. I am going to be creating a lab setup with 3 x Dell R710's with additional Quad Nics to make a team for the VM network and 2 x Cisco 2960S switches I shall be using 2 ports of each team to connect to each Cisco in switch independent mode, there should be no issues there, however from a VLAN perspective will there need to be any additional config to pass the VLAN information correctly, my real question would be does the switch require any more config does it still require populating with the VLAN id's?Anonymous
January 01, 2003
Hi Keith It looks like I will need to configure the ports in trunk mode as I will be using many VLANS. Am I correct in that I will need to populate the switch with the virtual machine VLANS so that it understands? Could you provide me with the correct syntax to create this on a Cisco 2960 as I have found there are many ways to do the same command, also can I split 1 team over 2 switches? I also suppose an alternative to the VLAN is to separate the networks in VMM 2012 and this would reduce complexity and allow me to utilise the switch independent mode, however the only issue I can see with this is that it would not work with VDOMS on my Fortigate appliance as it uses VLANS to separate the VDOMS.Anonymous
January 01, 2003
Hi keith , can you tell me why we dont have stand-by adapter in switch dependent nic teaming?? And can it be possible that we can develop it??Anonymous
January 01, 2003
Hi Sig, Yes, but if you want to extend 1 etherchannel across both switches, you'll need to stack them. Best regards, KeithAnonymous
January 01, 2003
Hi Edward, The switch teaming configuration in the article above is intended for use with static switch-dependent teaming. If you're using dynamic teaming with LACP, you should modify the switch configuration above to use "active" instead of "on" as the channel-group mode by substituting the existing channel-group commands on each interface with: channel-group 1 mode active This will configure the switch to use LACP link negotiation across each of the interfaces in the team. Hope this helps! KeithAnonymous
January 01, 2003
Thank you I think I have got this now you create separate channels per server? as on your second blog you was also using etherchannel for the server uplinks am I correct if I configured the following Switch 1 ports 1-4 Channel 1 relate to server 1 team Switch 2 ports 1-4 Channel 2 relate to server 2 team Switch 1 & 2 Ports 5 - 6 channel 3 relate to server 3 teamAnonymous
January 01, 2003
Hi Sig, Yes - you've got it! :-) One Etherchannel on the switch per NIC team on a server. Best regards, KeithAnonymous
January 01, 2003
Glad that I have got there your second post made everything a lot clearer. I would just like to confirm the split team outline below. Creating a channel on each switch with the same number and having 2 ports from 1 team on each is how I split them across?Anonymous
January 01, 2003
Excellent post I completely understand the new blog, I take it the etherchannel is created to pass the VLAN information between the switches hence needing to be created twice and I imagine this is created on the switch uplinks? As the stack function allows the switches to be seen as one is this why only one is required? Would it be possible for you to provide me with the Cisco command to create the etherchannel based upon configuration in your blog and my hardware?Anonymous
October 16, 2012
Great Post :) .Anonymous
November 21, 2012
The comment has been removedAnonymous
May 01, 2013
I enabled LACP on the switch configuration. The status says it's up. On the windows 2012 server, create an LACP team with 4 members. I see that the status is Faulted - Reason Not found. I see traffic on the 4 members but still faulted. What could be the reason?Anonymous
August 20, 2013
The comment has been removedAnonymous
January 03, 2014
Pingback from Microsoft – Preparing for free exam 74-409 Server Virtualization with Windows Server Hyper-V and System Center | blog.bjornhouben.comAnonymous
January 03, 2014
Pingback from Microsoft – Preparing for free exam 74-409 Server Virtualization with Windows Server Hyper-V and System Center | blog.bjornhouben.com