다음을 통해 공유


Demystify HLB and DNS Load Balancing - Lync 2013 Topology with High Availability (POOLs, DNS LB vs HLB)

The most misunderstood topic in Lync is the high availability. This is regardless of server count, positioning or DNS versus hardware load balancing.

Note

This document is neither a sizing nor a configuration guide. You should use this document only for your environment planning’s purposes.

In the first chapter the article needs to drive into the different understandings of high availability.


Client-to-Server and Server-to-Server traffic high availability

First I need to explain some generic settings about load balancing.

And one word before you read this article:

My personal recommended opinion is the DNS+HLB 1-armed solution, this makes trouble shooting much more simplified and do not require so much network knowledge for the Lync admins. On the Edge server recommend only 1-armed and purely HLB.

Client-to-Server

The only part, were the both option DNS+HLB and HLB come into place is for the Client to Server traffic ONLY.

Server-to-Server

Lync Server traffic is much different. As we setup the server first in the topology, Lync servers are aware about the entire topology setup with in the CMS or XDS database. This source will be queried to load balance server-to-server traffic fully automatically. That’s one part we have later to consider with LB 2-armed solution.


Lync Server Roles entitled for High Availability

Lync Server 2013 can be made high available by using redundant servers, which means we have to deploy multiple server in an array. This array in Lync topology is named POOL. A pool can have assigned 1 up to 12 server, regardless if we are talking about the different server roll, which are directors, frontends, mediations, group chat or edge servers. Beside Lync server, where the Lync application is installed on, we also have several other components, which I describe later in this passage.

Note

You should also understand the difference between High Availability and Resiliency. Please do not mess around with this differences

POOL Servers

All four server rolls described next are entitled for POOL based high availability.

Director Server

The incoming, general SIP requests from external and internal Clients will be redirected to its responsible Frontend system, most likely a Pool. Authentication is part of this server too.

The Director Server host both, Lync Application and IIS for Web Services.

Frontend Server

The core component of Lync. It hosts the users and every of their responsible functions Lync offers. They also will always have collocate the Conferencing part. Beside this, Archiving and Monitoring truly its hosted as well.

Optional, if it makes sense, the Meditation feature could be added to this system too.

Edge Server:

Lync component for external SIP related traffic. This is one of the only servers, which are not responsible for any kind of Web Services Lync has to offer.

But what’s about this server? Well, Lync Edge Server is a standalone, Non-Domain Joined System, designed to act as a true Application Proxy and Application Reverse Proxy.

Let me repeat some basic requirement: You MUST have 2x NICs and 2x dedicated/ separated/ isolated Networks and the internal DNS Suffix of your Active Directory Domain must be present, else Edge cannot work!

Mediation Server

As described, the Meditation server can also be separated from the Frontend Server. It is responsible for translating SIP to ISDN (also SIP Trunk), second it also doing transcoding from Lync codecs into G.711.

Other Servers

All of the server named above are pool servers, which can be configured under a single, addressable name space. All other system Lync need to operate with are different and will have different requirements to operate in high availability mode.

SQL

The SQL server, also called “Lync Backend” can be operated in two high availability modes. One is the well-known cluster, where we have two SQL servers in an active-passive mode. The other option the better choice, SQL server mirroring. Mirroring exists with two SQL server, where the primary server hosts the active database and do log shipping of all its transaction to the second, independent SQL server. Therefor a logical independent database copy exists.

Why this is even better than a cluster? First, we have a second database copy, independent from the other and second, if you only operate the mirror server for Lync, you do not need to pay the SQL server license.

File Share

Lync also need file storage for several features. Here we are able to utilize the most system opportunities. We can simply cluster the file storage, or we make use of Distributed File System (DFS) and its replication technology.

Other third party solutions as NetAPP or EMC can be used to. But you need to ensure the correct Lync requirements are full filled.

WAC

Office Web Application (former Office Web Companion) the server rendering in Lync 2013 all Power Point presentations into HTML5. Regarding high availability, we make use of “FARMS”, a farm are multiple WAC server installed and joined the farm, receiving the identical configuration. Since it is a Web based service, we can load balance this servers. The two methods are: hardware load balancing, or windows load balancing.

Note

This is the only system, which is entitled for Windows Load Balancing.

Gateways

At last, we have the Audio transcoding and protocol changes to ISDN or others. This is for some the most complex systems if we are talking about high availability.

Since the Gateway cannot be deployed in a Lync related pool, we need to create another kind of system collection.

In Lync 2013 we have several components/ elements in the IP flow. Starting with the mediation server, the mediation server has two sites and one of them is associated with the ISDN gateway. This does not mean we need two dedicated NICs, but it is in some circumstances possible or even necessary. In between of this path, we associate both sites with a Lync 2013 TRUNK. This are the three components involved.

Now we have to opportunity for high availability with a backup gateway (alternative gateway), the other option native in Lync if we deploy multiple paths and the third option is a load balanced solution between the mediation server and the gateways as bidirectional communication flow.


Lync High Availability

POOL

Every Lync pool servers are capable of both load balancing solutions, as long a supported client is used. The supported clients are Lync 2010, Lync 2013 and the App Store, as well the mobile clients.

Before I dig into the both scenarios, lets have a quick discussion about the two relevant components in Lync. Beside the Lync applications, we have web related services. Those service are e.g. Address Book Distribution, Distribution Group resolution, Meet and Dialin page, or Lync Mobile Clients Autodiscover and others. We remember the setup of our external facing systems, in our case the Reverse Proxy. Here a special mapping of the related http port 80 and 443 is needed within the redirection process, the port mapping will be 80 -> 8080 and 443 -> 4443. Why is this so?

The Lync Web Services are running inside the IIS, due to security reasons, Lync segregates all web traffic onto internal and external web sites. Since we have only a single IP address on the Lync servers, the only possibility is if its using different ports. Therefore the external connections run on the Lync external site under port 8080 and 4443.

After this excursion let’s come back to the main topics. In other words, we talk about SIP, server function and web services in Lync. All of them have to be high available. While SIP can be considered as a so call “persistent” protocol, if we use simple load balancing mechanism like DNS Round Robin, a TCP connection will with the “supported” Lync client always stay with its initiated Lync server. Well this is a very simple and rudimentary explanation, but should be solvent enough for this article. I more like to high light the word “persistent”, means continuously established IP session. Web Services in general are different. Web service must not and cannot be persistent. This is because of the IP session will always be reestablished after an http request is finished. In Lync we need to submit web requests on a session basis, but how this can be done, if we e.g. use a round robin process? Each time a client would start a new request and the server is not aware that within the same session and we would hit another server in a pool, the information stream within this session would have got lost. Therefore web services must stick to the same server. To do this and enable the client hitting the same server again, a supporting technology must be in place. Here we have several options, cookie based sessions or source IP address affinity. Now, since we understand this requirements, the recommendation for hardware load balancing is clearer. In the same way we can now understand why Windows Load Balancing cannot be used with Lync pool. Windows load balancing is based on shared information within the WLB cluster and communicated via multi/ unicast. This is never sufficient for the requirements of Lync.

Note

Windows Load Balancing can neither be used for Web service, nor Lync application services, even due to those services exist on the same server and cannot be separated.

Count of servers in a pool: 1, 2 or 3+ server in a pool

Very often I’m asked by other consultants: Why Microsoft recommends 3+ server in a pool.

I want to have a look what is happened to the pool with different server counts.

But this discussion is only related to Lync Frontend server pools, other pool follow the simple principals of at least 2 servers and +1 for each high availability user count. Say you have 10.000 users , so you need 2+1 Director server, since 1 server is allowed to fail until the load cannot be handled anymore. If you add more than +1 you increase the high availability but more related to SLAs since more server can fail.

Lync Frontend server are different, for user load and associated quorum. Truly the server to user count is quite similar. If you have 30.000 user for a single pool, you need at least 3 server +1 or more for high availability.

What is if you say, we have only 7.000 users on a single pool?

  1. Single Frontend server in a pool

    this single server is fully sufficient for 7k user and will run with this load. Also the necessary quorum is simple, it’s a single node system equivalent to what we understand from single node clusters.

  2. Dual Frontend server

    this is a working configuration, even if its not recommended. This also does not mean it would be unsupported. BUT:

    If we assign this solution, we need to keep in mind that under some circumstances we might run into issues. Those issues are probably happened, if one server fails and the left one is restarting. In this scenario when the server is restarted, the server is in an inconsistent state and will need up to 1 hour before the services will have started. The Lync Frontend quorum need to be rested.

    Command: Reset-CsPoolRegistrarState –ResetType QuorumLossRecovery

    As we can see, if we only install a pool of Frontend servers with 2 nodes, we need to be prepared handling cases like described.

  3. 3+ more Frontend server

    This is the recommended setup for high availability. If you are planning so for customers you will have the best in class solution with all necessary automated processes and less impact during system failures.

    Disadvantage is still the cost factor, since the server license count is higher.

I will now have the look into the high availability technology required to support those setups.

Hardware Load Balancing

We identified three components, the both IIS web site and the Lync application. Therefore a full hardware load balancing solution will cover all components for Lync. Sure the setup of HLB is a little bit more complex and requires more port which need to be load balanced. Also the load balancer will have more load compared with a setup of only web services. Since the Lync application do not need to off-load TLS encryption to the HLB, the impact is quite less.

I personally prefer this way of setup due to a more clearer setup and we know also in case of issues with load balancing, where to look for issues. The support process is straighter forward.

Other to this is, if we setup a fully load balance solution, we are able to implement 1- or 2-armed setup of load balancers.

Note

Another advantage of hardware load balancing is, if you need to take a pool server offline, this is purely done on the HLB and do not require DNS changes, which also need time to be replicated.

DNS Load Balancing

If you decide for DNS based load balancing, this setup make use of Lync application only. Due to the point we discussed more early, even if we decide for DNS based load balancing, the web services required a hardware load balancer.

For session based load balancing, it is a must that we have a look into the HTTPS data stream, which than require a SSL off-load to be configured.

If we do so, the shared pool server certificate must be copied and used on the hardware load balancer.

Note

Web Services must be HLB, it is necessary to change the INTERNAL and EXTERNAL Web Services to a dedicated name. Example: PoolName: FEPOOL01.<ad fqdn>, internal Web Services: intFEPOOL01.<sip domain>, external Web Services: extFEPOOL01.<sip domain>

Web Services External (Source: http://wenku.it168.com/d_001274570.shtml, p5 )

  • Use TCP idle timeout of 1800 seconds
  • Ports 8080 and 4443 for external web traffic
  • set cookie-based persistence on a per port basis
    • Cookies must not be marked httpOnly.
    • Cookies must not have an expiration time.
    • Cookies must be named MS-WSMAN.
Web Services Internal
  • Ports 80 and 443 for internal web traffic
  • set source_addr persistence
  • Exception for internal Lync Mobile clients » use cookie persistence instead

Specialties

Regardless with load balancing solution we chose, there are still more consideration we have to take in place. I separated this into three chapters

Generic Information Lync Mobility Services

Cookie-based persistence required for Lync Mobilityservices (Source: http://wenku.it168.com/d_001274570.shtml, p11)

  • Marked httpOnly, named MS-WSMAN and no expiration

Edge server

Edge pool server can also either be load balanced on hardware basis or via DNS. Important are here two points:

  1. If you decide for one of the both possibilities, you MUST stick to the same setup on both site of the Edge pool. Which means, the external and the internal site (NICs) must make use of the identical process.
  2. DNAT Load Balancing (half-NAT) for A/V Edge (external) (Ref. http://wenku.it168.com/d_001274570.shtml, p6)
  3. If you decide for DNS based load balancing you have some restrictions:

This restriction apply if the “core” server of the pool will fail.

  • Exchange Unified Messaging:

    If EUM is deployed and Exchange 2010 prior SP1 is used, the user will have no access to their voice mail

  • Office 365 and Public Instant Messaging (PIC):

    The communication is broken, since both systems do not support DNS load balanced Edge pool

  • Federation with partners with Office Communication Server 2007 / 2007 R2

    The communication is broken, since both systems do not support DNS load balanced Edge pool. DNS load balancing was not available with OCS

Mixed of DNS+HLB and HLB

Beside of the provided information for Edge servers and the internal and external site, you are entitled to make use of both solutions with in your Lync topology.

As a simple example:

If you have a topology with 2x Edge, 2x Director and 3x Frontend servers, you can do a setup of DNS LB for the Edge server, DNS LB+HLB for the Directors and HLB for the Frontend server.

Also this scenarios are supported

1-Armed or 2-Armed solution

Load balancer setup can vary in two different deployment, either 1-armed or 2-armed. The difference between both is how the IP flow has to run. While in a 1-armed solution the IP traffic will first hit the LB, than redirected based on the LB algorithm directly to the pool server. Afterwards the IP traffic is a direct communication between the source and target.

In a 2-amred setup the IP flow is different, the source will always communicate with the LB and the LB than starts, based on the algorithm again, with the target pool server.

A specialized setup if a LB between the PSTN site of a Mediation server and its associated gateways. I will describe this later.

Routing vs redirect

What is now this difference between a redirected and a routed setup? If we decide for the 1-armed solution, we enable the source for a direct traffic exchange with the target and we do not need a isolated network. The setup is more simplified compared with the 2-armed solution. Here we need the IP flow to be routed to its target.

This has a direct impact on the high availability setup we can chose. If a 2-armed is the choice, we cannot use DNS LB, this is because we can use multiple A records in DNS pointing to the different pool servers, because we can address them directly due to the LB with acts as a router.

PICTURE

If 2-armed only than u make use of server load analysis

Say, we decided for this option. We have a huge advantage coming from the LB. We are now able to fully control the traffic flowing through the LB. Since the LB continuously know the amount of data routed to the target pool server. The LB is aware of its load.

In scenarios, where we have to consider higher load on the pool, this might be the best choice, since we are able to redirect traffic much more efficient.

Say, one pool server host a huge conference and it load is now quite high, the LB is aware of this traffic and able to redirect other traffic to member server in this pool, which have lesser load.

Truly, this is not described in the Microsoft deployment guides, since this is comes from the networking prospective.

Note

If you are using a 2-armed solution, you cannot use DNS load balancing and you need to ensure, that server-to-server communication is possible and the HLB works for this fully transparent.

PICTURE!!!

Mediation server and gateways

The most complex setup is a load balancer in between of the Mediation server and the gateway. We have reduced ports we are utilizing here, mostly a single port 506x. But we need to plan the Mediation server, load balancer and gateway positioning in conjunction with the Lync TRUNK configuration. So it’s a more advanced config which provides the highest availability and routing of call to ISDN.

PICTURE

Source: http://wenku.it168.com/d_001274570.shtml, p5

  • SNAT Load Balancing (Full-NAT) for gateway/PBX side of Mediation Server Pool
  • Use if Gateway doesn’t support DNS LB to simplify Gateway/PBX configuration

Resiliency

I leave this chapter a bit shorter since it should be only understood as generic knowledge. If you design Resiliency it requires another longer article….

Backup Pool

Some consultants try to avoid HLB balancers at the customers and recommend a two standard edition server deployment, where the second standard server servers as a backup registrar. Sure you can consider, but this is not a god choice and requires much more knowledge for the customer as compared to the little knowledge of HLB.

SBS/SBA

Its really the most recommended solution, if you have branch offices with no admins and you preferable need resiliency for VOICE.

Pool

Assuming you have two datacenter, possibly far away from each other. Here you have the best option setting up this both dedicated pool in a relationship, acting as backup for each other. Combined with GEO-LB this is a solution of amazing high availability.


Technical requirements and ports for load balancing

More early I wrote an article about load balancing, you might consider this in your learning path too.

http://lyncuc.blogspot.de/2012/07/load-balancer-gateway-and-session.html

Certificates

Certificate requirements can become complex in mind of security. If you need to separate the certificates assigned in pools, you have so many choices how to define the certs for all services. But what I want to highlight here are the certificate for Web services.

You are able to assign each Web service an individual certificate, this means internal and external site of the IIS. If you do so or not, you MUST be able to export the certificate to the HLB for SSL OFF-LOADING, this also requires you to have the SAME certificate with the same private key assigned onto all pool servers.

Please note about this else you will run into serious issues.

OAuth protocol is not mentioned here, since this not an option to be load balanced. Just keep in mind, you should have assigned the same OAuth certificate on all pool server as well.

Ports

External Port Settings Required for Scaled Consolidated Edge, Hardware Load Balanced: External Interface Virtual IPs

The table below has been taken from Port Summary - Scaled Consolidated Edge with Hardware Load Balancers

Role/Protocol/TCP or UDP/Port

Source IP address

Destination IP address

Notes

XMPP/TCP/5269

Any

XMPP Proxy service (shares IP address with Access Edge service)

XMPP Proxy service accepts traffic from XMPP contacts in defined XMPP federations

XMPP/TCP/5269

XMPP Proxy service (shares IP address with Access Edge service)

Any

XMPP Proxy service sends traffic to XMPP contacts in defined XMPP federations

Access/SIP(TLS)/TCP/443

Any

Access Edge service public VIP address

Client-to-server SIP traffic for external user access

Access/SIP(MTLS)/TCP/5061

Any

Access Edge service public VIP address

SIP signaling, federated and public IM connectivity using SIP

Access/SIP(MTLS)/TCP/5061

Access Edge service public VIP address

Federated partner

SIP signaling, federated and public IM connectivity using SIP

Web Conferencing/PSOM(TLS)/TCP/443

Any

Edge Server Web Conferencing Edge service public VIP address

Web Conferencing media

A/V/STUN,MSTURN/UDP/3478

Any

Edge Server A/V Edge service public VIP address

STUN/TURN negotiation of candidates over UDP/3478

A/V/STUN,MSTURN/TCP/443

Any

Edge Server A/V Edge service public VIP address

STUN/TURN negotiation of candidates over TCP/443

Firewall Summary for Scaled Consolidated Edge, Hardware Load Balanced: Internal Interface Virtual IPs

The table below has been taken from Port Summary - Scaled Consolidated Edge with Hardware Load Balancers

Role/Protocol/TCP or UDP/Port

Source IP address

Destination IP address

Notes

Access/SIP(MTLS)/TCP/5061

Any (can be defined as Director, Director pool virtual IP address, Front End Server or Front End pool virtual IP address)

Edge Server Internal VIP interface

Outbound SIP traffic (from Director, Director pool virtual IP address, Front End Server or Front End pool virtual IP address)to Internal Edge VIP

Access/SIP(MTLS)/TCP/5061

Edge Server Internal VIP interface

Any (can be defined as Director, Director pool virtual IP address, Front End Server or Front End pool virtual IP address)

Inbound SIP traffic (to Director, Director pool virtual IP address, Front End Server or Front End pool virtual IP address) from Edge Server internal interface

SIP/MTLS/TCP/5062

Any (can be defined as Front End Server IP address, or Front End pool IP address or any Survivable Branch Appliance or Survivable Branch Server using this Edge Server)

Edge Server Internal VIP interface

Authentication of A/V users (A/V authentication service) from Front End Server or Front End pool IP address or any Survivable Branch Appliance or Survivable Branch Server using this Edge Server

STUN/MSTURN/UDP/3478

Any

Edge Server Internal VIP interface

Preferred path for A/V media transfer between internal and external users

STUN/MSTURN/TCP/443

Any

Edge Server Internal VIP interface

Fallback path for A/V media transfer between internal and external users if UDP communication cannot be established, TCP is used for file transfer and desktop sharing

STUN/MSTURN/TCP/443

Edge Server Internal VIP interface

Any

Fallback path for A/V media transfer between internal and external users if UDP communication cannot be established, TCP is used for file transfer and desktop sharing

Director Ports and Protocols for Firewall Definitions (DNS-HLB)

The table below has been taken from : Port Summary - Single Director

Role/Protocol/TCP or UDP/Port

Source IP address

Destination IP address

Notes

HTTP/TCP 8080

Reverse proxy internal interface

Director Hardware Load Balancer VIP

Initially received by the external side of the reverse proxy, the communication is sent on to the Director HLB VIP and Front End Server web services.

HTTPS/TCP 4443

Reverse proxy internal interface

Director Hardware Load Balancer VIP

Initially received by the external side of the reverse proxy, the communication is sent on to the Director HLB VIP and Front End Server web services.

HTTPS/TCP 444

Director

Front End pool or Front End Server

Inter-server communication between the Director HLB VIP and the Front End Server or Front End Servers.

HTTP/TCP 80

Internal Clients

Director Hardware Load Balancer VIP

The Director provides web services to internal as well as external clients.

HTTPS/TCP 443

Internal Clients

Director Hardware Load Balancer VIP

The Director provides web services to internal as well as external clients.

SIP/MTLS/TCP 5061

Edge Server internal interface

Director

SIP communication from the Edge Server to the Director, as well as the Front End Servers.

MTLS/TCP/50001

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

MTLS/TCP/50002

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

MTLS/TCP/50003

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

Director Ports and Protocols for Firewall Definitions (HLB)

The table below has been taken from : Port Summary - Single Director

Role/Protocol/TCP or UDP/Port

Source IP address

Destination IP address

Notes

HTTP/TCP 8080

Reverse proxy internal interface

Director Hardware Load Balancer VIP

Initially received by the external side of the reverse proxy, the communication is sent on to the Director HLB VIP and Front End Servers web services

HTTPS/TCP 4443

Reverse proxy internal interface

Director Hardware Load Balancer VIP

Initially received by the external side of the reverse proxy, the communication is sent on to the Director HLB VIP and Front End Servers web services

HTTPS/TCP 444

Director

Front End Server or Front End pool

Inter-server communication between the Director HLB VIP and the Front End Servers

HTTP/TCP 80

Internal Clients

Director Hardware Load Balancer VIP

The Director provides web services to internal as well as external clients.

HTTPS/TCP 443

Internal Clients

Director Hardware Load Balancer VIP

The Director provides web services to internal as well as external clients.

SIP/MTLS/TCP 5061

Edge Server internal interface

Director Hardware Load Balancer VIP

SIP communication from the Edge Server to the Director, and Front End Servers.

MTLS/TCP/50001

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

MTLS/TCP/50002

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

MTLS/TCP/50003

Any

Director

Centralized Logging Service controller (ClsController.exe) or agent (ClsAgent.exe)commands and log collection

Frontend Hardware Load Balancer Ports if Using Only Hardware Load Balancing (HLB)

Following table is taken from : Ports and Protocols for Internal Servers > Hardware Load Balancer Ports if Using Only Hardware Load Balancing

Load Balancer

Port

Protocol

Front End Server load balancer

5061

TCP (TLS)

Front End Server load balancer

444

HTTPS

Front End Server load balancer

135

DCOM and remote procedure call (RPC)

Front End Server load balancer

80

HTTP

Front End Server load balancer

8080

TCP - Client and device retrieval of root certificate from Front End Server – clients and devices authenticated by NTLM

Front End Server load balancer

443

HTTPS

Front End Server load balancer

4443

HTTPS (from reverse proxy)

Front End Server load balancer

5072

TCP

Front End Server load balancer

5073

TCP

Front End Server load balancer

5075

TCP

Front End Server load balancer

5076

TCP

Front End Server load balancer

5071

TCP

Front End Server load balancer

5080

TCP

Front End Server load balancer

448

TCP

Mediation Server load balancer

5070

TCP

Front End Server load balancer (if the pool also runs Mediation Server)

5070

TCP

Director load balancer

443

HTTPS

Director load balancer

444

HTTPS

Director load balancer

5061

TCP

Director load balancer

4443

HTTPS (from reverse proxy)

Frontend Hardware Load Balancer Ports if Using DNS Load Balancing (DNS+HLB)

The following table is taken from Ports and Protocols for Internal Servers > Hardware Load Balancer Ports if Using DNS Load Balancing

Load Balancer

Port

Protocol

Front End Server load balancer

80

HTTP

Front End Server load balancer

443

HTTPS

Front End Server load balancer

8080

TCP - Client and device retrieval of root certificate from Front End Server – clients and devices authenticated by NTLM

Front End Server load balancer

4443

HTTPS (from reverse proxy)

Director load balancer

443

HTTPS

Director load balancer

444

HTTPS

Director load balancer

4443

HTTPS (from reverse proxy)

References