Lync Server 2010: Security at the Edge
DoS and password brute-force attacks can be particularly nasty, but you can protect your network against these types of disruptions.
Rui Maximo
In this extraordinarily interconnected world, companies want to allow the utmost flexibility and mobility for their employees, many of whom may work remotely. Consequently, almost every organization exposes services to the Internet. However, there’s always the threat of attacks. Some companies are particularly concerned with Denial of Service (DoS) and password brute-force attacks. These are legitimate concerns. These types of attacks can be disruptive to users and consume internal server resources.
The primary trouble with DoS attacks is that they’re nearly indistinguishable from legitimate sign-in requests. The only differentiation is the frequency of sign-in attempts and their origin. A large number of sign-in attempts in rapid succession can be indicative of a DoS attack.
Most DoS attacks attempt to guess the user’s password to gain unauthorized access. They often result in locking out the user account if the security policy is enabled in Active Directory Domain Services, and has a maximum number of log-in attempts.
The Microsoft Lync Server 2010 Edge Server protects against unauthorized access using industry-standard security measures. It monitors sign-in requests and enforces account lockout at the network perimeter. All communications are encrypted and authenticated.
Edge Server does not protect against DoS attacks. However, Lync Server provides a flexible programming platform you can use to create server applications to intercept Session Initiation Protocol (SIP) messages on the server and perform specialized logic using the Microsoft SIP Processing Language (MSPL). This is how the security filter operates.
It inspects all incoming sign-in requests on the Edge Server. The remote user is not authenticated at the Edge Server, so the sign-in request is passed to the Director or directly to the internal pool, which then performs the authentication process. The response is then passed back to the Edge Server. The security filter inspects both the request and the response. If the sign-in fails, the security filter tracks the number of failed attempts for each user account.
The next time a client attempts to sign in to the same user account, and the number of failed attempts exceeds the maximum number of allowed sign-in attempts, the security filter immediately rejects the request without passing the request to the Director or internal pool for authentication. By enforcing account lockout at the Edge Server, the security filter blocks DoS attacks at the edge of the network perimeter. As a result, the security filter protects the internal Lync Server resources.
Using the security filter to prevent Windows NT LAN Manager (NTLM) version 2 authentication, companies can force users to only sign in from authorized company-issued laptops. With additional security measures (like using BitLocker and Group Policy to prevent users from installing unauthorized software), the corporate-issued laptops can themselves serve as a “smartcard” to provide two-factor authentication.
If at First You Don’t Succeed …
To prevent brute-force attacks on user accounts, many organizations enforce an Active Directory Group Policy to lock out the account after a certain number of failed attempts. The side effect of this countermeasure is that the attacker can lock out a user’s account by simply launching multiple attempts. This amounts to a DoS attack.
If the account isn’t protected by an Active Directory Group Policy, the attacker can use this type of brute-force attack on the user’s password. These attacks use up valuable internal server resources and deny users access to their account.
Uniquely identifying the user can prevent attacks on user accounts. There are several options with which to do this. You could use the source IP address, the sign-in name (that is, the SIP URI), the account name or even a combination of any of these options. After investigating each option, it seems that rogue clients mounting a DoS attack could spoof the source IP address, eliminating this choice as a way to uniquely identify the user.
The sign-in name, although required to successfully sign in to Lync Server, does not authenticate the user. A sign-in name can be varied during sign-in requests, yet still lock out the same user account. Therefore, neither the source IP address nor the sign-in name are good sources with which to identify the user. Only the account name uniquely identifies the user account.
You can only extract the account name, which consists of the user name and domain name, from the authentication protocol. Remote users trying to sign in and authenticate use the NTLM v2 protocol, not Kerberos. The NTLM protocol uses a three-stage handshake authentication process. The client passes the user’s credentials in the third stage of the NTLM handshake.
The security filter runs as a trusted server application on the Edge Server, so it’s allowed to intercept this sign-in request. The security filter decodes the user name and domain name from the NTLM authentication message. Because the account name isn’t available in the response, the security filter maps the response to the request using the message ID.
When either the internal pool or the Director sends the authentication response to the Edge Server, the security filter captures the Register response. If the sign-in failed, the security filter counts the failed attempts. If the sign-in succeeds, the security filter resets the count of failed attempts to zero.
Filtering at the Edge
Every time the Edge Server receives a sign-in request, it’s passed to the security filter. It checks whether the sign-in request has exceeded the maximum number allowed for that particular user account. If the request has not exceeded the maximum lockout count permitted, the security filter allows the request to continue to either the internal pool or the Director.
If the request exceeds the maximum lockout count permitted, the security filter blocks the request and returns a 403 response. This summarily rejects the request. Any further sign-in attempts are rejected for the duration of the lockout period. After the lockout period expires, it’s reset to permit new sign-in requests.
One problem can occur when users sign in from a computer not joined to the corporate Active Directory domain. Lync 2010 can automatically attempt to sign in using the user’s local computer credentials. Because those credentials aren’t corporate domain credentials, the authentication will fail. The user will eventually be blocked from signing in to Lync Server. To prevent the security filter from locking out valid users, it doesn’t count these attempts against the user.
Lync Server 2010 introduces support for an additional authentication protocol called TLS-DSK. This requires users to supply a client certificate for authentication. The Lync client requests certificates from Lync Server. This is an automatic process that happens the first time the user signs in to Lync Server from within the corporate network where the user is authenticated using Kerberos.
This client certificate is used for authentication with any subsequent log-in attempts. This is a self-signed certificate issued by Lync Server, not a Certificate Authority. If that same user tries to sign in to Lync from a different computer, he’s authenticated using Kerberos (if inside the corporate network) or using NTLM v2 (if outside the corporate network). The process of obtaining another client certificate starts all over.
TLS-DSK provides a level of security that’s very close to two-factor authentication. When combined with Windows BitLocker, the computer or laptop acts as the equivalent of a smartcard (something you have). The password that BitLocker requires to boot your computer is equivalent to the pin required to authorize the use of the smartcard (something you know).
There’s the remote possibility someone could steal the client certificate from the user’s computer, but you can mitigate this risk. Make sure corporate-issued computers are locked down to prevent users from downloading unauthorized applications.
You can force the Edge Server to negotiate the authentication protocol down from TLS-DSK to NTLM v2. In this case, the attacker can still target the user’s account, as discussed earlier. To prevent this scenario, the security filter provides an option to reject all NTLM v2 authentication requests, forcing TLS-DSK-only authentication. This doesn’t affect federated partner connections or PIC connections.
Configuration Considerations
Before you can run the security filter application, you must first register the application with your Edge Server. You only need to do this registration once by taking the following steps. Run these Lync Server 2010 Windows PowerShell cmdlets with Lync Server administrative permissions:
1. Run the following Windows PowerShell cmdlet to register the security_filter application from any Lync Server. Specify the fully qualified domain name (FQDN) of the Edge Server in the parameter <Edge Server FQDN>:
new-CsServerApplication -identity "EdgeServer:<Edge Server FQDN>/security_filter" -uri "https://www.maximo.ws/security_filter" -critical $false
2. Run the following Windows PowerShell cmdlet to initiate the replication of Central Management store configuration to the Edge Server:
invoke-CsManagementStoreReplication
3. Run the following Windows PowerShell cmdlet on the Edge Server to verify the proper registration of security_filter:
get-CsServerApplication -localstore
To run the security filter, there must be three parameters passed to the command-line version. For the Windows service version, the installer will prompt you for these parameters. The first parameter specifies a comma-delimited list of your internal Netbios domain names. These are the domain names remote users will use when authenticating to your internal Lync Servers connecting through your Edge Server.
Let’s imagine your company is called Woodgrove Bank, and has the following three internal Active Directory forests: woodgrovebank.com, contoso.com and fabrikam.com. Employees have accounts from each of these forests, so you should specify “woodgrovebank,contoso,fabrikam” as the value for this first parameter to the security filter.
These domain names verify that remote users trying to sign in to Lync Server are connecting by using credentials from one of these three domains (for example, contoso\bob, fabrikam\alice and so on).
The second parameter is the account lockout count. This is the number of failed sign-in attempts permitted before the account lockout is triggered.
The third parameter is the account lockout period. After an account is locked out, this specifies how long the account remains locked before it will accept another sign-in attempt. Any sign-in attempts during this lockout period are immediately rejected without verification.
Using the security filter and these authentication protocols with Microsoft Lync Server 2010 Edge Server, it becomes more difficult for attackers to either attempt to brute-force user passwords or stage a DoS attack.
.jpg "Rui Maximo")
Rui Maximo has more than 18 years in the technology industry, having worked with Microsoft, IBM, RSA, and several startups. His education is in computer science, mathematics and cryptography. His knowledge of Lync Server dates back to 2003, when it was originally called RTC. He worked on the RTC product team as a program manager and then lead program manager. Maximo is the primary author of five books on Microsoft Lync Server and its previous versions.