Security

Protect Instant Messaging with Forefront Security

Molly Gilmore

At a Glance:

• Protecting IM with FSOCS
• Deploying FSOCS in an OCS environment
• IM message flow and throughput
• Managing and protecting IM content

Contents

Optimal Antivirus Protection of IM
Deploying FSOCS in an OCS Environment
Securing IM-based File Transfers
File Transfers with External Users
Message Stamping
IM Message Flow within OCS
Managing Message Scan Throughput
Managing and Protecting IM Content
Configuration Examples
Take Away

Forefront Security for Office Communications Server (FSOCS) is a server-based antivirus product that is part of the Forefront Server product line. FSOCS provides effective protection against instant messaging—targeted malware within OCS 2007 and OCS 2007 R2 environments.

FSOCS scans all IM activity—including instant messaging content as well as IM-based transferred files—for viruses and prohibited content. Fine-grained control over where and how this is accomplished is exposed to the administrator through easy to use product configuration options. Real-time notifications of virus threat detections and breaches of policies set around IM content can be enabled within FSOCS so administrators can monitor the security activity within the system. In addition, FSOCS reports and performance counters are available for administrators to evaluate the ongoing health and performance of FSOCS and the security of the Instant Messaging flowing through the OCS infrastructure .

Optimal Antivirus Protection of IM

FSOCS integrates with leading third-party antivirus engines to ensure rapid and consistent responses to new and evolving IM-borne threats. Each instance of FSOCS can have up to five antivirus engines enabled; the OCS Standard Edition, Access Edge, Director, and Front-End servers are all protected by the Forefront Server multi-engine solution.

The antivirus engines are installed on the server with each install of FSOCS, and FSOCS interacts with each engine for IM virus scanning. FSOCS has the logic to evaluate the detection information each antivirus engine provides and to choose the scanning sequence most likely to result in an early, accurate detection of a virus. Administrators decide if they want to virus scan with all five antivirus engines or a subset of the available engines. Administrators do not have to configure or interact with these engines individually, as this is accomplished by FSOCS.

FSOCS includes components to manage the continual updating of antivirus signatures, as well as engine binary updates, so that real-time response to evolving threats is achieved. These update components communicate with a Microsoft-hosted system that constantly polls antivirus partner downloads sites to retrieve, test, and package engine and signature updates 24 hours a day, every day of the year. FCOCS administrators can determine how frequently their installations attempt to retrieve engine updates, but FCOCS handles the process of checking for downloads and installing the updates seamlessly.

In environments where the Forefront Server for Exchange or Forefront Server for SharePoint products are installed, FSOCS can be configured to retrieve updates from a previously configured redistribution server.

Deploying FSOCS in an OCS Environment

FSOCS can be deployed in both OCS 2007 and OCS 2007 R2 environments, and it supports the server specifications recommended for each OCS version. OCS 2007 deployments require Windows Server 2003 SP1 at the minimum, and Windows Server 2003 R2 is recommended. OCS 2007 and FSOCS will run on 64-bit versions of the Windows Server 2003 operating system when running in WOW64 mode. OCS 2007 R2 deployments require 64-bit hardware, as well as Windows Server 2008, Windows Server 2003 (SP2), or Windows Server 2003 R2 (SP2).

FSOCS should be installed on each instance of the OCS Standard Edition, Front-End, Access Edge, and Director server roles in both OCS 2007 and OCS 2007 R2 environments. In Enterprise Edition topologies, FSOCS should be installed on every server located within a Front-End, Director, or Access Edge Server Pool. The diagram in Figure 1 shows FSOCS components installed on each supported OCS server role.

Figure 1 FSOCS Components deployed on each supported OCS server role

The ForefrontRTCProxy integrates with OCS 2007 and OCS 2007 R2 through the Office Communications Server 2007 Server Application API. When FSOCS registers with OCS, it instantiates an MSFT_SIPApplicationSetting object (defined in the .Net Microsoft.RTC.Sip namespace) and sets the "Critical" attribute to true so that OCS will not start up without the ForefrontRTCProxy service running.

The ForefrontRTCProxy communicates with the FSCController to instantiate the FSOCScanner processes. It is the FSOCScanner that manages the virus-scanning calls to the enabled antivirus engines. In the FSOCS Administrative Console on the SETTINGS…General Options panel, the IM Process Count setting determines how many instances of the FSOCScanner get created. Additional instances increase scanning throughput, though at the cost of using more system resources.

Each IM message is routed through the ForefrontRTCProxy and onto an available FSOCScanner, which applies filtering rules first, then passes the message onto the configured antivirus engines to look for any viruses within the IM message or IM-transferred file.

If a filter rule is triggered or a virus detected, the FSOCS IM Notification Agent alerts the IM user that an IM message or file they attempted to send or receive contained a virus or restricted keyword or file type. Notifications are optionally sent to the sender and recipient via an IM session. The notification message content is customizable.

The Forefront IM Notification Agent is itself a client of OCS so that it can establish an IM session with the recipient of the IM notification. (For security reasons, FSOCS cannot insert SIP messages into the IM session it is filtering so it creates its own session with the user).

Securing IM-based File Transfers

By default, IM-based file transfers occur as a peer-to-peer TCP/FTP file copy between two instances of Office Communicator. The SIP messaging used to negotiate the file transfer is routed between the sending and receiving Office Communicators through OCS. Figure 2 shows the sequence of SIP messaging used to set up a file transfer.

Figure 2 Default OCS SIP messaging used to negotiate a file transfer through Office Communicator

The recipient uses the sender's IP address to initiate the TCP connection that will allow the FTP file transfer to occur.

As Figure 3 shows, when FSOCS is installed, the sequence is a bit different. FSOCS filters the SIP messaging in order to redirect the file transfer through the server by storing the original sender IP address and replacing it with the FSOCS IP address.

Figure 3 Modifications to SIP messaging for file transfers once FSOCS is installed

FSOCS uses the stored IP address to connect to the sender's computer and then copies the file to the server. FSOCS scans the file and if the file is clean, allows the recipient to transfer the file from the server to the desktop. If the file transfer fails as a result of a detect action, configured notifications will be sent to the sender, recipient, and/or administrator. No additional configuration to OCS or Office Communicator is needed to enable scanning of IM-based file transfers for internal users.

File Transfers with External Users

If the connection necessary to transfer files between internal and external users is successfully made, IM-transferred files will be protected by FSOCS across the Access Edge. At least one Access Edge server role needs to be available to allow IM with external users and each instance of the Access Edge server role needs to have FSOCS installed. If the administrator wants to facilitate file transfers across the Edge, the firewall should be configured to allow inbound connections to the Forefront application running on each Access Edge server. By default, this uses ports 6891-6900, but the port range can also be configured via two Registry keys: FileTransferStartPortRange and FileTransferMaxPorts

Message Stamping

In an Enterprise Edition topology, a SIP message could be routed across multiple instances of OCS and FSOCS. The first instance of FSOCS that determines a message is clean will add a message stamp to the SIP Message. It does this using the Message.Stamp property of the Message class in the Microsoft.RTC.Sip namespace. The Message.Stamp property can be updated, stored in the SIP message and subsequently passed forward through the SIP message routing sequence. FSOCS uses this property to indicate that a SIP message has already been scanned.

IM Message Flow within OCS

At the core of each OCS server role is an implementation of a SIP Server with Proxy, Redirect, and Registrar services. These services allow the OCS server to receive SIP messages bearing IM content, locate target User Agent End Points such as Office Communicator, and route or forward SIP messages accordingly. FSOCS secures all IM by inserting itself into the SIP message routing flow to scan and filter IM for viruses or other prohibited content so it can block compromised IM from being transmitted. By inserting itself right into the SIP message flow, FSOCS can secure IM with minimal impact on OCS performance.

Figure 4 shows the typical SIP message flow as it is routed across an OCS Enterprise Edition topology with FSOCS installed. IM has been enabled for a federated partner. In this scenario, a user belonging to the federated organization initiates an IM to an internal user. Here is the sequence of events:

1. 1.The IM entry point is the Access Edge server. The instance of FSOCS on the Access Edge server receives the IM, scans it for viruses and filtering rules and determines it is safe. The IM is stamped as clean and is routed through to the Director server role on its way to the intended recipient's computer on the internal network.
2. 2.The Director server role has been deployed as recommended to offload user authentication from the front-end servers. This server role is typically deployed on the internal network so that it can access the Active Directory instance storing OCS user configuration information. The IM is routed to this server from the Access Edge. Typically, the internally facing next hop server for an Access Edge is the Director server role. FSOCS checks to see if the IM has already been stamped as clean by an earlier instance of FSOCS and if so, it will avoid processing the message further and will let OCS route it forward.
3. 3.The IM is routed next to the Front-End server pool. The intended recipient will be homed to one of the Front-End servers in the pool. The instance of FSOCS on the Front-End server that receives the IM will also bypass the IM as it has already been stamped as clean. The Front-End server locates the recipient and routes the IM forward.

Figure 4 SIP message flow in an OCS Enterprise Edition topology

All of this activity is transparent to the end user.

Managing Message Scan Throughput

When deployed within an OCS Enterprise Edition environment, FSOCS provides a mechanism to handle unexpectedly heavy IM activity. In these rare situations, the depth of the queue storing the IM messages to be scanned by FSOCS on a server can increase to the point that an IM message at the end of the queue will not be scanned within an acceptable amount of time.

There are default actions FSOCS will take if this occurs, as well as configuration options available to the administrator for managing the actions FSOCS will take. Since FSOCS is sensitive to the length of the IM message queue waiting to be scanned, a default threshold for maximum IM queue depth has been established to indicate when a queue has grown too large. The action FSOCS takes in response to a queue length threshold exceeded event depends on the server role where FSOCS is deployed. Here are the default actions taken on the indicated server role.

Access Edge FSOCS routes the IM forward unscanned to the next instance of FSOCS, without applying any message stamp.

Director FSOCS routes the IM forward unscanned and unstamped to the Front-End server role. The recipient will not receive the IM unless it has passed through an instance of the Front-End server role.

Front-End FSOCS scans the IM. In the unlikely event that the IM queue exceeds the threshold on this server role, FSOCS will take additional measures to resolve the situation but in the end will drop the message rather than letting it through unscanned.

Note that threshold values can be set on each server role through the MessageOverloadWatermark registry key. The value is stored as a DWORD with the following defaults: 1,000 for Access Edge, 3, 000 for Director, and 10,000 for Front-End server roles.

Message stamping can also be turned off by setting the DisableMessageStamp registry key. This is stored as a DWORD value; the default is 0 but can be set to 1 to disable message stamping.

Managing and Protecting IM Content

FSOCS provides an additional layer of control over the flow of information via IM both internally (between employees within the internal network) and across the network perimeter to external users. Within FSOCS, these controls are put into effect via three different filter types.

File Filters can be configured to prevent certain files from being sent between IM users. Administrators can identify the files they want restricted by specifying file names, file extensions, file sizes, or file types (true file-type detection is included). For example, FSOCS can be configured to block all executable files from being transferred via IM, even if the file extension has been changed from .exe to .txt.

Administrators can further control IM-based file transfers by identifying where a file filter rule is put into effect. File filter rules can be applied to all IM-transferred files or specifically to files sent between internal users, files heading outbound from internal users to external users, or files coming inbound from an external user across the network perimeter to an internal user. For example, FSOCS can be configured to allow all file types to be transferred via IM between internal users but to block all executable files coming from external users to internal users.

Keyword filters can be configured to block IM message content or text-based transferred files from being sent when they contain restricted words or phrases. Keywords or phrases can be defined in any language. In addition, FSOCS comes with an installable profanity list that administrators can optionally use to block offensive language within IM. Keyword filters can also be associated with the direction of the IM or transferred file; internal, inbound, or outbound.

Content filters allow the administrator to block IM or IM-transferred files based on the domain or SIP URI of the IM sender or recipient. By using a wildcard character, IM and transferred files can be blocked for all users of a specified domain.

For example, by configuring "*.unknown.com" within a content filter, all IM from or to users associated with the unknown.com domain will be blocked. IM can also be blocked at the user level by configuring an individual user's SIP URI.

Certain concepts are relevant to all filters within FSOCS:

Detect Actions determine the processing FSOCS will perform on any IM message or transferred file that matches configured filter criteria. For example, a keyword filter in FSOCS can be configured with a detect action of block so that an IM message that contains a restricted keyword will be prevented from reaching any user.

Notifications are the optional alerts generated by FSOCS when a detect action takes place. Alerts can be configured per filter and can always be sent to the administrator. Depending upon the filter type, the IM sender, the recipient, or both can be alerted about a detect action. For example, if a sender attempts to IM a restricted word or phrase, FSOCS can be configured to send an alert to the administrator and to the sender indicating that the IM was blocked due to the restricted keyword.

Notifications are always sent to the administrator via e-mail; senders and recipients receive the notification through an IM conversation initiated by FSOCS.

Quarantine is the repository for IM messages and transferred files that are blocked as a result of a detect action. Administrators have the opportunity to evaluate the quarantined items and resend them through e-mail to the original recipient and others if they deem the content or file to be appropriate.

Configuration Examples

Now let's consider how an administrator can ensure that sensitive or private information that is openly discussed by some employees does not get sent to anyone outside of the company through IM. This is important as OCS has been configured to allow IM with federated organizations and several public IM networks.

One way to accomplish this goal is to configure each instance of FSOCS that is deployed on the OCS Access Edge server role with a keyword filter that blocks any IM message or text-based transferred file containing the restricted words or phrases from being sent from an internal user to an external user. As noted earlier, all IM messages sent from an internal user to an external user are routed across the Access Edge server role before reaching the external user, so FSOCS will block this information at the network perimeter.

On the other hand, if the administrator wants to block offensive information from coming into the internal network from external users that could be transmitted through an IM message or transferred file, the keyword filters should be configured on the instances of FSOCS installed in the internal network on either the OCS Standard Edition or Front-End server roles.

Administrators gain even more control via Allowed User Lists, which provide the option of configuring FSOCS to exclude a set of users from having their IM evaluated against configured filters. Administrator can associate an Allowed User List with the type of filter they wish to bypass. For example, an administrator can configure FSOCS to block all IM from a public IM domain using a content filter, then identify a subset of users (through an Allowed User List) of that public IM domain that are allowed to send and receive IM from internal employees.

Take Away

As IM becomes ever more popular, it is increasingly important that administrators have a way to keep it secure. We hope the summary information presented on what Forefront Security for OCS does, and the additional details around configuration and performance, give insight into how FSOCS can provide confidence to administrators that IM in their organization is secure and within policy.

Molly Gilmore is a program manager working on the Forefront Security Rapid Response Engineering team located on Long Island, New York. In addition to working on the first release of Forefront Security for Office Communications Server, she also works directly with the software vendors who develop the antivirus and antispam engines distributed with the Forefront Server Security product line. Molly holds a Master of Engineering in Engineering Management and a graduate certificate in Quantitative Software Engineering from Stevens Institute of Technology. She started as a software developer in 1991 and worked in different roles and technologies before joining Microsoft in 2006.