Microsoft Forefront

Using Microsoft Forefront TMG 2010 as a Secure Web Gateway

Yuri Diogenes, Jim Harrison, and Mohit Saxena

When companies establish a security policy, a key goal is to enable employees to more safely browse the Internet. Of course, this encompasses a very broad area and covers a number of requirements, such as:

  • Ensure users adhere to company policy regarding Internet access.
  • Inspect the traffic and block potential malware and suspicious activities.
  • Make end users aware their Web traffic is inspected.

Fortunately, taking advantage of the secure Web gateway capabilities in Microsoft Forefront Threat Management Gateway (TMG) 2010 can help you meet these goals. Figure 1 shows the main Forefront TMG 2010 features you can use to implement a secure Web access gateway:



Figure 1 Core Forefront TMG 2010 Features for Secure Web Gateway Scenario

As you can see, Forefront TMG uses three main cloud components: Microsoft Update, Telemetry Service and Microsoft Reputation Service (MRS). Microsoft Update keeps anti-malware and Network Inspection System (NIS) signatures updated. The Microsoft malware response team uses telemetry reports provided by TMG to learn what attacks are being seen and thus enhance signature quality. MRS maintains a huge database of categorized URLs that Forefront TMG 2010 can query.

In this article we’ll focus on URL filtering and HTTPS inspection. We covered malware inspection in-depth in our previous  TechNet Magazine February 2009 article, and the feature hasn’t changed in Forefront TMG 2010.

Using URL Filtering for a Safer Browsing Experience

URL filtering can be viewed as the Forefront TMG first line of defense for helping to secure your organization’s Web access. By using URL filtering to block requests for undesirable Web sites, Forefront TMG can spend less time scanning for malware and more time delivering useful content.

URL filtering consists of two main parts—the Web proxy filter that evaluates Web requests, and MRS, which provides the category definitions from which the filter determines how a request is to be categorized.  Here’s a simplified form of the process that occurs for a Web request:

  1. The Web application sends a request for “” through TMG.

  2. The Web proxy filter passes the request through URL filtering.

  3. URL filtering splits the whole request into different parts. For our example, the parts would be something like:

    • Com
  4. URL filtering looks for each part into the local URL category cache.

    Note: if the request cannot be matched to a URL category within the local cache, URL filtering queries MRS. If MRS returns “unknown,” or if MRS can’t be contacted, URL filtering returns an “unknown” response to the Web proxy filter.

  5. Once the URL filtering knows the category associated with each URL part, it determines the category that will apply to the whole URL. To do this, TMG uses a predefined precedence list provided by MRS that lets TMG know which categories take precedence over others. For instance:

    • Com = Unknown
    • = General Business
    • = Malicious
    • = Unknown In this example, two categories are identified: General Business and Malicious. Based on the predefined precedence list, Malicious takes precedence over General Business; as a result, the whole URL will be designated as Malicious.
  6. If the resulting URL category set matches a deny rule, the request is rejected and an error response is returned to the Web application.

  7. If the URL category doesn’t match a deny rule and does match an allow rule, the request is served according to the matching rule’s remaining criteria.

  8. If the request fails to match any user-created deny or allow rule, TMG rule processing falls through to the default deny rule and returns a rejection response to the Web application.

Because users tend to be statistically habitual in their Web usage, the URL category cache will become more relevant to your organization over time as Forefront TMG processes user requests. In this way, the ratio of MRS queries to user requests will eventually decrease. Because URL categories are likely to change, MRS assigns a time-to-live for each entry. As such, the number of MRS requests will never completely reach zero, even if your users always go to the same sites.

To fully understand how URL filtering works, you have to also understand how Web applications create connections and issue requests. Web applications generally fall into two categories: those that are configured to act as Web proxy clients, and those that aren’t.

  • Web (CERN) Proxy: This type of Web application is configured so that it knows of a Web proxy and is able to behave accordingly. The Web application connects to the Web proxy listener and issues requests in a specific form:




In this case, Forefront TMG, and thus URL filtering, has the complete URL available for comparison with the URL filtering database.

Note: METHOD may be any valid HTTP method, such as GET, POST and so forth.




For these requests, Forefront TMG and URL filtering have only the hostname or IP address (depending on how the client issues the request) and port for comparison with the URL filtering database.

  • Non-Web Proxy: When the Web application is not configured to act as a CERN proxy client, it will try to resolve the name of the Web site to an IP address, and if this succeeds, will try to connect to that IP address using whatever port was part of the original URL. Regardless of whether the client is a TMG Client (formerly known as a Firewall client) or a SecureNET client (aka SecureNAT client), the request will be sent using the following form:
METHOD /path/page.aspx?querystring HTTP/1.x

In this case, the ability of URL filtering to evaluate the request is dependent on two criteria:

  • Is the connection directed to the default HTTP port? If so, the Web proxy may be able to intercept this request and pass it to URL filtering for comparison. If not, the request will not be seen by URL filtering and thus cannot be compared to the database.
  • If the connection is directed at the default HTTPS port, is HTTPS inspection enabled? If so, HTTPS inspection can bridge the connection, and URL filtering will have an opportunity to compare the request to the database.

What may not seem obvious is that URL filtering in and of itself does not provide any form of blocking mechanism—it merely responds to the Web proxy with the categories associated with the requested URL as the Web proxy (and thus URL filtering) understands it. In all cases, when URL filtering receives a client request, it functions as illustrated in Figure 2.


Figure 2 Basic Flow of URL Filtering

If Forefront TMG needs to query MRS, a request is made using a single Web Services call to the MRS Web services portal. MRS processes the data presented by Forefront TMG and responds with the current URL categories that apply to the data MRS received. Forefront TMG 2010 has a predefined domain set that includes the portal destinations in effect when Forefront TMG shipped (see Figure 3).


Figure 3 MRS Domain Set

If the URL categories returned by URL filtering are found in a “deny” rule, Forefront TMG sends a rejection response (HTTP result-code 502) to the client. Depending on whether the client is a browser or other Web application, the user may see the Forefront TMG response page; in the case of a non-browser application (such as Windows Media Player or a CERN-proxy FTP application), the user may simply see an error message from the application itself.

Because the most expensive task Forefront TMG has to perform in order to determine the URL category is to query MRS, this process is much cheaper in terms of CPU, memory and network resources than allowing the user to reach the Web site, then having to perform malware scanning on all of the content they request.

Control over the Encrypted Channel Using HTTPS Inspection

Users have been advised for many years that to perform a secure online transaction, they have to use HTTP with SSL (HTTPS). However, this secure channel has also been used for malicious purposes. When users start a transaction using HTTPS, this traffic is normally encrypted end-to-end (from user to destination server), making the content being exchanged between them unreadable to any device in between. While this is a desirable behavior, because you don’t want anybody looking at your online credit-card transaction, the drawback is that any malicious operation within this channel can’t be evaluated. Figure 4 highlights the main parts of this operation using ISA Server 2006 as the firewall.


Figure 4 Traditional HTTPS Scenario

Figure 4 illustrates a full proxy scenario in which the client is accessing an HTTPS site. The summarized steps are divided in two main phases, as outlined here:

Phase 1 – SSL Tunnel

  1. Client connects to the Web proxy.
  2. Client issues an SSL tunnel request: CONNECT HTTP/1.1.
  3. Web proxy resolves to IP address
  4. Web proxy connects to on TCP port 443.
  5. Web proxy sends “200 OK” to the client.

Phase 2 – Encrypted conversation

  1. Client and Web server exchange SSL handshake messages, encryption keys and certificates.
  2. Client now has an encrypted tunnel end-to-end with the destination server and it will start send-and-receive traffic through this tunnel.
  3. ISA Server will keep that tunnel open between client and server but it will not inspect the traffic because it doesn’t have this capability.

The potential risk in this scenario is that as of phase 2, the edge firewall has no knowledge of what really is being transmitted on that channel. In a scenario where the destination server is hijacked and malicious code inserted, there’s a potential risk that the destination server will send malware through this encrypted channel that the client will receive without hesitation because it comes through a supposedly trusted connection.

Forefront TMG HTTPS Inspection reduces this threat by inspecting the traffic and maintaining separate encrypted channels with the user and server. Figure 5 shows how Forefront TMG accomplishes that.


Figure 5 HTTPS Inspection in Action

The summarized steps are still divided in two main phases; however, the process now includes inspection:

Phase 1 – Client Request

  1. Client connects to the TMG Web proxy listener.
  2. Client issues an SSL tunnel request: CONNECT HTTP/1.1.
  3. TMG resolves to IP address
  4. TMG connects to on TCP port 443.
  5. TMG negotiates an SSL connection with the server and evaluates the certificate.
  6. If the certificate is valid and trusted, TMG responds “200 OK” to the client.

Phase 2 – Encrypted conversation with Inspection

  1. Client and TMG exchange SSL handshake messages, encryption keys and certificates. Note that because TMG builds a server certificate using information derived from the Web server certificate, the client believes it is communicating with the Web server.

  2. Client now has an encrypted tunnel with Forefront TMG and Forefront TMG has an encrypted tunnel with the destination server. Forefront TMG will be able to inspect all traffic sent between client and server by following this sequence:

    a)     TMG receives and decrypts the encrypted traffic from the destination server.

    b)     TMG applies Malware Inspection and NIS filters to the traffic.

    c)      If the malware and NIS filters allow, TMG will encrypt the results and send it to the client workstation.

    d)     Client will receive, decrypt and process the traffic.

To establish the SSL handshake with the client, Forefront TMG needs a server certificate. In order to create it, Forefront TMG creates a spoof certificate based on the original server certificate data and signs it with the HTTPS inspection CA certificate.  To ensure maximum performance, Forefront TMG maintains a cache of duplicate server certificates. TMG will search for a duplicate server certificate in the local cache and if one is not found, it will duplicate the upstream server certificate and place it in the cache. The cache is stored only in memory. Thus, the spoof certificate cache is empty after the Firewall service is restarted.

Note: The size of cache (number of certificates) is controlled by the LowLevelSettings.ClonedCertificatesCacheSize COM property.

HTTPS Inspection Options

Before configuring HTTPS inspection, it is important to understand the full feature set and options that comprise this feature. Figure 6 shows the areas where HTTPS Inspection can be configured.


Figure 6 HTTPS Inspection Feature Set

When planning your HTTPS inspection implementation, you need to first consider the certificate settings, to decide whether you will use a self-signed certificate or a certificate issued by an internal CA. When importing an existing trusted certificate authority, you need a PFX file that contains the certificate of the authority along with its private key. You need this private key in order to sign the spoof certificate issued by TMG. You must also make sure the certificate’s key usage is set for certificate signing. This CA certificate must then be deployed on the client computers (under “Trusted Root Certification Authorities” of the local computer certificates store); otherwise, the client won’t trust the server certificate received from TMG.

On Forefront TMG, HTTPS inspection can be enabled through the Web Access Policy. Follow these steps to enable this feature:

  1. From the Forefront TMG console, click Web Access Policy and select Configure HTTPS Inspection under Web Protection Task on the Tasks pane.
  2. On the General tab of the HTTPS Outbound Inspection screen, select the Enable HTTPS inspection checkbox as shown in Figure 7.


Figure 7 Enabling HTTPS Inspection

For this example, we’ll use a Forefront TMG self-signed certificate. Click Generate and a page similar to the one in Figure 8 will appear.


Figure 8 Generate Certificate Window

  1. On the Generate Certificate page, fill in the issuer name, expiration date and the Issuer statement according to your company needs, then click Generate Certificate Now.
  2. A new certificate will be generated and the Certificate page pops up. Verify the certificate configuration and click Close.
  3. Click OK to close the window.
  4. On the HTTPS Outbound Inspection window, click HTTPS Inspection Trusted Root CA Certificate Options button. The Certificate Deployment Options window appears as shown in Figure 9.


Figure 9 Choosing How to Deploy the Certificate

  1. If TMG belongs to a domain, the recommended deployment method is “Automatically through Active Directory.” When you choose this option, the TMG CA certificate will be automatically deployed to the client computers using Group Policy. Click the Domain Administrator Credentials button and type the credentials that will be used for this operation. Click OK. Note that you should not include the domain in the user name field.
  2. Because Forefront TMG uses the certutil tool to publish the TMG CA certificate to Active Directory, you may notice a brief command prompt window. Click OK on the “Automatic certificate deployment succeeded” message box and OK on the Certificate Deployment Options window.
  3. Click OK to finish.

Important: Besides satisfying your organization’s security policies, you also need to evaluate any legal and compliance regulations before you enable HTTPS inspection. Any sites where HTTPS inspection is deemed inappropriate can be added to the exception list.

Enhanced User Experience

HTTPS inspection and URL filtering in Forefront TMG not only help provide a protected surfing environment for end users, they also include features that enhance the end-user experience through informational messages and by providing custom or precise error messages the end user can directly relate to.  

HTTPS Client Notification

To remain in compliance with corporate privacy policies, you can turn on client-side notification, which alerts the end user when an SSL site is being inspected and gives him the option to leave the site in order to protect information he deems personal or classified. Figure 10 illustrates this behavior.


Figure 10 HTTPS Inspection Client-Side Notification

For the user to receive HTTPS inspection notification, the client computer must have Forefront TMG Client installed, and it must have the HTTPS inspection trusted root certification authority installed in the local computer Trusted Root Certification Authorities certificate store. Remember that if you have upstream and downstream TMG configuration, HTTPS notifications need to be enabled on the downstream proxy rather than the upstream proxy.

To implement HTTPS inspection client-side notification, you must enable it on both the Forefront TMG server and client. To enable HTTPS inspection notifications on the server:

  1. In the Forefront TMG Management console, click Web Access Policy.
  2. Under Tasks in the right pane, click Configure HTTPS Inspection.
  3. On the Client Notification tab, click Notify users that HTTPS content is being inspected, and then click OK.

To enable notifications on Forefront TMG Client:

  1. Right-click on the Forefront TMG Client icon in the system tray and click Configure.
  2. On the Secure Connection Inspection tab, select Notify me when content sent to secure Web sites is inspected and click OK.

URL Filtering Error Messages

An administrator can allow or deny Web access to end users based on URL categories. If a user tries to go to a site that has been denied, she will get an error page like the one in Figure 11, showing that access to the site was denied because it had been categorized as such by the Forefront TMG administrator.


Figure 11 Access Denied Web Page

The error message can be customized by the TMG administrator. To do so:

  1. Open Forefront TMG management console and click Web Access Policy.
  2. Double-click to open the rule created by the administrator to allow or deny access.
  3. Click on Action tab as shown in Figure 12.


Figure 12 Customizing a URL Filtering Error Message

You can add any custom message to the Display denial notification to user text box. You can even use HTML content as long as the HTML is valid within the context of an HTML <body> element, but you can’t include any scripts.

You can also choose to show the category under which access to the site was denied. To do so, click the checkbox for Add denied request category to notification. This is particularly useful if a site is categorized incorrectly so the end user can let you know. You, in turn, can send this feedback to Microsoft through telemetry and at the same time set up an override category to rectify the categorization till it is fixed.

If you’d like to change the look and feel of the entire URL filtering error message page, you’ll need to edit the 12232.htm HTML page. You’ll find information about this on the Deny Page Customization on Forefront TMG 2010 page of the Forefront TMG (ISA Server) Product Team Blog.

Jim Harrison joined the ISA Server Sustained Engineering team as a QFE tester in January 2003. He is now a program manager with Forefront Edge CS and an avid ISA server supporter and systems implementer, as well as the coauthor of the Forefront Community Page called “Tales from the Edge.”

**Yuri Diogenes  ****works for Microsoft as a senior security support escalation engineer on the ISA Server/IAG Team. Diogenes is a coauthor of the Forefront Community Page called “Tales from the Edge.”

**Mohit Saxena  ****is the technical lead for the Microsoft ISA Server Support Team, a group of Support and Escalation engineers that works with customers on break fix issues, bugs and design change requests.