Understanding the Re-Injection Mechanism Improvement on Forefront TMG

Authors
Yuri Diogenes - Sr Security Support Escalation Engineer, Microsoft CSS Forefront Security Edge Team
Thomas Detzner - Escalation Engineer, Microsoft CSS Forefront Security Edge Team

Technical Reviewers
Doron Juster - Senior SDE, Microsoft Forefront Edge CS Team
Amit Finkelstein - Director SDET, Microsoft Forefront Edge Team
Masoud Hoghooghi - Escalation Engineer, Microsoft CSS Forefront Security Edge Team

Editor
James Kilner - Technical Writer, Microsoft Forefront Edge Team

Reviewing Kernel Mode Policy

The firewall engine (fweng.sys) and firewall service (wspsrv.exe) share an implementation of the rules engine (the component that decides if the current policy allows the traffic or not). When matching traffic to policy rules, some aspects cannot be matched by the firewall engine because performing blocked operations is not allowed in the kernel-mode context of the firewall engine. Specifically, matching user identity (authentication) and performing reverse name lookups (name resolution) are both blocking operations (APIs which involve I/O) and therefore can’t be done by the firewall engine.

The operating system network stack does not allow the driver to “delay” a packet at this stage (kernel). What the driver actually does is copy the packet to its own memory, tell the OS network stack to drop the packet, and then ask the firewall service to determine what to do next with the packet. If the firewall service decides to let this traffic through, it creates all of the necessary objects, and then tells the firewall engine to inject this packet into the OS network stack (at the firewall hook), as if it has just arrived from the lower network layers. This mechanism is called the re-inject mechanism and here are some core definitions about this mechanism:

• Re-injection is done only once for a session, when getting the TCP SYN, or first UDP (or ICMP) packet in a given session. For UDP and ICMP, a session means one minute of activity after the first packet.

• Re-injection is also done if the protocol has an application filter, regardless of whether name resolution or authentications are needed.

As can be seen from this description, connection elements are created because there is an allowing policy rule (checked by either the driver or the service), or when there is a creation element. The former represents static rules configured by the administrator. The latter is a dynamic mechanism through which firewall service components can allow traffic that they anticipate. For example, the firewall service instructs the driver to create one creation element per published server. When a client attempts to connect to the published server, the creation element allows the creation of a new connection element for this connection.

Note

For more information on ISA Server architecture review the ISA Server 2006 Firewall Core document.

ISA Server Limitation

There are some scenarios where ISA Server could stop answering requests due to failures on those key points mentioned earlier: authentication and name resolution. Here are some common scenarios where this issue can appear:

• Authentication delay: If ISA Server has access rules that require authentication in order to allow the access, ISA must validate a user’s credentials against the domain controller. If the domain controller does not respond in timely manner, ISA might start to accumulate requests. Eventually, ISA may reset the secure channel with the domain controller causing the end user to receive a prompt for credentials (even though the user is already authenticated). For more information, see Improving Web Proxy Client Authentication Performance on ISA Server 2006.

• Name resolution problems: Another scenario where ISA Server can stop answering requests is when name resolution is not working properly. There are two key scenarios where this can happen. For more information, read the following posts on the ISA/TMG Team blog:

  • Side Effects of Incorrect DNS configuration on ISA Server: 10060 Connection Timeout Scenario

  • ISA Server 2006 stops answering requests

By default ISA Server has five re-injections threads and if all of the threads are in use, ISA can’t answer any other request. Here is an example where this can happen:

1. All five re-injection worker threads call the LookupAccountName() function.

2. Internally, the OS implements this as an RPC call from the ISA firewall service to the local netlogon service.

3. The netlogon service needs to call the domain controller (via RPC).

4. That RPC (from local netlogon) starts a new TCP session over port 135 (RPC port mapper).

5. The ISA driver (Fweng) needs to re-inject this to user mode (firewall service).

6. Since all five re-injection threads are already busy with the LookupAccountName() API call, ISA can’t process any other request.

When this occurs, the only way to fix it is to wait until the re-injection threads are freed (you don’t know when this will occur, because ISA could be waiting for an answer from the domain controller or from DNS) or force a cleanup of all of the threads by restarting firewall service.

There are a few ways to identify if your environment is encountering this re-injection limitation by monitoring the following counters/thresholds:

Object	Counter	Description
ISA Server/Forefront TMG Firewall Packet Engine	Backlogged Packets	A value higher than 10 is considered bad.
ISA Server/Forefront TMG Firewall Service	Worker Threads	A value higher than 100 is considered bad.
ISA Server/Forefront TMG Firewall Packet Engine	ReInject Available IRPs *	For ISA Server a value of 5 is the default result, it reflects the available reinjection threads. For Forefront TMG 2010 a value of 50 is the default result.

* To enable this counter you need to add the following registry entry:

Location: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\RAT\Stingray\Debug\FWSRV

Key: FWS_PRIVATE_PERFORMANCE_COUNTERS

Type: DWORD

Value: 1

Note

If you do not have the path \RAT\Stingray\Debug\FWSRV you need to create it.

This registry change does not require a restart of the system.

The following figure shows an example where ISA Server stopped answering because the domain controller wasn’t answering authentication requests:

Note

For more counters and thresholds, see Best Practices for Performance in ISA Server 2006.

Forefront TMG 2010 Improvement

On Forefront TMG 2010, the number of default rejection threads changed to fifty (50). This change means that even when authentication delays or name resolution delays are present, Forefront TMG still handles other requests since there are more re-injection threads available. Using the counter ReInject Available IRPs you can see this change using Performance Monitor as shown below:

Another change that was made in Forefront TMG to solve this issue was that by default it reserves 10% of the re-injection threads for localhost traffic. For the scenario given in the perfmon snapshot above, Forefront TMG is still able to handle the traffic since it has the reserved re-injection threads available to handle the traffic.

It is very important to emphasize that although TMG enhanced the re-injection mechanism, authentication and name resolution issues can still impact overall performance. Therefore, you should resolve the root cause for those problems before performance can be improved further. You should also correctly size Forefront TMG 2010 for the environment where it will be implemented. The best way to do that is by knowing your traffic profile, the applications that will be published through TMG, and the overall expected number of users that will be using those resources. Once you have this information, you can use the Forefront TMG 2010 Capacity Planning Tool to verify which hardware is best for your TMG deployment.