Using High Performance Filtering

Article
08/05/2010

There are certain scenarios where the High Performance Filtering feature added in Netmon 3.4 will provide the best performance for capturing with a filter. The idea is to filter frames before they hit the disk which can improve your performance by reducing the impact on the capturing machine. High Performance filtering can be performed with both the UI and NMCap. However, if performance is a concern, NMCap provides the best capturing performance overall. The following provides some guidance on when and how to use this feature. You should also consider reading our Avoiding Dropped Frames blog as background.

This article is split into 3 general areas. The first few sections describe what High Performance mode is. The next sections following that describe when to use the feature. And finally the last few sections describe how to use the feature in more detail.

Capture Filter Overview

First let’s review the various capture filtering methods:

Capturing Mode	Description	Usage
Normal Buffered Capturing (Default Mode)	Frames are buffered to disk first and then evaluated afterwards. If your machine is unable to keep up with incoming traffic, your pending frame count will continue to grow. But as we buffer frames first, it’s much better at handling bursts of network traffic.	Any capture filter with at least one NON fully qualified data field or property.
High Performance Profile	This is an optimized parser set which provides better performance, but with less depth in terms of fields you can parse on. This profile is automatically used with High Performance Capture filters, but can also be used with normal buffered capturing.	You can enable this parser set using the /UseProfile with NMCap or selecting the profile in the UI. You should also turn off conversations for best performance. With NMCap this means adding the /DisableConversation switch.*
High Performance Capturing	Frames are evaluated before buffering as long as we detect we can keep up. This saves a write to the disk and uses a high performance filter to evaluate the frames.	Any capture filter where ALL fields are fully qualified and supported. See High Performance Capturing in the help file for a full list. Also reference the NM34 High Perf Filters in the standard filter list under the Load Filter drop down button.

*Note: That in general you can increase the performance of NMCap by disabling conversations and not enabling process tracking. Conversations are enabled by default when any filter is used, even in StopWhen/StartWhen filters. If you can verify conversations are not really needed then explicitly add the /DisableConversation switch. Conversations are not needed for TCP, IP or Ethernet data fields. Process tracking is disabled by default.

General Limitations of Capturing

There are 3 external factors which contribute to the ability for your machine to keep up and filter incoming network traffic.

Network – Obviously the greater the load/speed of your network the greater the chance that you have of dropping frames or filling up your driver with buffered frames. Also the type of traffic can affect your ability to keep up with incoming packets.
CPU - The speed and load on your CPU affect the ability to filter traffic. The faster your CPU, the more packets you can parse per second.
Disk – In the case where frames are being buffered and then written to disk, the load and speed of your disk can impact your ability to keep up with incoming packets.

What is High Performance Capturing

High Performance filtering is composed of two separate optimizations.

With specific fully qualified fields, like Frame.Ethernet.Ipv4.TCP.Port, packets can be filtered out before they are buffered to disk. This avoids a disk write can lower CPU/disk load. However if we detect we can't keep up, we revert to the buffering behavior. A type of "flow control" is used to control when we revert and when we can continue filtering before we buffer.
Second, we include a high performance parser profile which is automatically used when High Performance filters are used. While this parser set is limited in detail, it is able to parse more frames per second than the Default or Windows Profile.

When to Use High Performance Filtering

The range of system capturing performance, network throughput and network burst creates a huge matrix of possibilities. Rather than trying to provide guidance in terms of the types of machines and networks in which this feature will benefit, we instead provide a list of conditions to explain when and when not to use the various components of the feature.

For the most part, this feature is targeted at cases where you have a busy server with high traffic and you want to capture a narrow slice of the total network traffic. In cases where we can avoid buffering to disk, we lower the disk load. And in the case where we use the High Performance Profile we lower the load on the CPU.

When NOT to use High Performance Filtering

If you are capturing a majority of the traffic you get the best performance using NMCap with no filter. That includes StopWhen or StartWhen filters as these will cause the parsing engine to engage. By writing directly to disk we don’t have to evaluate frames and therefore we don’t have to read and write the capture after a filter evaluation.

When to Use High Performance Filtering and NMCap

When using High Performance Filtering you should meet all of the following conditions. The main difference in this scenario when compared to the next is that we cannot handle network bursts as reliably. When filtering before we buffer to disk, all buffering is done in memory and this is a limited resource when compared to your disk space. For more information on configuring this buffer please see the Avoiding Dropped Frames blog.

CPU Can Keep Up with Traffic - The ability to keep up with incoming frames depends on the pace at which frames are arriving and the ability for your CPU(s) to keep up with that traffic. The complexity of the filter also affects how well a CPU can keep up. See the “How To Optimize Your Filter” section below.
Filtering a Large Portion of the Traffic - The more you filter out, the less you have to write to disk. By skipping the disk buffer path, you can gain more performance. The greater number of frames you can filter out, the less work your system has to do.
Supported Filter – There is a subset of fields that are supported, mainly address and ports. So your filter has to be supported in high performance filtering. To see a complete list, please see the “High Performance Capturing” section in the help.
/Startwhen or /Stopwhen NOT Required - These triggering options are still done after the frames are buffered. If you set a capture filter which blocks the triggering conditions, they will never be hit. You can mitigate this problem by including the trigger filter in your capture filter. It makes the most sense to filter as narrowly as possible so that as few as possible extraneous frames are let through.

When to Use High Performance Profile Only and NMCap

You can also separately specify the High Performance Profile when running NMCap with the /UseProfile switch. Using this high performance filter set alone, you are able to get better capture performance than the default configuration.

CPU CANNOT Keep Up with Traffic Bursts - A loaded CPU(s) or network burst can affect the ability to keep up with traffic. In this case it might be better to buffer packets to make sure we don't drop any. By using the High Performance Profile explicitly you get maximum parser performance. See the “How to Optimize Your Filter” section below. And indication that your CPU can’t keep up is a constant increase in the pending frames counter.
Filter Supported by High Performance Filter Profile - In order to optimize the High Performance filter profile, we removed many higher level parsers. The filter will simply fail if it is not supported. However, because we restrict the fields in the standard High Performance Filtering, there is greater flexibility in what filters you can specify by using the profile alone. For instance, you can filter on TCP.Flags.Reset using the High Performance Profile but you cannot use this as a High Performance Filter.

When to Use High Performance Filtering in the UI

When you are using the UI and want to view traffic live, you can use a fully qualified filter to restrict the amount of traffic you are looking at. But you might still want to use the full parsing to view the data. For this scenario here are the conditions which would warrant the High Performance feature.

Not Concerned with Impacting Server -The UI has a larger footprint in terms of CPU and Memory Load. If you are concerned at all with impacting the capturing machine, you should use NMCap instead.
Not a Long Term Capture Session - Related to the previous condition, capturing in the UI for long periods of time might consume some many resources that the UI becomes unresponsive and you might not be able to save the resulting trace. As the UI collects frames forever, there is no way to release any resources related to the incoming trace information.

How to Optimize Your Filter (Weaker Filters)

As mentioned in the Avoiding Dropped Frames blog, the filter complexity affects the CPU load. And one possible solution is to use “weaker” filters. The type of filter you use affects your filtering speed. The more complex and deep a filter is, the longer it takes to parse a frame. For instance, parsing all the way to TCP.Port takes a lot longer parse than Ethernet.Address. But first let’s determine how to tell if you need to optimize your filter.

The first step is determining if your machine can keep up based on a specific filter. The only way to know for sure is to run NMCap using a high performance filter during the type of network traffic you expect to see. Then view the statistics that are printed out and view the Dropped and Pending frame counters.

If the Pending count continues to grow over time, then this is an indication your CPU cannot evaluate the filter quickly enough to keep up with the incoming traffic. It might also be an indication that the default settings for flow control are not ideal for your machine.
If the Drop count rises, then this is an indication that either your CPU can’t keep up or that the High Performance mode is unable to account for and detect the inability to keep up and automatically switch to back to buffering the packets to disk.

For either of these cases, it might be possible to optimize the buffers and flow control to avoid dropping frames or make sure you stay in high performance mode rather than reverting back to buffering. See the “Modifying the High Performance Buffer Options” below for more details.

Picking a Simpler Filter

In some cases you can provide a simpler filter to achieve what you need. For instance, if you are filtering on an IPv4 or IPv6 address, it might be possible to use an Ethernet address instead. Ethernet is simpler to parse and therefore faster.

Also keep in mind that it does take longer to evaluate multiple fields than just one. While you might not have an option to simplify in this manner, this might be a place you can consider simplifying. For instance, rather than looking for both the source and destination addresses you might be able to get away with looking for only the destination. Since the source is often your machine, and traffic to the capturing machine assumes your address, it would be better to use Frame.Ethernet.DestinationAddress rather than Frame.Ethernet.Address.

Using Blob to Filter a Pattern Offset Match

If there is a filter that is not supported using High Performance mode, or you need even higher performance, you can use a pattern offset filter. The caveat here is that you have to validate if your pattern and offset is accurate for the scenario you are capturing. For instance, if you are filtering on a TCP port with the ESP protocol in the stack, there might be situations where IP options move the location and block traffic that you'd normally capture with a normal filter. You must evaluate the possibility that you might drop frames you'd normally capture and understand the impact of missing these frames. If ANY of these following conditions are true, then you might consider this option.

Filtering Speed Still too Slow - A simple pattern offset can improve performance by an order or magnitude or two.
High Performance Filter not Supported - In some cases, the high performance filter doesn't include a protocol, for instance ESP. In that case, it might be possible to rely on a pattern offset filter to perform high performance filtering for these cases.

To create a filter using the Blob, you need to know the offset and length of the pattern you are matching. Often, the simplest way to do this is open a trace you’ve taken from the network you are interested in, and click on the field in question. Then look in the hex details for that location and offset.

For instance, if you wanted the IPv4 Destination Address you would see that on an Ethernet network that this is at offset 30 and the size is 4 octets. So the filter would be Blob(FrameData, 30, 4) == X. For X you would have to determine the 4 octet value that represents your IP address. Again if you already have a trace, looking at the hex details on an example filter is one easy way to find out what that value is. But if need be, you’ll have to translate the value to hex. If in this example the address was 1.2.3.4, the associated hex value is 0x01020304. So your blob filter would be:

Blob(FrameData, 30, 4) == 0x01020304

With most high performance filters you must preface them with a fully qualified path. But Blob is the one exception. If it is used as a capture filter in the UI or NMCap, and the rest of the filter string qualifies, then the filter will be attempted using the high performance mode.

High Performance Filtering to the Rescue

As networks continue to get faster, capturing that traffic becomes more and more challenging. With this new feature you have the ability to filter before you buffer to disk thus giving you the flexibility to control the load on your system. This can give you the ability to capture faster traffic with a lower tax on the capturing system.