Sampling and filtering events

Article
07/30/2014

The majority of scenarios for using the Semantic Logging Application Block involve collecting events raised by your own custom event source classes within your applications. However, there are some scenarios where you need to capture events generated by other event sources, such as those built into the operating system or in other applications you cannot modify. For example, you might want to capture events from the ASP.NET runtime if you are monitoring a web application.

The event listeners provided with the Semantic Logging Application Block, for both in-process and out-of-process use, can capture events from any standard event source. The capability to specify the event source by name in the EnableEvents method (in-process), and in the configuration file for the Out-of-Process Host, means that you can collect events from event sources even when you cannot obtain a reference to that event source. For an example of this, see Correlating events.

Typical Goals

You are logging events from an application and you also want to capture events from event sources that are not defined within the application, but are relevant to the application. For example, you want to capture the RequestStarted event raised by ASP.NET when a request is received by your application. However, you do not want to collect all of these events because the volume generated by these additional sources would overwhelm your logging store or would make it very difficult to analyse the logging information. To resolve this you need to collect only a sample of the events, rather than all of them, and also be able to filter the processes from which you collect events.

Notes

Event sampling is only possible when using version 4.5.1 or higher of the .NET Framework.

Event sampling is dependent on the event source accepting specific arguments when you enable listening to that event source. There are some common scenarios, but the you will need to determine the arguments for the event sources you want to use, and ensure that these arguments are valid and do not impact other processes. You use arguments with event sources at your own risk.

When sampling is configured, the event sources involved expose events from only specific requests. When sampling is specified for a request, all events related to the request are logged regardless of the thread from which they are raised. No events are logged from requests where sampling is not specified.

Not all event sources are required to participate in sampling. For example, you might choose to collect only a sample of the events raised by the Task Parallel Library (TPL) while capturing all the events raised by your application’s custom event sources. However, if you need to correlate events from both, you will be able to do this only for the requests that were sampled.

An added benefit of sampling is that it is likely to result in the exclusion of the TPL events originated by the Semantic Logging Application Block sinks, as long as no sampling start event was logged before the sinks were created. If you specify a sampling rate of 1 when you enable a listener for the TPL event source, you will receive the TPL events for all requests but not the TPL events from the application block sinks.

Solution

The following procedure shows how you can initiate event sampling and filtering for the ASP.NET local development server.

To sample and filter events for the ASP.NET local development server

Create a listener for the event source named Microsoft-Windows-ASPNET, and pass the argument ActivitySamplingStartEvent = RequestStarted:10 to this event source. The value for this argument is a space delimited list of event identifiers (either the name or the ID of each event), optionally followed by a sampling range. This example specifies the trigger event to start sampling every tenth event.
- If you are using the in-process approach, use the following code to create a listener.
```
var listener = new ObservableEventListener();
listener.EnableEvents("Microsoft-Windows-ASPNET ", 
         EventLevel.Informational, Keywords.None,
         new Dictionary<string, string> {
           { "ActivitySamplingStartEvent", "RequestStarted:10" }
         });
```
- If you are using the out-of-process approach, include the following extract in the configuration file for the Out-of-Process Host.
```
<eventSource name="Microsoft-Windows-ASPNET" level="Informational">
  <arguments>
    <add key="ActivitySamplingStartEvent" value="RequestStarted:10"> 
  </arguments>
  <filter>
    <process name="iisexpress.exe" />
  </filter>
</eventSource>
```

Start sampling by passing the argument ActivitySampling=true to your application’s custom event source. When you do this, all events raised from the source (for the sample interval) that have the same activity identifier are included in the sample.

If you are using the in-process approach, use the following code.

var listener = new ObservableEventListener();
listener.EnableEvents("MyCustomEventSource", 
         EventLevel.Informational, Keywords.None,
         new Dictionary<string, string> {
           { "ActivitySampling", "true" }
         });

If you are using the out-of-process approach, include the following extract in the configuration file for the Out-of-Process Host.

<eventSource name="MyCustomEventSource" level="Informational">
  <arguments>
    <add key="ActivitySampling" value="true"> 
  </arguments>
  <filter>
    <process name="iisexpress.exe" />
  </filter>
</eventSource>

Note

ASP.NET events are a good candidate for sampling because they are directly related to requests, but other events can be used as long as an activity ID is set when they are logged. Different events in an event source can start sampling at different rates.
If you are using asynchronous code and waiting on completion of tasks, and you want to log the transitional events, you must enable the TPL event source with the argument ActivitySampling=true, as described in the topic Correlating events.

The following log extract shows the values of the EventId, ActivityId, and RelatedActivityId generated from an application that uses a custom event source named MyEventSource and TPL Tasks to spawn new threads.

ProviderName	EventId	ActivityId	RelatedActivityId
Microsoft-Windows-ASPNET	2	415a6a4f-93ad-4f13-bd82-3abc1cf46940
System.Threading.Tasks.TplEventSource	12	415a6a4f-93ad-4f13-bd82-3abc1cf46940	00000003-0002-0000-ec2f-0000ffdcd7b5
MyCustomEventSource	1	415a6a4f-93ad-4f13-bd82-3abc1cf46940
System.Threading.Tasks.TplEventSource	10	415a6a4f-93ad-4f13-bd82-3abc1cf46940	00000005-0002-0000-ec2f-0000ffdcd7b5
System.Threading.Tasks.TplEventSource	12	415a6a4f-93ad-4f13-bd82-3abc1cf46940	00000006-0002-0000-ec2f-0000ffdcd7b5
MyCustomEventSource	2	00000005-0002-0000-ec2f-0000ffdcd7b5
System.Threading.Tasks.TplEventSource	7	00000005-0002-0000-ec2f-0000ffdcd7b5	00000006-0002-0000-ec2f-0000ffdcd7b5
...	...	...	...

Here, the TPL events indicate the transfer from activity ID 415a6a4f-93ad-4f13-bd82-3abc1cf46940 to activity ID 00000005-0002-0000-ec2f-0000ffdcd7b5. This event is logged between two events from MyCustomEventSource. The first is logged before awaiting on a Task, and the second is logged afterwards, so they have different activity IDs even though they belong to the same request.

Next Topic | Previous Topic | Home | Community

Sampling and filtering events

Typical Goals

Notes

Solution

Additional resources