Monitoring StreamInsight Performance Counters and Events

  • Article

You can now use your existing diagnostic infrastructure to monitor and troubleshoot StreamInsight processes and queries with Performance Monitor and Event Viewer.

In This Topic

  • Performance Counters

    • Configuring performance counters

    • List of performance counters

    • How can performance counters help me?

  • Administrative Events

    • Configuring administrative logging

    • List of administrative events

Performance Counters

StreamInsight installs and configures performance counters. You can monitor these counters with Performance Monitor or another monitoring tool.

Performance counters are available for the following StreamInsight objects:

  • Servers

  • Queries

  • Input Adapters

  • Resiliency

Configuring performance counters

The performance counters are installed and configured when you install StreamInsight. This setup option is not configurable.

Counters in the Servers category are on by default and cannot be turned off. Counters in the Queries and Input Adapters categories are off by default. When you turn on counters for a query, then counters for the associated input adapters are turned on also. When you restart an instance of StreamInsight, these default behaviors are restored.

How to turn on counters for a single query

You turn on counters for a specific query and its associated input adapters by enabling a new Aspect in a call to Server.SetDiagnosticSettings.

For example, you can use the following code to turn on the performance counters for the query with the Uri, cep:/Server/Application/MyApp/Query/MyQuery.

DiagnosticSettings settings = server.GetDiagnosticSettings(new Uri(“cep:/Server/Application/MyApp/Query/MyQuery”);
Settings.Aspects |= DiagnosticAspect.PerformanceCounters;
server.SetDiagnosticSettings(new Uri(“cep:/Server/Application/MyApp/Query/MyQuery”), settings);

You can use the following code to turn off the performance counters for the same query.

DiagnosticSettings settings = server.GetDiagnosticSettings(new Uri(“cep:/Server/Application/MyApp/Query/MyQuery”);
Settings.Aspects & ~DiagnosticAspect.PerformanceCounters;
server.SetDiagnosticSettings(new Uri(“cep:/Server/Application/MyApp/Query/MyQuery”), settings);

How to turn on counters for all queries

You can also use a foreach loop to turn on the performance counters for all the queries and input adapters in an application.

foreach (var q in application.Queries)
   {
       DiagnosticSettings settings = server.GetDiagnosticSettings(q.Value.Name);
       settings.Aspects |= DiagnosticAspect.PerformanceCounters;
       server.SetDiagnosticSettings(q.Value.Name, settings);
   }

For more information about the classes and methods used in the preceding code samples, see the following topics:

[TOP]

List of performance counters

Naming conventions

Strings such as application names that are longer than 16 characters are shortened according to the following formula: first 10 characters + 2 periods + last 4 characters.

In the specific naming conventions described below, uniqueid refers to the ID of the object at run time.

Server counters

Naming convention for server instances: %PROCESSNAME%[uniqueid], where %PROCESSNAME% is the name of the hosting process.

Name

Description

Events in input queues

Number of events in input queues of all queries.

Events in output queues

Number of events in output queues of all queries.

Memory

Total memory (in bytes) used for all streams, events, and operator indexes.

Running queries

Number of queries currently running.

Query counters

Naming convention for query instances: %APPLICATION NAME%,%QUERY NAME%[uniqueid],%SERVER_INSTANCE_NAME%, where %SERVER_INSTANCE_NAME% is the instance name of the server to which the query belongs.

Name

Description

Average produced event latency

The average latency (in milliseconds) of events produced by the query each second.

CTIs produced

Number of CTIs produced.

Events in output queue

Number of events in output queue.

Events produced

Number of events produced.

Memory

Total memory (in bytes) used, including events, indexes, and streams.

Produced events/sec

Number of events produced per second.

Input Adapter counters

Naming convention for adapter instances: %APPLICATION NAME%,%QUERY NAME%,%ADAPTER NAME%[uniqueid],%SERVER_INSTANCE_NAME%.

Name

Description

Adjusted events

Number of events whose temporal metadata was adjusted according to the AdvanceTimeSettings policy.

CTIs input

Number of CTIs sent to input queue.

Dropped events

Number of events dropped according to the AdvanceTimeSettings policy.

Events in input queue

Number of events in input queue (including CTIs).

Incoming events/sec

Number of events received by the adapter per second.

Resumes/sec

Number of state transitions into the running state per second.

Suspensions/sec

Number of state transitions into the suspended state per second.

Total events enqueued

Number of events enqueued.

[TOP]

How can performance counters help me?

Here are some examples of the scenarios in which performance counters can help you to understand, monitor, and troubleshoot your StreamInsight applications:

  • Server scenarios

    • Memory footprint. How much memory is the embedded instance of StreamInsight, separately from the hosting application?

    • Capacity planning. How many queries can I run on this server before performance degrades?

    • Post-mortem analysis. How many queries were running when the server crashed?

  • Query scenarios

    • Throughput monitoring. Where is the bottleneck?

    • Latency monitoring.

[TOP]

Administrative Events

StreamInsight logs events to the Windows Application event log. You can review this log with Event Viewer or another monitoring tool. The logged events help you to monitor state changes and troubleshoot anomalies that may occur in your StreamInsight applications.

Events are logged by the following StreamInsight objects:

  • Server

  • Query

Configuring administrative logging

Administrative logging is on by default. You can turn administrative logging off and on only at the level of the StreamInsight instance (cep:/Server/).

How to turn off administrative logging

DiagnosticSettings settings = server.GetDiagnosticSettings(new Uri("cep:/Server/"));
settings.Aspects &= ~DiagnosticAspect.AdminLog;
server.SetDiagnosticSettings(new Uri("cep:/Server/"), settings);

How to turn on administrative logging

DiagnosticSettings settings = server.GetDiagnosticSettings(new Uri("cep:/Server/"));
settings.Aspects |= DiagnosticAspect.AdminLog;
server.SetDiagnosticSettings(new Uri("cep:/Server/"), settings);

[TOP]

List of administrative events

The following table lists the events that you may see in the event log.

The Source for all StreamInsight administrative events is StreamInsight.

Query events

Symbol

Event ID

Level

Message

Parameters

QueryInitializing

11001

Information

Query {0} is initializing.

Query Uri

QueryRunning

11002

Information

Query {0} is running.

Query Uri

QueryCheckpointing

11003

Information

Query [0} is checkpointing.

Query Uri

QueryStopping

11004

Information

Query {0} is stopping.

Query Uri

QuerySuspended

11005

Information

Query {0} is suspended.

Query Uri

QueryCompleted

11006

Information

Query {0} is completed.

Query Uri

QueryAborted

11007

Warning

Query {0} is aborted.

Query Uri

QueryStopped

11008

Information

Query {0} is stopped.

Query Uri

QueryError

11500

Warning

Query {0} encountered an exception. Exception details: {1}

Query Uri, exception message.

QueryRecoveryError

11501

Error

Query {0} encountered an exception during recovery. Exception details: {1}

Query Uri, exception message.

QueryCheckpointError

11502

Error

Query {0} encountered an exception during checkpointing. Exception details: {1}

Query Uri, exception message.

Server events

Symbol

Event ID

Level

Message

Parameters

ServerCreated

1000

Information

Server created.

ServerDisposed

1001

Information

Server disposed.

[TOP]