Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Monitoring the state of a StreamInsight server involves tracking the overall health of the system and query performance. The state of a StreamInsight server is captured by monitoring the StreamInsight queries running on the server and by monitoring how the entities that compose a StreamInsight query are using system resources.
Creating Diagnostic Views
You can obtain monitoring information by using the diagnostic views API. As a prerequisite to using this API, you must have the management Web service enabled on both the embedded and stand-alone modes of server deployment and your client application must be connected to the server through this Web service. For more information, see Publishing and Connecting to the StreamInsight Server.
Alternatively, you can monitor the server and query by using the diagnostics features in the StreamInsight Event Flow Debugger, which uses the diagnostics API to return the results in a GUI interface. You should connect the debugger to a live StreamInsight server and use the Object Explorer to view the various objects in the server. Right-click any object to obtain the runtime diagnostics of that entity. For more information, see Using the StreamInsight Event Flow Debugger.
All objects in the server are accessed by using Uniform Resource Identifiers (URI) based on a hierarchical naming schema. This naming schema starts with the server and continues down to query operators and event streams. You can retrieve server level diagnostics from the following objects:
cep:/Server
cep:/Server/PlanManager
cep:/Server/EventManager
cep:/Server/Query
To refer to a specific query, use the naming scheme cep:/Server/Application/ApplicationName/Query/QueryName.
To refer to a specific entity, use the naming scheme cep:/Server/Application/ApplicationName/Entity/EntityName
To refer to specific operators and streams that belong to a query, use the following naming schemes. Note that adapters are also considered operators in a query - use the operator nomenclature for adapters.
cep:/Server/Application/ApplicationName/Query/QueryName/Operator/OperatorName
cep:/Server/Application/ApplicationName/Query/QueryName/Stream/StreamName
To refer to a specific query or subscription that belongs to an entity (process or subject), use the following naming schemes.
cep:/Server/Application/ApplicationName/Entity/EntityName/Query/QueryName
cep:/Server/Application/ApplicationName/Entity/EntityName/Subscription/SubscriptionName
To refer to a specific operator that belongs to a query within an entity (process or subject), use the following naming scheme.
- cep:/Server/Application/ApplicationName/Entity/EntityName/Query/QueryName/Operator/OperatorName
For example, to reference a query named “TrafficSensorQuery” in the application named “ObjectModelSample”, use cep:/Server/Application/ObjectModelSample/Query/TrafficSensorQuery.
Or, as another example, to reference a query named “NewProgrammingModelQuery” in the entity (process) “NewProgrammingModelProcess” that belongs to application “App”, usecep:/Server/Application/App/Entity/NewProgrammingModelProcess/Query/NewProgrammingModelQuery.
The diagnostic information for these objects is obtained by calling the GetDiagnosticView() method. You can filter the retrieval of diagnostic settings along two dimensions - the aspect (memory, CPU, and other attributes specific to that setting), and level (of criticality). You can set these filtering conditions using the SetDiagnosticsSettings() and ClearDiagnosticSettings() methods to set or clear specific settings for a specified query. The following example is taken from the sample 'ExplicitServer.cs' and outputs the diagnostics across various objects for a running query. The example assumes a registered StreamInsight instance named "MyInstance".
The following example is based on the sample “TrafficJoinQuery”. The following code could be used to output the diagnostics across various objects for a running query. The example assumes a registered StreamInsight instance named “Default”.
namespace StreamInsight.Samples.TrafficJoinQuery
{
//list of usings
internal class ExplicitBinding
{
internal static void Main()
{
using (Server server = Server.Create("Default"))
{
try
{
// Create application in the server.
// Create query logic as a query template
// Register adapter factories
// bind query to event producers and consumers
// Create bound query that can be run
// Start the query
query.Start();
// Wait for the query to be suspended
// Retrieve some diagnostic information from the CEP server
// about the query.
Console.WriteLine(string.Empty);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/EventManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/PlanManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/TrafficJoinSample/Query/TrafficSensorQuery")), Console.Out);
DiagnosticSettings settings = new DiagnosticSettings(DiagnosticAspect.GenerateErrorReports, DiagnosticLevel.Always);
server.SetDiagnosticSettings(new Uri("cep:/Server"), settings);
Console.WriteLine("Global Server Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/EventManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/PlanManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Query")), Console.Out);
Console.WriteLine("Summary Query Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/TrafficJoinSample/Query/TrafficSensorQuery")), Console.Out);
Console.WriteLine("Operator Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/TrafficJoinSample/Query/TrafficSensorQuery/Operator/sensorInput")), Console.Out);
query.Stop();
}
catch (Exception e)
{
Console.WriteLine(e);
Console.ReadLine();
}
}
Console.WriteLine("\npress enter to exit application");
Console.ReadLine();
}
/// <summary>
/// Takes a diagnostic view and dumps all its entries to the given
/// text writer.
/// </summary>
/// <param name="diagview">Diagnostic view to dump.</param>
/// <param name="traceListener">Tracer to receive the diagnostic data.</param>
private static void RetrieveDiagnostics(DiagnosticView diagview, System.IO.TextWriter traceListener)
{
// Display diagnostics for diagnostic view object
traceListener.WriteLine("Diagnostic View for '" + diagview.ObjectName + "':");
foreach (KeyValuePair<string, object> diagprop in diagview)
{
traceListener.WriteLine(" " + diagprop.Key + ": " + diagprop.Value);
}
}
}
}
The following example is based on the sample “ComposingQueries”. The following code could be used to output the diagnostics across various objects for a running query. The example assumes a registered StreamInsight instance named “Default”.
namespace StreamInsight.Samples.ComposingQueries
{
//list of usings
internal class Program
{
internal static void Main()
{
using (Server server = Server.Create("Default"))
{
try
{
Application myApp = server.CreateApplication("DeviceReadings");
// Configuration of the data generator input adapter.
// Define a source stream based on an adapter implementation, which requests point events from the simulator
// Configuration of the tracer output adapter 1.
// Configuration of the tracer output adapter 2.
// Annotate the original values with the delta events by joining them.
// Hydra is used to bind multiple queries
using (annotatedValues.Bind(outputStream1).With(maxDelta.Bind(outputStream2)).Run("process"))
{
Thread.Sleep(1000);
Console.WriteLine(string.Empty);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/EventManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/PlanManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/DeviceReadings/Entity/process/Query/StreamableBinding_1")), Console.Out);
DiagnosticSettings settings = new DiagnosticSettings(DiagnosticAspect.GenerateErrorReports, DiagnosticLevel.Always);
server.SetDiagnosticSettings(new Uri("cep:/Server"), settings);
Console.WriteLine("Global Server Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/EventManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/PlanManager")), Console.Out);
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Query")), Console.Out);
Console.WriteLine("Summary Process Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/DeviceReadings/Entity/process")), Console.Out);
Console.WriteLine("Summary Subscription Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/DeviceReadings/Entity/process/Subscription/StreamableBinding_1")), Console.Out);
Console.WriteLine("Summary Query Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/DeviceReadings/Entity/process/Query/StreamableBinding_1")), Console.Out);
Console.WriteLine("Operator Diagnostics");
RetrieveDiagnostics(server.GetDiagnosticView(new Uri("cep:/Server/Application/DeviceReadings/Entity/process/Query/StreamableBinding_1/Operator/Aggregate.2")), Console.Out);
// Wait for keystroke to end.
Console.WriteLine("Press enter to stop.");
Console.ReadLine();
}
Console.WriteLine("Process stopped. Press enter to exit application.");
Console.ReadLine();
}
catch (Exception e)
{
Console.WriteLine(e);
Console.ReadLine();
}
host.Close();
}
}
}
private static void RetrieveDiagnostics(DiagnosticView diagview, System.IO.TextWriter traceListener)
{
// Display diagnostics for diagnostic view object
traceListener.WriteLine("Diagnostic View for '" + diagview.ObjectName + "':");
foreach (KeyValuePair<string, object> diagprop in diagview)
{
traceListener.WriteLine(" " + diagprop.Key + ": " + diagprop.Value);
}
}
}
For more information about the API used in the preceding example, see T:Microsoft.ComplexEventProcessing.ManagementService.DiagnosticView. For more information about using trace listeners, see Trace Listeners.
What Problems Can I Troubleshoot with Diagnostic Views?
You can use diagnostic views to troubleshoot several kinds of issues with a StreamInsight application. For example:
Root Cause | Symptom(s) |
---|---|
User-defined extension is slow | High system latency |
User-defined extension has raised an exception | Hung or aborted query |
Input adapter is not feeding data quickly | High data latency |
Input adapter is not generating CTIs | Hung query |
Output adapter is not keeping up | High system latency |
However you typically cannot use diagnostic views to troubleshoot issues at the level of the server, or of the local network, which may be affecting your StreamInsight applications. Issues of this kind include a slow network, slow disk i/o, low memory, and resource contention on the server.
In This Topic
Query States
A query passes through several states during its lifetime. These states are described in the following table.
Query State | Description |
---|---|
Stopped | The query is no longer active. No query plan is retained. This state typically exists after a call to the query's Stop method. |
Initializing | This state typically exists after one of the following events: - The query was stopped and the user has called its Start method. The query will proceed to the Running state without user interaction. - A resilient query was running and the server is recovering from failure. To change the state of the query, the user must call its Start method. |
Running | The query is processing events. This state typically exists after a call to the query's Start method. |
Checkpointing | The query is running and a checkpoint is in progress. This state typically exists after a call to the BeginCheckpoint method. |
Stopping | The user has requested that the query stop. This state typically exists after a call to the query's Stop method. |
Suspended | A resilient query has failed recovery, or could not be recovered because the server was started without resiliency. |
Completed | The query has completed in a normal manner. That is, the input adapters enqueued end-of-stream markers, and the output adapter consumed them. |
Aborted | A failure occurred. The query is still available in the plan manager. |
In This Topic
Understanding Query-Level Monitoring
When you monitor query performance in the StreamInsight server, you can use the query-level monitoring attributes that the ManagementService API provides. Use the following event flow illustration to help you understand how you can use these monitoring attributes.
Based on the illustration, imagine that there are four measurement points that record the passage of events (from the left to the right) from the input adapter through the query to the output adapter. Based on these measurement points, the following metrics can be derived:
Incoming — Records incoming events across all input adapters. This is a measure of the raw input event arrival rate into the query from the input adapter.
Consumed — Records events consumed by the StreamInsight server, that is, across all the operators that immediately follow the input adapters. This is a measure of the events enqueued into the server.
Produced — Records all events that are leaving the last operator that immediately precedes the output adapter. This is a measure of the events dequeued from the query.
Outgoing — Records all events that leave the output adapter. This is a measure of the event departure rate from the StreamInsight server to the sink.
By using the illustration as a guide, you can determine the appropriate attribute to use based on the area of event flow in which you are interested. For example, if you are interested in the number of events consumed by the query, use the attribute QueryTotalConsumedEventCount; or, if you are interested in the events that were produced by the query, use the attribute QueryTotalProducedEventCount.
In This Topic
Monitoring for Query Latency
Latency is a cumulative number computed across all event arrivals and departures at a particular gate (incoming, produced, consumed, or outgoing). You can compute the average latency between measurements made at any two points in time as (S2 – S1) / (C2 – C1), where S is the cumulative sum of latency at any given gate and C is the count of events at that gating point.
For example, to compute the average consumed latency, measure the cumulative consumed latency (QueryTotalConsumedEventLatency) at time stamp t1 (S1) together with the total consumed event count (QueryTotalConsumedEventCount) at time stamp t1 (C1). Then, you repeat the same measurements at a different time stamp (t2), and then calculate the average consumed event latency as (S2 – S1)/ (C2 – C1).
You can use the attributes starting from QueryTotalIncomingEventCount through QueryLastProducedCtiTimestamp to determine the efficiency of the adapters in transferring events into or out of the query, and the rate at which the StreamInsight server can process the events.
You can determine the total memory consumed by operators by summing the values of the attributes OperatorEventMemory and OperatorIndexMemory.
In This Topic
Diagnostic Properties
Diagnostics views return attributes at several levels of object granularity: server, process, subject, subscription, query, published streams, operator, and adapter.
The diagnostics are designed in a manner that they can aggregate up from objects of finer granularity to objects of coarser granularity in the hierarchy. For each of these levels, you can obtain the following kinds of diagnostic information:
Static information (S) returns the property of the object. Static information does not change with the varying conditions of query operation.
Nonaggregated information (N) returns statistics that are not aggregated from child objects to their parent objects.
Aggregated information (A) returns aggregated statistics from child objects to their parent object.
Note that all the diagnostic view properties listed below are available in DiagnosticViewProperty.
Operator Diagnostics
Metadata
The following table lists the metadata properties that describe individual operators in a query. The values for these properties do not change.
Property Name | Type | Description |
---|---|---|
OperatorId | Int64 | Identifier for an operator. |
OperatorKind | String | Type of operator. |
OperatorQueryId | Int64 | Identifier for the query that the current operator resides. |
OperatorEventType | String | XML representation of the output type of the operator. For G&A, it is the type of the grouping field, not the output. |
Non-Aggregated Statistics
The following table lists the statistics that are aggregated across all the logical instances of an operator, but are not aggregated up into the query statistics.
Property Name | Type | Description |
---|---|---|
OperatorTotalInputEventCount | Int64 | Total number of input events for the operator. |
OperatorMinInputEventCountAcrossWorkers | Int64 | The minimum number of input events processed among all the workers for an operator. |
OperatorMaxInputEventCountAcrossWorkers | Int64 | The maximum number of input events processed among all the workers for an operator. |
OperatorTotalOutputEventCount | Int64 | Total number of output events for the operator. |
OperatorMinOutputEventCountAcrossWorkers | Int64 | The minimum number of output events generated among all the workers for an operator. |
OperatorMaxOutputEventCountAcrossWorkers | Int64 | The maximum number of output events generated among all the workers for an operator. |
OperatorLastOutputCtiTimestamp | DateTime | The timestamp of the last CTI (in application time) produced by the operator. |
OperatorTotalOutputCtiCount | Int64 | The total number of CTI events produced by the operator. |
OperatorMinOutputCtiCountAcrossWorkers | Int64 | The minimum number of CTI events produced among all the workers for an operator. |
OperatorMaxOutputCtiCountAcrossWorkers | Int64 | The maximum number of CTI events produced among all the workers for an operator. |
OperatorEventCountSinceLastCti | Int64 | The aggregated number of events produced by the operator since the last CTI for all workers of an operator. For the Cleanse operator, this value is typically 0 (zero). |
OperatorMinIndexEventCountAcrossWorkers | Int64 | The minimum number of events in the indexes among the workers of the operator. |
OperatorMaxIndexEventCountAcrossWorkers | Int64 | The maximum number of events in the indexes among the workers of the operator. |
OperatorMinEventMemoryAcrossWorkers | Int64 | The minimum amount of memory used (in bytes) by the events in the indexes among all workers of an operator. |
OperatorMaxEventMemoryAcrossWorkers | Int64 | The maximum amount of memory used (in bytes) by the events in the indexes among all workers of an operator. |
OperatorMinIndexMemoryAcrossWorkers | Int64 | The minimum amount of memory used (in bytes) by the indexes among all workers of an operator. |
OperatorMaxIndexMemoryAcrossWorkers | Int64 | The maximum amount of memory used (in bytes) by the indexes among all workers of an operator. |
OperatorNumberOfWorkers | Int32 | The number of computational units that are executing the operator. |
OperatorGroupIdField | String | The name of the group ID field for a Group and Apply operator. |
OperatorMinCpuUsageAcrossWorkers | Int64 | The minimum CPU usage, in milliseconds, among the workers of the operator. |
OperatorMaxCpuUsageAcrossWorkers | Int64 | The maximum CPU usage, in milliseconds, among the workers of the operator. |
OperatorMinEventAdjustedCount | Int64 | The minimum number of events adjusted among the workers of the operator. |
OperatorMaxEventAdjustedCount | Int64 | The maximum number of events adjusted among the workers of the operator. |
OperatorTotalEventAdjustedCount | Int64 | The total number of events adjusted among the workers of the operator. |
OperatorMinEventDroppedCount | Int64 | The minimum number of events dropped among the workers of the operator. |
OperatorMaxEventDroppedCount | Int64 | The maximum number of events dropped among the workers of the operator. |
OperatorTotalEventDroppedCount | Int64 | The total number of events dropped among the workers of the operator. |
Aggregated Statistics
The following table lists the statistics that are aggregated across all the logical instances of an operator and are aggregated up into the query statistics.
Property Name | Type | Description |
---|---|---|
OperatorIndexEventCount | Int64 | Memory used by the indexes across all active logical instances of the operator. |
OperatorEventMemory | Int64 | The amount of memory used (in bytes) by the events in the indexes across all logical instances of the operator. |
OperatorIndexMemory | Int64 | The amount of memory used (in bytes) by the indexes in the operator. |
OperatorTotalCpuUsage | Int64 | The total CPU usage of the operator in milliseconds. |
OperatorTotalScheduledCount | Int64 | The total number of times the operator was scheduled. |
In This Topic
Adapter Diagnostics
This section lists the diagnostic properties that are specific to adapters. Adapters are a special type of operator and therefore include all the diagnostic properties listed for operators.
Metadata
The following table lists the metadata properties that describe individual adapters. Note: the AdapterState property value can change.
Property Name | Type | Description |
---|---|---|
AdapterStateTransitionHistory | String | The XML representation of the last few adapter transitions for each adapter. |
Statistics
The following table lists the statistics that are specific to adapters.
Property Name | Type | Description |
---|---|---|
AdapterTotalSuspendCount | Int64 | The total number of times that all instances of the adapter have suspended. |
AdapterMinSuspendCountAcrossWorkers | Int64 | The minimum number of times that an instance of the adapter has been suspended. |
AdapterMinSuspendCountAcrossWorkers | Int64 | The maximum number of times that an instance of the adapter has been suspended. |
AdapterTotalTimeInSuspendedState | TimeSpan | The total time all instances of the adapter was in the suspended state. |
AdapterMinTimeInSuspendedStateAcrossWorkers | TimeSpan | The minimum time that an instance of the adapter was in the suspended state. |
AdapterMaxTimeInSuspendedStateAcrossWorkers | TimeSpan | The maximum time that an instance of the adapter was in the suspended state. |
AdapterTotalTimeInNonSuspendedState | TimeSpan | The total time all instances of the adapter was in a non-suspended state. |
AdapterMinTimeInNonSuspendedStateAcrossWorkers | TimeSpan | The minimum time that an instance of the adapter was in a non-suspended state. |
AdapterMaxTimeInNonSuspendedStateAcrossWorkers | TimeSpan | The maximum time that an instance of the adapter was in a non-suspended state. |
AdapterFirstCtiTimestamp | DateTime | The timestamp of the first CTI (in application time) produced or consumed by the adapter. |
AdapterNumberOfRunningWorkers | Int32 | Number of instances of the adapter in the Running state. |
AdapterNumberOfStoppedWorkers | Int32 | Number of instances of the adapter in the Stopped state. |
AdapterNumberOfSuspendedWorkers | Int32 | Number of instances of the adapter in the Suspended state. |
Adapter resiliency
Attribute | Information type |
.NET Framework type | Description |
---|---|---|---|
InputAdapterLastCheckpointHighWaterMark | N | DateTime | The high-water mark of application time used for the last checkpoint of this stream. This uniquely identifies a point in the input stream, and the stream should replay all events after this point during recovery. |
OutputAdapterLastCheckpointEventOffset | N | Int32 | The number of events since the application time high-water mark specifying the place in the output stream where the last checkpoint was taken. |
OutputAdapterLastCheckpointHighWaterMark | N | DateTime | The application time high-water mark of the output stream for the last checkpoint taken by the query. |
In This Topic
Stream Diagnostics
Metadata
The following table lists the metadata properties that describe individual streams in a query. The values for these properties do not change.
Property Name | Type | Description |
---|---|---|
StreamId | Int64 | The ID of the stream. |
StreamQueryId | Int64 | The ID of the query for the stream. |
StreamSourceOperatorId | Int64 | The ID of the source operator for the stream. |
StreamTargetOperatorId | Int64 | The ID of the target operator for the stream. |
StreamSystemInstance | Boolean | Boolean value indicating whether the stream is a system stream. |
Non-Aggregated Statistics
The following table lists the statistics that are aggregated across all the logical instances of a stream, but are not aggregated up into the query statistics.
Property Name | Type | Description |
---|---|---|
StreamTotalInputEventCount | Int64 | The total number of input events for the stream. |
StreamMinEventCountAcrossWorkers | Int64 | The minimum number of events in all instances of the stream. |
StreamMaxEventCountAcrossWorkers | Int64 | The maximum number of events in all instances of the stream. |
StreamNumberOfWorkers | Int32 | The number of instances of this stream. |
StreamMinInputEventCountAcrossWorkers | Int64 | The minimum number of input events across the instances of the stream. |
StreamMaxInputEventCountAcrossWorkers | Int64 | The maximum number of input events across the instances of the stream. |
StreamMinMemoryIncludingEventsAcrossWorkers | Int64 | The minimum amount of memory used (in bytes) across the instances of the stream. |
StreamMaxMemoryIncludingEventsAcrossWorkers | Int64 | The maximum amount of memory used (in bytes) across the instances of the stream. |
Aggregated Statistics
The following table lists the statistics that are aggregated across all the logical instances of a stream and are aggregated up into the query statistics.
Property Name | Type | Description |
---|---|---|
StreamEventCount | Int64 | The total number of events across all instances of the stream. |
StreamMemoryIncludingEvents | Int64 | The amount of memory (in bytes) used by the stream and all events in it. |
In This Topic
Query Diagnostics
Queries contain the aggregate statistics from their constituent operators and streams, along with their own statistics. The following sections detail the statistics specific to queries.
When a query is not running, the diagnostics return only the QueryState (either Suspended or Stopped) and the IsResilient (either true or false) properties.
Metadata
The following table lists the metadata properties that describe individual queries. The values for these properties do not change.
Property Name | Type | Description |
---|---|---|
QueryState | String | The current state of the query. |
QueryStartTime | DateTime | The start time of the query. |
QueryEndTime | DateTime | The end time of the query. |
QueryException | String | The last exception that occurred in the query. |
QueryCreationTime | DateTime | The time when the query instance was created. |
QueryId | Int64 | The ID of the query. |
Statistics
The following table lists the statistics specific to the query.
Property Name | Type | Description |
---|---|---|
QueryTotalIncomingEventCount | Int64 | The total number of incoming events for the query. |
QueryTotalConsumedEventCount | Int64 | The total number of events consumed by the query. |
QueryTotalProducedEventCount | Int64 | The total number of events produced by the query. |
QueryTotalOutgoingEventCount | Int64 | The total number of outgoing events for the query. |
QueryLastIncomingEventTimestamp | DateTime | The system time of the last incoming event for the query. |
QueryLastConsumedEventTimestamp | DateTime | The system time of the last consumed event for the query. |
QueryLastProducedEventTimestamp | DateTime | The system time of the last produced event for the query. |
QueryLastOutgoingEventTimestamp | DateTime | The system time of the last outgoing event for the query. |
QueryTotalConsumedEventLatency | Double | The total latency (in milliseconds) of all events consumed by the query. |
QueryTotalProducedEventLatency | Double | The total latency (in milliseconds) of all events produced by the query. |
QueryTotalOutgoingEventLatency | Double | The total latency (in milliseconds) of all outgoing events for the query. |
QueryLastProducedCtiTimestamp | DateTime | The timestamp (in application time) of the last CTI produced by the query. |
Query resiliency
Attribute | Information type |
.NET Framework type | Description |
---|---|---|---|
QueryLastCheckpointBeginTime | N | DateTime | The time when the last checkpoint of the query began. This will not be present if a checkpoint has never been taken, if the query is stopped, or if the query is aborted. |
QueryLastCheckpointEndTime | N | DateTime | The time when the last checkpoint of the query completed. This will not be present if a checkpoint has never been taken, if the query is stopped, or if the query is aborted. |
QueryLastCheckpointSize | N | Int64 | The size in bytes of the last checkpoint of the query. This will not be present if a checkpoint has never been taken. |
QueryIsResilient | N | Boolean | True if the query is configured for resiliency (regardless of the server state). False if not. |
In This Topic
Published Stream Diagnostics
The following table lists the metrics that are returned for a published stream. See Composing Queries at Runtime in the Developer's Guide to learn more about published streams. Note that each query also has an implicit published stream - the details of this published stream will be displayed as part of the query diagnostics.
Attribute | Information type | Data type | .NET Framework type | Description |
---|---|---|---|---|
PublishedStreamId | S | Number | Int64 | The ID of the published stream. |
PublishedStreamEventShape | S | String | String | The shape of the event processed by the published stream - one of Point, Interval or Edge. |
PublishedStreamEventType | S | String | String | A string containing the event type represented in XML. |
PublishedStreamProducerCount | S | Number | Int32 | Count of producers of events into the stream; value of 0 or 1. |
PublishedStreamConsumerCount | S | Number | Int32 | Count of consumers of events from the stream. |
PublishedStreamEventCount | S | Number | Int64 | Count of events in the published stream. |
PublishedStreamTotalEventCount | S | Number | Int64 | Total count of events across all published streams. |
In This Topic
Server Diagnostics
Server
The following table lists the server-wide (StreamInsight server) metrics.
Attribute | Data type | .NET Framework type | Description |
---|---|---|---|
ServerVersion | String | String | Assembly version string of the server. |
Event Manager
The following table lists the server-wide (StreamInsight server) metrics that can be returned for the event manager.
Attribute | Data type | .NET Framework type | Description |
---|---|---|---|
AllEventsCount | Number | Int64 | Count of events that are alive (allocated) in the StreamInsight server. |
AllEventsMemory | Number | Int64 | Amount of the memory used by all the events that are alive in the system. |
Query Manager
The following table lists the server-wide (StreamInsight server) metrics that can be returned for the query plan manager.
Attribute | Data type | .NET Framework type | Description |
---|---|---|---|
AllQueriesCount | Number | Int64 | Total running, aborted, or suspended queries across the StreamInsight server (running and finished). A finished query implies that the adapter has invoked the Stopped() method in response to a query shutdown request, or the query was aborted. |
AllQueriesStreamCount | Number | Int64 | Total operators in the queries. |
AllQueriesOperatorCount | Number | Int64 | Total streams in the queries. |
Queries (server-wide)
The server-wide (StreamInsight server) query diagnostic metrics are a union of the three tables showing the aggregate metrics for the query, the operator, and the stream. In addition, query-wide latency metrics are also provided by this diagnostic view. Two of the metrics are explained here to emphasize their significance.
Attribute | Data type | .NET Framework type | Description |
---|---|---|---|
StreamEventCount | Number | Int64 | Total number of events in all the logical streams across all queries. This is the cumulative count of events across all the streams. Streams from different queries can maintain references to these events, Therefore, if you total this count across queries, it might appear that the stream memory for the total event count exceeds the physical capacity of the computer that is running the StreamInsight server. |
OperatorIndexEventCount | Number | Int64 | Cumulative event count across all operators. This is the cumulative count of events across all the operators. Operators from different queries can maintain references to these events. Therefore, if you total this count across queries, it might appear that the memory for the total event count exceeds the physical capacity of the computer that is running the StreamInsight server. So it is important to interpret this number as being localized to a particular query and not across queries. |
Resiliency Manager
Attribute | Information type |
.NET Framework type | Description |
---|---|---|---|
ServerCheckpointMemory | A | Int64 | The amount of memory taken up for checkpointing |
In This Topic
Entity Properties
Source/Sink Entity
Property | Type | Description |
---|---|---|
EntityId | Int64 | The ID of the entity. |
EntityCreationTime | DateTime | The creation time of the entity. |
Process Entity
Property | Type | Description |
---|---|---|
EntityId | Int64 | The ID of the entity (process). |
EntityCreationTime | DateTime | The creation time of the entity (process). |
ActivationStartTime | DateTime | The date and time of activation creation. |
Subject Entity
Property | Type | Description |
---|---|---|
EntityId | Int64 | The ID of the entity (subject). |
EntityCreationTime | DateTime | The creation time of the entity (subject). |
ActivationStartTime | DateTime | The date and time of activation creation. |
SubjectInputCount | Int64 | The number of OnNext calls made to the subject. |
SubjectInputErrorCount | Int64 | The number of OnError calls made to the subject. |
SubjectInputCompletedCount | Int64 | The number of OnComplete calls made to the subject. |
SubjectSubscriberCount | Int64 | The number of subscribers to the subject. |
SubjectLastError | String | The last unexpected exception thrown by the subject. |
Subscription Properties
The following table lists the diagnostic properties for subscriptions.
Property | Type | Description |
---|---|---|
SubscriptionId | Int64 | The ID of the subscription. |
SubscriptionOutputCount | Int64 | The number of events that have been output by the subscription. |
SubscriptionLastError | String | The last exception that went through the subscription. |
SubscriptionErrorCount | Int64 | The number of times onError was called on the subscription. |
SubscriptionCompletedCount | Int64 | The number of times onComplete was called on the subscription. |
SubscripitonKind | String | The kind of the subscription. |
SubscriptionQueryUri | URI | The URI of the StreamInsight temporal query that represents this subscription, if one exists (URI). |
SubscriptionLastEventReceivedTime | DateTime | The DateTime when the last event was received by this subscription. |
SubscriptionDisposed | Boolean | Whether the subscription has been disposed. |
Accessing Diagnostics by Using PowerShell
You can use PowerShell to access manageability information or manage metadata for a running, hosted instance of StreamInsight. The following examples use Powershell 2.0. To query these, use the sample application ObjectModel.cs running on a StreamInsight host.
The following example loads the Microsoft.ComplexEventProcessing
DLLs from the Global Assembly Cache (GAC).
PS C:\> [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.ComplexEventProcessing")
Here is the result set.
GAC
Version
Location
------ ----------- -------------------------------------------------------------------------
True v2.0.50727 C:\Windows\assembly\GAC_MSIL\Microsoft.ComplexEventProcessing\10.0.0.0__89845dcd8080cc91\Micro...
The following example returns the methods and properties that can be accessed on the running instance of StreamInsight.
PS C:\> $server = [Microsoft.ComplexEventProcessing.Server]::Connect("https://localhost/StreamInsight")
PS C:\> $server | gm
TypeName: Microsoft.ComplexEventProcessing.Server
Here is the result set.
Name MemberType Definition
------------------------------ ---------- ----------------
ClearDiagnosticSettings Method System.Void ClearDiagnosticSettings(System.Uri name)
CreateApplication Method Microsoft.ComplexEventProcessing.Application CreateApplication(string name)
CreateManagementService Method Microsoft.ComplexEventProcessing.ManagementService.IManagementService CreateManag...
Dispose Method System.Void Dispose()
Equals Method bool Equals(System.Object obj)
GetDiagnosticSettings Method Microsoft.ComplexEventProcessing.DiagnosticSettings GetDiagnosticSettings(System....
GetDiagnosticView Method Microsoft.ComplexEventProcessing.DiagnosticView GetDiagnosticView(System.Uri name)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
SetDiagnosticSettings Method System.Void SetDiagnosticSettings(System.Uri name, Microsoft.ComplexEventProcessi...
ToString Method string ToString()
Applications Property System.Collections.Generic.IDictionary
2[[System.String, mscorlib, Version=2.0.0....`
IsEmbedded Property System.Boolean IsEmbedded {get;}
The following example returns the server-wide (StreamInsight server) metrics for the event manager.
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/EventManager")
PS C:\> $dv
Here is the result set.
Key Value
------------------------------ -----
AllEventsCount 19
AllEventsMemory 249856
The following example returns the server-wide (StreamInsight server) metrics for the plan manager.
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/PlanManager")
PS C:\> $dv
Here is the result set.
Key Value
----------------------- -----
AllQueriesCount 14
AllQueriesStreamCount 50
AllQueriesOperatorCount 38
The following example returns the query metrics for the query TrafficSensorQuery
.
PS C:\> $dv = $server.GetDiagnosticView("cep:/Server/Application/ObjectModelSample/Query/TrafficSensorQuery")
PS C:\> $dv
Here is the result set.
Key Value
------------------------------------------ --------------
QueryState Suspended
QueryStartTime 9/22/2009 5:34:02 PM
QueryEndTime 9/22/2009 5:34:03 PM
StreamEventCount 0
OperatorCount 0
QueryTotalIncomingEventCount 553
QueryTotalConsumedEventCount 553
QueryTotalProducedEventCount 192
QueryTotalOutgoingEventCount 192
QueryLastIncomingEventSystemTime 9/22/2009 5:34:02 PM
QueryLastConsumedEventSystemTime 9/22/2009 5:34:02 PM
QueryLastProducedEventSystemTime 9/22/2009 5:34:03 PM
QueryLastOutgoingEventSystemTime 9/22/2009 5:34:03 PM
QueryTotalConsumedEventsLatency 14527.833
QueryTotalProducedEventsLatency 62457.0953
QueryTotalOutgoingEventsLatency 63553.2049
QueryLastProducedCTITimestamp 12/31/9999 11:59:59 PM
StreamMemoryIncludingEvents 0
OperatorIndexEventCount 0
OperatorEventMemory 0
OperatorIndexMemory 65870
OperatorTotalScheduledCount 708
OperatorTotalCpuUsage 670
In This Topic