Diagnostics and performance monitoring for Reliable Service Remoting
The Reliable ServiceRemoting runtime emits performance counters. These provide insights into how the ServiceRemoting is operating and help with troubleshooting and performance monitoring.
Performance counters
The Reliable ServiceRemoting runtime defines the following performance counter categories:
| Category | Description |
|---|---|
| Service Fabric Service | Counters specific to Azure Service Fabric Service Remoting , for example, average time taken to process request |
| Service Fabric Service Method | Counters specific to methods implemented by Service Fabric Remoting Service, for example, how often a service method is invoked |
Each of the preceding categories has one or more counters.
The Windows Performance Monitor application that is available by default in the Windows operating system can be used to collect and view performance counter data. Azure Diagnostics is another option for collecting performance counter data and uploading it to Azure tables.
Performance counter instance names
A cluster that has a large number of ServiceRemoting services or partitions have a large number of performance counter instances. The performance counter instance names can help in identifying the specific partition and Service method (if applicable) that the performance counter instance is associated with.
Service Fabric Service category
For the category Service Fabric Service, the counter instance names are in the following format:
ServiceFabricPartitionID_ServiceReplicaOrInstanceId_ServiceRuntimeInternalID
ServiceFabricPartitionID is the string representation of the Service Fabric partition ID that the performance counter instance is associated with. The partition ID is a GUID, and its string representation is generated through the Guid.ToString method with format specifier "D".
ServiceReplicaOrInstanceId is the string representation of the Service Fabric Replica/Instance ID that the performance counter instance is associated with.
ServiceRuntimeInternalID is the string representation of a 64-bit integer that is generated by the Fabric Service runtime for its internal use. This is included in the performance counter instance name to ensure its uniqueness and avoid conflict with other performance counter instance names. Users should not try to interpret this portion of the performance counter instance name.
The following is an example of a counter instance name for a counter that belongs to the Service Fabric Service category:
2740af29-78aa-44bc-a20b-7e60fb783264_635650083799324046_5008379932
In the preceding example, 2740af29-78aa-44bc-a20b-7e60fb783264 is the string representation of the Service Fabric partition ID, 635650083799324046 is string representation of Replica/InstanceId and 5008379932 is the 64-bit ID that is generated for the runtime's internal use.
Service Fabric Service Method category
For the category Service Fabric Service Method, the counter instance names are in the following format:
MethodName_ServiceRuntimeMethodId_ServiceFabricPartitionID_ServiceReplicaOrInstanceId_ServiceRuntimeInternalID
MethodName is the name of the service method that the performance counter instance is associated with. The format of the method name is determined based on some logic in the Fabric Service runtime that balances the readability of the name with constraints on the maximum length of the performance counter instance names on Windows.
ServiceRuntimeMethodId is the string representation of a 32-bit integer that is generated by the Fabric Service runtime for its internal use. This is included in the performance counter instance name to ensure its uniqueness and avoid conflict with other performance counter instance names. Users should not try to interpret this portion of the performance counter instance name.
ServiceFabricPartitionID is the string representation of the Service Fabric partition ID that the performance counter instance is associated with. The partition ID is a GUID, and its string representation is generated through the Guid.ToString method with format specifier "D".
ServiceReplicaOrInstanceId is the string representation of the Service Fabric Replica/Instance ID that the performance counter instance is associated with.
ServiceRuntimeInternalID is the string representation of a 64-bit integer that is generated by the Fabric Service runtime for its internal use. This is included in the performance counter instance name to ensure its uniqueness and avoid conflict with other performance counter instance names. Users should not try to interpret this portion of the performance counter instance name.
The following is an example of a counter instance name for a counter that belongs to the Service Fabric Service Method category:
ivoicemailboxservice.leavemessageasync_2_89383d32-e57e-4a9b-a6ad-57c6792aa521_635650083804480486_5008380
In the preceding example, ivoicemailboxservice.leavemessageasync is the method name, 2 is the 32-bit ID generated for the runtime's internal use, 89383d32-e57e-4a9b-a6ad-57c6792aa521 is the string representation of the Service Fabric partition ID,635650083804480486 is the string representation of the Service Fabric Replica/Instance ID and 5008380 is the 64-bit ID generated for the runtime's internal use.
List of Performance counters
Service method performance counters
The Reliable Service runtime publishes the following performance counters related to the execution of service methods.
| Category name | Counter name | Description |
|---|---|---|
| Service Fabric Service Method | Invocations/Sec | Number of times that the service method is invoked per second |
| Service Fabric Service Method | Average milliseconds per invocation | Time taken to execute the service method in milliseconds |
| Service Fabric Service Method | Exceptions thrown/Sec | Number of times that the service method threw an exception per second |
Service request processing performance counters
When a client invokes a method via a service proxy object, it results in a request message being sent over the network to the remoting service. The service processes the request message and sends a response back to the client. The Reliable ServiceRemoting runtime publishes the following performance counters related to service request processing.
| Category name | Counter name | Description |
|---|---|---|
| Service Fabric Service | # of outstanding requests | Number of requests being processed in the service |
| Service Fabric Service | Average milliseconds per request | Time taken (in milliseconds) by the service to process a request |
| Service Fabric Service | Average milliseconds for request deserialization | Time taken (in milliseconds) to deserialize service request message when it is received at the service |
| Service Fabric Service | Average milliseconds for response serialization | Time taken (in milliseconds) to serialize the service response message at the service before the response is sent to the client |