Events
Power BI DataViz World Championships
Feb 14, 4 PM - Mar 31, 4 PM
With 4 chances to enter, you could win a conference package and make it to the LIVE Grand Finale in Las Vegas
Learn moreThis browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Note
This isn't the latest version of this article. For the current release, see the .NET 9 version of this article.
Warning
This version of ASP.NET Core is no longer supported. For more information, see the .NET and .NET Core Support Policy. For the current release, see the .NET 9 version of this article.
Important
This information relates to a pre-release product that may be substantially modified before it's commercially released. Microsoft makes no warranties, express or implied, with respect to the information provided here.
For the current release, see the .NET 9 version of this article.
This article provides guidance for gathering diagnostics from a gRPC app to help troubleshoot issues. Topics covered include:
DiaganosticSource
and Activity
. Traces from diagnostic source are commonly used to collect app telemetry by libraries such as Application Insights and OpenTelemetry.EventCounter
and can be observed using dotnet-counters command-line tool or with Application Insights.gRPC services and the gRPC client write logs using .NET Core logging. Logs are a good place to start when debugging unexpected behavior in service and client apps.
Warning
Server-side logs may contain sensitive information from your app. Never post raw logs from production apps to public forums like GitHub.
Since gRPC services are hosted on ASP.NET Core, it uses the ASP.NET Core logging system. In the default configuration, gRPC logs minimal information, but logging can be configured. See the documentation on ASP.NET Core logging for details on configuring ASP.NET Core logging.
gRPC adds logs under the Grpc
category. To enable detailed logs from gRPC, configure the Grpc
prefixes to the Debug
level in the appsettings.json
file by adding the following items to the LogLevel
subsection in Logging
:
{
"Logging": {
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information",
"Grpc": "Debug"
}
}
}
Logging can also be configured in Program.cs
with ConfigureLogging
:
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.AddFilter("Grpc", LogLevel.Debug);
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
When not using JSON-based configuration, set the following configuration value in the configuration system:
Logging:LogLevel:Grpc
= Debug
Check the documentation for your configuration system to determine how to specify nested configuration values. For example, when using environment variables, two _
characters are used instead of the :
(for example, Logging__LogLevel__Grpc
).
We recommend using the Debug
level when gathering detailed diagnostics for an app. The Trace
level produces low-level diagnostics and is rarely needed to diagnose issues.
Here is an example of console output at the Debug
level of a gRPC service:
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/2 POST https://localhost:5001/Greet.Greeter/SayHello application/grpc
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'gRPC - /Greet.Greeter/SayHello'
dbug: Grpc.AspNetCore.Server.ServerCallHandler[1]
Reading message.
info: GrpcService.GreeterService[0]
Hello World
dbug: Grpc.AspNetCore.Server.ServerCallHandler[6]
Sending message.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'gRPC - /Greet.Greeter/SayHello'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.4113ms 200 application/grpc
How server-side logs are accessed depends on the app's environment.
If you're running in a console app, the Console logger should be enabled by default. gRPC logs will appear in the console.
If the app is deployed to another environment (for example, Docker, Kubernetes, or Windows Service), see Logging in .NET Core and ASP.NET Core for more information on how to configure logging providers suitable for the environment.
Warning
Client-side logs may contain sensitive information from your app. Never post raw logs from production apps to public forums like GitHub.
To get logs from the .NET client, set the GrpcChannelOptions.LoggerFactory
property when the client's channel is created. When calling a gRPC service from an ASP.NET Core app, the logger factory can be resolved from dependency injection (DI):
[ApiController]
[Route("[controller]")]
public class GreetingController : ControllerBase
{
private ILoggerFactory _loggerFactory;
public GreetingController(ILoggerFactory loggerFactory)
{
_loggerFactory = loggerFactory;
}
[HttpGet]
public async Task<ActionResult<string>> Get(string name)
{
var channel = GrpcChannel.ForAddress("https://localhost:5001",
new GrpcChannelOptions { LoggerFactory = _loggerFactory });
var client = new Greeter.GreeterClient(channel);
var reply = await client.SayHelloAsync(new HelloRequest { Name = name });
return Ok(reply.Message);
}
}
An alternative way to enable client logging is to use the gRPC client factory to create the client. A gRPC client registered with the client factory and resolved from DI will automatically use the app's configured logging.
If the app isn't using DI, then create a new ILoggerFactory
instance with LoggerFactory.Create. To access this method, add the Microsoft.Extensions.Logging package to your app.
var loggerFactory = LoggerFactory.Create(logging =>
{
logging.AddConsole();
logging.SetMinimumLevel(LogLevel.Debug);
});
var channel = GrpcChannel.ForAddress("https://localhost:5001",
new GrpcChannelOptions { LoggerFactory = loggerFactory });
var client = Greeter.GreeterClient(channel);
The gRPC client adds a logging scope to logs made during a gRPC call. The scope has metadata related to the gRPC call:
Grpc.Core.MethodType
enum. For example, Unary
.Here is an example of console output at the Debug
level of a gRPC client:
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Starting gRPC call. Method type: 'Unary', URI: 'https://localhost:5001/Greet.Greeter/SayHello'.
dbug: Grpc.Net.Client.Internal.GrpcCall[6]
Sending message.
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Reading message.
dbug: Grpc.Net.Client.Internal.GrpcCall[4]
Finished gRPC call.
gRPC services and the gRPC client provide information about gRPC calls using DiagnosticSource and Activity.
gRPC services are hosted on ASP.NET Core, which reports events about incoming HTTP requests. gRPC specific metadata is added to the existing HTTP request diagnostics that ASP.NET Core provides.
Microsoft.AspNetCore
.Microsoft.AspNetCore.Hosting.HttpRequestIn
.
grpc.method
.grpc.status_code
.The .NET gRPC client uses HttpClient
to make gRPC calls. Although HttpClient
writes diagnostic events, the .NET gRPC client provides a custom diagnostic source, activity, and events so that complete information about a gRPC call can be collected.
Grpc.Net.Client
.Grpc.Net.Client.GrpcOut
.
grpc.method
.grpc.status_code
.The easiest way to use DiagnosticSource
is to configure a telemetry library such as Application Insights or OpenTelemetry in your app. The library will process information about gRPC calls along-side other app telemetry.
Tracing can be viewed in a managed service like Application Insights, or run as your own distributed tracing system. OpenTelemetry supports exporting tracing data to Jaeger and Zipkin.
DiagnosticSource
can consume tracing events in code using DiagnosticListener
. For information about listening to a diagnostic source with code, see the DiagnosticSource user's guide.
Note
Telemetry libraries do not capture gRPC specific Grpc.Net.Client.GrpcOut
telemetry currently. Work to improve telemetry libraries capturing this tracing is ongoing.
Metrics is a representation of data measures over intervals of time, for example, requests per second. Metrics data allows observation of the state of an app at a high level. .NET gRPC metrics are emitted using EventCounter
.
gRPC server metrics are reported on Grpc.AspNetCore.Server
event source.
Name | Description |
---|---|
total-calls |
Total Calls |
current-calls |
Current Calls |
calls-failed |
Total Calls Failed |
calls-deadline-exceeded |
Total Calls Deadline Exceeded |
messages-sent |
Total Messages Sent |
messages-received |
Total Messages Received |
calls-unimplemented |
Total Calls Unimplemented |
ASP.NET Core also provides its own metrics on Microsoft.AspNetCore.Hosting
event source.
gRPC client metrics are reported on Grpc.Net.Client
event source.
Name | Description |
---|---|
total-calls |
Total Calls |
current-calls |
Current Calls |
calls-failed |
Total Calls Failed |
calls-deadline-exceeded |
Total Calls Deadline Exceeded |
messages-sent |
Total Messages Sent |
messages-received |
Total Messages Received |
dotnet-counters is a performance monitoring tool for ad-hoc health monitoring and first-level performance investigation. Monitor a .NET app with either Grpc.AspNetCore.Server
or Grpc.Net.Client
as the provider name.
> dotnet-counters monitor --process-id 1902 Grpc.AspNetCore.Server
Press p to pause, r to resume, q to quit.
Status: Running
[Grpc.AspNetCore.Server]
Total Calls 300
Current Calls 5
Total Calls Failed 0
Total Calls Deadline Exceeded 0
Total Messages Sent 295
Total Messages Received 300
Total Calls Unimplemented 0
Another way to observe gRPC metrics is to capture counter data using Application Insights's Microsoft.ApplicationInsights.EventCounterCollector package. Once setup, Application Insights collects common .NET counters at runtime. gRPC's counters are not collected by default, but App Insights can be customized to include additional counters.
Specify the gRPC counters for Application Insight to collect in Startup.cs
:
using Microsoft.ApplicationInsights.Extensibility.EventCounterCollector;
public void ConfigureServices(IServiceCollection services)
{
//... other code...
services.ConfigureTelemetryModule<EventCounterCollectionModule>(
(module, o) =>
{
// Configure App Insights to collect gRPC counters gRPC services hosted in an ASP.NET Core app
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "current-calls"));
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "total-calls"));
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "calls-failed"));
}
);
}
This article provides guidance for gathering diagnostics from a gRPC app to help troubleshoot issues. Topics covered include:
DiaganosticSource
and Activity
. Traces from diagnostic source are commonly used to collect app telemetry by libraries such as Application Insights and OpenTelemetry.EventCounter
and can be observed using dotnet-counters command line tool or with Application Insights.gRPC services and the gRPC client write logs using .NET Core logging. Logs are a good place to start when debugging unexpected behavior in service and client apps.
Warning
Server-side logs may contain sensitive information from your app. Never post raw logs from production apps to public forums like GitHub.
Since gRPC services are hosted on ASP.NET Core, it uses the ASP.NET Core logging system. In the default configuration, gRPC logs minimal information, but logging can be configured. See the documentation on ASP.NET Core logging for details on configuring ASP.NET Core logging.
gRPC adds logs under the Grpc
category. To enable detailed logs from gRPC, configure the Grpc
prefixes to the Debug
level in your appsettings.json
file by adding the following items to the LogLevel
subsection in Logging
:
{
"Logging": {
"LogLevel": {
"Default": "Debug",
"System": "Information",
"Microsoft": "Information",
"Grpc": "Debug"
}
}
}
You can also configure this in Startup.cs
with ConfigureLogging
:
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.ConfigureLogging(logging =>
{
logging.AddFilter("Grpc", LogLevel.Debug);
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
If you aren't using JSON-based configuration, set the following configuration value in your configuration system:
Logging:LogLevel:Grpc
= Debug
Check the documentation for your configuration system to determine how to specify nested configuration values. For example, when using environment variables, two _
characters are used instead of the :
(for example, Logging__LogLevel__Grpc
).
We recommend using the Debug
level when gathering detailed diagnostics for an app. The Trace
level produces low-level diagnostics and is rarely needed to diagnose issues.
Here is an example of console output at the Debug
level of a gRPC service:
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/2 POST https://localhost:5001/Greet.Greeter/SayHello application/grpc
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[0]
Executing endpoint 'gRPC - /Greet.Greeter/SayHello'
dbug: Grpc.AspNetCore.Server.ServerCallHandler[1]
Reading message.
info: GrpcService.GreeterService[0]
Hello World
dbug: Grpc.AspNetCore.Server.ServerCallHandler[6]
Sending message.
info: Microsoft.AspNetCore.Routing.EndpointMiddleware[1]
Executed endpoint 'gRPC - /Greet.Greeter/SayHello'
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 1.4113ms 200 application/grpc
How you access server-side logs depends on the environment in which you're running.
If you're running in a console app, the Console logger should be enabled by default. gRPC logs will appear in the console.
If the app is deployed to another environment (for example, Docker, Kubernetes, or Windows Service), see Logging in .NET Core and ASP.NET Core for more information on how to configure logging providers suitable for the environment.
Warning
Client-side logs may contain sensitive information from your app. Never post raw logs from production apps to public forums like GitHub.
To get logs from the .NET client, set the GrpcChannelOptions.LoggerFactory
property when the client's channel is created. When calling a gRPC service from an ASP.NET Core app, the logger factory can be resolved from dependency injection (DI):
[ApiController]
[Route("[controller]")]
public class GreetingController : ControllerBase
{
private ILoggerFactory _loggerFactory;
public GreetingController(ILoggerFactory loggerFactory)
{
_loggerFactory = loggerFactory;
}
[HttpGet]
public async Task<ActionResult<string>> Get(string name)
{
var channel = GrpcChannel.ForAddress("https://localhost:5001",
new GrpcChannelOptions { LoggerFactory = _loggerFactory });
var client = new Greeter.GreeterClient(channel);
var reply = await client.SayHelloAsync(new HelloRequest { Name = name });
return Ok(reply.Message);
}
}
An alternative way to enable client logging is to use the gRPC client factory to create the client. A gRPC client registered with the client factory and resolved from DI will automatically use the app's configured logging.
If your app isn't using DI, then you can create a new ILoggerFactory
instance with LoggerFactory.Create. To access this method, add the Microsoft.Extensions.Logging package to your app.
var loggerFactory = LoggerFactory.Create(logging =>
{
logging.AddConsole();
logging.SetMinimumLevel(LogLevel.Debug);
});
var channel = GrpcChannel.ForAddress("https://localhost:5001",
new GrpcChannelOptions { LoggerFactory = loggerFactory });
var client = Greeter.GreeterClient(channel);
The gRPC client adds a logging scope to logs made during a gRPC call. The scope has metadata related to the gRPC call:
Grpc.Core.MethodType
enum. For example, Unary
.Here is an example of console output at the Debug
level of a gRPC client:
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Starting gRPC call. Method type: 'Unary', URI: 'https://localhost:5001/Greet.Greeter/SayHello'.
dbug: Grpc.Net.Client.Internal.GrpcCall[6]
Sending message.
dbug: Grpc.Net.Client.Internal.GrpcCall[1]
Reading message.
dbug: Grpc.Net.Client.Internal.GrpcCall[4]
Finished gRPC call.
gRPC services and the gRPC client provide information about gRPC calls using DiagnosticSource and Activity.
gRPC services are hosted on ASP.NET Core, which reports events about incoming HTTP requests. gRPC specific metadata is added to the existing HTTP request diagnostics that ASP.NET Core provides.
Microsoft.AspNetCore
.Microsoft.AspNetCore.Hosting.HttpRequestIn
.
grpc.method
.grpc.status_code
.The .NET gRPC client uses HttpClient
to make gRPC calls. Although HttpClient
writes diagnostic events, the .NET gRPC client provides a custom diagnostic source, activity, and events so that complete information about a gRPC call can be collected.
Grpc.Net.Client
.Grpc.Net.Client.GrpcOut
.
grpc.method
.grpc.status_code
.The easiest way to use DiagnosticSource
is to configure a telemetry library such as Application Insights or OpenTelemetry in your app. The library will process information about gRPC calls along-side other app telemetry.
Tracing can be viewed in a managed service like Application Insights, or you can choose to run your own distributed tracing system. OpenTelemetry supports exporting tracing data to Jaeger and Zipkin.
DiagnosticSource
can consume tracing events in code using DiagnosticListener
. For information about listening to a diagnostic source with code, see the DiagnosticSource user's guide.
Note
Telemetry libraries do not capture gRPC specific Grpc.Net.Client.GrpcOut
telemetry currently. Work to improve telemetry libraries capturing this tracing is ongoing.
Metrics is a representation of data measures over intervals of time, for example, requests per second. Metrics data allows observation of the state of an app at a high level. .NET gRPC metrics are emitted using EventCounter
.
gRPC server metrics are reported on Grpc.AspNetCore.Server
event source.
Name | Description |
---|---|
total-calls |
Total Calls |
current-calls |
Current Calls |
calls-failed |
Total Calls Failed |
calls-deadline-exceeded |
Total Calls Deadline Exceeded |
messages-sent |
Total Messages Sent |
messages-received |
Total Messages Received |
calls-unimplemented |
Total Calls Unimplemented |
ASP.NET Core also provides its own metrics on Microsoft.AspNetCore.Hosting
event source.
gRPC client metrics are reported on Grpc.Net.Client
event source.
Name | Description |
---|---|
total-calls |
Total Calls |
current-calls |
Current Calls |
calls-failed |
Total Calls Failed |
calls-deadline-exceeded |
Total Calls Deadline Exceeded |
messages-sent |
Total Messages Sent |
messages-received |
Total Messages Received |
dotnet-counters is a performance monitoring tool for ad-hoc health monitoring and first-level performance investigation. Monitor a .NET app with either Grpc.AspNetCore.Server
or Grpc.Net.Client
as the provider name.
> dotnet-counters monitor --process-id 1902 Grpc.AspNetCore.Server
Press p to pause, r to resume, q to quit.
Status: Running
[Grpc.AspNetCore.Server]
Total Calls 300
Current Calls 5
Total Calls Failed 0
Total Calls Deadline Exceeded 0
Total Messages Sent 295
Total Messages Received 300
Total Calls Unimplemented 0
Another way to observe gRPC metrics is to capture counter data using Application Insights's Microsoft.ApplicationInsights.EventCounterCollector package. Once setup, Application Insights collects common .NET counters at runtime. gRPC's counters are not collected by default, but App Insights can be customized to include additional counters.
Specify the gRPC counters for Application Insight to collect in Startup.cs
:
using Microsoft.ApplicationInsights.Extensibility.EventCounterCollector;
public void ConfigureServices(IServiceCollection services)
{
//... other code...
services.ConfigureTelemetryModule<EventCounterCollectionModule>(
(module, o) =>
{
// Configure App Insights to collect gRPC counters gRPC services hosted in an ASP.NET Core app
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "current-calls"));
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "total-calls"));
module.Counters.Add(new EventCounterCollectionRequest("Grpc.AspNetCore.Server", "calls-failed"));
}
);
}
ASP.NET Core feedback
ASP.NET Core is an open source project. Select a link to provide feedback:
Events
Power BI DataViz World Championships
Feb 14, 4 PM - Mar 31, 4 PM
With 4 chances to enter, you could win a conference package and make it to the LIVE Grand Finale in Las Vegas
Learn moreTraining
Module
Implement observability in a .NET cloud-native application with OpenTelemetry - Training
Learn about observability and how to implement it in a cloud-native application. Use OpenTelemetry packages to output logs, metrics, and tracing data and analyze the data in Application Insights and third-party applications.