Example: Use OpenTelemetry with OTLP and the standalone Aspire Dashboard

This article is one of a series of examples to illustrate .NET observability with OpenTelemetry.

In addition to being a standard part of Aspire, the Aspire Dashboard is available as a standalone Docker container, which provides an OTLP endpoint that telemetry can be sent to. The dashboard visualizes the logs, metrics, and traces. Using the dashboard in this way has no dependency on Aspire, and it visualizes telemetry from any application that sends it telemetry via OTLP. It works equally well for applications written in Java, GoLang, or Python provided they can send their telemetry to an OTLP endpoint.

Using the Aspire Dashboard has less configuration and setup steps than using Open Source solutions such as Prometheus, Grafana, and Jaeger. But unlike those tools, the Aspire Dashboard is intended as a developer visualization tool, and not for production monitoring.

1. Create the project

Create a simple web API project by using the ASP.NET Core Empty template in Visual Studio or the following .NET CLI command:

dotnet new web

2. Add metrics and activity definitions

The following code defines a new metric (greetings.count) for the number of times the API has been called, and a new activity source (Otel.Example).

// Custom metrics for the application
var greeterMeter = new Meter("OTel.Example", "1.0.0");
var countGreetings = greeterMeter.CreateCounter<int>("greetings.count", description: "Counts the number of greetings");

// Custom ActivitySource for the application
var greeterActivitySource = new ActivitySource("OTel.Example");

3. Create an API endpoint

Insert the following code between builder.Build(); and app.Run()

app.MapGet("/", SendGreeting);

Insert the following function at the bottom of the file:

async Task<string> SendGreeting(ILogger<Program> logger)
{
    // Create a new Activity scoped to the method
    using var activity = greeterActivitySource.StartActivity("GreeterActivity");

    // Log a message
    logger.LogInformation("Sending greeting");

    // Increment the custom counter
    countGreetings.Add(1);

    // Add a tag to the Activity
    activity?.SetTag("greeting", "Hello World!");

    return "Hello World!";
}

Note

The endpoint definition doesn't use anything specific to OpenTelemetry. It uses the .NET APIs for observability.

4. Reference the OpenTelemetry packages

Use the NuGet Package Manager or command line to add the following NuGet packages:

  <ItemGroup>
    <PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.9.0" />
  </ItemGroup>

Note

Use the latest versions, as the OTel APIs are constantly evolving.

5. Configure OpenTelemetry with the correct providers

Insert the following code before builder.Build();:

// Setup logging to be exported via OpenTelemetry
builder.Logging.AddOpenTelemetry(logging =>
{
    logging.IncludeFormattedMessage = true;
    logging.IncludeScopes = true;
});

var otel = builder.Services.AddOpenTelemetry();

// Add Metrics for ASP.NET Core and our custom metrics and export via OTLP
otel.WithMetrics(metrics =>
{
    // Metrics provider from OpenTelemetry
    metrics.AddAspNetCoreInstrumentation();
    //Our custom metrics
    metrics.AddMeter(greeterMeter.Name);
    // Metrics provides by ASP.NET Core in .NET 8
    metrics.AddMeter("Microsoft.AspNetCore.Hosting");
    metrics.AddMeter("Microsoft.AspNetCore.Server.Kestrel");
});

// Add Tracing for ASP.NET Core and our custom ActivitySource and export via OTLP
otel.WithTracing(tracing =>
{
    tracing.AddAspNetCoreInstrumentation();
    tracing.AddHttpClientInstrumentation();
    tracing.AddSource(greeterActivitySource.Name);
});

// Export OpenTelemetry data via OTLP, using env vars for the configuration
var OtlpEndpoint = builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"];
if (OtlpEndpoint != null)
{
    otel.UseOtlpExporter();
}

This code sets up OpenTelemetry with the different sources of telemetry:

• It adds a OTel provider to ILogger to collect log records.
• It sets up metrics, registering instrumentation providers and Meters for ASP.NET and our custom Meter.
• It sets up tracing, registering instrumentation providers and our custom ActivitySource.

It then registers the OTLP exporter using env vars for its configuration.

6. Configure OTLP Environment variables

The OTLP exporter can be configured via APIs in code, but it's more common to configure it via environment variables. Add the following to AppSettings.Development.json

"OTEL_EXPORTER_OTLP_ENDPOINT": "http://localhost:4317",
"OTEL_SERVICE_NAME": "OTLP-Example"

You can add additional environment variables for the .NET OTLP Exporter or common OTel variables such as OTEL_RESOURCE_ATTRIBUTES to define resource attributes.

Note

A common gotcha is to mix up AppSettings.json and AppSettings.Development.json. If the latter is present, it will be used when you F5 from Visual Studio, and any settings in AppSettings.json will be ignored.

7. Start the Aspire Dashboard container

Use docker to download and run the dashboard container.

docker run --rm -it `
-p 18888:18888 `
-p 4317:18889 `
--name aspire-dashboard `
mcr.microsoft.com/dotnet/aspire-dashboard:latest

Data displayed in the dashboard can be sensitive. By default, the dashboard is secured with authentication that requires a token to log in. The token is displayed in the resulting output when running the container.

Copy the URL shown, and replace 0.0.0.0 with localhost, for example, http://localhost:18888/login?t=123456780abcdef123456780, and open that in your browser. Or you can also paste the key after /login?t= when the login dialog is shown. The token changes each time you start the container.

8. Run the project

Run the project and then access the API with the browser or curl.

curl -k http://localhost:7275

Each time you request the page, it increments the count for the number of greetings that have been made.

8.1 Log output

The logging statements from the code are output using ILogger. By default, the Console Provider is enabled so that output is directed to the console.

There are a few options for how logs can be egressed from .NET:

• stdout and stderr output is redirected to log files by container systems such as Kubernetes.
• Using logging libraries that integrate with ILogger. These libraries include Serilog and NLog.
• Using logging providers for OTel such as OTLP. The logging section in the code from step 5 adds the OTel provider.

The logs are shown in the dashboard as structured logs - any properties you set in the log message are extracted as fields in the log record.

8.2 Viewing the metrics

The Aspire dashboard shows metrics on a per resource basis (a resource being the OTel way of talking about sources of telemetry such as a process). When a resource is selected, the dashboard enumerates each metric that has been sent to its OTLP endpoint by the resource. The list of metrics is dynamic, and is updated as new metrics are received.

The view for the metrics depends on the type of metric that's being used:

• Counters are shown directly.
• Histograms that track a value per request, such as a timespan or bytes sent per request, are collected into a series of buckets. The dashboard graphs the P50, P90, and P99 percentiles. Histogram results can include exemplars, which are individual datapoints together with the trace/spanId for that request. These are shown as dots on the graph. Selecting one navigates to the respective trace so you can see what happened to cause that value. This is useful for diagnosing outliers.
• Metrics can include dimensions, which are key/value pairs associated with individual values. The values are aggregated per dimension. Using the dropdowns in the view, you can filter the results to look at specific dimensions, such as only GET requests, or those for a specific URL route in ASP.NET.

8.3 Viewing the tracing

The tracing view shows a list of traces. Each trace is a set of activities that share the same traceId. Work is tracked with spans, which represent a unit of work. Processing an ASP.NET request creates a span. Making an HttpClient request is a span. By tracking the span's parent, a hierarchy of spans can be visualized. By collecting spans from each resource (process), you can track the work that happens across a series of services. HTTP requests have a header that's used to pass the traceId and parent spanId to the next service. Each resource needs to collect telemetry and send it to the same collector. It will then aggregate and present a hierarchy of the spans.

The dashboard shows a list of traces with summary information. Whenever spans with a new traceId are seen, they get a row in the table. Clicking view shows all the spans in the trace.

Selecting a span shows its details including any properties on the span, such as the greeting tag that you set in step 3.