OpenTelemetry for .NET Microservices: Guide

OpenTelemetry is a powerful open-source framework for monitoring .NET microservices. It simplifies tracking performance and debugging issues by collecting traces, metrics, and logs in a standardized way. Here's what you need to know:

• Why Use OpenTelemetry?
  • Tracks requests across microservices with distributed tracing.
  • Offers a unified, vendor-neutral monitoring solution.
  • Works with tools like Jaeger, Prometheus, and Grafana.
• How to Get Started:
  1. Install NuGet packages like OpenTelemetry.Extensions.Hosting and OpenTelemetry.Instrumentation.AspNetCore.
  2. Configure tracing and metrics in your Program.cs or Startup.cs.
  3. Use exporters like Jaeger for tracing and Prometheus for metrics.
• Key Features:
  • Automatic instrumentation for ASP.NET Core and HTTP clients.
  • Custom spans for tracking specific operations.
  • Sampling and batching to reduce performance overhead.
• Tools Integration:
  • Jaeger: Visualize and analyze traces.
  • Prometheus & Grafana: Collect and display metrics.

Practical OpenTelemetry in .NET 8

Setting Up OpenTelemetry in a .NET Project

Installing Required OpenTelemetry Packages

To monitor your .NET microservices with OpenTelemetry, start by installing the necessary NuGet packages. These packages enable core functionality, ASP.NET Core instrumentation, and HTTP client monitoring ^[1][3]:

dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http

After installing these packages, you can move on to configuring OpenTelemetry in your application.

Configuring OpenTelemetry in ASP.NET Core Applications

To enable instrumentation for ASP.NET Core and HTTP clients, add the following configuration to your Program.cs or Startup.cs file ^[1][3]:

services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSource("YourServiceName")
        .AddJaegerExporter());

The AddSource method helps identify your service in traces, while exporters like Jaeger allow you to visualize those traces ^[2].

Adding Custom Instrumentation

Automatic instrumentation handles many common scenarios, but custom instrumentation gives you control over monitoring specific application logic. This is especially useful for capturing key events and metrics that are vital to your business.

Here’s an example of using custom spans to track specific operations:

private static readonly ActivitySource MyActivitySource = new("YourServiceName");

public void ProcessOrder(Order order)
{
    using var activity = MyActivitySource.StartActivity("ProcessOrder");
    activity?.SetTag("orderId", order.Id);
    activity?.SetTag("orderAmount", order.Amount);

    // Your order processing logic here
}

To make the most of your observability setup:

• Use OpenTelemetry's naming conventions for spans and attributes to ensure consistency.
• Batch telemetry data where possible to improve performance.
• Include meaningful attributes to add context to your traces.

The OpenTelemetry .NET SDK offers APIs for handling spans, attributes, and metrics, giving you the tools to fine-tune your monitoring setup ^[3].

sbb-itb-29cd4f6

Connecting OpenTelemetry to Monitoring Tools

Using Prometheus and Grafana for Metrics

Combining metrics and traces gives you a full view of your system's performance. To start, add the Prometheus exporter package to your project and configure it in your startup file:

dotnet add package OpenTelemetry.Exporter.Prometheus

builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics => metrics
        .AddPrometheusExporter()
        .AddMeter("YourServiceName")
        .AddAspNetCoreInstrumentation());

In Grafana, you can create dashboards by following these steps:

• Create a new dashboard.
• Add panels using PromQL queries.

For example, this PromQL query calculates the 95th percentile of HTTP request duration over 5-minute intervals:

histogram_quantile(0.95, 
    rate(http_server_duration_ms_bucket{service_name="your-service"}[5m]))

Prometheus and Grafana are excellent for metrics, while Jaeger is a go-to tool for tracing, offering detailed insights into request flows.

Exporting Traces to Jaeger

To set up tracing with Jaeger, install the Jaeger exporter package and configure it in your application:

dotnet add package OpenTelemetry.Exporter.Jaeger

services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddJaegerExporter(opts =>
        {
            opts.AgentHost = "localhost";
            opts.AgentPort = 6831;
        })
        .AddSource("YourServiceName")
        .AddAspNetCoreInstrumentation());

Run Jaeger locally using Docker:

docker run -d --name jaeger \
  -p 6831:6831/udp \
  -p 16686:16686 \
  jaegertracing/all-in-one:latest

Once running, access the Jaeger UI at http://localhost:16686. This interface allows you to:

• Pinpoint performance bottlenecks.
• Dive deep into span details.
• Trace request flows across services.

"OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software's performance and behavior." ^[5]

For production environments, consider using Docker Compose to streamline the management of Prometheus, Grafana, and Jaeger. Together, these tools form a powerful observability stack for .NET microservices.

Tips for Monitoring .NET Microservices with OpenTelemetry

Using Distributed Tracing for Issue Detection

OpenTelemetry makes it easier to track requests across different services, giving you a better view of your microservices setup. To get the most out of distributed tracing, here are some practical tips:

• Keep service names consistent

.AddService("payment-service-v1", serviceVersion: "1.0.0")

• Add custom activity tracking

using var activity = MyActivitySource.StartActivity("ProcessPayment");
activity?.SetTag("payment.id", paymentId);
activity?.SetTag("payment.amount", amount);

While tracing highlights how requests move through your system, combining it with logs and metrics paints a fuller picture of how things are running.

Combining Logs and Metrics for Better Insights

To truly understand your system's behavior, you need to link logs, metrics, and traces. By tying these elements together using trace IDs, you can pinpoint performance problems more easily:

logger.LogInformation("Processing order {OrderId} - Trace: {TraceId}", 
    orderId, Activity.Current?.TraceId);

However, balancing these telemetry types requires planning to avoid slowing down your system.

Reducing Performance Overhead

If not configured carefully, OpenTelemetry can impact your system's performance. Here’s how to keep things running smoothly:

• Use sampling to limit data collection

services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .SetSampler(new TraceIdRatioBasedSampler(0.25)) // Samples 25% of traces
        .AddAspNetCoreInstrumentation());

• Batch data for efficient exporting

builder.Services.AddOpenTelemetry()
    .WithTracing(builder => builder
        .AddOtlpExporter(opts => {
            opts.BatchExportProcessorOptions = new BatchExportProcessorOptions<Activity>
            {
                MaxQueueSize = 2048,
                ScheduledDelayMilliseconds = 5000
            };
        }));

Batching groups your telemetry data and sends it at set intervals, helping to reduce resource usage and keep things efficient.

Summary and Additional Resources

Key Takeaways

OpenTelemetry plays a crucial role in monitoring .NET microservices, offering tools for standardized telemetry collection and export. The OpenTelemetry SDK supports distributed tracing, metrics collection, and logging in .NET applications ^[1]. From setup to integration with tools like Jaeger and Prometheus, each step strengthens your monitoring capabilities. Here’s what to focus on:

• Streamlined Data Collection: Achieve comprehensive monitoring with minimal code changes ^[3].
• Optimized Performance: Use batching and sampling to reduce overhead.
• Consistency: Follow semantic conventions to maintain uniform telemetry data across services.

Keep Up with .NET Updates

Stay informed about the latest tools and practices by subscribing to resources like the .NET Newsletter. It provides daily updates on OpenTelemetry implementations and other .NET technologies.

Topics for Advanced Exploration

For those looking to deepen their expertise, exploring advanced topics can help refine your monitoring setup and address challenges like scaling or customization. Below are some areas worth investigating:

The OpenTelemetry GitHub repositories and SDK documentation offer detailed guides and examples for these advanced features ^[4]. The MicroserviceExample project is particularly useful, showcasing practical applications of OpenTelemetry in distributed .NET systems.

Related posts

• WCF Performance Monitoring: Top Tools & Counters
• WCF Message Size Reduction Techniques
• gRPC with .NET Core: Step-by-Step Guide