Google.Cloud.Diagnostics.AspNetCore

Google.Cloud.Diagnostics.AspNetCore is an ASP.NET Core instrumentation library for Google Logging, Error Reporting and Tracing. It allows for simple integration of Google observability components into ASP.NET Core 2.1+ applications with minimal code changes.

Note: ASP.NET Core 2.1 is the long-term support release of ASP.NET Core 2.x.

Note: This documentation is for version 4.2.0 of the library. Some samples may not work with other versions.

Installation

Install the Google.Cloud.Diagnostics.AspNetCore package from NuGet. Add it to your project in the normal way (for example by right-clicking on the project in Visual Studio and choosing "Manage NuGet Packages...").

Authentication

When running on Google Cloud Platform, no action needs to be taken to authenticate.

Otherwise, the simplest way of authenticating your API calls is to download a service account JSON file then set the GOOGLE_APPLICATION_CREDENTIALS environment variable to refer to it. The credentials will automatically be used to authenticate. See the Getting Started With Authentication guide for more details.

See API Permissions for entries.write for the permissions needed for Logging and Error Reporting.

See API Permissions for PatchTraces for the permissions needed for Tracing.

Note

The Google.Cloud.Diagnostics.AspNetCore package attempts to collect the filename and line number when entries are collected. However, to be able to collect this information PDBs must be included with the deployed code.

Note

When running on environments that limit or disable CPU usage for background activities, for instance Google Cloud Run, take care not to use the timed buffer options for any of Logging, Tracing or Error Reporting. Take into account that the timed buffer is used for all of these components by default so you will need to explicitly configure the buffers by using the Google.Cloud.Diagnostics.AspNetCore.LoggerOptions, Google.Cloud.Diagnostics.Common.TraceOptions and Google.Cloud.Diagnostics.Common.ErrorReportingOptions classes. Below you'll find examples of how to configure the buffers.

Getting started

Initializing Google Diagnostics

The easiest way to initialize Google Diagnostics services is using the UseGoogleDiagnostics extentension method on IWebHostBuilder. This configures Logging, Tracing and Error Reporting middleware.

If your application is runnng on Google App Engine, Google Kubernetes Engine, Google Cloud Run or Google Compute Engine, you don't need to provide a value for ProjectId, Service and Version since they can be automatically obtained by the UseGoogleDiagnostics method as far as they make sense for the environment. (Not every environment has the concept of a "service" or "version".) The values used will be the ones associated with the running application.

If your application is running outside of GCP, including when it runs locally, then you'll need to provide the ProjectId of the Google Cloud Project in which to store the diagnostic information as well as the Service and Version with which to identify your application.

var hostBuilder = new WebHostBuilder()
    // Replace ProjectId with your Google Cloud Project ID.
    // Replace Service with a name or identifier for the service.
    // Replace Version with a version for the service.
    .UseGoogleDiagnostics(ProjectId, Service, Version)
    .UseStartup<Startup>();

You can still initialize the separate components using the extension methods below. This can be useful if you only need to use some of the observability components.

Optional parameters on UseGoogleDiagnostics are also available to specify options for each of the components (logging, tracing and error reporting). This is typically useful for diagnosing problems, as described below.

Error Reporting

Registering Error Reporting

public void ConfigureServices(IServiceCollection services)
{
    services.AddGoogleExceptionLogging(options =>
    {
        // Replace ProjectId with your Google Cloud Project ID.
        options.ProjectId = ProjectId;
        // Replace Service with a name or identifier for the service.
        options.ServiceName = Service;
        // Replace Version with a version for the service.
        options.Version = Version;
    });

    // Add any other services your application requires, for instance,
    // depending on the version of ASP.NET Core you are using, you may
    // need one of the following:

    // services.AddMvc();

    // services.AddControllersWithViews();
}

public void Configure(IApplicationBuilder app)
{
    // Use before handling any requests to ensure all unhandled exceptions are reported.
    app.UseGoogleExceptionLogging();

    // Add any other configuration your application requires, for instance,
    // depending on the verson of ASP.NET Core you are using, you may
    // need one of the following:

    //app.UseMvc(routes =>
    //{
    //    routes.MapRoute(
    //        name: "default",
    //        template: "{controller=Home}/{action=Index}/{id?}");
    //});

    //app.UseRouting();
    //app.UseEndpoints(endpoints =>
    //{
    //    endpoints.MapControllerRoute(
    //        name: "default",
    //        pattern: "{controller=Home}/{action=Index}/{id?}");
    //    endpoints.MapRazorPages();
    //});
}

Log Exceptions

/// <summary>
/// This method catches an exception thrown by another method and explicitly
/// logs that exception.
/// The <see cref="IExceptionLogger"/> is populated by dependency injection 
/// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
/// </summary>
public void CatchesAndLogsException(string id, [FromServices]IExceptionLogger exceptionLogger)
{
    try
    {
        // This method call throws an exception.
        ThrowsException(id);
    }
    catch (Exception e)
    {
        exceptionLogger.Log(e);
    }
}

Logging

Initializing Logging

When configuring an IWebHostBuilder Logging can be initialized in two different ways:

Using ConfigureServices:

return new WebHostBuilder()
    .ConfigureServices(services =>
    {
        // Replace projectId with your Google Cloud Project ID.
        services.AddSingleton<ILoggerProvider>(sp => GoogleLoggerProvider.Create(sp, projectId));
    })
    .UseStartup<Startup>();

Or using ConfigureLogging:

return new WebHostBuilder()
    .ConfigureLogging(builder => builder.AddProvider(GoogleLoggerProvider.Create(serviceProvider: null, projectId)))
    .UseStartup<Startup>();

Note that this approach does not support custom services (such as log entry label providers) being used as the service provider is not available within ConfigureLogging.

Alternatively, logging can be configured within the application's Startup.Configure method:

public void ConfigureServices(IServiceCollection services)
{
    // No specific services are needed to use Google Logging.

    // Add any services your application requires, for instance,
    // depending on the version of ASP.NET Core you are using, you may
    // need one of the following:

    // services.AddMvc();

    // services.AddControllersWithViews();
}

public void Configure(IApplicationBuilder app, ILoggerFactory loggerFactory)
{
    // Replace ProjectId with your Google Cloud Project ID.
    loggerFactory.AddGoogle(app.ApplicationServices, ProjectId);

    // Add any other configuration your application requires, for instance,
    // depending on the verson of ASP.NET Core you are using, you may
    // need one of the following:

    //app.UseMvc(routes =>
    //{
    //    routes.MapRoute(
    //        name: "default",
    //        template: "{controller=Home}/{action=Index}/{id?}");
    //});

    //app.UseRouting();
    //app.UseEndpoints(endpoints =>
    //{
    //    endpoints.MapControllerRoute(
    //        name: "default",
    //        pattern: "{controller=Home}/{action=Index}/{id?}");
    //    endpoints.MapRazorPages();
    //});
}

Log

public class LoggingSamplesController : Controller
{
    /// <summary>
    /// This method logs an Information level message.
    /// The <see cref="ILogger"/> is populated by dependency injection
    /// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
    /// </summary>
    public void LogInformation(string id, [FromServices]ILogger<LoggingSamplesController> logger)
    {
        // Log whatever message you want to log.
        logger.LogInformation(id);
    }
}

Troubleshooting Logging

Sometimes it is neccessary to diagnose log operations. It might be that logging is failing or that we simply cannot find where the logs are being stored in GCP. What follows are a couple of code samples that can be useful to find out what might be wrong with logging operations.

Propagating Exceptions

By default the Google Logger won't propagate any exceptions thrown during logging. This is to protect the application from crashing if logging is not possible. But logging is an important aspect of most applications so at times we need to know if it's failing and why. The following example shows how to configure Google Logger so that it propagates exceptions thrown during logging.

// Explicitly create logger options that will propagate any exceptions thrown
// during logging.
RetryOptions retryOptions = RetryOptions.NoRetry(ExceptionHandling.Propagate);
// Also set the no buffer option so that writing the logs is attempted immediately.
BufferOptions bufferOptions = BufferOptions.NoBuffer();
LoggerOptions loggerOptions = LoggerOptions.Create(bufferOptions: bufferOptions, retryOptions: retryOptions);
return new WebHostBuilder()
    .UseGoogleDiagnostics(projectId, loggerOptions: loggerOptions)
    .UseStartup<Startup>();

The same LoggerOptions can be specified in any of the other ways of registering logging.

Finding out the URL where logs are written

Depending on where your code is running and the options you provided for creating a Google Logger, it might be hard to find your logs in the GCP Console. We have provided a way for you to obtain the URL where your logs can be found.

As the following code sample shows, you only need to pass a System.IO.TextWriter (typically Console.Out or Console.Error) as part of the options when registering logging. When the GoogleLoggerProvider is initialized, the URL where its logs can be found will be written to the given text writer.

return new WebHostBuilder()
    .UseGoogleDiagnostics(projectId, loggerOptions: LoggerOptions.Create(loggerDiagnosticsOutput: Console.Out))
    .UseStartup<Startup>();

Please note that since this is a Google Logger diagnostics feature, we don't respect settings for exception handling, i.e. we propagate any exception thrown while writing the URL to the given text writer so you know what might be happening. This feature should only be activated as a one off, if you are having trouble trying to find your logs in the GCP Console, and not as a permanent feature in production code. To deactivate this feature simply stop passing a System.IO.TextWriter as part of the options when creating a Google Logger.

Tracing

Initializing Tracing

public void ConfigureServices(IServiceCollection services)
{
    // The line below is needed for trace ids to be added to logs.
    services.AddHttpContextAccessor();

    // Replace ProjectId with your Google Cloud Project ID.
    services.AddGoogleTrace(options =>
    {
        options.ProjectId = ProjectId;
    });

    // Add any other services your application requires, for instance,
    // depending on the version of ASP.NET Core you are using, you may
    // need one of the following:

    // services.AddMvc();

    // services.AddControllersWithViews();
}

public void Configure(IApplicationBuilder app)
{
    // Use at the start of the request pipeline to ensure the entire request is traced.
    app.UseGoogleTrace();

    // Add any other configuration your application requires, for instance,
    // depending on the verson of ASP.NET Core you are using, you may
    // need one of the following:

    //app.UseMvc(routes =>
    //{
    //    routes.MapRoute(
    //        name: "default",
    //        template: "{controller=Home}/{action=Index}/{id?}");
    //});

    //app.UseRouting();
    //app.UseEndpoints(endpoints =>
    //{
    //    endpoints.MapControllerRoute(
    //        name: "default",
    //        pattern: "{controller=Home}/{action=Index}/{id?}");
    //    endpoints.MapRazorPages();
    //});
}

Troubleshooting Tracing

Just as with logging, trace is most easily diagnosed by removing buffering and propagating exceptions immediately.

// Explicitly create trace options that will propagate any exceptions thrown during tracing.
RetryOptions retryOptions = RetryOptions.NoRetry(ExceptionHandling.Propagate);
// Also set the no buffer option so that tracing is attempted immediately.
BufferOptions bufferOptions = BufferOptions.NoBuffer();
TraceOptions traceOptions = TraceOptions.Create(bufferOptions: bufferOptions, retryOptions: retryOptions);

The options can be specified wherever you are configuring tracing.

Tracing in Controllers

To use the IManagedTracer in controllers you can either inject the singleton instance of IManagedTracer into the controller's constructor (see TraceSamplesConstructorController) or you can inject the IManagedTracer into the action method using the [FromServices] attribute (see TraceSamplesMethodController).

public class TraceSamplesConstructorController : Controller
{
    private readonly IManagedTracer _tracer;

    /// <summary>
    /// The <see cref="IManagedTracer"/> is populated by dependency injection.
    /// </summary>
    public TraceSamplesConstructorController(IManagedTracer tracer)
    {
        _tracer = tracer;
    }

    public void TraceHelloWorld(string id)
    {
        // Change the name of the span to what makese sense in your context.
        using (_tracer.StartSpan(id))
        {
            // The code whose execution is to be included in the span goes here.
            ViewData["Message"] = "Hello World.";
        }
    }
}

public class TraceSamplesMethodController : Controller
{
    /// <summary>
    /// Manually trace a set of operations.
    /// The <see cref="IManagedTracer"/> is populated by dependency injection
    /// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
    /// </summary>
    public void TraceHelloWorld(string id, [FromServices] IManagedTracer tracer)
    {
        // Change the name of the span to what makese sense in your context.
        using (tracer.StartSpan(id))
        {
            // The code whose execution is to be included in the span goes here.
            ViewData["Message"] = "Hello World.";
        }
    }
}

Manual Tracing

/// <summary>
/// Manually trace a set of operations.
/// The <see cref="IManagedTracer"/> is populated by dependency injection
/// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
/// </summary>
public void TraceHelloWorld(string id, [FromServices] IManagedTracer tracer)
{
    // Change the name of the span to what makese sense in your context.
    using (tracer.StartSpan(id))
    {
        // The code whose execution is to be included in the span goes here.
        ViewData["Message"] = "Hello World.";
    }
}

/// <summary>
/// Manually trace a specific Action or Func<T>.
/// The <see cref="IManagedTracer"/> is populated by dependency injection
/// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
/// </summary>
public void TraceHelloWorldRunIn(string id, [FromServices]IManagedTracer tracer)
{
    tracer.RunInSpan(
        // The Action or Func<T> to be traced.
        () =>
        {
            // The code whose execution is to be included in the span goes here.
            ViewData["Message"] = "Hello World.";
        },
        // The name of the span.
        id);
}

Trace Outgoing HTTP Requests (recommended)

The recommended way of creating HttpClient in ASP.NET Core 2.0 and upwards is to use the System.Net.Http.IHttpClientFactory defined in the Microsoft.Extensions.Http package. The following example demonstrates how to register and use an HttpClient using Google Trace so that it traces outgoing requests.

Configuration

public void ConfigureServices(IServiceCollection services)
{
    // The line below is needed for trace ids to be added to logs.
    services.AddHttpContextAccessor();

    // Replace ProjectId with your Google Cloud Project ID.
    services.AddGoogleTrace(options =>
    {
        options.ProjectId = ProjectId;
    });

    // Register an HttpClient for outgoing requests.
    services.AddHttpClient("tracesOutgoing")
        // The next call guarantees that trace information is propagated for outgoing
        // requests that are already being traced.
        .AddOutgoingGoogleTraceHandler();

    // Add any other services your application requires, for instance,
    // depending on the version of ASP.NET Core you are using, you may
    // need one of the following:

    // services.AddMvc();

    // services.AddControllersWithViews();
}

public void Configure(IApplicationBuilder app)
{
    // Use at the start of the request pipeline to ensure the entire request is traced.
    app.UseGoogleTrace();

    // Add any other configuration your application requires, for instance,
    // depending on the verson of ASP.NET Core you are using, you may
    // need one of the following:

    //app.UseMvc(routes =>
    //{
    //    routes.MapRoute(
    //        name: "default",
    //        template: "{controller=Home}/{action=Index}/{id?}");
    //});

    //app.UseRouting();
    //app.UseEndpoints(endpoints =>
    //{
    //    endpoints.MapControllerRoute(
    //        name: "default",
    //        pattern: "{controller=Home}/{action=Index}/{id?}");
    //    endpoints.MapRazorPages();
    //});
}

Usage

/// <summary>
/// Use the <see cref="IHttpClientFactory"/> to create an HttpClient that will guarantee
/// the tracing of outgoing requests.
/// The <see cref="IHttpClientFactory"/> is populated by dependency injection
/// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
/// </summary>
public async Task<HttpResponseMessage> TraceOutgoingClientFactory([FromServices] IHttpClientFactory clientFactory)
{
    var httpClient = clientFactory.CreateClient("tracesOutgoing");
    // Any code that makes outgoing requests.
    return await httpClient.GetAsync("https://weather.com/");
}

Trace Outgoing HTTP Requests (alternative)

Alternatively, if you need to construct HttpClient objects manually, TraceHeaderPropagatingHandler can be used to propagate trace headers:

/// <summary>
/// Add a handler to trace outgoing requests and to propagate the trace header.
/// The <see cref="TraceHeaderPropagatingHandler"/> is populated by dependency injection
/// thanks to the use of the <see cref="FromServicesAttribute"/> attribute.
/// </summary>
public async Task<HttpResponseMessage> TraceOutgoing([FromServices] TraceHeaderPropagatingHandler traceHeaderHandler)
{
    using var httpClient = new HttpClient(traceHeaderHandler);
    // Any code that makes outgoing requests.
    return await httpClient.GetAsync("https://weather.com/");
}