Provide guidance on logging frameworks

[tomkerkhove]

Provide guidance on logging frameworks for ie logging exceptions.

Ideas:

• Cloud:
  • Application Insights
• On-Prem:
  • ETW?

[samneirinck]

Do we limit this to exceptions, or logging in general?

For cloud, I'd also vouch for a console logger in addition to Application Insights. This is especially handy when running in a container, or running the api locally. This allows
you to see the error details directly, without having to wait for AI to index your logs.

For on-prem, ETW is cool, I'm a fan, but it does involve users having to know tools to parse these ETW events, and it's harder to view a live trace of your current system. Is it worth it? Also, for on-prem I think Application Insights might still make sense.

With regards to using an actual logging framework, I'd suggest one that promotes the use of structured logging. I'd also make it clear that the logging framework chosen it's just a suggestion. I've used a bazillion logging frameworks already, and each one has its own benefits.

[tomkerkhove]

It certainly is not limited to exceptions and changed the description to reflect this.

• Cloud - Would you provide parity on both or use on over the other depending on the scenario?
• On-Prem - That's a fair point. We could, question is what with does that can't?

[MassimoC]

I'd also make it clear that the logging framework chosen it's just a suggestion

I agree, on the other hand it would be nice to have a a single approach (cloud and on-prem) - e.g. Serilog - and use different sink. Does it make sense?

I'd suggest one that promotes the use of structured logging.

great point

[samneirinck]

I like Serilog yes, although we'd need to take a look if it integrates correctly with Application Insights (operation_Id and operation_ParentId specifically).

We could use 2 sinks (console+AI) if it works correctly.

[samneirinck]

Had a quick look at Serilog + AI yesterday, below my findings:

• Current AI sink for Serilog is just for traces/events/exceptions, no requests/dependencies
• Default AI sink does not populate the operation_Id and operation_ParentId properties
• Serilog recommends disabling the application insights nuget packages, and use their sink (without the request/dependency tracking)

Open questions:

• Where/how do we split up logging vs request/dependency tracking? Do we want it all in Serilog?
• How much custom work do we want to do vs using the default setup (1-liner to get application insights working without serilog)

Conceptually I think everything should be in Serilog, and the configuration can decide on what to log where. However that seems like a huge task, and pretty much guaranteed to be out-of-date as soon as we release it.

Is there a middle ground? Is there benefit in still using Serilog just for messages?

[tomkerkhove]

Thanks for sharing your findings. I could live with a mixture to be honest where Serilog is doing traces, events and exceptions but the question is - Will they work together if we use AI packages for request & dependency tracking?

Another downside is the lack of support for operation_id but maybe there is a way to fix that.

[tomkerkhove]

Created an issue with Serilog for the operation_Id

[MassimoC]

Ok to use AI with the default logging mechanism of aspnetcore. It's important to be practical so the logging mechanism should be easily configurable to sink to AI, console and file system.

[MassimoC]

For practical reasons, let’s move this to the “should have” (level two)

[fgheysels]

In the API guidelines, I would emphasize on how / what to log (using operation-id's, correlation-ids, ...) and not advice on using a certain Logging Framework.

Instead, I'd suggest to make use of Microsoft.Extensions.Logging.ILogger interface which abstracts away the specific logging framework that is being used. Most modern logging frameworks have extensions that implement this interface, so you're nog tightly coupled on a specific implementation. (SeriLog), NLog.

[tomkerkhove]

Sounds good to me, leaning towards that approach as well nowadays.

We should also provide guidance what not to log ie due to GDPR