Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add execution context to log records #60122

Open
joshdover opened this issue Mar 13, 2020 · 8 comments
Open

Add execution context to log records #60122

joshdover opened this issue Mar 13, 2020 · 8 comments
Labels
enhancement New value added to drive a business result Feature:Logging Feature:New Platform NeededFor:Security Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc

Comments

@joshdover
Copy link
Contributor

In order to support enhanced observability of Kibana itself, we should add a richer "execution context" to the New Platform's logging service that allows correlation between different log records that occur during an "execution context" such as a request or background task.

This context should automatically be attached to any log records produced during an execution context, ideally by leveraging the RequestHandlerContext pattern.

Fields that should be part of this context:

  • A unique identifier for the execution context, such as the X-Opaque-Id header

Plugins should be able to leverage this identifier to find other useful information about these log events, such as the current user.

Related issue: #52125

@joshdover joshdover added Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc Feature:New Platform enhancement New value added to drive a business result labels Mar 13, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-platform (Team:Platform)

@pgayvallet
Copy link
Contributor

Do we have an idea how we would 'bind' the logger's records to the request execution context in a node environment without tools such as Java's threadlocal ?

@joshdover
Copy link
Contributor Author

@thomheymann is this a blocker for security anymore? As I understand it, this only affects regular log messages and should not affect the audit trail logs at all.

While I agree that adding the request ID to regular logs is very useful, I think we are fine if we delay this past 7.10.

@pgayvallet
Copy link
Contributor

@elastic/kibana-security do you knoe if that is still a requierement on your side?

@legrego
Copy link
Member

legrego commented Jul 8, 2024

@elastic/kibana-security do you knoe if that is still a requierement on your side?

@pgayvallet I'm not sure what the original goal was. Our audit logs already include a trace id to understand everything that happens within the context of a specific KibanaRequest.

If we're looking for something that spans multiple requests (such as the loading of a dashboard), then we don't have a mechanism for that, yet. I think this would be a welcome addition to our audit events, but this isn't something we are currently prioritizing.

@pgayvallet
Copy link
Contributor

I'm not sure what the original goal was. Our audit logs already include a trace id to understand everything that happens within the context of a specific KibanaRequest.

The idea is to be able to attach a traceId to all our log entries, so that we can track execution of a specific request more easily (so basically what you did for audit logging, but for all our logging system). This is similar to log4js' MDC.

@rudolf
Copy link
Contributor

rudolf commented Jul 18, 2024

It surprises me that we don't have a logger on the RequestHandlerContext that would be a natural place to enrich the logger with the request context.

The most frequent logging pattern in routes is probably to log on error, so it seems like we could perhaps get a lot of the value by just enriching our 500 / error logger with the trace id similar to additional fields we added in #169552

@pgayvallet
Copy link
Contributor

pgayvallet commented Aug 30, 2024

Yeah, I agree that just exposing a scoped logger via the request handler context would already do a lot, while being way less complex technically (no need for async context or anything).

The downside of the approach would be that we need all route handlers to change their code to use this RHC's scoped logger, where a more complex approach (such as async context) might be able to do it automatically without any code change in any handler.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Logging Feature:New Platform NeededFor:Security Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc
Projects
None yet
Development

No branches or pull requests

6 participants