Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(instr-knex): implement requireParentSpan config flag #2288

Merged

Conversation

reberhardt7
Copy link
Contributor

Which problem is this PR solving?

Many other instrumentations have a requireParentSpan flag, including http, undici, and pg, which helps to reduce tracing noise and load when we are only trying to instrument a small part of an existing system. This PR adds the flag for knex.

Short description of the changes

Adds a flag to avoid generating spans for knex queries if no parent span is present

Copy link

codecov bot commented Jun 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.40%. Comparing base (dfb2dff) to head (bd1622b).
Report is 195 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2288      +/-   ##
==========================================
- Coverage   90.97%   90.40%   -0.58%     
==========================================
  Files         146      149       +3     
  Lines        7492     7359     -133     
  Branches     1502     1527      +25     
==========================================
- Hits         6816     6653     -163     
- Misses        676      706      +30     
Files Coverage Δ
...emetry-instrumentation-knex/src/instrumentation.ts 98.79% <100.00%> (+0.01%) ⬆️

... and 65 files with indirect coverage changes

@reberhardt7 reberhardt7 force-pushed the rebs/knex-implement-requireParent branch from 6dbb1a6 to c5a9f39 Compare June 20, 2024 21:49
Copy link
Member

@blumamir blumamir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for opening the PR and your contribution.
The implementation looks good to me, and I understand the usecase and that this practice is already inmplemented in other instrumentations.

Having said that, I think it's worth mentioning here few of the past concerns with this config option:

  • while requireParentSpan can untrace one specific span, users might still get spans for this execution if any downstream library is instrumented. Not sure how it works with knex, but if it uses a db client underneath then we can still get a span for that with non-complete context which requires yet another configuration.
  • I think that the right place to control which spans are being untraced in opentelemetry is via samplers, and really wish we can find a practical way to move this logic there and deprecate all instrumentaiton config options that controls when a span is skipped and when not. unfortunately, since the samplers do not receive the instrumentation scope, this task becomes harder.

I am also curious if you can share a bit information about your motivation to create this PR - what is the untraced trigger in your service that makes knex calls without parent context?
is it from timers? does the context breaks somewhere along the way? are those heartbeats? is it your buisness logic or some framework? Did you observe cases in other instrumentations where you had to configure this option to reduce noise?

I wonder if we can solve the issue by supplying the context instead of ignoring the trace.

@trentm
Copy link
Contributor

trentm commented Jun 25, 2024

See also my comment at open-telemetry/opentelemetry-js#4788 (comment)

@reberhardt7
Copy link
Contributor Author

reberhardt7 commented Jun 25, 2024

@blumamir some responses inline:

I am also curious if you can share a bit information about your motivation to create this PR - what is the untraced trigger in your service that makes knex calls without parent context? is it from timers? does the context breaks somewhere along the way? are those heartbeats? is it your buisness logic or some framework? Did you observe cases in other instrumentations where you had to configure this option to reduce noise?

We were adding instrumentation to an existing complicated service, and I wanted to slowly add instrumentation to reduce the risk of degrading prod performance from suddenly generating lots of spans everywhere. At first, we only wanted to trace one specific request handler. We want database spans descending from that request handler, but not from any other handlers, which would be overwhelming.

Having said that, I think it's worth mentioning here few of the past concerns with this config option:

  • while requireParentSpan can untrace one specific span, users might still get spans for this execution if any downstream library is instrumented. Not sure how it works with knex, but if it uses a db client underneath then we can still get a span for that with non-complete context which requires yet another configuration.

That's definitely a valid concern. We are using knex with pg, and did not add the @opentelemetry/instrumentation-pg auto-instrumentation since the knex instrumentation is sufficient. However, if we did enable that, it has a requireParentSpan option that we would enable as well. Not sure about other database clients.

  • I think that the right place to control which spans are being untraced in opentelemetry is via samplers, and really wish we can find a practical way to move this logic there and deprecate all instrumentaiton config options that controls when a span is skipped and when not. unfortunately, since the samplers do not receive the instrumentation scope, this task becomes harder.

I think the sampler gets passed the context, so it could check whether there is a parent span that way, but I don't think it can see which instrumentation library or tracer generated the span. (Maybe that was what you were already pointing out.) I think writing a sampler is a much more general way to solve this problem, but it seems much simpler to get started tracing an existing app with a requireParentSpan flag.

@trentm
Copy link
Contributor

trentm commented Jun 27, 2024

This is perhaps getting off-track for this issue, but I wonder if the following would work for the use case:

  • A custom Sampler (call it InterestingRequestPathsSampler) that returns true from shouldSample if the spanKind is SERVER (i.e. this is an incoming request) and the span attributes include a url.path [*] that matches one of a given set of "interesting" request paths.
  • This sampler could be used in combination with ParentBasedSampler to respect an already set sampling decision on a parent span.

Anyway, just musing.

@github-actions github-actions bot added the pkg-status:unmaintained This package is unmaintained. Only bugfixes may be acceped until a new owner has been found. label Jul 23, 2024
@blumamir blumamir merged commit 6e8989d into open-telemetry:main Jul 23, 2024
19 checks passed
@dyladan dyladan mentioned this pull request Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pkg:instrumentation-knex pkg-status:unmaintained This package is unmaintained. Only bugfixes may be acceped until a new owner has been found.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants