filter_log_to_metrics: add new option 'discard_logs' #9150
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
edsiper
requested review from
leonardo-albertovich,
fujimotos and
koleini
as code owners
| @@ -902,8 +909,7 @@ static int cb_log_to_metrics_filter(const void *data, size_t bytes, | |||
| } | |||
|
|
|||
| ret = flb_input_metrics_append(ctx->input_ins, ctx->tag, strlen(ctx->tag), ctx->cmt); | |||
|
|
|||
| if (ret != 0) { | |||
| if (ret != 0) { | |||
There was a problem hiding this comment.
Is this indentation correct?
edsiper
force-pushed
the
log-to-metrics-discard-logs
branch
from
0885027
to
93daf38
Compare
edsiper
force-pushed
the
log-to-metrics-discard-logs
branch
from
4e0e77b
to
f86d39d
Compare
By default, the filter don't drop the logs from the pipeline, there are cases where only metrics are desired to reduce log volume. This patch introduces a new option called discard_logs (default: false) which when enabled will discard the logs once they are processed. In addition, other changes were made: - replacement of flb_malloc() with flb_calloc() - context structure definition changes: e.g: do not malloc ints - fix double free on cmetrics context destroy (histograms) - general cleanups... NOTE: I noticed that record accessor contexts are being created for fixed patterns (e.g: $kubernetes['pod_name']), we will need to fix this since the creation of those contexts are expensive, is times better to have them in the context and use it on demand. Signed-off-by: Eduardo Silva <edsiper@gmail.com>
edsiper
force-pushed
the
log-to-metrics-discard-logs
branch
from
f86d39d
to
ab9927c
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[updated]
By default, the filter don't drop the logs from the pipeline, there are cases where only metrics are desired to reduce log volume.
This patch introduces a new option called
discard_logs(default: false) which when enabled will discard the logs once they are processed.usage:
In addition, other changes were made:
NOTE: I noticed that record accessor contexts are being created for fixed patterns (e.g: $kubernetes['pod_name']), we will need to fix this since the creation of those contexts are expensive, is times better to have them in the context and use it on demand.
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.