out_stackdriver: Add partialSuccess: true to all logs sent to Google Cloud Logging API #8002
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
avilevy18
requested review from
braydonk,
igorpeshansky and
qingling128
as code owners
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
49fec1c
to
759a2e8
Compare
braydonk
changed the title
partialSuccess: true to all logs sent to Google Cloud Logging APIpartialSuccess: true to all logs sent to Google Cloud Logging API
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
759a2e8
to
fea1d2c
Compare
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
da3738b
to
5980688
Compare
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
5980688
to
b379865
Compare
avilevy18
changed the title
partialSuccess: true to all logs sent to Google Cloud Logging API
braydonk
reviewed
Oct 11, 2023
There was a problem hiding this comment.
General note, please change these style guide things where necessary:
- braces opening a new function (and only when opening a new function) should go on a newline
- Line length should not exceed 90 columns, so long lines need to be reformatted so they don't exceed that (see other examples where arguments to functions or in function signatures are put on newlines to fit this requirement)
| @@ -2637,6 +2728,10 @@ static void cb_stackdriver_flush(struct flb_event_chunk *event_chunk, | |||
| } | |||
| else if (c->resp.status >= 400 && c->resp.status < 500) { | |||
| ret_code = FLB_ERROR; | |||
| if(c->resp.status == 400){ | |||
| ret = parse_partial_success_response(c, ctx, &partial_failure); | |||
There was a problem hiding this comment.
Should you be checking the value of ret here to see if it worked?
| @@ -2637,6 +2728,10 @@ static void cb_stackdriver_flush(struct flb_event_chunk *event_chunk, | |||
| } | |||
| else if (c->resp.status >= 400 && c->resp.status < 500) { | |||
| ret_code = FLB_ERROR; | |||
| if(c->resp.status == 400){ | |||
| ret = parse_partial_success_response(c, ctx, &partial_failure); | |||
| cmt_counter_add(ctx->cmt_failed_requests, ts, (double)partial_failure, 1, (char *[]) {name}); | |||
There was a problem hiding this comment.
We're still counting this as one "failed request". We're already going to be increasing this counter further down this function.
The number of failures in the partial failure should be added to cmt_dropped_records. You can access that counter through ctx->ins->cmt_dropped_records.
There was a problem hiding this comment.
I thought the point was to count the request as "successful", but count the records themselves as erroneous?..
| @@ -2637,6 +2728,10 @@ static void cb_stackdriver_flush(struct flb_event_chunk *event_chunk, | |||
| } | |||
| else if (c->resp.status >= 400 && c->resp.status < 500) { | |||
| ret_code = FLB_ERROR; | |||
| if(c->resp.status == 400){ | |||
| ret = parse_partial_success_response(c, ctx, &partial_failure); | |||
| cmt_counter_add(ctx->cmt_failed_requests, ts, (double)partial_failure, 1, (char *[]) {name}); | |||
There was a problem hiding this comment.
I thought the point was to count the request as "successful", but count the records themselves as erroneous?..
braydonk
reviewed
Nov 13, 2023
braydonk
reviewed
Nov 20, 2023
braydonk
previously approved these changes
Nov 20, 2023
|
Before merge, could you flatten all the "Addressed comments" commit into the base one? |
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
b38ead0
to
329a565
Compare
Add partialSuccess: true to all logs sent to Google Cloud Logging API Signed-off-by: avilevy <avilevy@google.com>
avilevy18
force-pushed
the
avilevy-stackdriver-partial-success
branch
from
329a565
to
83f8c4c
Compare
9 tasks
igorpeshansky
pushed a commit
to igorpeshansky/fluent-bit
that referenced
this pull request
Nov 29, 2023
…tial-success out_stackdriver: Add partialSuccess: true to all logs sent to Google Cloud Logging API
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adding
partialSuccess: trueas part of the request to Google Cloud Logging API.Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
valgrind --leak-check=full ./fluent-bit --conf=<path>If this is a change to packaging of containers or native binaries then please confirm it works for all targets.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.