Batch Processor: Log Support #1723
Conversation
keitwb
requested review from
bogdandrutu,
dmitryax,
james-bebbington,
nilebox,
owais,
pjanotti and
tigrannajaryan
as code owners
Codecov Report
@@ Coverage Diff @@
## master #1723 +/- ##
=======================================
Coverage 92.37% 92.38%
=======================================
Files 258 258
Lines 18192 18239 +47
=======================================
+ Hits 16805 16850 +45
- Misses 975 976 +1
- Partials 412 413 +1
Continue to review full report at Codecov.
|
| } | ||
|
|
||
| func (bm *batchLogs) size() int { | ||
| return bm.logData.ResourceLogs().Len() |
There was a problem hiding this comment.
What is size() supposed to return?
There was a problem hiding this comment.
Based on the metrics batcher, the size in bytes of the internal data representation of logs. I added a helper method to pdata to derive it without the batch processor having to convert to internal.
There was a problem hiding this comment.
Do we have batching by size in bytes? I thought we only support batching by the number of items. Can you point me to the code which does byte-based batching? I don't see anything mentioned in the config of the batcher.
There was a problem hiding this comment.
It is just used for telemetry at the detailed level to record how many bytes the batches are.
I'll look into this. The trace correctness test uses batch processor. |
|
@keitwb Please resolve the conflict. |
This mimics the same logic as metrics for logs in the batchprocessor.
Yeah it is probably best to add e2e tests for logging (beyond the existing perf test) with another PR/issue as it looks quite involved. |
This mimics the same logic as metrics for logs in the batchprocessor.