Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[processor/logstransform]: Logstransform processor does not properly drain data #31159

Open
BinaryFissionGames opened this issue Feb 9, 2024 · 5 comments
Labels
bug Something isn't working processor/logstransform Logs Transform processor Stale

Comments

@BinaryFissionGames
Copy link
Contributor

Component(s)

processor/logstransform

What happened?

Description

The logstransform processor does not drain log data that it has accepted in ConsumeLogs.
This means that when the collector is stopped, some log data may be lost.

I noticed this when I was writing a test to reproduce #31139 that I was getting 0 logs half the time, 500 logs the other half here:
975a39d#diff-2de55e867acbc6d11c6db4e7470806017e911f59d334e99352f19f7c18ed35b6R287

There are a couple of places where we bail out early when stopping, but I think it's preferable to keep going in these areas so we don't drop any logs on shutdown, e.g.:

for {
select {
case <-c.stopChan:
return
case workerItem, ok := <-c.workerChan:
if !ok {
return
}
select {
case c.entriesChan <- convertFromLogs(workerItem):
case <-c.stopChan:
return
}
}
}

Collector version

8ccadc3

Environment information

No response

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

@BinaryFissionGames BinaryFissionGames added bug Something isn't working needs triage New item requiring triage labels Feb 9, 2024
@github-actions github-actions bot added the processor/logstransform Logs Transform processor label Feb 9, 2024
Copy link
Contributor

github-actions bot commented Feb 9, 2024

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@BinaryFissionGames
Copy link
Contributor Author

This also seems related to #31074, since stanza adapter code is shared with this component.

@djaglowski djaglowski removed the needs triage New item requiring triage label Feb 12, 2024
Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Copy link
Contributor

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working processor/logstransform Logs Transform processor Stale
Projects
None yet
Development

No branches or pull requests

2 participants