[pkg/stanza] not drop logs on timestamp parsing error

[newly12]

Component(s)

pkg/stanza

Is your feature request related to a problem? Please describe.

in case of timestmap parsing error, fall back to observed timestamp instead of dropping logs

Describe the solution you'd like

add a config for this behavior

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• pkg/stanza: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

I think there are two separate proposals here.

1. Ensure logs are not dropped. This should be possible today but it sounds like you've experienced otherwise? Would you be willing to provide a test case which demonstrates this?

2. Fall back to observed timestamp. This is not the appropriate behavior as defined by our data model. The timestamp should remain empty and it is up to exporters to determine what to do. See ObservedTimestamp: For converting OpenTelemetry log data to formats that support only one timestamp or when receiving OpenTelemetry log data by recipients that support only one timestamp internally the following logic is recommended: Use Timestamp if it is present, otherwise use ObservedTimestamp.

[crobert-1]

@newly12 Can you confirm whether or not logs were being dropped, as asked in question 1 above?

[newly12]

@djaglowski sorry for late response. you are correct, if I have on_error set to send, the entry is still sent.

Fall back to observed timestamp. This is not the appropriate behavior as defined by our data model. The timestamp should remain empty and it is up to exporters to determine what to do. See ObservedTimestamp: For converting OpenTelemetry log data to formats that support only one timestamp or when receiving OpenTelemetry log data by recipients that support only one timestamp internally the following logic is recommended: Use Timestamp if it is present, otherwise use ObservedTimestamp.

sounds good.