Documentation for reading rotated logs using file log receiver

[zekai-rai]

Component(s)

receiver/filelog

Describe the issue you're reporting

The readme page of the file log receiver states that "File Log Receiver can read files that are being rotated." I am trying to read more about how it's done and whether there are any limitations.

Using Logrotate, for example, when rotating logs using create mode, it rotates the main log file by renaming the original one and creating a new one under the original name. If the process is actively writing to it while the file being rotated. Log might continue writing to the already rotated file. In a case like that,

1. Should filelog receiver be configured to read from all logs -- main log and rotated logs? I suppose so. If not, filelog receiver will miss some logs.
2. How does filelog receiver avoid reading duplicated content from the rotated files? It relies on the fingerprint setting.
3. If yes to 2, does using fingerprint and reading from all logs add much burden to the pipeline?

This is probably a question for stackoverflow, but I either way, I would like some pointers to read more about the supporting of reading rotated files in general. Thank you.

[github-actions]

Pinging code owners:

• receiver/filelog: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

There is some detailed documentation here which you may find helpful.

In general, the design tries to make minimal assumptions about log rotation mechanics and instead relies on identifying files based on the first N bytes of the file, which should not change under rotation. (So the assumption is just that log files are append-only.)

rotates the main log file by renaming the original one and creating a new one under the original name. If the process is actively writing to it while the file being rotated. Log might continue writing to the already rotated file. In a case like that,

On nix systems, we keep file handles open in between polls. Then we check if the same file was not found in the most recent poll. If not found, then we read the remainder of the file before closing it. This works quite well to ensure that files are consumed completely even in the scenario you describe. In theory, it's possible that a delayed write to the file may be missed, but at some point we must stop watching files that are moved out of the matching pattern.