io::Stderr should be line-buffered by default, not unbuffered

[zackw]

std::io::stderr does no buffering at all by default, which means that a Rust program that uses the most obvious way to print error messages (eprintln!) will make individual write syscalls not only for each call to the macro, but for each subcomponent of the formatted string:

$ cat test.rs
fn main() {
  eprint!("partial ");
  eprintln!("line");
  eprintln!("a one and a two and a {} and a {}", 3, 4.0);
}
$ rustc test.rs
$ strace -e trace=write sh -c 'exec ./test 2> /dev/null'
write(2, "partial ", 8)                 = 8
write(2, "line\n", 5)                   = 5
write(2, "a one and a two and a ", 22)  = 22
write(2, "3", 1)                        = 1
write(2, " and a ", 7)                  = 7
write(2, "4", 1)                        = 1
write(2, "\n", 1)                       = 1

This behavior is undesirable in any context where multiple programs might be emitting error messages to the same terminal, logfile, or whatever at the same time (for instance, make -j) because partial lines from different programs can get mixed up with each other. If stderr instead buffered up a full line and wrote it to the OS all at once, then different programs' output could only get mixed together on a line-by-line basis, which is usually much less confusing for a person reading the logs.

This behavior is also troublesome for programs that incrementally parse stderr output; for instance, it may be the reason why emacs' compilation-mode occasionally doesn't detect all of the diagnostics in cargo build output (I don't know if there's an existing bug report for this).

(There is a strong case for having eprint! flush out the partial line that it generates, but that could be handled inside of eprint!.)

(This is closely related to, but not the same as, #60673, which is about stdout. Block buffering is not normally appropriate for stderr, even when writing to a file.)

[cuviper]

FWIW, C++ cerr behaves similarly:

#include <iostream>

using namespace std;

int main() {
    cerr << "partial ";
    cerr << "line" << endl;
    cerr << "a one and a two and a " << 3 << " and a " << 4.0 << endl;
    return 0;
}

$ strace -e write sh -c 'exec ./test 2> /dev/null'
write(2, "partial ", 8)                 = 8
write(2, "line", 4)                     = 4
write(2, "\n", 1)                       = 1
write(2, "a one and a two and a ", 22)  = 22
write(2, "3", 1)                        = 1
write(2, " and a ", 7)                  = 7
write(2, "4", 1)                        = 1
write(2, "\n", 1)                       = 1

The only difference is that "line" and its "\n" are written separately, but this is between using eprintln!'s format string (which can append newline directly) vs. my explicit string and endl.

In C, it does appear to buffer the format string before writing:

#include <stdio.h>

int main() {
    fprintf(stderr, "partial ");
    fprintf(stderr, "line\n");
    fprintf(stderr, "a one and a two and a %d and a %f\n", 3, 4.0);
    return 0;
}

$ strace -e write sh -c 'exec ./test 2> /dev/null'
write(2, "partial ", 8)                 = 8
write(2, "line\n", 5)                   = 5
write(2, "a one and a two and a 3 and a 4."..., 39) = 39

[zackw]

@cuviper Yeah, I guess what I'm saying is I think the C behavior is preferable to the C++ behavior.

[Lucretiel]

FWIW, C++ cerr behaves similarly:

It's worth noting, though, that C++ has both cerr and clog. They both write to stderr, but the latter is buffered; the logic is that error messages should be shown ASAP, even if it means a performance hit.

[nox]

stderr in C is supposed to be unbuffered too AFAIK.

[sfackler]

stderr is unbuffered in C:

Notes
The stream stderr is unbuffered.

https://linux.die.net/man/3/stderr