cargo test occasionally leaves a file descriptor open when exec'ing a binary on Mac

[alecmocatta]

Problem
cargo test occasionally leaves a file descriptor open when execing a binary.

Steps
The following as a test with harness = false in the Cargo.toml:

use std::path::Path;

fn main() {
    for fd in 3..1024 {
        assert!(!Path::new(&format!("/dev/fd/{}", fd)).exists());
    }
}

sometimes balks on Mac, specifically at fd 6, which if you try to open it with File::open("/dev/fd/6") is most of the time successful but sometimes returns ErrorKind::PermissionDenied aka EPERM.

Notes
Witnessed on nightlies for the last ~4 months.

I see this on travis (at least on macos-10.13-xcode9.4.1) running this test occasionally but haven't observed locally. Will update if/when I have more details; documenting in case anyone else observes in the meantime.

cargo 1.32.0-nightly (241fac0e3 2018-11-09)

[alexcrichton]

This is likely an unfortunate inevitability because OSX doesn't have atomic open-and-set-CLOEXEC APIs, meaning we're fundamentally always capable of leaking fds to child processes

[dwijnand]

Is that still true? 10 minutes ago I didn't know what CLOEXEC was, but it looks like it's available in modern macOS?

The answers in https://stackoverflow.com/q/1309521/463761 seem to be happily talking about calling fcntl(fd, F_SETFD, FD_CLOEXEC), and https://bugs.python.org/issue26343 says "Apple added O_CLOEXEC to OS X in 10.7" (though the SO answers are from 2009 and 10.7 was released in 2011 🤷‍♂️).

[alecmocatta]

@dwijnand Specifically @alexcrichton is referring to atomic open-and-set-CLOEXEC, rather than just set-CLOEXEC.

For example, whereas on linux and many other unices socket(2) accepts SOCK_CLOEXEC as a flag to atomically make the socket CLOEXEC, on mac this has to be done un-atomically with a call to socket(2) and then a call to fcntl(2).

Ths issue is that if a fork occurs on thread A, while thread B is midway between these two calls, then a fd is leaked into the fork without CLOEXEC set, which is thus inherited by the execd process.

@alexcrichton There are workarounds, e.g. performing the two calls in a fork and sending the fd back to the main process over a UnixDatagram::pair(), or ensuring that no threads remain at the time of the fork(2). Any thoughts as to whether it's worth pursuing, or is this (understandably) a wontfix?

[dwijnand]

TIL. Thank you.

[alexcrichton]

@alecmocatta heh unfortunately we'd have to do that in the standard library so that may be a bit difficult to stomach at this point unfortunately :(

Note though that the window for this multithreaded problem is quite small so in theory it shouldn't happen all that much. If a file descriptor is deterministically leaked into the child on OSX that probably means we have an actual problem which we should fix as well!