zig tracking issue

[motiejus]

This issue lists what's important to hermetic_cc_toolchain from upstream zig. It is linked from zig wiki.

Universal headers

Users of hermetic_cc_toolchain tend to run on older versions of glibc (e.g. 2.24 is still known to be used by some as of writing). The bigger the discrepancy between the used header and the target version, the higher the incompatibility risk of each glibc upgrade.

Universal headers would mitigate the glibc upgrade risks. We are eaglerly waiting.

Pre-filling zig cache folder

Bazel is in principle a hermetic toolchain: it needs to know all inputs and outputs of each target (target in this context is a Makefile-target). In the ideal world, when building a binary, Bazel creates an empty directory, puts in all inputs and outputs, chroots there, and invokes the build command. The only allowed external ABI is linux kernel.

However, zig's cache folder goes against that: for Bazel, Zig cache folder is an opaque directory that we have to pass through. We must share the Zig cache, otherwise each linking action takes an extra ~1 minute (mostly for glibc and libcxx). The current hermetic_cc_toolchain instructions ask the user to pass-through /tmp from their chroot to the host:

build --sandbox_add_mount_pair=/tmp

ZIG_GLOBAL_CACHE_DIR is configured to be /tmp/zig-cache. With this flag, different Zig invocations can share the cache. This approach has the following disadvantages:

• if /tmp is shared across different Unix users, it will spew "permission denied" errors.
• does not work well with remote execution.
• every user of hermetic_cc_toolchain is non-hermetic by design. If you look at the name of the project, it's more than ironic.
• folks from Bazel community, please chime in more in the comments.

Note that we only care for glibc (including crt*), libcxx and similar. We do not care about caching of the intermediate object files that the user builds: Bazel is aware of them and caches it on it's own.

Suggestion

Create a zig sub-command that pre-compiles all the necessary "system" stubs for a given -target and -mcpu into a directory, tarball or whatever. The following zig cc commands should accept the path to the artifact as an environment variable. That way, Bazel knows all the requisites it needs to build a specific target.

Prior art: that's what rules_go does for Go. In Go case it's simpler, because there is ineed a single .a file for each GOOS (linux, windows, etc) and GOARCH (arm64, amd64, etc) combination.

Possible complications

System stubs rely on more than -target and -mcpu: optimization flags, debug info and much more are all taken into account when compiling, say, crt*.o. We need to find the balance between the number of the flags we want to support versus building the most optimized intermediate artifact that zig dynamically or statically links.