Measure binary size impact of implicit caller location

[anp]

The largest unanswered questions about implicit caller location are about the binary size impact of the feature and how to offer users control over it.

There are three ways I see that #[track_caller] inflates binaries:

1. code bloat from panicking branches that were previously fused
2. encoding the Location structs themselves
3. encoding the &'static str for every filename referenced by a Location

We need to determine the impact to the ecosystem for each of these before deciding on mitigation strategy.

(1) was the main concern in the RFC but it was predicated on the assumption that #[track_caller] would affect inlining (an artifact of the implementation proposed). Since it doesn't affect inlining now, the only lost optimization in panicking branches would be for cases where LLVM previously used knowledge of a single constant panic location to make the failure path of a function smaller.

We've not yet had any reports of this and some ad-hoc measurements don't show any difference. This matches my mental model of LLVM's ability to optimize calling conventions which is "very expensive magic". I'm open to ideas for how to better assess this, including "wait and see".

TLDR Re (2) and (3),

Locations are 24 bytes when size_of::<usize>() == size_of::<u64>() and we don't currently measure how many we encode. We should put these in their own section to make it easier to measure.

Which filenames do we encode? In practice: probably all source files. We should put these in their own section too.

[anp]

@eddyb could you provide some pointers on how to put static allocations in their own sections? I assume it's a change to librustc_codegen_ssa?

[eddyb]

@anp Yeah, look at how #[link_section] is handled and do the same.

[Mark-Simulacrum]

Have we looked at a diff of a stage1 (or stage2) rustc build? rustc has a lot of expect/unwrap etc code so I suspect if there's essentially no delta then I'm not too worried. I haven't noticed nightlies getting much bigger so I suspect we in practice aren't growing by too much.

[eddyb]

@Mark-Simulacrum this reminds me that we're not measuring size changes on perf.rust-lang.org.

[anp]

Filed: rust-lang/rustc-perf#643

[anp]

OK, here's a gist documenting an instrumentation-based approach. TLDR: rustc with #[track_caller] in the standard library uses ~0.68% of its binary size on the static Location allocations and their filenames but the binary is still quite a bit smaller than a matching stage1 build. I haven't yet been able to observe a conclusive net increase in binary size due to the feature.

The fact that intuitively we know this could cause a net binary bloat makes me think that we want to polish up the custom sections patch in my gist. It's useful instrumentation for the future.

I'm not sure whether this is sufficient motivation to block stabilization on #70580. Thoughts?

[tmandry]

The best way of calculating the impact of the feature itself would be doing two stage2 builds, one with track_caller turned on and one with it off. From your analysis now we know it's no more than 810 kB for rustc.

Percentage of the overall binary size is probably useful, but it's hard to say without seeing more binaries. For a large enough system I'd guess it'd be approximately the same overall, but for smaller binaries it could take up a higher percentage.

[hudson-ayers]

I took a stab at measuring this for a TockOS kernel binary. I did so by following the recommendation at #70580 (comment) , modifying those lines to replace all filenames with "", all line numbers with 0, and all column numbers with 0. I then built a new stage 1 compiler, and used that compiler to re-compile the tock "Imix" kernel binary (https://github.com/tock/tock/tree/master/boards/imix).

I chose to use a Tock kernel binary as the subject of this test because many uses of Tock are severely size-constrained. The Tock Imix kernel is required to remain smaller than 180 kB to fit in the space available on that platform. Other Tock deployments require even smaller sizes. I consider binary size impact on these classes of embedded devices more important than the impact on traditional binaries, where size is less likely to be a significant concern anyway.

• Tock Imix kernel size, rust nightly-2021-10-11: 175,304 bytes
• Tock Imix kernel size, all Panic locations fused: 158,356 bytes (-16,948 bytes / -9.6%)
• Tock Imix kernel size, keeping filenames but setting all line/column numbers to 0: 163,752 bytes (-11,552 bytes / -6.6%)
• Tock Imix kernel size, keeping only line numbers but setting filenames to "" and column to 0: 170,040 bytes (-5,264 bytes / -3%)
• Tock Imix kernel size, keeping line numbers + column numbers, setting filenames to "": 172,856 bytes (-2,448 bytes / -1.4%)
• Tock Imix kernel size, keeping filename only but trimming paths (e.g. kernel/src/sched.rs --> sched.rs): 162,196 bytes (-13,108 bytes / -7.5%)

Key things to note:

• Tock uses -Z build-std=core,compiler-builtins by default to rebuild the core library with size optimizations. This means that not only does this approach remove all panic strings from the Tock binary, but also from the core library. Severely size constrained applications are likely to do this as well.
• Tock uses remap-path-prefix for both the Tock root directory and the core library (necessary bc of the use of build-std), so paths are already shorter than they might be in more typical projects
• When this issue was first created, I suspect the goal was to measure the overhead of #[track_caller] compared to the status quo before #[track_caller]. However, since https://github.com/libstd uses core::panic::Location where possible. #67137 , all panic locations are derived via #[track_caller], so any estimate of code size without #[track_caller] is really an estimate of code size with no panic locations in the binary at all (as opposed to an estimate where all unwrap/expect calls just use the same location in the core library).

Finally, I also took a measurement of code size with the compiler configured such that requires_caller_location() always returns false. This means that functions annotated with #[track_caller] do not get an extra argument added to pass a location. This is nice, because if we aren't using the location anyway the extra argument is just wasted size. The downside is that this change would break code not using build-std, and would break dylibs (I think).

• Tock imix kernel size, track_caller completely disabled: 157,204 bytes (-18,100 bytes / -10.3%)

[benpye]

I think it would be interesting to allow putting the CallerLocation allocations in a different section - similar to the instrumentation change in #70579 (comment) . My thinking is that I would log the pointer to the core::panic::Location structure which would be in a no-load section of the binary - and so would be stripped from the flash image. A post processing tool could them take the log messages + the unstripped ELF and give you the original filename:line:column.