thread 'rustc' has overflowed its stack

[avl]

I just tried the latest nightly (rustc 1.45.0-nightly (ad4bc33 2020-06-01)) and noticed it can't compile one of the crates I work on.

The crate does have an enum with a a huge number of variants (almost 6000), for which I use serde to derive Serialize and Deserialize.

When building, even with just 'cargo check', the build fails with

thread 'rustc' has overflowed its stack
fatal runtime error: stack overflow
error: could not compile `<crate_name>`.

It works on stable 1.43.1.

I tried running rustc under gdb. It didn't crash exactly the same way, I got SIGSEGV instead.

Looking at the callstack in gdb, it is suspiciously deep (15000 functions deep). The top most functions are:

#0  0x00007ff0bae9925b in rustc_mir_build::build::matches::simplify::<impl rustc_mir_build::build::Builder>::simplify_candidate ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#1  0x00007ff0bae9e7ad in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::match_candidates ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#2  0x00007ff0baef7fe4 in <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#3  0x00007ff0baea2f93 in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::test_candidates::{{closure}} ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#4  0x00007ff0bae9b2f1 in rustc_mir_build::build::matches::test::<impl rustc_mir_build::build::Builder>::non_scalar_compare ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#5  0x00007ff0baea1362 in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::match_simplified_candidates ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#6  0x00007ff0baea2fdc in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::test_candidates::{{closure}} ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#7  0x00007ff0bae9b2f1 in rustc_mir_build::build::matches::test::<impl rustc_mir_build::build::Builder>::non_scalar_compare ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#8  0x00007ff0baea1362 in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::match_simplified_candidates ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#9  0x00007ff0baea2fdc in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::test_candidates::{{closure}} ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#10 0x00007ff0bae9b2f1 in rustc_mir_build::build::matches::test::<impl rustc_mir_build::build::Builder>::non_scalar_compare ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so
#11 0x00007ff0baea1362 in rustc_mir_build::build::matches::<impl rustc_mir_build::build::Builder>::match_simplified_candidates ()
   from /home/anders/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/../lib/librustc_driver-b6bcd19e02bce685.so

If I reduce the number of enum variants to 5000, or stop using serde to derive Serialize/Deserialize, it starts working again.

Since it works using stable, but not using nightly, I thought it might be some kind of regression. I totally realize this may not be a bug and that what I'm doing (Serde + giant enum) is just not a good idea. Accept my apologies if this kind of thing shouldn't be reported here!

[Mark-Simulacrum]

cc @oli-obk -- I believe #55617 should have fixed this but I guess we're missing at least this case?

[avl]

As an experiment I tried doubling the number of enum variants and compiling using stable. That still worked. So it seems the code wasn't just on the brink to stop working using the stable compiler.

[avl]

Here is a github repo with a minimized test case for this issue:
https://github.com/avl/repro_enum_serde_stackoverflow72933

[nagisa]

@Mark-Simulacrum It introduced a mechanism for dealing with this kind of issue, but I don’t doubt that we might have missed instances where the compiler is doing something recursive. MIR construction for matches appears to be one of these cases.

[Mark-Simulacrum]

Yeah, that's what I expected, I just recalled seeing some commentary suggesting that we'd perhaps covered all cases. Definitely not unexpected that we didn't, though, nor did I intend to imply we should have :)

[avl]

The beta seems to not have the problem. The problem does not occur with "rustc 1.44.0-beta.4 (02c25b3 2020-05-23)".

[avl]

I saw the 'E-needs-mcve'-tag. @nagisa Did you see my repro-case?

https://github.com/avl/repro_enum_serde_stackoverflow72933

(but maybe you're talking about something more self-contained, not referencing 'serde'?)

[nagisa]

I saw the repo, but it still needs some work to be reduced to a viable regression test case that could be included into the repo along with fix. This is what I’m working on now as I already have the fix itself.

[LeSeulArtichaut]

Assigning P-medium as discussed as part of the Prioritization WG procedure.

[avl]

If anyone else gets here having the same problem, manually implementing Serialize and Deserialize rather than relying on serde-derive, is one way around the problem. I adapted the examples at https://serde.rs/custom-serialization.html for my enum and this made the compile time insignificant. It also improved performance using the old rust considerably.

[LeSeulArtichaut]

Mhh, I remember having seen some other issues related to derive macros recently... cc @petrochenkov, who might know.