run-make-fulldeps/foreign-exceptions loops on armv7-unknown-linux-gnueabihf

[cuviper]

When I tested 1.40.0-beta.5 on Fedora rawhide, run-make-fulldeps/foreign-exceptions never completed on armv7-unknown-linux-gnueabihf. When I run that test on arm directly, I get:

throwing C++ exception
DropCheck::drop
caught C++ exception
throwing rust panic
thread 'main' panicked at 'Box<Any>', foo.rs:42:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
caught foreign exception in catch (...)
caught foreign exception in catch (...)
caught foreign exception in catch (...)
caught foreign exception in catch (...)
caught foreign exception in catch (...)
caught foreign exception in catch (...)
[... and so on ...]

The same happens when I use official builds of Rust 1.40-beta and 1.41-nightly, so it's not just a problem with Fedora's build of rustc or LLVM. This test didn't exist before #65646, but running 1.39 on the current test just fails even earlier, as expected, "entered unreachable code: catch_unwind should not have caught foreign exception."

I tried compiling the C++ part with GCC 9.2.1 and Clang 9.0.0, both with the same result. That's using libstdc++ either way though, because we don't have LLVM's libc++ available.

cc @Amanieu

[Mark-Simulacrum]

cc @rust-lang/wg-ffi-unwind

[cuviper]

PS: while I initially saw this on Fedora rawhide, my followup testing has been on stable Fedora 31.

[Amanieu]

That is very strange. And this behavior only occurs on ARM?

I'm tempted to say that this is a bug in GCC's unwinder (in libgcc). Can you try this C++-only code to see if it has the same issue?

a.cpp

void foo() { throw 1234; }

b.cpp

void foo();
int main() {
    try {
        foo();
    } catch (...) {
        printf("caught exception in catch (...)\n");
        throw;
    }
}

This should print the message once and then fail with an uncaught exception.

[cuviper]

And this behavior only occurs on ARM?

Yes, the other arches completed and passed that test.

Can you try this C++-only code to see if it has the same issue?

Needs #include <cstdio> for printf, but then it works:

$ g++ -c a.cpp
$ g++ -c b.cpp
$ g++ a.o b.o -o foo
$ ./foo
caught exception in catch (...)
terminate called after throwing an instance of 'int'
Aborted (core dumped)

[cuviper]

This test didn't exist before #65646, but running 1.39 on the current test just fails even earlier, as expected, "entered unreachable code: catch_unwind should not have caught foreign exception."

I tried stable 1.39 again commenting out the call to throw_cxx_exception(), then throw_rust_panic() gets stuck in the same "caught foreign exception in catch" loop on the C++ side. In fact, this happens on every compiler I tried back to 1.13, which is the earliest I could get working at all!

So at least it's not a new problem. Maybe we're generating bad unwind data, or maybe my libgcc has a bug in handling foreign exceptions, I don't know...

[Amanieu]

I think this might be a bug in the way the personality routine in libstdc++ (libsupc++ actually) handles foreign exceptions on ARM.

Can you try using libc++ instead of libstdc++? Just change the linker args for rustc from -lstdc++ -lgcc_s to -lc++ -lc++abi.

[cuviper]

Oh, I found that Fedora does have libc++ packaged, libcxx-devel and libcxxabi-devel. But when I tried to use it simply as you indicate, foreign exceptions don't seem to be caught at all, in either direction. I see the binary still gets gcc_s linked implicitly though, I think because libunwind/build.rs adds it to std. I'll try to get a full build of rustc with just libc++ instead.

[Amanieu]

It's fine to link to gcc_s, since that contains the libunwind implementation. The actual C++ personality functions are in libstd++/libc++abi.

Can you post the test output with libc++?

[cuviper]

Maybe this is obvious, but I have to compile the C++ part with clang++, not g++, or it gets "undefined reference to __cxa_end_cleanup". I wasn't sure whether to also use -stdlib=libc++, but either way the C++ exception is not caught by Rust:

terminating with uncaught exception of type exception
throwing C++ exception

So I commented out throw_cxx_exception again to just try throw_rust_panic:

throwing rust panic
thread 'main' panicked at 'Box<Any>', foo.rs:42:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
fatal runtime error: failed to initiate panic, error 9

[mati865]

@cuviper you may need to explicitly link libcxxabi depending on how libcxx was built. That should fix undefined reference error.

[cuviper]

@mati865 I did use -lc++ -lc++abi -- do you mean more than that?

If anyone else is interested in arm-linux, it would be great to get more hands on this, because that target is not a high priority for me. I will probably disable this test for arm in Fedora in the meantime.

[Amanieu]

I managed to reproduce this on Arch Linux with qemu-arm, so it's definitely not a Fedora-only issue.

[mati865]

@cuviper no, that should be fine. Guess passing --unwindlib=[libgcc|compiler-rt] to the Clang (supported since 9.0.0) doesn't help either?

[cuviper]

The link error was only when using g++ to compile foo.cpp. That much was ok with clang++. I just tried adding --unwindlib to the clang command too, but got "argument unused during compilation".

[infinity0]

I'm hitting this on Debian too. Running foo by itself does result in infinite-loop printing caught foreign exception in catch (...) however the rust test suite is actually running /bin/sh -c LD_LIBRARY_PATH="/home/infinity0/rustc/build/armv5te-unknown-linux-gnueabi/test/run-make-fulldeps/foreign-exceptions/foreign-exceptions:/home/infinity0/rustc/build/armv5te-unknown-linux-gnueabi/stage2/lib/rustlib/armv5te-unknown-linux-gnueabi/lib:/home/infinity0/rustc/build/armv5te-unknown-linux-gnueabi/stage0-bootstrap-tools/armv5te-unknown-linux-gnueabi/release/deps:/usr/lib" /home/infinity0/rustc/build/armv5te-unknown-linux-gnueabi/test/run-make-fulldeps/foreign-exceptions/foreign-exceptions/foo. Directly running this however exits immediately, so I don't know why the test is hanging. I'll see if I can grab the environment from /proc/$pid/env next time.

[infinity0]

Ah nvmd, I just had to wrap the thing in single-quotes to reproduce the loop, the output of "pstree" is misleading.

[infinity0]

Since this test is new in 1.40 I'll just # ignore-arm for the time being in Debian.

[Amanieu]

Should be fixed by #67779