`tests/ui/allocator/no_std-alloc-error-handler*` fail when `download-rustc` is enabled #108767

jyn514 · 2023-03-05T10:30:59Z

cc #81930

I tried this:

x test tests/ui/allocator/no_std-alloc-error-handler*

I expected to see this happen: The tests pass.

Instead, this happened: The tests fail:

Check compiletest suite=ui mode=ui (aarch64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)

running 22 tests
iiiiiiiiiiiiiiiiiiiiFF
failures:

---- [ui] tests/ui/allocator/no_std-alloc-error-handler-default.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/home/gh-jyn514/rust/tests/ui/allocator/no_std-alloc-error-handler-default.rs" "-Zthreads=1" "--target=aarch64-unknown-linux-gnu" "-O" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Zdeduplicate-diagnostics=no" "-Cstrip=debuginfo" "--remap-path-prefix=/home/gh-jyn514/rust/tests/ui=fake-test-src-base" "-C" "prefer-dynamic" "-o" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-default/a" "-Crpath" "-Cdebuginfo=0" "-Lnative=/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-default/auxiliary" "-C" "panic=abort"
stdout: none
--- stderr -------------------------------
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

error: aborting due to previous error

For more information about this error, try `rustc --explain E0464`.
------------------------------------------


---- [ui] tests/ui/allocator/no_std-alloc-error-handler-custom.rs stdout ----

error: test compilation failed although it shouldn't!
status: exit status: 1
command: "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "/home/gh-jyn514/rust/tests/ui/allocator/no_std-alloc-error-handler-custom.rs" "-Zthreads=1" "--target=aarch64-unknown-linux-gnu" "-O" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Zdeduplicate-diagnostics=no" "-Cstrip=debuginfo" "--remap-path-prefix=/home/gh-jyn514/rust/tests/ui=fake-test-src-base" "-C" "prefer-dynamic" "-o" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-custom/a" "-Crpath" "-Cdebuginfo=0" "-Lnative=/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/test/ui/allocator/no_std-alloc-error-handler-custom/auxiliary" "-C" "panic=abort"
stdout: none
--- stderr -------------------------------
error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-custom.rs:16:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

error: aborting due to previous error

For more information about this error, try `rustc --explain E0464`.
------------------------------------------



failures:
    [ui] tests/ui/allocator/no_std-alloc-error-handler-custom.rs
    [ui] tests/ui/allocator/no_std-alloc-error-handler-default.rs

test result: FAILED. 0 passed; 2 failed; 20 ignored; 0 measured; 14535 filtered out; finished in 0.13s

These tests are pretty complicated but the error message looks correct:

error[E0464]: multiple candidates for `rlib` dependency `libc` found
  --> fake-test-src-base/allocator/no_std-alloc-error-handler-custom.rs:16:1
   |
LL | extern crate libc;
   | ^^^^^^^^^^^^^^^^^^
   |
   = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib
   = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta

I'm not sure why both files are there and particularly why they have different hashes. tar -tf build/cache/35636f9459720513ca4ed19373a4a7c9e0ea3c46//rustc-dev-nightly-aarch64-unknown-linux-gnu.tar.xz | grep liblibc shows that
rustc-dev-nightly-aarch64-unknown-linux-gnu/rustc-dev/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta is being downloaded from CI, and I guess the .rlib is coming from ensure(compile::Std) somewhere - maybe we should clear the rustlib/ directory before doing that?

Meta

HEAD is 35636f9

The text was updated successfully, but these errors were encountered:

KittyBorgX · 2023-03-07T03:04:10Z

@rustbot claim

jyn514 · 2023-04-09T20:46:04Z

Ok, it turns out both versions are coming from CI:

; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc
rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta

I am ... surprised this hasn't broken anything else? I don't know why only this specific test would be failing.

jyn514 · 2023-04-09T21:20:40Z

The difference between this test and others is that somehow in other tests rustc_resolve knows ahead of time which hash it should use:

INFO rustc_metadata::creader resolving dep crate libc hash: `75267ce7b5b7c8d8` extra filename: `-68a2d9e195dd6ed2`
INFO rustc_metadata::creader resolving crate `libc`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::locator rlib reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::creader register crate `libc` (cnum = 6. private_dep = false)

for the allocator tests, it doesn't know, and gives an ambiguity error:

INFO rustc_metadata::creader resolving crate `libc`
INFO rustc_metadata::creader falling back to a load
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::locator lib candidate: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
INFO rustc_metadata::locator rmeta reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta
INFO rustc_metadata::locator rlib reading metadata from: /home/jyn/src/rust2/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib
INFO rustc_metadata::creader resolving crate `core`
error[E0464]: multiple candidates for `rlib` dependency `libc` found

jyn514 · 2023-04-09T21:28:06Z

ok, here's a minimal reproducer:

#![crate_type = "lib"]
#![no_std]
#![feature(rustc_private)]
extern crate libc;

I think what's going on is for the working tests, the implicit extern crate std injected by the compiler also loads libc, because std depends on libc:

; cargo tree -p std -i libc --depth 1
libc v0.2.140
├── panic_abort v0.0.0 (/home/jyn/src/rust2/library/panic_abort)
├── std v0.0.0 (/home/jyn/src/rust2/library/std)

Then, when the compiler sees extern crate libc, it's already seen the version std depends on and resolves the ambiguity by reusing that crate.

But for the failing test, we never load std, so we don't have a pre-existing hash to default to, so it gives a hard ambiguity error.

…tlarsan68 Fix no_std tests that load libc from the sysroot when download-rustc is enabled There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`): ```rust #![crate_type = "lib"] #![no_std] #![feature(rustc_private)] extern crate libc; ``` Before, this would give an error about duplicate versions of libc: ``` error[E0464]: multiple candidates for `rlib` dependency `libc` found --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1 | LL | extern crate libc; | ^^^^^^^^^^^^^^^^^^ | = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib = note: candidate #2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta ``` Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`: ``` ; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib ; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta ``` The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`. To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot. This also allows reverting the hack in rust-lang#110121; now that we only copy rustc-dev on-demand, we can correctly add the `Rustc` check artifacts into the sysroot, so that this works correctly even when `download-rustc` is forced to `true` and some tool depends on a local change to `compiler`. --- See rust-lang#108767 (comment) for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important. Fixes rust-lang#108767.

Fix no_std tests that load libc from the sysroot when download-rustc is enabled There were a series of unfortunate interactions here. Here's an MCVE of the test this fixes (committed as `tests/ui/meta/no_std-extern-libc.rs`): ```rust #![crate_type = "lib"] #![no_std] #![feature(rustc_private)] extern crate libc; ``` Before, this would give an error about duplicate versions of libc: ``` error[E0464]: multiple candidates for `rlib` dependency `libc` found --> fake-test-src-base/allocator/no_std-alloc-error-handler-default.rs:15:1 | LL | extern crate libc; | ^^^^^^^^^^^^^^^^^^ | = note: candidate #1: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-358db1024b7d9957.rlib = note: candidate rust-lang#2: /home/gh-jyn514/rust/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib/liblibc-ebc478710122a279.rmeta ``` Both these versions were downloaded from CI, but one came from the `rust-std` component and one came from `rustc-dev`: ``` ; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc rust-std-nightly-x86_64-unknown-linux-gnu/rust-std-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-68a2d9e195dd6ed2.rlib ; tar -tf build/cache/f2d9a3d0771504f1ae776226a5799dcb4408a91a/rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | grep liblibc rustc-dev-nightly-x86_64-unknown-linux-gnu/rustc-dev/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-f226c9fbdd92a0fd.rmeta ``` The fix was to only copy files from `rust-std` unless a Step explicitly requests for the `rustc-dev` components to be available by calling `builder.ensure(compile::Rustc)`. To avoid having to re-parse the `rustc-dev.tar.xz` tarball every time, which is quite slow, this adds a new `build/host/ci-rustc/.rustc-dev-contents` cache file which stores only the names of files we need to copy into the sysroot. This also allows reverting the hack in rust-lang/rust#110121; now that we only copy rustc-dev on-demand, we can correctly add the `Rustc` check artifacts into the sysroot, so that this works correctly even when `download-rustc` is forced to `true` and some tool depends on a local change to `compiler`. --- See rust-lang/rust#108767 (comment) for why `no_std` is required for the MCVE test to fail; it's complicated and not particularly important. Fixes rust-lang/rust#108767.

Copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component in the sysroot. Signed-off-by: onur-ozkan <work@onurozkan.dev>

Since rust-lang#127188, copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component. Signed-off-by: onur-ozkan <work@onurozkan.dev>

…, r=Kobzol don't copy `.rustc-dev-contents` from CI rustc Since rust-lang#127188, copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component. Blocker for rust-lang#122709

Rollup merge of rust-lang#129311 - onur-ozkan:multiple-candidates-fix, r=Kobzol don't copy `.rustc-dev-contents` from CI rustc Since rust-lang#127188, copying files from `.rustc-dev-contents` regressed rust-lang#108767 again. Since `rustc-src` is already included in the CI rustc sysroot, we don't need to copy these files to have `rustc-src` component. Blocker for rust-lang#122709

jyn514 added E-hard Call for participation: Hard difficulty. Experience needed to fix: A lot. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. labels Mar 5, 2023

jyn514 mentioned this issue Mar 5, 2023

More config.toml.example cleanups #108625

Merged

rustbot assigned KittyBorgX Mar 7, 2023

KittyBorgX removed their assignment Mar 16, 2023

jyn514 added A-metadata Area: Crate metadata A-testsuite Area: The testsuite used to check the correctness of rustc labels Apr 9, 2023

jyn514 self-assigned this Apr 9, 2023

jyn514 added the A-download-rustc Area: Related to the `rust.download-rustc` build option label Apr 12, 2023

jyn514 mentioned this issue Apr 12, 2023

Fix no_std tests that load libc from the sysroot when download-rustc is enabled #110229

Merged

bors closed this as completed in 71f04bd Apr 19, 2023

intelfx mentioned this issue Mar 10, 2024

tests/ui/allocator/no_std-*, tests/ui/extern-flag/*force-extern, tests/ui/meta/no_std-* fail when bootstrapping stage 2 #122263

Open

onur-ozkan mentioned this issue Aug 20, 2024

don't copy .rustc-dev-contents from CI rustc #129311

Merged

Kobzol mentioned this issue Sep 4, 2024

copy rustc rustlib artifacts from ci-rustc #129942

Merged

onur-ozkan mentioned this issue Sep 17, 2024

skip in-tree compiler build for llvm-bitcode-linker if ci-rustc is on #130479

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`tests/ui/allocator/no_std-alloc-error-handler*` fail when `download-rustc` is enabled #108767

`tests/ui/allocator/no_std-alloc-error-handler*` fail when `download-rustc` is enabled #108767

jyn514 commented Mar 5, 2023 •

edited

Loading

KittyBorgX commented Mar 7, 2023

jyn514 commented Apr 9, 2023

jyn514 commented Apr 9, 2023 •

edited

Loading

jyn514 commented Apr 9, 2023 •

edited

Loading

tests/ui/allocator/no_std-alloc-error-handler* fail when download-rustc is enabled #108767

tests/ui/allocator/no_std-alloc-error-handler* fail when download-rustc is enabled #108767

Comments

jyn514 commented Mar 5, 2023 • edited Loading

Meta

KittyBorgX commented Mar 7, 2023

jyn514 commented Apr 9, 2023

jyn514 commented Apr 9, 2023 • edited Loading

jyn514 commented Apr 9, 2023 • edited Loading

`tests/ui/allocator/no_std-alloc-error-handler*` fail when `download-rustc` is enabled #108767

`tests/ui/allocator/no_std-alloc-error-handler*` fail when `download-rustc` is enabled #108767

jyn514 commented Mar 5, 2023 •

edited

Loading

jyn514 commented Apr 9, 2023 •

edited

Loading

jyn514 commented Apr 9, 2023 •

edited

Loading