fix(build-std): remove hack on creating virtual std workspace #14358

weihanglo · 2024-08-06T03:47:15Z

What does this PR try to resolve?

Starting from rust-lang/rust#128534 (nightly-2024-08-05),
standard library has its own Cargo workspace.
Hence -Zbuild-std no longer need to fake a virtual workspace.

This also adjusts Cargo.toml in mock-std to align with std's Cargo.toml

How should we test and review this PR?

I haven't done any e2e test for -Zbuild-std manually. Would appreciate if someone would like to.

Additional information

There might be more clean-up we could do.

Starting from rust-lang/rust#128534 (nightly-2024-08-05), stnadard library has its own Cargo workspace. Hence `-Zbuild-std` no longer need to fake a virtual workspace. This also adjusts Cargo.toml in `mock-std` to align with std's Cargo.toml

It was designed for `-Zbuild-std`

rustbot · 2024-08-06T03:47:20Z

r? @ehuss

rustbot has assigned @ehuss.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

weihanglo · 2024-08-06T03:49:03Z

See build failures from other PRs that this patch aims to fix:

glandium · 2024-08-06T04:36:27Z

---- expected: tests/build-std/main.rs:116:27
++++ actual:   stderr
        1 + [UPDATING] crates.io index

That's an interesting error. -Zbuild-std should probably not touch crates.io

bjorn3 · 2024-08-06T05:39:47Z

src/cargo/core/compiler/standard_lib.rs

    // TODO: Consider doing something to enforce --locked? Or to prevent the
    // lock file from being written, such as setting ephemeral.
-    let mut std_ws = Workspace::new_virtual(src_path, current_manifest, virtual_manifest, gctx)?;
+    let mut std_ws = Workspace::new(&std_ws_manifest_path, gctx)?;


Can you build this workspace with --locked to avoid masking any future case where the lockfile is not up to date for whatever reason?

I understand the needs, though that requires more tweaks. I slightly lean toward get this fix first to unblock Cargo CI pipeline and then we make it --locked.

Regardless, which one is preferable?

Build succeeded. Lockfile got update but was written to file system.

Build failed because of any lockfile update.

Note that tests/build-std/main.rs should have guarded basic cases that won't update registry index, though they are not run in rust-lang/rust main CI pipeline.

I think the build should fail. An outdated lockfile means that something went wrong when building the rust-src component (it should be a verbatim copy of the lockfile used while building the standard library in the rust build system, which uses --locked on CI afaik) or that the rust-src component was (accidentally) tampered with on the user's system.

Tracking issue of it: rust-lang/wg-cargo-std-aware#38

epage · 2024-08-06T12:10:28Z

@bors r+

bors · 2024-08-06T12:10:32Z

📌 Commit 7902bf8 has been approved by epage

It is now in the queue for this repository.

bors · 2024-08-06T12:11:44Z

⌛ Testing commit 7902bf8 with merge 2cd657c...

bors · 2024-08-06T12:42:39Z

☀️ Test successful - checks-actions
Approved by: epage
Pushing 2cd657c to master...

Update cargo 3 commits in fa646583675d7c140482bd906145c71b7fb4fc2b..94977cb1fab003d45eb5bb108cb5e2fa0149672a 2024-08-02 16:08:06 +0000 to 2024-08-06 21:42:10 +0000 - Don't specify the depedency name in the `cargo add` inferred name test (rust-lang/cargo#14357) - Fix renamed disallowed cfg lint name (rust-lang/cargo#14352) - fix(build-std): remove hack on creating virtual std workspace (rust-lang/cargo#14358) r? ghost

rust-lang#14358 didn't check the correct Cargo.lock existence Perhaps it was there so the test passed, but after a new nightly is out it is gone. ``` Blocking waiting for file lock on package cache error: "/home/user/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try: rustup component add rust-src --toolchain nightly-aarch64-apple-darwin note: run with `RUST_BACKTRACE=1` environment variable to display a backtrac ```

rust-lang#14358 didn't check the correct Cargo.lock existence Perhaps it was there so the test passed, but after a new nightly is out it is gone. ``` Blocking waiting for file lock on package cache error: "/home/user/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try: rustup component add rust-src --toolchain nightly-aarch64-apple-darwin note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ```

fix: std Cargo.lock moved to `library` dir #14358 didn't check the correct Cargo.lock existence Perhaps it was there so the test passed, but after a new nightly is out it is gone. ``` Blocking waiting for file lock on package cache error: "/home/user/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try: rustup component add rust-src --toolchain nightly-aarch64-apple-darwin note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` Fixes rust-lang/rust#128808 ### How to test the change: The current nightly `cargo 1.82.0-nightly (94977cb 2024-08-06)` would fail when running ``` cargo +nightly build -Zbuild-std --target <host-triple> ``` After this fix, it should just work ``` RUSTC=~/.rustup/toolchains/nightly-<host-triple>/bin/rustc ./target/debug/cargo build -Zbuild-std --target <host-triple> ```

Update cargo 3 commits in fa646583675d7c140482bd906145c71b7fb4fc2b..94977cb1fab003d45eb5bb108cb5e2fa0149672a 2024-08-02 16:08:06 +0000 to 2024-08-06 21:42:10 +0000 - Don't specify the depedency name in the `cargo add` inferred name test (rust-lang/cargo#14357) - Fix renamed disallowed cfg lint name (rust-lang/cargo#14352) - fix(build-std): remove hack on creating virtual std workspace (rust-lang/cargo#14358) r? ghost

glandium · 2024-09-03T07:27:48Z

FWIW, this didn't change anything wrt -Zbuild-std not working with the use of a vendoring tree. Parts of rust-lang/rust#78790 and/or #8834 are probably still necessary.

weihanglo · 2024-09-03T07:46:18Z

@glandium
Could you open a new issue with reproduction if it is not supported?

glandium · 2024-09-03T08:03:43Z

The issue already exists as rust-lang/wg-cargo-std-aware#23

rust-lang#14358 didn't check the correct Cargo.lock existence Perhaps it was there so the test passed, but after a new nightly is out it is gone. ``` Blocking waiting for file lock on package cache error: "/home/user/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/Cargo.lock" does not exist, unable to build with the standard library, try: rustup component add rust-src --toolchain nightly-aarch64-apple-darwin note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ```

weihanglo added 3 commits August 5, 2024 23:22

test(build-std): expand assertion to ensure no index update

9af3930

refactor: remove unused Workspace::new_virtual

7902bf8

It was designed for `-Zbuild-std`

rustbot assigned ehuss Aug 6, 2024

rustbot added A-workspaces Area: workspaces S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 6, 2024

weihanglo mentioned this pull request Aug 6, 2024

Move the standard library to a separate workspace rust-lang/rust#128534

Merged

bjorn3 reviewed Aug 6, 2024

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 6, 2024

bors merged commit 2cd657c into rust-lang:master Aug 6, 2024
24 checks passed

weihanglo deleted the build-std branch August 6, 2024 12:45

bors mentioned this pull request Aug 6, 2024

Verify build-std resolve against original lockfile #13916

Closed

This was referenced Aug 6, 2024

Consider doing a forced lock of the standard library. rust-lang/wg-cargo-std-aware#38

Open

Update cargo rust-lang/rust#128763

Merged

rustbot added this to the 1.82.0 milestone Aug 7, 2024

weihanglo mentioned this pull request Aug 8, 2024

fix: std Cargo.lock moved to library dir #14370

Merged

weihanglo added the Z-build-std Nightly: build-std label Aug 8, 2024

davidhewitt mentioned this pull request Aug 9, 2024

Please release a version with rustc-build-sysroot 0.5.3 RalfJung/cargo-careful#36

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(build-std): remove hack on creating virtual std workspace #14358

fix(build-std): remove hack on creating virtual std workspace #14358

weihanglo commented Aug 6, 2024

rustbot commented Aug 6, 2024

weihanglo commented Aug 6, 2024 •

edited

Loading

glandium commented Aug 6, 2024

bjorn3 Aug 6, 2024

weihanglo Aug 6, 2024

bjorn3 Aug 6, 2024

weihanglo Aug 6, 2024

epage commented Aug 6, 2024

bors commented Aug 6, 2024

bors commented Aug 6, 2024

bors commented Aug 6, 2024

glandium commented Sep 3, 2024

weihanglo commented Sep 3, 2024

glandium commented Sep 3, 2024

fix(build-std): remove hack on creating virtual std workspace #14358

fix(build-std): remove hack on creating virtual std workspace #14358

Conversation

weihanglo commented Aug 6, 2024

What does this PR try to resolve?

How should we test and review this PR?

Additional information

rustbot commented Aug 6, 2024

weihanglo commented Aug 6, 2024 • edited Loading

glandium commented Aug 6, 2024

bjorn3 Aug 6, 2024

Choose a reason for hiding this comment

weihanglo Aug 6, 2024

Choose a reason for hiding this comment

bjorn3 Aug 6, 2024

Choose a reason for hiding this comment

weihanglo Aug 6, 2024

Choose a reason for hiding this comment

epage commented Aug 6, 2024

bors commented Aug 6, 2024

bors commented Aug 6, 2024

bors commented Aug 6, 2024

glandium commented Sep 3, 2024

weihanglo commented Sep 3, 2024

glandium commented Sep 3, 2024

weihanglo commented Aug 6, 2024 •

edited

Loading