Stabilize cache-messages #7450

ehuss · 2019-09-27T00:12:58Z

This stabilizes the -Zcache-messages feature, making it always enabled.

What is stabilized?

This feature is intended to redisplay previous warnings on a "fresh" build instead of displaying no output.

Users have occasionally indicated frustration when they know there are warnings, but no output is displayed when the build is fresh. This also improves the interaction between cargo check and cargo clippy-preview. This also simplifies the code, and opens more possibilities for rustc to send side-channel messages to Cargo.

Cargo will now use JSON output from rustc and rustdoc 100% of the time (rustdoc --test does not use JSON). Previously Cargo would only use JSON for pipelined crates.

Cargo will save the JSON output into a file named output in the .fingerprint directory. This file is only created when the compiler outputs a diagnostic message.

If a crate is being recompiled, and Cargo considers it to be "fresh", it will replay the output file to the console.

Notable changes in this PR

Fixed a bug where replays were erroneously including pipeline rmeta artifact json messages.
clippy-preview is now included in the metadata hash, to force its artifacts to be separate from cargo check.
clippy-preview is no longer force-enabled, under the assumption that caching and fingerprinting is accurate, and the cached messages will be replayed.
clippy-preview's arguments are included in the fingerprint now that it is not force-enabled.
Rustdoc colors and short messages were fixed when pipelining was stabilized, so updated tests.

Closes #6986
Closes #6848
Closes #6664
Closes #2458

Concerns

The only notable issue with this is that switching between short and long human messages only replays the format from the first invocation. That is, if you do cargo build and it generates warnings, then running again with --message-format=short will still show the full length human messages. I'm personally fine with that behavior, even though it is not ideal. I think this feature overall improves the situation (where before no output was displayed). Being able to re-render between short/long is a very difficult problem, and unlikely to be fixable in the foreseeable future.

There was some concern expressed about being able to disable this. I think that would only be necessary if a severe bug is discovered. I do not feel that this change is risky enough to warrant a configurable option. If it does cause a problem, it can be quickly reverted with a one-line change to set OutputOptions::cache_cell to None. Since pipelining has been using JSON output for a while now without complaints, I feel pretty confident in it.

rust-highfive · 2019-09-27T00:13:09Z

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2019-09-27T14:11:12Z

@rfcbot fcp merge

This all looks great to me, thanks @ehuss for pushing on this! Since it's a pretty big behavior change in rustc I'm just gonna confirm with fcp merge here.

rfcbot · 2019-09-27T14:31:38Z

Team member @alexcrichton has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

bors · 2019-09-30T20:58:46Z

☔ The latest upstream changes (presumably #6989) made this pull request unmergeable. Please resolve the merge conflicts.

rfcbot · 2019-09-30T21:41:06Z

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot · 2019-10-10T21:41:23Z

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

alexcrichton · 2019-10-15T13:48:05Z

@bors: r+

bors · 2019-10-15T13:48:06Z

📌 Commit bd73e8d has been approved by alexcrichton

bors · 2019-10-15T13:48:18Z

⌛ Testing commit bd73e8d with merge b03182a...

Stabilize cache-messages This stabilizes the -Zcache-messages feature, making it always enabled. ## What is stabilized? This feature is intended to redisplay previous warnings on a "fresh" build instead of displaying no output. Users have occasionally indicated frustration when they know there are warnings, but no output is displayed when the build is fresh. This also improves the interaction between `cargo check` and `cargo clippy-preview`. This also simplifies the code, and opens more possibilities for `rustc` to send side-channel messages to Cargo. Cargo will now use JSON output from `rustc` and `rustdoc` 100% of the time (`rustdoc --test` does not use JSON). Previously Cargo would only use JSON for pipelined crates. Cargo will save the JSON output into a file named `output` in the `.fingerprint` directory. This file is only created when the compiler outputs a diagnostic message. If a crate is being recompiled, and Cargo considers it to be "fresh", it will replay the output file to the console. ## Notable changes in this PR - Fixed a bug where replays were erroneously including pipeline rmeta artifact json messages. - clippy-preview is now included in the metadata hash, to force its artifacts to be separate from `cargo check`. - clippy-preview is no longer force-enabled, under the assumption that caching and fingerprinting is accurate, and the cached messages will be replayed. - clippy-preview's arguments are included in the fingerprint now that it is not force-enabled. - Rustdoc colors and short messages were fixed when pipelining was stabilized, so updated tests. Closes #6986 Closes #6848 Closes #6664 Closes #2458 ## Concerns The only notable issue with this is that switching between short and long human messages only replays the format from the first invocation. That is, if you do `cargo build` and it generates warnings, then running again with `--message-format=short` will still show the full length human messages. I'm personally fine with that behavior, even though it is not ideal. I think this feature overall improves the situation (where before *no* output was displayed). Being able to re-render between short/long is a very difficult problem, and unlikely to be fixable in the foreseeable future. There was some concern expressed about being able to disable this. I think that would only be necessary if a severe bug is discovered. I do not feel that this change is risky enough to warrant a configurable option. If it does cause a problem, it can be quickly reverted with a one-line change to set `OutputOptions::cache_cell` to `None`. Since pipelining has been using JSON output for a while now without complaints, I feel pretty confident in it.

bors · 2019-10-15T14:14:55Z

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing b03182a to master...

…chton Update cargo, cc, books Update `cc` to include new parallel building support. ## nomicon 3 commits in 4374786f0b4bf0606b35d5c30a9681f342e5707b..5004ad30d69f93553ceef74439fea2159d1f769e 2019-09-17 18:33:21 +0200 to 2019-10-12 19:52:40 +0200 - further clarify C11 and C/C++11 terminology (rust-lang/nomicon#169) - atomics: C11 -> C++20 (rust-lang/nomicon#168) - use sound/unsound terminology ## cargo 12 commits in a429e8cc4614a46a86322a0777a477e2baa83f1c..3a9abe3f065554a7fbc59f440df2baba4a6e47ee 2019-10-04 17:36:12 +0000 to 2019-10-15 15:55:35 +0000 - Fix typo in git index initialization error path (rust-lang/cargo#7512) - Reject feature flags in a virtual workspace. (rust-lang/cargo#7507) - Rename `overrides` to `package` in profiles. (rust-lang/cargo#7504) - Allow publishing with dev-dependencies without a version. (rust-lang/cargo#7333) - Stabilize cache-messages (rust-lang/cargo#7450) - don't lock the package cache when cleaning target dir. (rust-lang/cargo#7502) - Document rustc wrapper (rust-lang/cargo#7499) - Migrate towards exclusively using serde for `Config` (rust-lang/cargo#7456) - Re-enable some MSVC tests. (rust-lang/cargo#7492) - when -Z unstable-options not specified, don't validate --profile (rust-lang/cargo#7489) - Improve error message for cyclic dependencies (rust-lang/cargo#7470) - Some minor clippy fixes. (rust-lang/cargo#7484) ## book 7 commits in 04806c80be0f54b1290287e3f85e84bdfc0b6ec7..9bb8b161963fcebc9d9ccd732ba26f42108016d5 2019-10-01 20:20:22 -0400 to 2019-10-14 18:42:55 -0500 - Make a portion of text less ambiguous (rust-lang/book#2092) - fix heading level (rust-lang/book#2117) - Add missing "of" before `"duck typing"`. (rust-lang/book#1951) - ch18-03: no need to debug print destructured int (rust-lang/book#1991) - Subtle fix to introduce ? on Option in Chapter 9.2 (rust-lang/book#2047) - make wording clearer (rust-lang/book#1976) - Update the version of rand we use ## rust-by-example 5 commits in a6288e7407a6c4c19ea29de6d43f40c803883f21..0b111eaae36cc4b4997684be853882a59e2c7ca7 2019-10-01 10:09:14 -0300 to 2019-10-14 18:34:25 -0300 - Some fix to three files (rust-lang/rust-by-example#1280) - Add reference to Generics (rust-lang/rust-by-example#1281) - Confusing and long sentence (rust-lang/rust-by-example#1282) - Explicit mention of slice range meaning (rust-lang/rust-by-example#1277) - Updated aliasing for nll (rust-lang/rust-by-example#1276)

Update cargo, books ## nomicon 3 commits in 4374786f0b4bf0606b35d5c30a9681f342e5707b..5004ad30d69f93553ceef74439fea2159d1f769e 2019-09-17 18:33:21 +0200 to 2019-10-12 19:52:40 +0200 - further clarify C11 and C/C++11 terminology (rust-lang/nomicon#169) - atomics: C11 -> C++20 (rust-lang/nomicon#168) - use sound/unsound terminology ## cargo 12 commits in a429e8cc4614a46a86322a0777a477e2baa83f1c..3a9abe3f065554a7fbc59f440df2baba4a6e47ee 2019-10-04 17:36:12 +0000 to 2019-10-15 15:55:35 +0000 - Fix typo in git index initialization error path (rust-lang/cargo#7512) - Reject feature flags in a virtual workspace. (rust-lang/cargo#7507) - Rename `overrides` to `package` in profiles. (rust-lang/cargo#7504) - Allow publishing with dev-dependencies without a version. (rust-lang/cargo#7333) - Stabilize cache-messages (rust-lang/cargo#7450) - don't lock the package cache when cleaning target dir. (rust-lang/cargo#7502) - Document rustc wrapper (rust-lang/cargo#7499) - Migrate towards exclusively using serde for `Config` (rust-lang/cargo#7456) - Re-enable some MSVC tests. (rust-lang/cargo#7492) - when -Z unstable-options not specified, don't validate --profile (rust-lang/cargo#7489) - Improve error message for cyclic dependencies (rust-lang/cargo#7470) - Some minor clippy fixes. (rust-lang/cargo#7484) ## book 7 commits in 04806c80be0f54b1290287e3f85e84bdfc0b6ec7..9bb8b161963fcebc9d9ccd732ba26f42108016d5 2019-10-01 20:20:22 -0400 to 2019-10-14 18:42:55 -0500 - Make a portion of text less ambiguous (rust-lang/book#2092) - fix heading level (rust-lang/book#2117) - Add missing "of" before `"duck typing"`. (rust-lang/book#1951) - ch18-03: no need to debug print destructured int (rust-lang/book#1991) - Subtle fix to introduce ? on Option in Chapter 9.2 (rust-lang/book#2047) - make wording clearer (rust-lang/book#1976) - Update the version of rand we use ## rust-by-example 5 commits in a6288e7407a6c4c19ea29de6d43f40c803883f21..0b111eaae36cc4b4997684be853882a59e2c7ca7 2019-10-01 10:09:14 -0300 to 2019-10-14 18:34:25 -0300 - Some fix to three files (rust-lang/rust-by-example#1280) - Add reference to Generics (rust-lang/rust-by-example#1281) - Confusing and long sentence (rust-lang/rust-by-example#1282) - Explicit mention of slice range meaning (rust-lang/rust-by-example#1277) - Updated aliasing for nll (rust-lang/rust-by-example#1276)

Version 1.40.0 (2019-12-19) =========================== Language -------- - [You can now use tuple `struct`s and tuple `enum` variant's constructors in `const` contexts.][65188] e.g. ```rust pub struct Point(i32, i32); const ORIGIN: Point = { let constructor = Point; constructor(0, 0) }; ``` - [You can now mark `struct`s, `enum`s, and `enum` variants with the `#[non_exhaustive]` attribute to indicate that there may be variants or fields added in the future.][64639] For example this requires adding a wild-card branch (`_ => {}`) to any match statements on a non-exhaustive `enum`. [(RFC 2008)] - [You can now use function-like procedural macros in `extern` blocks and in type positions.][63931] e.g. `type Generated = macro!();` - [Function-like and attribute procedural macros can now emit `macro_rules!` items, so you can now have your macros generate macros.][64035] - [The `meta` pattern matcher in `macro_rules!` now correctly matches the modern attribute syntax.][63674] For example `(#[$m:meta])` now matches `#[attr]`, `#[attr{tokens}]`, `#[attr[tokens]]`, and `#[attr(tokens)]`. Compiler -------- - [Added tier 3 support\* for the `thumbv7neon-unknown-linux-musleabihf` target.][66103] - [Added tier 3 support for the `aarch64-unknown-none-softfloat` target.][64589] - [Added tier 3 support for the `mips64-unknown-linux-muslabi64`, and `mips64el-unknown-linux-muslabi64` targets.][65843] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [The `is_power_of_two` method on unsigned numeric types is now a `const` function.][65092] Stabilized APIs --------------- - [`BTreeMap::get_key_value`] - [`HashMap::get_key_value`] - [`Option::as_deref_mut`] - [`Option::as_deref`] - [`Option::flatten`] - [`UdpSocket::peer_addr`] - [`f32::to_be_bytes`] - [`f32::to_le_bytes`] - [`f32::to_ne_bytes`] - [`f64::to_be_bytes`] - [`f64::to_le_bytes`] - [`f64::to_ne_bytes`] - [`f32::from_be_bytes`] - [`f32::from_le_bytes`] - [`f32::from_ne_bytes`] - [`f64::from_be_bytes`] - [`f64::from_le_bytes`] - [`f64::from_ne_bytes`] - [`mem::take`] - [`slice::repeat`] - [`todo!`] Cargo ----- - [Cargo will now always display warnings, rather than only on fresh builds.][cargo/7450] - [Feature flags (except `--all-features`) passed to a virtual workspace will now produce an error.][cargo/7507] Previously these flags were ignored. - [You can now publish `dev-dependencies` without including a `version`.][cargo/7333] Misc ---- - [You can now specify the `#[cfg(doctest)]` attribute to include an item only when running documentation tests with `rustdoc`.][63803] Compatibility Notes ------------------- - [As previously announced, any previous NLL warnings in the 2015 edition are now hard errors.][64221] - [The `include!` macro will now warn if it failed to include the entire file.][64284] The `include!` macro unintentionally only includes the first _expression_ in a file, and this can be unintuitive. This will become either a hard error in a future release, or the behavior may be fixed to include all expressions as expected. - [Using `#[inline]` on function prototypes and consts now emits a warning under `unused_attribute` lint.][65294] Using `#[inline]` anywhere else inside traits or `extern` blocks now correctly emits a hard error. [65294]: rust-lang/rust#65294 [66103]: rust-lang/rust#66103 [65843]: rust-lang/rust#65843 [65188]: rust-lang/rust#65188 [65092]: rust-lang/rust#65092 [64589]: rust-lang/rust#64589 [64639]: rust-lang/rust#64639 [64221]: rust-lang/rust#64221 [64284]: rust-lang/rust#64284 [63931]: rust-lang/rust#63931 [64035]: rust-lang/rust#64035 [63674]: rust-lang/rust#63674 [63803]: rust-lang/rust#63803 [cargo/7450]: rust-lang/cargo#7450 [cargo/7507]: rust-lang/cargo#7507 [cargo/7525]: rust-lang/cargo#7525 [cargo/7333]: rust-lang/cargo#7333 [(rfc 2008)]: https://rust-lang.github.io/rfcs/2008-non-exhaustive.html [`f32::to_be_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.to_be_bytes [`f32::to_le_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.to_le_bytes [`f32::to_ne_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.to_ne_bytes [`f64::to_be_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.to_be_bytes [`f64::to_le_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.to_le_bytes [`f64::to_ne_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.to_ne_bytes [`f32::from_be_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.from_be_bytes [`f32::from_le_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.from_le_bytes [`f32::from_ne_bytes`]: https://doc.rust-lang.org/std/primitive.f32.html#method.from_ne_bytes [`f64::from_be_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.from_be_bytes [`f64::from_le_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.from_le_bytes [`f64::from_ne_bytes`]: https://doc.rust-lang.org/std/primitive.f64.html#method.from_ne_bytes [`option::flatten`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.flatten [`option::as_deref`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.as_deref [`option::as_deref_mut`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.as_deref_mut [`hashmap::get_key_value`]: https://doc.rust-lang.org/std/collections/struct.HashMap.html#method.get_key_value [`btreemap::get_key_value`]: https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#method.get_key_value [`slice::repeat`]: https://doc.rust-lang.org/std/primitive.slice.html#method.repeat [`mem::take`]: https://doc.rust-lang.org/std/mem/fn.take.html [`udpsocket::peer_addr`]: https://doc.rust-lang.org/std/net/struct.UdpSocket.html#method.peer_addr [`todo!`]: https://doc.rust-lang.org/std/macro.todo.html

`cargo fix` had some special handling to deal with ensuring that the `--error-format=json` flag was passed to rustc. However, cargo was changed in rust-lang#7450 to always pass that flag. This special handling is no longer necessary, so this PR removes it.

cleanup: Remove error-format special-case in `cargo fix` `cargo fix` had some special handling to deal with ensuring that the `--error-format=json` flag was passed to rustc. However, cargo was changed in #7450 to always pass that flag. This special handling is no longer necessary, so this PR removes it.

rust-highfive assigned alexcrichton Sep 27, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 27, 2019

ehuss force-pushed the stabilize-cache-messages branch from 5bd089d to da5d65a Compare September 27, 2019 03:01

alexcrichton added the T-cargo Team: Cargo label Sep 27, 2019

rfcbot added proposed-final-comment-period An FCP proposal has started, but not yet signed off. disposition-merge FCP with intent to merge labels Sep 27, 2019

Stabilize cache-messages

bd73e8d

ehuss force-pushed the stabilize-cache-messages branch from da5d65a to bd73e8d Compare September 30, 2019 21:21

rfcbot added the final-comment-period FCP — a period for last comments before action is taken label Sep 30, 2019

rfcbot removed the proposed-final-comment-period An FCP proposal has started, but not yet signed off. label Sep 30, 2019

rfcbot added the finished-final-comment-period FCP complete label Oct 10, 2019

rfcbot removed the final-comment-period FCP — a period for last comments before action is taken label Oct 10, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 15, 2019

bors merged commit bd73e8d into rust-lang:master Oct 15, 2019

This was referenced Oct 15, 2019

Pass the current shell width to rustc #7315

Closed

Rename clippy-preview to clippy and stabilize it #7382

Closed

Stabilize fix --clippy #7383

Closed

ehuss mentioned this pull request Oct 15, 2019

Update cargo, books rust-lang/rust#65445

Merged

apiraino mentioned this pull request Oct 15, 2019

Tracking issue for cache-messages #6986

Closed

4 tasks

SimonSapin mentioned this pull request Oct 18, 2019

"cannot specify the --color option with --json" without --color being passed rust-lang/rust#65558

Closed

mchernyavsky mentioned this pull request Nov 26, 2019

Cached compilation warnings aren't shown in Build Tool Window intellij-rust/intellij-rust#4697

Closed

ebroto mentioned this pull request Dec 8, 2019

No warnings raised when the binary is run twice o0Ignition0o/cargo-scout#36

Closed

bjorn3 mentioned this pull request Jan 9, 2020

Warnings don't show up until cargo clean or file change in VSCode rust-lang/rust-analyzer#1929

Closed

foriequal0 mentioned this pull request Jan 13, 2020

MSBuild parameter -p:RunAnalyzers=false is ignored dotnet/roslyn#40926

Closed

nipunn1313 mentioned this pull request Feb 10, 2020

Update to mem::MaybeUninit and Once::new() sfackler/rust-openssl#1230

Closed

ehuss mentioned this pull request Dec 19, 2020

Message caching does not allow switching --message-format between short and human #9003

Open

ehuss added this to the 1.40.0 milestone Feb 6, 2022

ehuss mentioned this pull request Dec 30, 2023

cleanup: Remove error-format special-case in cargo fix #13224

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Stabilize cache-messages #7450

Stabilize cache-messages #7450

ehuss commented Sep 27, 2019 •

edited

Loading

rust-highfive commented Sep 27, 2019

alexcrichton commented Sep 27, 2019

rfcbot commented Sep 27, 2019 •

edited by ehuss

Loading

bors commented Sep 30, 2019

rfcbot commented Sep 30, 2019

rfcbot commented Oct 10, 2019

alexcrichton commented Oct 15, 2019

bors commented Oct 15, 2019

bors commented Oct 15, 2019

bors commented Oct 15, 2019

Stabilize cache-messages #7450

Stabilize cache-messages #7450

Conversation

ehuss commented Sep 27, 2019 • edited Loading

What is stabilized?

Notable changes in this PR

Concerns

rust-highfive commented Sep 27, 2019

alexcrichton commented Sep 27, 2019

rfcbot commented Sep 27, 2019 • edited by ehuss Loading

bors commented Sep 30, 2019

rfcbot commented Sep 30, 2019

rfcbot commented Oct 10, 2019

alexcrichton commented Oct 15, 2019

bors commented Oct 15, 2019

bors commented Oct 15, 2019

bors commented Oct 15, 2019

ehuss commented Sep 27, 2019 •

edited

Loading

rfcbot commented Sep 27, 2019 •

edited by ehuss

Loading