Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coverage: Initial support for branch coverage instrumentation #122322

Merged
merged 9 commits into from
Mar 15, 2024

Conversation

Zalathar
Copy link
Contributor

@Zalathar Zalathar commented Mar 11, 2024

(This is a review-ready version of the changes that were drafted in #118305.)

This PR adds support for branch coverage instrumentation, gated behind the unstable flag value -Zcoverage-options=branch. (Coverage instrumentation must also be enabled with -Cinstrument-coverage.)

During THIR-to-MIR lowering (MIR building), if branch coverage is enabled, we collect additional information about branch conditions and their corresponding then/else blocks. We inject special marker statements into those blocks, so that the InstrumentCoverage MIR pass can reliably identify them even after the initially-built MIR has been simplified and renumbered.

The rest of the changes are mostly just plumbing needed to gather up the information that was collected during MIR building, and include it in the coverage metadata that we embed in the final binary.

Note that llvm-cov show doesn't print branch coverage information in its source views by default; that needs to be explicitly enabled with --show-branches=count or similar.


The current implementation doesn't have any support for instrumenting if let or let-chains. I think it's still useful without that, and adding it would be non-trivial, so I'm happy to leave that for future work.

@rustbot
Copy link
Collaborator

rustbot commented Mar 11, 2024

r? @nnethercote

rustbot has assigned @nnethercote.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 11, 2024
@rustbot
Copy link
Collaborator

rustbot commented Mar 11, 2024

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

Some changes occurred in match lowering

cc @Nadrieril

@rustbot rustbot added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Mar 11, 2024
@Zalathar
Copy link
Contributor Author

The changes that are most important in terms of reviewer attention are the alterations to non-coverage-specific code in rustc_mir_build, since they have the most potential to step on other people's toes.

(Whereas the coverage-specific modules are mostly self-contained, so there's less impact on the rest of the compiler.)

@Zalathar
Copy link
Contributor Author

Made some style improvements to coverageinfo.rs, since I wasn't happy with Inversions as a name (diff).

@@ -837,7 +842,9 @@ impl<'a, 'tcx> Builder<'a, 'tcx> {
self.fn_span,
self.coroutine,
None,
)
);
body.coverage_hir_branch_info = self.coverage_branch_info.and_then(|b| b.into_done());
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here I had a choice between two ways to set this field:

  • Add an extra parameter to Body::new, and None to a half-dozen callers that don't care.
  • Leave the constructor as-is, and do an explicit field write in the one caller that does care.

I think I prefer the explicit write, as shown here.

@nnethercote
Copy link
Contributor

I skimmed through this and it looks reasonable and is well-commented, and I like that you added the test first so the changes to it are clear later on. But I know very little about MIR and so am not a good reviewer for this. According to triagebot.toml the MIR-iest reviewer we have is...

r? @oli-obk

@rustbot rustbot assigned oli-obk and unassigned nnethercote Mar 12, 2024
@Zalathar
Copy link
Contributor Author

#122226 has been approved, so once that lands I'll rebase this and make the necessary adjustments.

matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 13, 2024
…hercote

coverage: Remove or migrate all unstable values of `-Cinstrument-coverage`

(This PR was substantially overhauled from its original version, which migrated all of the existing unstable values intact.)

This PR takes the three nightly-only values that are currently accepted by `-Cinstrument-coverage`, completely removes two of them (`except-unused-functions` and `except-unused-generics`), and migrates the third (`branch`) over to a newly-introduced unstable flag `-Zcoverage-options`.

I have a few motivations for wanting to do this:

- It's unclear whether anyone actually uses the `except-unused-*` values, so this serves as an opportunity to either remove them, or prompt existing users to object to their removal.
- After rust-lang#117199, the stable values of `-Cinstrument-coverage` treat it as a boolean-valued flag, so having nightly-only extra values feels out-of-place.
  - Nightly-only values also require extra ad-hoc code to make sure they aren't accidentally exposed to stable users.
- The new system allows multiple different settings to be toggled independently, which isn't possible in the current single-value system.
- The new system makes it easier to introduce new behaviour behind an unstable toggle, and then gather nightly-user feedback before possibly making it the default behaviour for all users.
- The new system also gives us a convenient place to put relatively-narrow options that won't ever be the default, but that nightly users might still want access to.
- It's likely that we will eventually want to give stable users more fine-grained control over coverage instrumentation. The new flag serves as a prototype of what that stable UI might eventually look like.

The `branch` option is a placeholder that currently does nothing. It will be used by rust-lang#122322 to opt into branch coverage instrumentation.

---

I see `-Zcoverage-options` as something that will exist more-or-less indefinitely, though individual sub-options might come and go as appropriate. I think there will always be some demand for nightly-only toggles, so I don't see `-Zcoverage-options` itself ever being stable, though we might eventually stabilize something similar to it.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 13, 2024
…hercote

coverage: Remove or migrate all unstable values of `-Cinstrument-coverage`

(This PR was substantially overhauled from its original version, which migrated all of the existing unstable values intact.)

This PR takes the three nightly-only values that are currently accepted by `-Cinstrument-coverage`, completely removes two of them (`except-unused-functions` and `except-unused-generics`), and migrates the third (`branch`) over to a newly-introduced unstable flag `-Zcoverage-options`.

I have a few motivations for wanting to do this:

- It's unclear whether anyone actually uses the `except-unused-*` values, so this serves as an opportunity to either remove them, or prompt existing users to object to their removal.
- After rust-lang#117199, the stable values of `-Cinstrument-coverage` treat it as a boolean-valued flag, so having nightly-only extra values feels out-of-place.
  - Nightly-only values also require extra ad-hoc code to make sure they aren't accidentally exposed to stable users.
- The new system allows multiple different settings to be toggled independently, which isn't possible in the current single-value system.
- The new system makes it easier to introduce new behaviour behind an unstable toggle, and then gather nightly-user feedback before possibly making it the default behaviour for all users.
- The new system also gives us a convenient place to put relatively-narrow options that won't ever be the default, but that nightly users might still want access to.
- It's likely that we will eventually want to give stable users more fine-grained control over coverage instrumentation. The new flag serves as a prototype of what that stable UI might eventually look like.

The `branch` option is a placeholder that currently does nothing. It will be used by rust-lang#122322 to opt into branch coverage instrumentation.

---

I see `-Zcoverage-options` as something that will exist more-or-less indefinitely, though individual sub-options might come and go as appropriate. I think there will always be some demand for nightly-only toggles, so I don't see `-Zcoverage-options` itself ever being stable, though we might eventually stabilize something similar to it.
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 13, 2024
Rollup merge of rust-lang#122226 - Zalathar:zcoverage-options, r=nnethercote

coverage: Remove or migrate all unstable values of `-Cinstrument-coverage`

(This PR was substantially overhauled from its original version, which migrated all of the existing unstable values intact.)

This PR takes the three nightly-only values that are currently accepted by `-Cinstrument-coverage`, completely removes two of them (`except-unused-functions` and `except-unused-generics`), and migrates the third (`branch`) over to a newly-introduced unstable flag `-Zcoverage-options`.

I have a few motivations for wanting to do this:

- It's unclear whether anyone actually uses the `except-unused-*` values, so this serves as an opportunity to either remove them, or prompt existing users to object to their removal.
- After rust-lang#117199, the stable values of `-Cinstrument-coverage` treat it as a boolean-valued flag, so having nightly-only extra values feels out-of-place.
  - Nightly-only values also require extra ad-hoc code to make sure they aren't accidentally exposed to stable users.
- The new system allows multiple different settings to be toggled independently, which isn't possible in the current single-value system.
- The new system makes it easier to introduce new behaviour behind an unstable toggle, and then gather nightly-user feedback before possibly making it the default behaviour for all users.
- The new system also gives us a convenient place to put relatively-narrow options that won't ever be the default, but that nightly users might still want access to.
- It's likely that we will eventually want to give stable users more fine-grained control over coverage instrumentation. The new flag serves as a prototype of what that stable UI might eventually look like.

The `branch` option is a placeholder that currently does nothing. It will be used by rust-lang#122322 to opt into branch coverage instrumentation.

---

I see `-Zcoverage-options` as something that will exist more-or-less indefinitely, though individual sub-options might come and go as appropriate. I think there will always be some demand for nightly-only toggles, so I don't see `-Zcoverage-options` itself ever being stable, though we might eventually stabilize something similar to it.
@Zalathar
Copy link
Contributor Author

Zalathar commented Mar 13, 2024

I've rebased and updated the tests for #122226, but I still need to update the “placeholder” docs for -Zcoverage-options=branch.

EDIT: Updates should be complete now.

@Zalathar Zalathar force-pushed the branch branch 2 times, most recently from 1085c76 to a043c19 Compare March 14, 2024 02:03
@Zalathar
Copy link
Contributor Author

Shortened some identifiers, mostly by removing “hir” from them (diff).

@oli-obk
Copy link
Contributor

oli-obk commented Mar 14, 2024

@bors r+ rollup

@bors
Copy link
Contributor

bors commented Mar 14, 2024

📌 Commit 060c7ce has been approved by oli-obk

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 14, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 14, 2024
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#117118 ([AIX] Remove AixLinker's debuginfo() implementation)
 - rust-lang#121650 (change std::process to drop supplementary groups based on CAP_SETGID)
 - rust-lang#121764 (Make incremental sessions identity no longer depend on the crate names provided by source code)
 - rust-lang#122212 (Copy byval argument to alloca if alignment is insufficient)
 - rust-lang#122322 (coverage: Initial support for branch coverage instrumentation)
 - rust-lang#122373 (Fix the conflict problem between the diagnostics fixes of lint `unnecessary_qualification`  and  `unused_imports`)
 - rust-lang#122479 (Implement `Duration::as_millis_{f64,f32}`)
 - rust-lang#122487 (Rename `StmtKind::Local` variant into `StmtKind::Let`)
 - rust-lang#122498 (Update version of cc crate)
 - rust-lang#122503 (Make `SubdiagMessageOp` well-formed)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 15, 2024
…iaskrgr

Rollup of 10 pull requests

Successful merges:

 - rust-lang#117118 ([AIX] Remove AixLinker's debuginfo() implementation)
 - rust-lang#121650 (change std::process to drop supplementary groups based on CAP_SETGID)
 - rust-lang#121764 (Make incremental sessions identity no longer depend on the crate names provided by source code)
 - rust-lang#122212 (Copy byval argument to alloca if alignment is insufficient)
 - rust-lang#122322 (coverage: Initial support for branch coverage instrumentation)
 - rust-lang#122373 (Fix the conflict problem between the diagnostics fixes of lint `unnecessary_qualification`  and  `unused_imports`)
 - rust-lang#122479 (Implement `Duration::as_millis_{f64,f32}`)
 - rust-lang#122487 (Rename `StmtKind::Local` variant into `StmtKind::Let`)
 - rust-lang#122498 (Update version of cc crate)
 - rust-lang#122503 (Make `SubdiagMessageOp` well-formed)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 54a5a49 into rust-lang:master Mar 15, 2024
11 checks passed
@rustbot rustbot added this to the 1.78.0 milestone Mar 15, 2024
@Zalathar Zalathar deleted the branch branch March 15, 2024 02:04
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 15, 2024
Rollup merge of rust-lang#122322 - Zalathar:branch, r=oli-obk

coverage: Initial support for branch coverage instrumentation

(This is a review-ready version of the changes that were drafted in rust-lang#118305.)

This PR adds support for branch coverage instrumentation, gated behind the unstable flag value `-Zcoverage-options=branch`. (Coverage instrumentation must also be enabled with `-Cinstrument-coverage`.)

During THIR-to-MIR lowering (MIR building), if branch coverage is enabled, we collect additional information about branch conditions and their corresponding then/else blocks. We inject special marker statements into those blocks, so that the `InstrumentCoverage` MIR pass can reliably identify them even after the initially-built MIR has been simplified and renumbered.

The rest of the changes are mostly just plumbing needed to gather up the information that was collected during MIR building, and include it in the coverage metadata that we embed in the final binary.

Note that `llvm-cov show` doesn't print branch coverage information in its source views by default; that needs to be explicitly enabled with `--show-branches=count` or similar.

---

The current implementation doesn't have any support for instrumenting `if let` or let-chains. I think it's still useful without that, and adding it would be non-trivial, so I'm happy to leave that for future work.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants