Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustdoc: don't process Crate::external_traits when collecting intra-doc links #58972

Merged
merged 2 commits into from
Apr 11, 2019

Conversation

QuietMisdreavus
Copy link
Member

@QuietMisdreavus QuietMisdreavus commented Mar 6, 2019

Part of #58745, closes #58917

The collect-intra-doc-links pass keeps track of the modules it recurses through as it processes items. This is used to know what module to give the resolver when looking up links. When looking through the regular items of the crate, this works fine, but the DocFolder trait as written doesn't just process the main crate hierarchy - it also processes the trait items in the external_traits map. This is useful for other passes (so they can strip out #[doc(hidden)] items, for example), but here it creates a situation where we're processing items "outside" the regular module hierarchy. Since everything in external_traits is defined outside the current crate, we can't fall back to finding its module scope like we do with local items.

Skipping this collection saves us from emitting some spurious warnings. We don't even lose anything by skipping it, either - the docs loaded from here are only ever rendered through html::render::document_short which strips any links out, so the fact that the links haven't been loaded doesn't matter. Hopefully this removes most of the remaining spurious resolution warnings from intra-doc links.

r? @GuillaumeGomez

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 6, 2019
@GuillaumeGomez
Copy link
Member

Wo, nice!

r=me once CI passed

@QuietMisdreavus
Copy link
Member Author

Whoops, i just noticed this doesn't totally fix the linked issue. We can merge this on its own, but i'll keep digging to see what i can do about the last few warnings from rand.

@GuillaumeGomez
Copy link
Member

We can also merge mine. I prefer to wait until you have the full thing. ;)

@QuietMisdreavus
Copy link
Member Author

I'm hesitant to merge your PR. Muting all resolution errors from non-local items would hide the warnings that were not actually spurious (see the issue linked in the PR description for a couple examples). I think properly solving all the non-spurious warnings would require adding a much larger mechanism, or fixing rand itself.

@QuietMisdreavus
Copy link
Member Author

I still think this is mergeable as-is; the "full thing" would add too much on top of it, since the remaining warnings in rand were not spurious. (Even if they were the result of shortcomings in the intra-doc links feature itself, they still created broken links or erroneous [link targets] with square brackets around them.)

@Centril Centril added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 30, 2019
@GuillaumeGomez
Copy link
Member

Let's merge it then, sorry for the time it took!

@bors: r+

@bors
Copy link
Contributor

bors commented Apr 8, 2019

📌 Commit 49cde40 has been approved by GuillaumeGomez

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 8, 2019
Centril added a commit to Centril/rust that referenced this pull request Apr 8, 2019
…rts, r=GuillaumeGomez

rustdoc: don't process `Crate::external_traits` when collecting intra-doc links

Part of rust-lang#58745, closes rust-lang#58917

The `collect-intra-doc-links` pass keeps track of the modules it recurses through as it processes items. This is used to know what module to give the resolver when looking up links. When looking through the regular items of the crate, this works fine, but the `DocFolder` trait as written doesn't just process the main crate hierarchy - it also processes the trait items in the `external_traits` map. This is useful for other passes (so they can strip out `#[doc(hidden)]` items, for example), but here it creates a situation where we're processing items "outside" the regular module hierarchy. Since everything in `external_traits` is defined outside the current crate, we can't fall back to finding its module scope like we do with local items.

Skipping this collection saves us from emitting some spurious warnings. We don't even lose anything by skipping it, either - the docs loaded from here are only ever rendered through `html::render::document_short` which strips any links out, so the fact that the links haven't been loaded doesn't matter. Hopefully this removes most of the remaining spurious resolution warnings from intra-doc links.

r? @GuillaumeGomez
@bors
Copy link
Contributor

bors commented Apr 9, 2019

⌛ Testing commit 49cde40 with merge ef9be56...

bors added a commit that referenced this pull request Apr 9, 2019
…laumeGomez

rustdoc: don't process `Crate::external_traits` when collecting intra-doc links

Part of #58745, closes #58917

The `collect-intra-doc-links` pass keeps track of the modules it recurses through as it processes items. This is used to know what module to give the resolver when looking up links. When looking through the regular items of the crate, this works fine, but the `DocFolder` trait as written doesn't just process the main crate hierarchy - it also processes the trait items in the `external_traits` map. This is useful for other passes (so they can strip out `#[doc(hidden)]` items, for example), but here it creates a situation where we're processing items "outside" the regular module hierarchy. Since everything in `external_traits` is defined outside the current crate, we can't fall back to finding its module scope like we do with local items.

Skipping this collection saves us from emitting some spurious warnings. We don't even lose anything by skipping it, either - the docs loaded from here are only ever rendered through `html::render::document_short` which strips any links out, so the fact that the links haven't been loaded doesn't matter. Hopefully this removes most of the remaining spurious resolution warnings from intra-doc links.

r? @GuillaumeGomez
@bors
Copy link
Contributor

bors commented Apr 9, 2019

💔 Test failed - checks-travis

@rust-highfive
Copy link
Collaborator

The job dist-x86_64-linux-alt of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:54:55]    Compiling rustc_macros v0.1.0 (/checkout/src/librustc_macros)
[00:55:55] [RUSTC-TIMING] syntax test:false 81.995
[00:55:55]    Compiling syntax_ext v0.0.0 (/checkout/src/libsyntax_ext)
[00:56:31] [RUSTC-TIMING] syntax_ext test:false 36.907
No output has been received in the last 30m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 9, 2019
@GuillaumeGomez
Copy link
Member

@bors: retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 9, 2019
@bors
Copy link
Contributor

bors commented Apr 9, 2019

⌛ Testing commit 49cde40 with merge 8f332ec...

bors added a commit that referenced this pull request Apr 9, 2019
…laumeGomez

rustdoc: don't process `Crate::external_traits` when collecting intra-doc links

Part of #58745, closes #58917

The `collect-intra-doc-links` pass keeps track of the modules it recurses through as it processes items. This is used to know what module to give the resolver when looking up links. When looking through the regular items of the crate, this works fine, but the `DocFolder` trait as written doesn't just process the main crate hierarchy - it also processes the trait items in the `external_traits` map. This is useful for other passes (so they can strip out `#[doc(hidden)]` items, for example), but here it creates a situation where we're processing items "outside" the regular module hierarchy. Since everything in `external_traits` is defined outside the current crate, we can't fall back to finding its module scope like we do with local items.

Skipping this collection saves us from emitting some spurious warnings. We don't even lose anything by skipping it, either - the docs loaded from here are only ever rendered through `html::render::document_short` which strips any links out, so the fact that the links haven't been loaded doesn't matter. Hopefully this removes most of the remaining spurious resolution warnings from intra-doc links.

r? @GuillaumeGomez
@Manishearth Manishearth mentioned this pull request Apr 9, 2019
@rust-highfive
Copy link
Collaborator

The job dist-x86_64-apple of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_fold:start:worker_info
Worker information
hostname: c7b69a8a-0430-4fdd-8dc5-52b7d2147819@1.worker-custom-2-7846647cfb-d9k8w.macstadium-prod-2
instance: af9c1fc5-1aaf-4e52-9031-20111c0f48df moar-travis-ci-macos10.13-xcode9.3-1516902186 (via amqp)
startup: 1m2.174357354s
travis_fold:end:worker_info
Ignoring bigdecimal-1.3.4 because its extensions are not built.  Try: gem pristine bigdecimal --version 1.3.4
---
voltdb
wry
zxing-cpp
==> Downloading https://homebrew.bintray.com/bottles/xz-5.2.4.high_sierra.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/e7/e7be50f4ee00e35887f3957263334eb3baba59e8c061919060f9259351be6880?__gda__=exp=1554812628~hmac=d6760e45058bb207c45dc9c407319dff4e1f8e62242645337e8449f9bd16571b&response-content-disposition=attachment%3Bfilename%3D%22xz-5.2.4.high_sierra.bottle.tar.gz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX18gWTji7ZR19qBWrt8dNk6QAxYHt4THwOFL-SyXXLK0mYHNKWgaYsYiUgNkS5iEl-CWEagxu4Hf-72WyX638Kvwai4Qop4mtLUgf232qH9ntGw9pSSFr5LlL4xhOSl8WUDxwLZ0ltroWw&response-X-Checksum-Sha1=32dc0b28e61f32b40c20e2993418aa8cb6e746d5&response-X-Checksum-Sha2=e7be50f4ee00e35887f3957263334eb3baba59e8c061919060f9259351be6880
🍺  /usr/local/Cellar/xz/5.2.4: 92 files, 1MB
==> `brew cleanup` has not been run in 30 days, running now...
Removing: /Users/travis/Library/Caches/Homebrew/boost-1.66.0.high_sierra.bottle.tar.gz... (84.6MB)
Removing: /Users/travis/Library/Caches/Homebrew/carthage-0.28.0.high_sierra.bottle.tar.gz... (8.3MB)
---
Pruned 0 symbolic links and 5 directories from /usr/local
==> Installing dependencies for swig: pcre
==> Installing swig dependency: pcre
==> Downloading https://homebrew.bintray.com/bottles/pcre-8.43.high_sierra.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/03/0389911a93a88efd4a69b52dea8ecb872fdb55bcfff45d2f7313be5f79730861?__gda__=exp=1554812643~hmac=fd3935991efb788f2aea572541d253729cd85d5eddf8c986a91162c02de3669a&response-content-disposition=attachment%3Bfilename%3D%22pcre-8.43.high_sierra.bottle.tar.gz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX18NUDNvRWYfwX5cYHUhG68tIhiGaonUy13BZEC4GeUndI9LyaeByoUo7RptmBZUF6BD9RaugBDAfvDfC7d-w4BGWqoM5PrpkxJbdnklkFZUiHoC-MCD65Sh0UnLsQZVrXHjIUIYQZEjMA&response-X-Checksum-Sha1=c67d4b99bb245f0ea56b34118dd6325b06a7250c&response-X-Checksum-Sha2=0389911a93a88efd4a69b52dea8ecb872fdb55bcfff45d2f7313be5f79730861
🍺  /usr/local/Cellar/pcre/8.43: 204 files, 5.5MB
==> Installing swig
==> Downloading https://homebrew.bintray.com/bottles/swig-3.0.12.high_sierra.bottle.tar.gz
==> Downloading https://homebrew.bintray.com/bottles/swig-3.0.12.high_sierra.bottle.tar.gz
==> Downloading from https://akamai.bintray.com/c0/c0e2656fd10d57281280d20ce8bf9a060cf8714f4283dd1dfde383b3688d9ed1?__gda__=exp=1554812646~hmac=012703f4506b6898534ede680b1eee2081ce3b62bcd3a26863e22dd25228a6f7&response-content-disposition=attachment%3Bfilename%3D%22swig-3.0.12.high_sierra.bottle.tar.gz%22&response-content-type=application%2Fgzip&requestInfo=U2FsdGVkX1-4mvhzXlNZ0P-XTpKMAExU4OHc8fnDbIbXu-I6AzlWDiQcs5rPXG0QdSUW2jKJ029SzBNO8XSjeefAeg0xpcX9gxf0lmQx1RFBlBrxLmn8Xo-sqCMwCsp6ku58CfJFJg2dXQ52qBMEZQ&response-X-Checksum-Sha1=db6e6ed21965214d5f9fba1b180517bb2587ef59&response-X-Checksum-Sha2=c0e2656fd10d57281280d20ce8bf9a060cf8714f4283dd1dfde383b3688d9ed1
🍺  /usr/local/Cellar/swig/3.0.12: 755 files, 5.5MB
travis_time:end:0b37f223:start=1554811520454400000,finish=1554811968457518000,duration=448003118000
travis_fold:end:install
travis_fold:start:before_script.1
---
[02:51:10]    Compiling clippy_lints v0.0.212 (/Users/travis/build/rust-lang/rust/src/tools/clippy/clippy_lints)
[02:51:10]    Compiling cargo v0.36.0 (/Users/travis/build/rust-lang/rust/src/tools/cargo)
[02:51:10]    Compiling rustfmt-nightly v1.2.0 (/Users/travis/build/rust-lang/rust/src/tools/rustfmt)
[02:51:10]    Compiling racer v2.1.21
The job exceeded the maximum time limit for jobs, and has been terminated.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Apr 9, 2019

💔 Test failed - checks-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 9, 2019
@kennytm
Copy link
Member

kennytm commented Apr 10, 2019

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 10, 2019
@bors
Copy link
Contributor

bors commented Apr 11, 2019

⌛ Testing commit 49cde40 with merge ee1474a...

bors added a commit that referenced this pull request Apr 11, 2019
…laumeGomez

rustdoc: don't process `Crate::external_traits` when collecting intra-doc links

Part of #58745, closes #58917

The `collect-intra-doc-links` pass keeps track of the modules it recurses through as it processes items. This is used to know what module to give the resolver when looking up links. When looking through the regular items of the crate, this works fine, but the `DocFolder` trait as written doesn't just process the main crate hierarchy - it also processes the trait items in the `external_traits` map. This is useful for other passes (so they can strip out `#[doc(hidden)]` items, for example), but here it creates a situation where we're processing items "outside" the regular module hierarchy. Since everything in `external_traits` is defined outside the current crate, we can't fall back to finding its module scope like we do with local items.

Skipping this collection saves us from emitting some spurious warnings. We don't even lose anything by skipping it, either - the docs loaded from here are only ever rendered through `html::render::document_short` which strips any links out, so the fact that the links haven't been loaded doesn't matter. Hopefully this removes most of the remaining spurious resolution warnings from intra-doc links.

r? @GuillaumeGomez
@bors
Copy link
Contributor

bors commented Apr 11, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: GuillaumeGomez
Pushing ee1474a to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 11, 2019
@bors bors merged commit 49cde40 into rust-lang:master Apr 11, 2019
@QuietMisdreavus QuietMisdreavus deleted the intra-doc-link-imports branch April 11, 2019 13:49
@jyn514 jyn514 added A-intra-doc-links Area: Intra-doc links, the ability to link to items in docs by name T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Aug 29, 2020
jyn514 added a commit to jyn514/rust that referenced this pull request Aug 29, 2020
 rust-lang#58972 ignored extern_traits because before rust-lang#65983 was fixed, they
would always fail to resolve, giving spurious warnings.
This undoes that change, so extern traits are now seen by the
`collect_intra_doc_links` pass. There are also some minor changes in
librustdoc/fold.rs to avoid borrowing the extern_traits RefCell more
than once at a time.
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 30, 2020
…trochenkov

Fix intra-doc links for cross-crate re-exports of default trait methods

The original fix for this was very simple: rust-lang#58972 ignored `extern_traits` because before rust-lang#65983 was fixed, they would always fail to resolve, giving spurious warnings. So the first commit just undoes that change, so extern traits are now seen by the `collect_intra_doc_links` pass. There are also some minor changes in `librustdoc/fold.rs` to avoid borrowing the `extern_traits` RefCell more than once at a time.

However, that brought up a much more thorny problem. `rustc_resolve` started giving 'error: cannot find a built-in macro with name `cfg`' when documenting `libproc_macro` (I still haven't been able to reproduce on anything smaller than the full standard library). The chain of events looked like this (thanks @eddyb for the help debugging!):

0. `x.py build --stage 1` builds the standard library and creates a sysroot
1. `cargo doc` does something like `cargo check` to create `rmeta`s for all the crates (unrelated to what was built above)
2. the `cargo check`-like `libcore-*.rmeta` is loaded as a transitive dependency *and claims ownership* of builtin macros
3. `rustdoc` later tries to resolve some path in a doc link
4. suggestion logic fires and loads "extern prelude" crates by name
5. the sysroot `libcore-*.rlib` is loaded and *fails to claim ownership* of builtin macros

`rustc_resolve` gives the error after step 5. However, `rustdoc` doesn't need suggestions at all - `resolve_str_path_error` completely discards the `ResolutionError`! The fix implemented in this PR is to skip the suggestion logic for `resolve_ast_path`: pass `record_used: false` and skip `lookup_import_candidates` when `record_used` isn't set.

It's possible that if/when rust-lang#74207 is implemented this will need a more in-depth fix which returns a `ResolutionError` from `compile_macro`, to allow rustdoc to reuse the suggestions from rustc_resolve. However, that's a much larger change and there's no need for it yet, so I haven't implemented it here.

Fixes rust-lang#73829.

r? @GuillaumeGomez
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-intra-doc-links Area: Intra-doc links, the ability to link to items in docs by name merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants