cargo metadata: Don't show `null` deps. #6534

ehuss · 2019-01-10T05:05:27Z

If a package has a dependency without a library target, the "name" field was
showing up as null in resolve.nodes.deps. At this time (AFAIK), binary-only
dependencies are always ignored. Instead of making users filter out this entry
(or more commonly, crash), just don't include it.

rust-highfive · 2019-01-10T05:05:30Z

r? @Eh2406

(rust_highfive has picked a reviewer for you, use r? to override)

src/etc/man/cargo-metadata.1

dwijnand · 2019-01-10T10:51:08Z

Field missing vs field set to null.. Hmm, not sure which is best or if there's an established convention in we're using and should continue to follow.

ehuss · 2019-01-10T14:17:49Z

Field missing vs field set to null.. Hmm, not sure which is best or if there's an established convention in we're using and should continue to follow.

It's not just the field, it's the entire Dep value. It's the difference between:

"deps": [
    {
        "name": null,
        "pkg": "bdep 0.5.0 (path+file:///bdep)"
    }
],

and

"deps": [],

This causes things like cargo_metadata to fail because they are requiring a string value for name.

dwijnand · 2019-01-10T14:23:03Z

What is cargo_metadata?

dwijnand · 2019-01-10T14:24:08Z

Oh right, https://github.com/oli-obk/cargo_metadata. So this is resolving an incoherence between cargo and that crate.

Eh2406 · 2019-01-10T15:00:57Z

I would love to see us more involved in making sure that cargo and cargo_metadata always work together.

What is the use case for depending on a crate without a library? In that use case should cargo_metadata's users have to be aware that that crate exists?

ehuss · 2019-01-10T15:08:31Z

What is the use case for depending on a crate without a library? In that use case should cargo_metadata's users have to be aware that that crate exists?

I can't think of a legitimate case. I scanned all of crates.io for packages with binary dependencies, and all I found were:

./aquaengine-0.0.2 on clippy 0.0.302
./cargo-name-1.0.1 on clippy 0.0.302
./collapse-crate-0.1.0 on clippy 0.0.302
./dagon-0.0.2 on clippy 0.0.302
./dat-0.0.0 on clippy 0.0.302
./github-release-0.1.2 on clippy 0.0.302
./k-bucket-0.1.0 on clippy 0.0.302
./pop-trait-0.2.1 on clippy 0.0.302
./prefix-map-0.3.1 on clippy 0.0.302
./rand-pop-0.1.1 on clippy 0.0.302
./rrun-ssh-0.3.0 on clippy 0.0.302
./rtps-0.2.3 on clippy 0.0.302
./rumqtt-0.10.1 on clippy 0.0.302
./speedometer-0.2.2 on clippy 0.0.302
./statics-0.2.0 on cargo-watch 6.0.0
./version-sort-0.1.0 on clippy 0.0.302

The clippy ones are because a special version of clippy was published that says "do not use me". I suspect the cargo-watch thing was just a mistake.

ehuss · 2019-01-10T15:10:05Z

I would love to see us more involved in making sure that cargo and cargo_metadata always work together.

Oh, and the reason I found this is because I was working on a new test suite for cargo_metadata. 😆

dwijnand · 2019-01-10T15:41:36Z

Maybe we should depend on cargo_metadata, or maybe we should have a crate inside this repo that cargo_metadata depends on (or is replaced by). (sorry, veering off-topic)

Eh2406 · 2019-01-10T16:09:03Z

If the team is on board with having closer ties then we should open up a discussion of how that looks. (I have thoughts, but it is off topic here.)

I feel a little uncomfortable removing information from our output. This is a thing a project can do, and metadata is how tooling will find out about it. Like I am not sure the bug is on the cargo side.

That being said I don't want to make cargo_metadata have to use an Option<String> just to support this very niche case (not to mention the breaking change). Maybe cargo_metadata can convert null to ""?

dwijnand · 2019-01-10T16:12:04Z

Yeah, I think that's a sound argument.

I'm ambivalent, personally.

matklad · 2019-01-10T20:38:14Z

This was semi intentional, to make deps strictly better than dependencies. If we filter dep without a library, the user wouldn't be able to figure out the dependency at all (field in Package specifies only requirement, not dependency).

It is true that library-less dependencies are weird, so perhaps it makes sense to put the fix closer to the core, report an error if we see such a dependency and just assume, everywhere, that every dependency has a library target?

matklad · 2019-01-10T21:06:30Z

Semi unrelated, but, while we are at it, perhaps it makes sense to add a dependency from a package on itself, while we are at it? That way clients won't have to deal with replacing - with _ and dealing with [lib] name = "other" for current package?

ehuss · 2019-01-10T22:13:22Z

It is true that library-less dependencies are weird, so perhaps it makes sense to put the fix closer to the core, report an error if we see such a dependency and just assume, everywhere, that every dependency has a library target?

I'm not very excited about making a hard error. I'm often wary of adding new errors since they can cause trouble, and the warning→error transition is a hassle. It would break any package that had clippy="*" in them. I think a warning would be a very good idea, though.

It might help me if I knew who is currently using this information and for what purpose. In the past I've tried to use it, but it doesn't provide enough information for the things I wanted to do. The entries can't be (reliably) matched to the Package dependencies, so you can't tell what Kind they are, or anything else. My ideal situation would be to have access to the unit dependencies map.

the user wouldn't be able to figure out the dependency at all

I'm still of the position that these binary dependencies are useless information that is not actionable, and just requires all users to filter them out. I don't know what purpose they serve to know about a dependency that can't be used.

bors · 2019-02-05T16:07:06Z

☔ The latest upstream changes (presumably #6631) made this pull request unmergeable. Please resolve the merge conflicts.

If a package has a dependency without a library target, the "name" field was showing up as null in `resolve.nodes.deps`. At this time (AFAIK), binary-only dependencies are always ignored. Instead of making users filter out this entry (or more commonly, crash), just don't include it.

matklad · 2019-03-16T13:37:31Z

Heh, hit this in rust-analyzer: rust-lang/rust-analyzer#979

The current PR definitely looks good, r=me.

Adding a warning can done separately.

matklad · 2019-03-16T13:41:36Z

tests/testsuite/metadata.rs

+    let nodes = &meta["resolve"]["nodes"];
+    assert!(nodes[0]["deps"].as_array().unwrap().is_empty());
+    assert!(nodes[1]["deps"].as_array().unwrap().is_empty());
+}


We might have a test for this (I remember dealing with this problem explicitelly) already. Might be a good idea to put .unwrap on the name and run the test-suite.

I tried with an unwrap, but none of the tests failed.

ehuss · 2019-03-26T17:38:31Z

Ping @Eh2406 or @alexcrichton, just wondering if we can make a decision on this. I see three options:

Do nothing, force all users to filter out these null entries.
Merge as-is.
Enhance or redesign how dependency information is exposed.

I still lean for 2, for the reasons outlined above. I don't think we have the list of use cases to properly do 3.

Eh2406 · 2019-03-26T18:00:24Z

I don't understand this well enough to have a deciding opinion. Looks like the knowledgeable people seem to like merging.

alexcrichton · 2019-03-26T18:19:13Z

I'd personally be fine with option 2 myself as recommended!

Is this just waiting on an r+ otherwise?

ehuss · 2019-03-26T18:35:50Z

Is this just waiting on an r+ otherwise?

Yep.

alexcrichton · 2019-03-26T18:51:30Z

@bors: r+

bors · 2019-03-26T18:51:31Z

📌 Commit a2fc5cf0241ff8841519fb8ca682a4b8f740640b has been approved by alexcrichton

ehuss · 2019-03-26T22:26:09Z

@bors r=alexcrichton

bors · 2019-03-26T22:26:10Z

📌 Commit c3f4b0d has been approved by alexcrichton

bors · 2019-03-26T23:22:13Z

⌛ Testing commit c3f4b0d with merge b3b65c3...

cargo metadata: Don't show `null` deps. If a package has a dependency without a library target, the "name" field was showing up as null in `resolve.nodes.deps`. At this time (AFAIK), binary-only dependencies are always ignored. Instead of making users filter out this entry (or more commonly, crash), just don't include it.

bors · 2019-03-26T23:45:43Z

☀️ Test successful - checks-travis, status-appveyor
Approved by: alexcrichton
Pushing b3b65c3 to master...

Update cargo Update cargo 22 commits in 0e35bd8af0ec72d3225c4819b330b94628f0e9d0..63231f438a2b5b84ccf319a5de22343ee0316323 2019-03-13 06:52:51 +0000 to 2019-03-27 12:26:45 +0000 - Code cleanup (rust-lang/cargo#6787) - Add cargo:rustc-link-arg to pass custom linker arguments (rust-lang/cargo#6298) - Testsuite: remove some unnecessary is_nightly checks. (rust-lang/cargo#6786) - cargo metadata: Don't show `null` deps. (rust-lang/cargo#6534) - Some fingerprint cleanup. (rust-lang/cargo#6785) - Fix fingerprint for canceled build script. (rust-lang/cargo#6782) - Canonicalize default target if it ends with `.json` (rust-lang/cargo#6778) - Fix setting `panic=unwind` compiling lib a extra time. (rust-lang/cargo#6781) - Always nicely show errors from crates.io if possible (rust-lang/cargo#6771) - Testsuite: Make `cwd()` relative to project root. (rust-lang/cargo#6768) - Allow `cargo fix` if gitignore matches root working dir. (rust-lang/cargo#6767) - Remove redundant imports (rust-lang/cargo#6763) - Handle backcompat hazard with `toml` crate (rust-lang/cargo#6761) - Fix spurious error in dirty_both_lib_and_test. (rust-lang/cargo#6756) - Update toml requirement from 0.4.2 to 0.5.0 (rust-lang/cargo#6760) - Reuse std::env::consts::EXE_SUFFIX (rust-lang/cargo#6758) - Proptest 0.9.1 (rust-lang/cargo#6753) - Don't need extern crate in 2018 (rust-lang/cargo#6752) - Release a jobserver token while locking a file (rust-lang/cargo#6748) - Minor doc fix for publish command synopsis (rust-lang/cargo#6749) - Stricter package change detection. (rust-lang/cargo#6740) - Fix resolving yanked crates when using a local registry. (rust-lang/cargo#6742)

rust-highfive assigned Eh2406 Jan 10, 2019

ehuss force-pushed the meta-deps-name-null branch from b7630d6 to a2fc5cf Compare January 10, 2019 06:50

dwijnand reviewed Jan 10, 2019

View reviewed changes

src/etc/man/cargo-metadata.1 Show resolved Hide resolved

ehuss mentioned this pull request Jan 10, 2019

Add some more fields. oli-obk/cargo_metadata#61

Merged

Eh2406 mentioned this pull request Jan 16, 2019

How can Cargo help oli-obk/cargo_metadata#63

Open

ehuss force-pushed the meta-deps-name-null branch from a2fc5cf to c3f4b0d Compare February 5, 2019 16:25

ehuss mentioned this pull request Feb 10, 2019

Error on crates depending on binary crates oli-obk/cargo_metadata#66

Closed

ehuss added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 7, 2019

matklad mentioned this pull request Mar 16, 2019

loading workspace failed: cargo metadata failed: invalid type: null, expected a string at line 1 column 236445 rust-lang/rust-analyzer#979

Closed

matklad approved these changes Mar 16, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 26, 2019

ehuss closed this Mar 26, 2019

ehuss reopened this Mar 26, 2019

bors merged commit c3f4b0d into rust-lang:master Mar 26, 2019

ehuss mentioned this pull request Mar 29, 2019

Update cargo rust-lang/rust#59516

Merged

ehuss added this to the 1.35.0 milestone Feb 6, 2022

ByteSecurity mentioned this pull request Jul 6, 2023

Prevent execution to stop when pair that doesn't exist encountered fuzzland/ityfuzz#100

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cargo metadata: Don't show `null` deps. #6534

cargo metadata: Don't show `null` deps. #6534

ehuss commented Jan 10, 2019

rust-highfive commented Jan 10, 2019

dwijnand commented Jan 10, 2019

ehuss commented Jan 10, 2019

dwijnand commented Jan 10, 2019

dwijnand commented Jan 10, 2019

Eh2406 commented Jan 10, 2019

ehuss commented Jan 10, 2019

ehuss commented Jan 10, 2019

dwijnand commented Jan 10, 2019

Eh2406 commented Jan 10, 2019

dwijnand commented Jan 10, 2019

matklad commented Jan 10, 2019

matklad commented Jan 10, 2019

ehuss commented Jan 10, 2019

bors commented Feb 5, 2019

matklad commented Mar 16, 2019

matklad Mar 16, 2019

ehuss Mar 16, 2019

ehuss commented Mar 26, 2019

Eh2406 commented Mar 26, 2019

alexcrichton commented Mar 26, 2019

ehuss commented Mar 26, 2019

alexcrichton commented Mar 26, 2019

bors commented Mar 26, 2019

ehuss commented Mar 26, 2019

bors commented Mar 26, 2019

bors commented Mar 26, 2019

bors commented Mar 26, 2019

cargo metadata: Don't show null deps. #6534

cargo metadata: Don't show null deps. #6534

Conversation

ehuss commented Jan 10, 2019

rust-highfive commented Jan 10, 2019

dwijnand commented Jan 10, 2019

ehuss commented Jan 10, 2019

dwijnand commented Jan 10, 2019

dwijnand commented Jan 10, 2019

Eh2406 commented Jan 10, 2019

ehuss commented Jan 10, 2019

ehuss commented Jan 10, 2019

dwijnand commented Jan 10, 2019

Eh2406 commented Jan 10, 2019

dwijnand commented Jan 10, 2019

matklad commented Jan 10, 2019

matklad commented Jan 10, 2019

ehuss commented Jan 10, 2019

bors commented Feb 5, 2019

matklad commented Mar 16, 2019

matklad Mar 16, 2019

Choose a reason for hiding this comment

ehuss Mar 16, 2019

Choose a reason for hiding this comment

ehuss commented Mar 26, 2019

Eh2406 commented Mar 26, 2019

alexcrichton commented Mar 26, 2019

ehuss commented Mar 26, 2019

alexcrichton commented Mar 26, 2019

bors commented Mar 26, 2019

ehuss commented Mar 26, 2019

bors commented Mar 26, 2019

bors commented Mar 26, 2019

bors commented Mar 26, 2019

cargo metadata: Don't show `null` deps. #6534

cargo metadata: Don't show `null` deps. #6534