Tracking issue for namespaced-features

[djc]

Docs: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#namespaced-features
Original issue: #1286
Implementation: #5300

Not sure what the process is for stabilizing experimental Cargo features, I figured a tracking issue might be useful? @alexcrichton any guide on what should be done to move this towards stabilization?

Unresolved issues

• Figure out how to deal with breakage of Cargo versions older than 1.19. Add schema field and features2 to the index. #9161

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

[alexcrichton]

Indeed a tracking issue is useful! I think though this still has a large piece to implement which is to propagate namespaced features to the registry index

[djc]

Yes, I figured as much. So do you have any ideas/guidance about how to go about that? Are there established ideas on how to evolve the registry index?

[alexcrichton]

The evolution here is just a new key in the registry which we've done a bunch of times before, so it's fine to do whenever once there's an agreed up on design. Most of the time this is hard to undo though so we want to be pretty sure of the design before it lands. Technically it'll involve modifying crates.io as well as the publishing code in Cargo to send more data to the registry

[djc]

Can you be a bit more specific about which parts of the cargo and/or crates.io code need changing?

[alexcrichton]

You may want to hop onto the crates.io channel on discord as they may know more, but it's all around krate.rs and git.rs IIRC

[djc]

@alexcrichton okay, so, I'm still not entirely clear on what needs to be done on the crates.io side here. Is it just propagating the package.namespaced-features value? The feature values can still be serialized as a string, so that's the least invasive change. Alternatively, we could add more structure to the feature values and use serde's enum handling to have a new key that's more explicit about what's what.

Either way, I'm not sure how the compatibility with old cargo is supposed to work, or who all needs to be on board with a supposed design.

[alexcrichton]

@djc there's a few things here I think that we'd need to consider:

• One aspect is that we need to consider what happens when older versions of Cargo are used. For example if I use Cargo from 1.20.0 with a crate that has a namespaced feature, what happens? Is an error generated? Does the package not exist? Does it silently work sometimes and not others? My preferred way to solve this would be to add support for this today but don't actually allow publishing crates just yet. That way after a few cycles and Cargo rides the trains older Cargo will be able to understand newer crates in the future.
• Next is the actual registry format itself. Right now the features in the registry index are just an array of strings, but presumably yeah we'll need to propagate the namespaced features boolean as well. New registry fields could be used as well.

Before we go much further though I think it'd be good to get weigh-in from @rust-lang/cargo because once we start changing the registry it's sort of the point of no return

[joshtriplett]

@alexcrichton So, this is exactly the kind of change that really motivated the idea of supporting Cargo version dependencies in a way that older versions of Cargo can handle.

We can't make old versions of Cargo unable to parse new versions of the index; that's a serious problem.

If we're going to do this, we need to version the index, such that older versions of Cargo don't see crates that use this feature at all.

[alexcrichton]

@joshtriplett indeed! To be clear though the main issue is that an older Cargo won't understand the newer index, but we get to control how it doesn't understand the new index as well (aka no features, a mapping of features, or something like that). Having a fully versioned index seems far off still so I'd be fine implementing this before that comes along.

Now having a versioned index is a good idea though and we can start at any time to add safeguards into Cargo to avoid looking at future revisions!

[alexreg]

Is this going to move towards stabilisation soon?

[djc]

Probably not -- I'm still motivated to write code to move this forward, but I don't know exactly what needs to be done.

[alexreg]

Oh, I thought it was already implemented, just needed stabilising.

[djc]

The Cargo part is implemented -- it apparently needs complementary stuff in crates.io.

[alexreg]

Aha.

What I still don't understand is how this issue relates to implicit features for dependencies, exactly. (I was linked here from that issue.)

[ExpHP]

What I still don't understand is how this issue relates to implicit features for dependencies, exactly. (I was linked here from that issue.)

I agree that more could be said here. I find it odd how none of the examples I can locate for this feature show how it can be used to overcome the limitations of optional dependencies (which were its primary motivation)!

See this page:

• Docs for namespaced-features.

And now let me try to put it in context:

---

Suppose you have this:

[dependencies]
rand = { version = "0.5.5", optional = true }

Without namespaced-features, an optional dep implicitly defines a feature that cannot be overridden:

[features]
rand = [] # implicitly generated, always

...and rand is added to the dependency tree whenever the feature rand is required.

With namespaced-features, an optional dep implicitly defines a feature that can be overridden:

[features]
rand = ["crate:rand"]  # implicitly generated, unless you write your own rand feature

...and rand is added to the dependency tree whenever the feature crate:rand is required.

---

If you look closely, the actual namespacing of the dependencies does not, in itself, have anything to do with the issue of implicit features. It is the other changes enabled by the namespaced-features feature (notably the ability to override the implicit feature) that make a difference.

The justification for tying these changes together was given in #1286. Er... maybe. Actually, I can't really find it explicitly spelled out there. But the proposal is strictly more powerful than if we were to simply allow the implicit feature to be overridden with no other changes (because this also allows you to have a rand feature that does not toggle the rand dependency. Why? Idunno.).

[alexreg]

@ExpHP Thanks, that explanation makes a lot more sense to me. Was there a reason @sfackler's original proposal wasn't chosen though? I kind of like the idea of doing something like

[features]
a = { features = ["b", "c/d"], dependencies = ["c"] }

or such.

[tikue]

Is it possible to publish a crate that uses this feature? I'm getting:

error: api errors: invalid upload request: invalid value: string "crate:serde", expected a valid feature name at line 1 column 1862

[alexcrichton]

@tikue I believe not currently. In addition to crates.io checks the registry index isn't ready for namespaced features

[tikue]

@alexcrichton thanks, it was easy enough to work around, just had to name a serde feature serde1.

[alexreg]

@alexcrichton Does an issue for that exist on the crates.io repo?

[alexcrichton]

@alexreg I think that's this issue, the tracking issue for this feature. The feature needs to be designed for crates.io first.

[alexreg]

@alexcrichton Hmm? That's what I mean, we need to implement namespaced features on crates.io. There should be a separate issue for that over on that repo, I think, no?

[djc]

FWIW, I'd like to move this forward, but I'm currently having a hard time figuring out exactly what needs to be done about the index and crates.io. If someone can provide me with more guidance on that, that would be great.

[alexcrichton]

@djc I think that rust-lang/crates.io#1487 may be helpful in providing a rough template for how to make progress?

[est31]

Regarding the crate:foo syntax, I think it'll confuse a lot of people that crate:foo in Cargo.toml refers to the foo external crate (in fact, package), while crate::foo in mod.rs refers to the crate's internal module named foo.

Maybe it would be better to use pkg:foo instead of crate:foo?

[ehuss]

As discovered in #8832, this unfortunately breaks Cargo versions older than 1.19, even if they have a Cargo.lock file. We'll need to figure out how to address that concern before this can move forward.

[ehuss]

I forgot to post a followup here. #9161 implemented changes to support namespaced features in the index and avoid breaking older versions of cargo. There is some risk that versions older than 1.51 may resolve packages using namespaced features in such a way that those features don't get activated. I'll feel a little more comfortable once more time has passed so that 1.51 is at least a few months old.

[ehuss]

I have posted a proposal to stabilize this in rust-lang/rfcs#3143.

[taralx]

Proposal passed, what's next? If there are work items people could pick up, could they be added to the initial comment?

[taralx]

Also, out of curiosity, why doesn't #[cfg(feature="dep:bar")] work?

[ehuss]

Proposal passed, what's next?

This is just waiting for a review on a PR on crates.io to accept the new syntax.

Also, out of curiosity, why doesn't #[cfg(feature="dep:bar")] work?

I'm not sure I fully understand your question, since the point of the dep: syntax is to avoid setting features, and dep:bar would be invalid syntax for a cfg expression. The intent is that cfg(accessible(...)) will be the intended way in the future for detecting the presence of a dependency, which is discussed in the rfc.

[ehuss]

Posted #10269 as a proposal to stabilized this.

[runiq]

Huzzah, #10269 has been merged! :)

[ehuss]

Oops, looks like I closed the wrong issue. Thanks!