Macro expansion bypasses #[derive_*] stability checks.

[eddyb]

One can use macro expansion to bypass all early attribute checks, directly:

macro_rules! evil {
    () => {
        #[derive_Debug]
        struct Foo;
    }
}
evil!();
fn main() {}

Or by using the built-in include! macro.

// foo.rs
#[derive_Debug]
struct Foo; 

include!("foo.rs");
fn main() {}

Found while investigating libpnet breakage.

This affects any use of #[derive(...)] in syntex, as virtually everyone uses include!.
Even though I don't like stabilizing this, there may be too many cases in the wild.

[pnkfelix]

cc #29644

[pnkfelix]

there are different possible outcomes for what we do here, depending on how hard it is to come up with a fix (@eddyb may already have one in the works), and what the impact of such a fix is according to crater.

[pnkfelix]

(assigning P-high to reflect the priority of deciding what to do.)

[eddyb]

Attempt at a fix and crater results can be found at #32671.
It looks like all the regressions are due to serde_codegen - sadly, that doesn't count crates which weren't compiling anyway (like pnet, where I observed this issue, which would compile if syntex were fixed).

In IRC @durka and I looked at the crates that depend on syntex and while most of them could be fixed by releasing a new version, pnet of all things has the serde version pinned, without even allowing patch releases.

And that doesn't count anyone who has a Cargo.lock around. The whole reason they're using syntex is to escape instability.

Some food for thought: @nrc's macro plans could eventually allow for #[derive(foo::Bar)]. At first you might think #[derive_foo::Bar] just doesn't work, and you'd be right, but what about #[foo::derive_Bar]?

It's not the worst that could happen, and I haven't seen specific plans for making #[derive] pluggable, other than that "template-based" scheme @Manishearth and I were discussing, which would still work fine (#[derivable(...)] trait Foo {...} on the trait would simply produce a derive_Foo attribute).

Note that you can't use #[derive] with non-builtin traits anyway without using a plugin, which is unstable, so the #[derive(SomeCustomTrait)] gating is pretty pointless by itself (and my tentative fix doesn't apply to it).
You could instead gate registering #[derive_*] decorator plugins, if you really wanted to.

That said, I leave this to @rust-lang/lang to decide. Do we just upload a new version of serde and/or do we ungate #[derive_*]?
I would personally do both (given the future-compatibility point above) and warn when #[derive_*] doesn't come from the compiler, just to make sure, but that's a bit overkill.

[mrmonday]

Chiming in, as primary libpnet author:

• Syntex is pinned because each release can break the API, causing the build to fail - it gets updated whenever I get around to it, and that doesn't necessarily imply a new cargo release (the cargo release process isn't streamlined yet, so happens when I have time or someone asks nicely)
• libpnet hasn't hit 1.0 yet, so breaking things isn't the worst thing, though I would rather it didn't happen.

Please contact me whenever about this - I believe Dropbox is using some subset of libpnet, but I don't know exact details.

I suspect there's a nice way to deal with this (at least in terms of libpnet), though I admittedly haven't been following particularly closely.

[eddyb]

@mrmonday You could have pinned just the minor version, not the patch, i.e. >= 0.26.0 < 0.27.0 instead of = 0.26.0, even if semver doesn't work before 1.0 (I can't remember if it does; some other crates do the >= 0.x.0 < 0.x+1.0 thing for syntex dependencies, instead of ^ 0.x.0).

[Manishearth]

Since the fix for this likely breaks syntex, cc #31645. Please don't merge the fix without letting me know first.

[erickt]

I'm just on my way out, but as long as we can fix syntex first, I think it'd be fine to make this change in the compiler. The syntex consumers are pretty familiar with breaking changes in order to track stable and nightly plugin support, so doing a version bump to avoid this error wouldn't be that bad.

How about we fix this in syntex, do a serde version bump for both v6 and v7, then see how things look in crater?

[eddyb]

@erickt One way that would work pretty well IMO is to replace all #[derive_Trait] attributes with a single combined #[derive(...)], using a Folder with a fold_item_simple method, just before pretty-printing the result.

[erickt]

@eddyb: I've landed syntex v0.29.2 and v0.30.1 which both implement squashing #[derive_X] #[derive_Y] into #[derive(X, Y)]. However according to crates.io it looks like there are still some projects that are pinned to syntex v0.29.0 and v0.30.0, which would be broken by your fix.

[durka]

I actually don't see any crates pinned to 0.29.0 or 0.30.0, did I miss some? I see a bunch of ^0.29.0 and ^0.30.0, which would pick up 0.29.2 and 0.30.1 after cargo update. The only pinning I see is libnet's =0.26.0.

[erickt]

@durka: Not sure. I just saw that there were still some downloads for v0.29.0 and v0.30.0 even after I released my crate. It's possible those downloads are just from a lockfile still pointing at the older version, or they could be from unpublished crates.

[nrc]

Let's keep trying to land the fix, it sounds like it should be possible. I really don't want to stabilise derive_*, it's really hacky. I would like to think about some way to support custom derive as part of proc macro reform.

[mrmonday]

I've changed libpnet to depend on 0.30.*, I'll do a version bump shortly.

[pnkfelix]

(hmm it seems like this was fixed, but I'm not clear what PR actually fixed it. In particular, PR #32671 never landed...)

[pnkfelix]

well bisection indicates it was fixed (in the sense that we started issuing the lint even when the custom attribute was hidden) between nightly-2016-04-28 (cda7c1c 2016-04-27) and nightly-2016-05-07 (62e2b2f 2016-05-06).

rustup says we don't have prebuilt distributions for the dates between those two points.

... oh, maybe I should try looking at the slew of commits from LeoTestard referencing this issue: PR #32791 is what resolved this.