Rust 1.63.0 fails to build with: error: failed to download addr2line v0.16.0

[paulmenzel]

Trying to build Rust 1.63.0 it fails with the error below:

$ python3 ./x.py build --exclude src/tools/miri
Building rustbuild
    Updating crates.io index
  Downloaded cfg-if v1.0.0
  Downloaded opener v0.5.0
  Downloaded serde v1.0.125
  Downloaded sha2 v0.10.1
  Downloaded serde_derive v1.0.125
  Downloaded lazy_static v1.4.0
  Downloaded same-file v1.0.6
  Downloaded version_check v0.9.3
  Downloaded toml v0.5.7
  Downloaded xattr v0.2.2
  Downloaded pkg-config v0.3.18
  Downloaded unicode-width v0.1.8
  Downloaded tar v0.4.38
  Downloaded walkdir v2.3.2
  Downloaded regex v1.5.5
  Downloaded num_cpus v1.13.1
  Downloaded ryu v1.0.5
  Downloaded regex-syntax v0.6.25
  Downloaded cmake v0.1.44
  Downloaded typenum v1.12.0
  Downloaded itoa v0.4.6
  Downloaded proc-macro2 v1.0.37
  Downloaded xz2 v0.1.6
  Downloaded syn v1.0.91
  Downloaded log v0.4.14
  Downloaded cpufeatures v0.2.1
  Downloaded lzma-sys v0.1.16
  Downloaded libc v0.2.126
  Downloaded thread_local v1.1.4
  Downloaded crossbeam-utils v0.8.8
  Downloaded unicode-xid v0.2.2
  Downloaded cc v1.0.69
  Downloaded hex v0.4.2
  Downloaded generic-array v0.14.4
  Downloaded memchr v2.4.1
  Downloaded aho-corasick v0.7.18
  Downloaded block-buffer v0.10.2
  Downloaded fnv v1.0.7
  Downloaded getopts v0.2.21
  Downloaded regex-automata v0.1.10
  Downloaded quote v1.0.18
  Downloaded once_cell v1.12.0
  Downloaded ignore v0.4.17
  Downloaded globset v0.4.5
  Downloaded filetime v0.2.14
  Downloaded digest v0.10.2
  Downloaded crypto-common v0.1.2
  Downloaded bstr v0.2.13
  Downloaded serde_json v1.0.59
  Downloaded 49 crates (3.7 MB) in 0.48s
    Finished dev [unoptimized] target(s) in 44.74s
warning: x.py has made several changes recently you may want to look at
help: consider looking at the changes in `src/bootstrap/CHANGELOG.md`
note: to silence this warning, add `changelog-seen = 2` at the top of `config.toml`
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
error: failed to download `addr2line v0.16.0`

Caused by:
  attempting to make an HTTP request, but --frozen was specified
Build completed unsuccessfully in 0:00:45

[Mark-Simulacrum]

Hm, so I can't immediately reproduce -- can you say more about what sources you're using to build? https://static.rust-lang.org/dist/2022-08-11/rustc-1.63.0-src.tar.xz is the tarball I used to make an attempt at reproducing.

You might also want to check if you've enabled the use of vendored sources (config.toml should contain vendor = true under [build]).

[paulmenzel]

We are using https://static.rust-lang.org/dist/2022-08-11/rustc-1.63.0-src.tar.gz.

$ md5sum /package/pkg/rustc-1.63.0-0/build/rustc-1.63.0-src.tar.gz
8904e95118a2a53318c78532d3f33443  /package/pkg/rustc-1.63.0-0/build/rustc-1.63.0-src.tar.gz

You might also want to check if you've enabled the use of vendored sources (config.toml should contain vendor = true under [build]).

We did not have that in our build script. It worked without in the past. Could you refer me to further details, why it’s needed?

[Mark-Simulacrum]

That tarball should be content equivalent to .xz, just with gzip compression.

I don't see why we would've broken a workflow that worked in the past. Can you link me to the build script you're using?

--frozen is passed if we detect that you're running either with vendoring enabled or with sudo. Rustbuild checks for sudo here, while our Python script has a slightly different check here. Maybe your environment changed in such a way that the Python check isn't true while the rustbuild check is? That would explain the error here.

(Setting vendor = true in config.toml will override the detection logic here and just enable using the vendored sources from the tarball).

[paulmenzel]

The files are in https://github.molgen.mpg.de/mariux64/pkg-scripts, and it’s merge/pull request https://github.molgen.mpg.de/mariux64/pkg-scripts/pull/205/.

$ git clone https://github.molgen.mpg.de/mariux64/pkg-scripts
$ cd pkg-scripts
$ git checkout add-rustc-1.63.0-0-with-cbindgen-0.24.3
$ sudo tools/build.sh rustc-1.63.0-0.build.sh # … is what we use.

sudo ./rustc-1.63.0-0.build.sh should work in non-MarIuX environment (our distribution).

Adapt PREFIX= in the script to your liking.

[paulmenzel]

(Edited last comment.)

[Mark-Simulacrum]

It looks like you were previously on 1.61, right? #95671 landed in 1.62, which introduced the difference¹ between handling in bootstrap.py and the Rust component for whether we're running inside sudo (as you do). I suspect that difference made us start using vendoring (though not sure), or stop, but either way, it's likely the cause of your problem.

I'm going to go ahead and mark this as E-easy -- we should likely move the Rust code to use the same getuid() == 0 check. I believe we can do that with libc::getuid() and appropriately cfg that if needed (based on what CI complains about).

Footnotes

1. that I pointed out in https://github.com/rust-lang/rust/issues/100459#issuecomment-1213628089 ↩

[paulmenzel]

It looks like you were previously on 1.61, right?

Yes, we did.

Thank you for the analysis. If you want us to test anything, please tell us.

[Mark-Simulacrum]

Once #100852 merges (and we release a new nightly ~24 hours after that) it'd be great to retest with a nightly src tarball in your environment to confirm this is fixed, but also fine to just wait until the next stable release including it and file a new bug if not. That'll be 1.64, releasing September 22nd. (I marked the PR as beta-backport).