Build dist-android with --enable-profiler #70054

rojamd · 2020-03-16T19:41:35Z

This will make the runtime available to enable PGO for Rust code in Firefox on Android.

rust-highfive · 2020-03-16T19:41:38Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @michaelwoerister (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

michaelwoerister · 2020-03-17T14:22:17Z

@bors r+

bors · 2020-03-17T14:22:19Z

📌 Commit f8870bf has been approved by michaelwoerister

@michaelwoerister

Build dist-android with --enable-profiler This will make the runtime available to enable PGO for Rust code in Firefox on Android. r? @michaelwoerister

Centril · 2020-03-18T15:21:24Z

Failed in #70099 (comment), @bors r- rollup=never

michaelwoerister · 2020-03-18T15:37:08Z

This is the error it fails with:

error: failed to run custom build command for `profiler_builtins v0.0.0 (/checkout/src/libprofiler_builtins)`

Caused by:
  process didn't exit successfully: `/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/release/build/profiler_builtins-9bb15f3a7adeeb21/build-script-build` (exit code: 1)
--- stdout
TARGET = Some("arm-linux-androideabi")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-unknown-linux-gnu")
CC_arm-linux-androideabi = Some("sccache /android/ndk/arm-14/bin/arm-linux-androideabi-clang")
CFLAGS_arm-linux-androideabi = Some("-ffunction-sections -fdata-sections -fPIC --target=arm-linux-androideabi")
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
running: "sccache" "/android/ndk/arm-14/bin/arm-linux-androideabi-clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=arm-linux-androideabi" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=arm-linux-androideabi" "-I" "/checkout/src/llvm-project/compiler-rt/include" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/arm-linux-androideabi/release/build/profiler_builtins-48aac5a25541f1f3/out/GCDAProfiling.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/GCDAProfiling.c"
exit code: 0
running: "sccache" "/android/ndk/arm-14/bin/arm-linux-androideabi-clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=arm-linux-androideabi" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=arm-linux-androideabi" "-I" "/checkout/src/llvm-project/compiler-rt/include" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/arm-linux-androideabi/release/build/profiler_builtins-48aac5a25541f1f3/out/InstrProfiling.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/InstrProfiling.c"
exit code: 0
running: "sccache" "/android/ndk/arm-14/bin/arm-linux-androideabi-clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=arm-linux-androideabi" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=arm-linux-androideabi" "-I" "/checkout/src/llvm-project/compiler-rt/include" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/arm-linux-androideabi/release/build/profiler_builtins-48aac5a25541f1f3/out/InstrProfilingBuffer.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/InstrProfilingBuffer.c"
exit code: 0
running: "sccache" "/android/ndk/arm-14/bin/arm-linux-androideabi-clang" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "--target=arm-linux-androideabi" "-ffunction-sections" "-fdata-sections" "-fPIC" "--target=arm-linux-androideabi" "-I" "/checkout/src/llvm-project/compiler-rt/include" "-fno-builtin" "-fvisibility=hidden" "-fomit-frame-pointer" "-ffreestanding" "-DVISIBILITY_HIDDEN" "-DCOMPILER_RT_HAS_UNAME=1" "-DCOMPILER_RT_HAS_FCNTL_LCK=1" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-std/arm-linux-androideabi/release/build/profiler_builtins-48aac5a25541f1f3/out/InstrProfilingFile.o" "-c" "/checkout/src/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c"
cargo:warning=In file included from /checkout/src/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:27:
cargo:warning=In file included from /android/ndk/arm-14/bin/../sysroot/usr/include/unistd.h:35:
cargo:warning=In file included from /android/ndk/arm-14/bin/../sysroot/usr/include/sys/select.h:36:
cargo:warning=In file included from /android/ndk/arm-14/bin/../sysroot/usr/include/signal.h:193:
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:71:33: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  return (int)((local_set[bit / LONG_BIT] >> (bit % LONG_BIT)) & 1);
cargo:warning=                                ^
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:71:53: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  return (int)((local_set[bit / LONG_BIT] >> (bit % LONG_BIT)) & 1);
cargo:warning=                                                    ^
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:82:19: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  local_set[bit / LONG_BIT] |= 1UL << (bit % LONG_BIT);
cargo:warning=                  ^
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:82:46: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  local_set[bit / LONG_BIT] |= 1UL << (bit % LONG_BIT);
cargo:warning=                                             ^
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:94:19: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  local_set[bit / LONG_BIT] &= ~(1UL << (bit % LONG_BIT));
cargo:warning=                  ^
cargo:warning=/android/ndk/arm-14/bin/../sysroot/usr/include/android/legacy_signal_inlines.h:94:48: error: use of undeclared identifier 'LONG_BIT'
cargo:warning=  local_set[bit / LONG_BIT] &= ~(1UL << (bit % LONG_BIT));
cargo:warning=                                               ^
cargo:warning=6 errors generated.
exit code: 1

rojamd · 2020-03-18T15:46:43Z

At first glance this looks like an issue with the NDK headers on the builder. How can I find out the exact version of the headers being used on that machine?

rojamd · 2020-03-18T16:03:46Z

According to android/ndk#1117 this is caused by

rust/src/libprofiler_builtins/build.rs

Line 44 in e82febc

cfg.flag("-ffreestanding");

…e standard library

michaelwoerister · 2020-03-19T16:34:25Z

From looking at LLVM cmake files it seems like some of the sanitizers are built with -ffreestanding but not the profiling runtime. Let's see if removing this causes any trouble.

@bors r+

bors · 2020-03-19T16:34:27Z

📌 Commit 50c0562 has been approved by michaelwoerister

Dylan-DPC-zz · 2020-03-19T18:56:48Z

@bors p=1

bors · 2020-03-21T00:15:00Z

⌛ Testing commit 50c0562 with merge d2b9bfafa0e7beabfb8f76a1d05d17db58d4471c...

Dylan-DPC-zz · 2020-03-21T00:25:02Z

@bors retry yield for rollup

bors · 2020-03-21T02:18:53Z

⌛ Testing commit 50c0562 with merge 24f9c25ca02969ded44312e6154126d76aaf7618...

Centril · 2020-03-21T02:26:17Z

@bors retry yielding

bors · 2020-03-21T04:28:52Z

⌛ Testing commit 50c0562 with merge e98af6565c499091cfbd21060ff6f50711ba5a14...

Centril · 2020-03-21T04:33:58Z

@bors retry yielding

bors · 2020-03-21T07:46:51Z

⌛ Testing commit 50c0562 with merge 105c38f78bd50968ee2308fd41d5c096de28aa92...

Centril · 2020-03-21T07:53:06Z

@bors retry rollup=maybe

@ghost

Rollup of 8 pull requests Successful merges: - rust-lang#67888 (Prefetch some queries used by the metadata encoder) - rust-lang#69934 (Update the mir inline costs) - rust-lang#69965 (Refactorings to get rid of rustc_codegen_utils) - rust-lang#70054 (Build dist-android with --enable-profiler) - rust-lang#70089 (rustc_infer: remove InferCtxt::closure_sig as the FnSig is always shallowly known.) - rust-lang#70092 (hir: replace "items" terminology with "nodes" where appropriate.) - rust-lang#70138 (do not 'return' in 'throw_' macros) - rust-lang#70151 (Update stdarch submodule) Failed merges: - rust-lang#70074 (Expand: nix all fatal errors) r? @ghost

bors · 2020-03-21T11:11:35Z

⌛ Testing commit 50c0562 with merge 98803c1...

Trying to build a Rust project with `-Zprofile` for target x86_64-unknown-linux-musl using rustc 1.46.0-nightly (346aec9 2020-07-11), installed with rustup, results in the following error. ``` export RUSTFLAGS="-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"export CARGO_INCREMENTAL=0$ cargo build --target=x86_64-unknown-linux-muslCompiling hello_world v0.1.0 (…)error[E0463]: can't find crate for `profiler_builtins` | = note: the compiler may have been built without the profiler runtime error: aborting due to previous error For more information about this error, try `rustc --explain E0463`.error: could not compile `hello_world`. To learn more, run the command again with --verbose. ``` `-Zprofile` is required here to enable grcov profiling. This is similar in nature to issue rust-lang#57257, which has been fixed in asimilar way at rust-lang#60476 . A fix for Android has also landed not long ago: rust-lang#70054 . Signed-off-by: Tiago Lam <tiagol@hadean.com>

Build dist-x86_64-musl with --enable-profiler. Trying to build a Rust project with `-Zprofile` for target x86_64-unknown-linux-musl using rustc 1.46.0-nightly (346aec9 2020-07-11), installed with rustup, results in the following error. ``` export RUSTFLAGS="-Zprofile -Ccodegen-units=1 -Copt-level=0 -Clink-dead-code -Coverflow-checks=off -Zpanic_abort_tests -Cpanic=abort"export CARGO_INCREMENTAL=0$ cargo build --target=x86_64-unknown-linux-muslCompiling hello_world v0.1.0 (…)error[E0463]: can't find crate for `profiler_builtins` | = note: the compiler may have been built without the profiler runtime error: aborting due to previous error For more information about this error, try `rustc --explain E0463`.error: could not compile `hello_world`. To learn more, run the command again with --verbose. ``` `-Zprofile` is required here to enable grcov profiling. This is similar in nature to issue rust-lang#57257, which has been fixed in asimilar way at rust-lang#60476 . A fix for Android has also landed not long ago: rust-lang#70054 . Signed-off-by: Tiago Lam <tiagol@hadean.com>

Build dist-android with --enable-profiler

f8870bf

rust-highfive assigned michaelwoerister Mar 16, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 16, 2020

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 17, 2020

Centril mentioned this pull request Mar 18, 2020

Rollup of 8 pull requests #70099

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 18, 2020

Remove -ffreestanding from libprofiler_builtins because we do need th…

50c0562

…e standard library

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 19, 2020

Centril mentioned this pull request Mar 21, 2020

Rollup of 8 pull requests #70211

Merged

bors merged commit bbd1ca3 into rust-lang:master Mar 21, 2020

tiagolam mentioned this pull request Aug 28, 2020

Build dist-x86_64-musl with --enable-profiler. #76035

Merged

briansmith mentioned this pull request Nov 30, 2020

Code coverage and profiling don't work on non-X86_64 -musl targets because profiler_builtins isn't provided for them #79556

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build dist-android with --enable-profiler #70054

Build dist-android with --enable-profiler #70054

rojamd commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

michaelwoerister commented Mar 17, 2020

bors commented Mar 17, 2020

Centril commented Mar 18, 2020

michaelwoerister commented Mar 18, 2020

rojamd commented Mar 18, 2020

rojamd commented Mar 18, 2020

michaelwoerister commented Mar 19, 2020

bors commented Mar 19, 2020

Dylan-DPC-zz commented Mar 19, 2020

bors commented Mar 21, 2020

Dylan-DPC-zz commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020

Build dist-android with --enable-profiler #70054

Build dist-android with --enable-profiler #70054

Conversation

rojamd commented Mar 16, 2020

rust-highfive commented Mar 16, 2020

michaelwoerister commented Mar 17, 2020

bors commented Mar 17, 2020

Centril commented Mar 18, 2020

michaelwoerister commented Mar 18, 2020

rojamd commented Mar 18, 2020

rojamd commented Mar 18, 2020

michaelwoerister commented Mar 19, 2020

bors commented Mar 19, 2020

Dylan-DPC-zz commented Mar 19, 2020

bors commented Mar 21, 2020

Dylan-DPC-zz commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020

Centril commented Mar 21, 2020

bors commented Mar 21, 2020