Use RUSTFLAGS to set the optimization level #10719

dvdplm · 2019-06-05T08:17:34Z

Cargo has a quirk in how configuration settings are propagated when cargo test runs: local code respect the settings in [profile.test] but all dependencies use the [profile.dev] settings. Here we force opt-level=3 for all dependencies.

Cargo has a [quirk]() in how configuration settings are propagated when `cargo test` runs: local code respect the settings in `[profile.test]` but all dependencies use the `[profile.dev]` settings. Here we force `opt-level=3` for all dependencies.

TriplEight

I will deal with it.

TriplEight · 2019-06-05T12:40:49Z

Cargo.toml

-[profile.test]
-lto = false
-opt-level = 3 # makes tests slower to compile, but faster to run
-


This might work: RUSTFLAGS="-C opt-level=3 -C overflow-checks=off -C incremental=false -C debuginfo=2" time cargo test -v

These are the settings for release mode I think:

The release profile, used for cargo build --release (and the dependencies

for cargo test --release, including the local library or binary).

[profile.release]
opt-level = 3
debug = false
rpath = false
lto = false
debug-assertions = false
codegen-units = 16
panic = 'unwind'
incremental = false
overflow-checks = false

TriplEight · 2019-06-05T12:42:49Z

dvdplm
So cargo test --release will use the settings from [profile.release] which with androniks change includes lto = true, which in turn makes compilation take a really long time (but binary size is smaller). So for tests, where we do care about fast run speeds, we want the code optimizations from release mode but not lto = true.

The first idea was to stop passing --release to cargo test and instead just add opt-level = 3 to [profile.test]. That didn't work because cargo applies opt-level =3 only to the local code but not to dependencies. For dependencies it uses [profile.dev] (which is kind of crazy and probably a bug). To get around that we tried passing in the optimization level with RUST_FLAGS but as Kiril points out, we probably need more flags than just -C opt-level = 3.

dvdplm · 2019-06-05T13:09:33Z

@TriplEight pushed #d654d255f to see what happens.

TriplEight · 2019-06-06T08:44:34Z

I think it's still "not enough" optimized.
Since you are passing opt-level=3 to everything including deps, it must be faster, but it's still not.
There have to be something missing.

TriplEight

Approved. Looks good, but I feel there is some space for improve.

* master: SecretStore: non-blocking wait of session completion (#10303) removed secret_store folder (#10722) SecretStore: expose restore_key_public in HTTP API (#10241) Revert "enable lto for release builds (#10717)" (#10721)

scripts/gitlab/test-linux.sh

.gitlab-ci.yml

* Use RUSTFLAGS to set the optimization level Cargo has a [quirk]() in how configuration settings are propagated when `cargo test` runs: local code respect the settings in `[profile.test]` but all dependencies use the `[profile.dev]` settings. Here we force `opt-level=3` for all dependencies. * Remove unused profile settings * Maybe like this? * Turn off incremental compilation * Remove colors; try again with overflow-checks on * Use quiet CI machine * Turn overflow checking back on * Be explicit about what options we use * Remove "quiet machine" override

* version: bump stable to 2.4.7 * [CI] allow cargo audit to fail (#10676) * [CI] allow cargo audit to fail * [.gitlab-ci.yml] add a comment about cargo audit * [Cargo.lock] cargo update -p protobuf * new image (#10673) * Update publishing (#10644) * docker images are now built on k8s: test run * copy check_sync.sh in build-linux job * copy scripts/docker/hub/* in build-linux job * removed cache var * cleanup, no more nightly dockers * cleanup in dockerfile * some new tags * removed sccsche debug log, cleanup * no_gits, new artifacts dir, changed scripts. Test run. * define version once * one source for TRACK * stop kovan onchain updates * moved changes for two images to a new branch * rename Dockerfile * no need in libudev-dev * enable lto for release builds (#10717) * Use RUSTFLAGS to set the optimization level (#10719) * Use RUSTFLAGS to set the optimization level Cargo has a [quirk]() in how configuration settings are propagated when `cargo test` runs: local code respect the settings in `[profile.test]` but all dependencies use the `[profile.dev]` settings. Here we force `opt-level=3` for all dependencies. * Remove unused profile settings * Maybe like this? * Turn off incremental compilation * Remove colors; try again with overflow-checks on * Use quiet CI machine * Turn overflow checking back on * Be explicit about what options we use * Remove "quiet machine" override * ethcore: enable ECIP-1054 for classic (#10731) * config: enable atlantis on ethereum classic * config: enable atlantis on morden classic * config: enable atlantis on morden classic * config: enable atlantis on kotti classic * ethcore: move kotti fork block to 0xAEF49 * ethcore: move morden fork block to 0x4829BA * ethcore: move classic fork block to 0x81B320 * remove trailing comma * remove trailing comma * fix chainspec * ethcore: move classic fork block to 0x7fffffffffffffff

* version: bump beta to 2.5.2 * [CI] allow cargo audit to fail (#10676) * [CI] allow cargo audit to fail * [.gitlab-ci.yml] add a comment about cargo audit * [Cargo.lock] cargo update -p protobuf * Reset blockchain properly (#10669) * delete BlockDetails from COL_EXTRA * better proofs * added tests * PR suggestions * new image (#10673) * Update publishing (#10644) * docker images are now built on k8s: test run * copy check_sync.sh in build-linux job * copy scripts/docker/hub/* in build-linux job * removed cache var * cleanup, no more nightly dockers * cleanup in dockerfile * some new tags * removed sccsche debug log, cleanup * no_gits, new artifacts dir, changed scripts. Test run. * define version once * one source for TRACK * stop kovan onchain updates * moved changes for two images to a new branch * rename Dockerfile * no need in libudev-dev * enable lto for release builds (#10717) * Use RUSTFLAGS to set the optimization level (#10719) * Use RUSTFLAGS to set the optimization level Cargo has a [quirk]() in how configuration settings are propagated when `cargo test` runs: local code respect the settings in `[profile.test]` but all dependencies use the `[profile.dev]` settings. Here we force `opt-level=3` for all dependencies. * Remove unused profile settings * Maybe like this? * Turn off incremental compilation * Remove colors; try again with overflow-checks on * Use quiet CI machine * Turn overflow checking back on * Be explicit about what options we use * Remove "quiet machine" override * ethcore: enable ECIP-1054 for classic (#10731) * config: enable atlantis on ethereum classic * config: enable atlantis on morden classic * config: enable atlantis on morden classic * config: enable atlantis on kotti classic * ethcore: move kotti fork block to 0xAEF49 * ethcore: move morden fork block to 0x4829BA * ethcore: move classic fork block to 0x81B320 * remove trailing comma * remove trailing comma * fix chainspec * ethcore: move classic fork block to 0x7fffffffffffffff

* master: updater: fix static id hashes initialization (#10755) Use fewer threads for snapshotting (#10752) Die error_chain, die (#10747) Fix deprectation warnings on nightly (#10746) fix docker tags for publishing (#10741) DevP2p: Get node IP address and udp port from Socket, if not included in PING packet (#10705) ethcore: enable ECIP-1054 for classic (#10731) Stop breaking out of loop if a non-canonical hash is found (#10729) Refactor Clique stepping (#10691) Use RUSTFLAGS to set the optimization level (#10719) SecretStore: non-blocking wait of session completion (#10303) removed secret_store folder (#10722) SecretStore: expose restore_key_public in HTTP API (#10241) Revert "enable lto for release builds (#10717)" (#10721) enable lto for release builds (#10717) Merge `Notifier` and `TransactionsPoolNotifier` (#10591)

* master: Enable aesni (#10756) remove support of old SS db formats (#10757) [devp2p] Don't use `rust-crypto` (#10714) updater: fix static id hashes initialization (#10755) Use fewer threads for snapshotting (#10752) Die error_chain, die (#10747) Fix deprectation warnings on nightly (#10746) fix docker tags for publishing (#10741) DevP2p: Get node IP address and udp port from Socket, if not included in PING packet (#10705) ethcore: enable ECIP-1054 for classic (#10731) Stop breaking out of loop if a non-canonical hash is found (#10729) Refactor Clique stepping (#10691) Use RUSTFLAGS to set the optimization level (#10719) SecretStore: non-blocking wait of session completion (#10303) removed secret_store folder (#10722) SecretStore: expose restore_key_public in HTTP API (#10241) Revert "enable lto for release builds (#10717)" (#10721) enable lto for release builds (#10717) Merge `Notifier` and `TransactionsPoolNotifier` (#10591)

dvdplm requested a review from TriplEight June 5, 2019 08:17

dvdplm self-assigned this Jun 5, 2019

dvdplm added A0-pleasereview 🤓 Pull request needs code review. A2-insubstantial 👶 Pull request requires no code review (e.g., a sub-repository hash update). labels Jun 5, 2019

Remove unused profile settings

ba1ef0b

dvdplm added A1-onice 🌨 Pull request is reviewed well, but should not yet be merged. and removed A0-pleasereview 🤓 Pull request needs code review. labels Jun 5, 2019

TriplEight suggested changes Jun 5, 2019

View reviewed changes

Maybe like this?

d654d25

Turn off incremental compilation

391e9bc

TriplEight approved these changes Jun 6, 2019

View reviewed changes

dvdplm added A0-pleasereview 🤓 Pull request needs code review. and removed A1-onice 🌨 Pull request is reviewed well, but should not yet be merged. A2-insubstantial 👶 Pull request requires no code review (e.g., a sub-repository hash update). labels Jun 6, 2019

dvdplm requested a review from ordian June 6, 2019 15:54

ordian reviewed Jun 6, 2019

View reviewed changes

scripts/gitlab/test-linux.sh Outdated Show resolved Hide resolved

dvdplm added 2 commits June 6, 2019 18:11

Remove colors; try again with overflow-checks on

4918780

Use quiet CI machine

7c67c35

dvdplm commented Jun 6, 2019

View reviewed changes

.gitlab-ci.yml Outdated Show resolved Hide resolved

dvdplm added 3 commits June 6, 2019 20:43

Turn overflow checking back on

0be77df

Be explicit about what options we use

6c7416d

Remove "quiet machine" override

9daf394

ordian approved these changes Jun 7, 2019

View reviewed changes

ordian added this to the 2.6 milestone Jun 7, 2019

ordian added the A8-looksgood 🦄 Pull request is reviewed well. label Jun 7, 2019

ordian added B0-patch-stable 🕷 Pull request should also be back-ported to the stable branch. B1-patch-beta 🕷🕷 M0-build 🏗 Building and build system. and removed A0-pleasereview 🤓 Pull request needs code review. labels Jun 7, 2019

dvdplm merged commit 7827cc0 into master Jun 7, 2019

dvdplm deleted the dp/fix/force-tests-to-run-opt-level3 branch June 7, 2019 09:25

This was referenced Jun 11, 2019

backports for stable 2.4.7 #10736

Merged

backports for beta 2.5.2 #10737

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use RUSTFLAGS to set the optimization level #10719

Use RUSTFLAGS to set the optimization level #10719

dvdplm commented Jun 5, 2019 •

edited

Loading

TriplEight left a comment

TriplEight Jun 5, 2019

TriplEight commented Jun 5, 2019

dvdplm commented Jun 5, 2019 •

edited

Loading

TriplEight commented Jun 6, 2019

TriplEight left a comment

Use RUSTFLAGS to set the optimization level #10719

Use RUSTFLAGS to set the optimization level #10719

Conversation

dvdplm commented Jun 5, 2019 • edited Loading

TriplEight left a comment

Choose a reason for hiding this comment

TriplEight Jun 5, 2019

Choose a reason for hiding this comment

The release profile, used for cargo build --release (and the dependencies

for cargo test --release, including the local library or binary).

TriplEight commented Jun 5, 2019

dvdplm commented Jun 5, 2019 • edited Loading

TriplEight commented Jun 6, 2019

TriplEight left a comment

Choose a reason for hiding this comment

dvdplm commented Jun 5, 2019 •

edited

Loading

The release profile, used for `cargo build --release` (and the dependencies

for `cargo test --release`, including the local library or binary).

dvdplm commented Jun 5, 2019 •

edited

Loading