Attempt to make Box::default() construct in-place + Box::new_in_place() #64062

petertodd · 2019-09-01T12:19:44Z

To start with, I'll say straight up this is more of a sketch of a possible API then something I'm sure we should merge.

The first part here is the Box::new_in_place() API. Basically the idea is that we could provide a way to reliably create large boxed values in-place via return-value-optimization. This is something that often is never possible to optimize, even in theory, because allocation is fallible and if the creation of the value is itself fallible, the optimizer can't move the allocation to happen first because that would be an observable change.

Unfortunately this doesn't quite work if the closure can't be inlined sufficiently due to optimizer limitations. But hopefully this at least shows a way forward that might be interesting to others.

Secondly, I use this Box::new_in_place() API to optimize Box::default(). I'm not totally sure this is a good idea to actually merge, as currently this is kind of an observable change in behavior for things like large arrays that would otherwise blow up the stack; people might rely on it, and currently the optimization is unreliable.

Finally, this of course this could be extended to Rc and Arc as well in much the same fashion.

rust-highfive · 2019-09-01T12:19:47Z

r? @kennytm

(rust_highfive has picked a reviewer for you, use r? to override)

Centril · 2019-09-01T14:22:25Z

src/liballoc/boxed.rs

+    /// FIXME: This is intended to work via return-value-optimization, but due to compiler
+    /// limitations can't reliably do that yet.
+    #[inline(always)]
+    fn new_in_place(f: impl FnOnce() -> T) -> Box<T> {


This is semantically more like new_please_try_to_do_it_in_place_but_I_am_not_expecting_any_guarantee?
(Such a guarantee is not something we can reliably or would make on a language (not LLVM) level.)

Point is, if we had guaranteed copy elision, I think this is as close as you could get with an API that looked something like this given Rust's semantics. Even with out-references it's not clear to me you could get semantics that nicely guarantee no copies in an easy way, as you could still mess up and wind up with a copy prior to writing it to the out-reference.

Like I said above, it's not clear to me this is actually a good idea. But it does seem novel. :)

edit: fixed thinko: we do have return-value-optimization; it's the guaranteed copy elision that we're missing.

Given that we do not guarantee copy elision we'll need to at minimum call this something other than new_in_place which suggests that there actually is a guarantee. For example: new_hint_in_place.

It might be a good idea to just use this internally for now... let's see how this measures up in terms of perf for fn default.

It might be a good idea to just use this internally for now

Yup, it is private right now, as I didn't want to give people false hope even in nightly. I'd suggest that we leave it that way until RVO is more reliable (specifically named-return-value-optimization if I understand the terminology right?).

Centril · 2019-09-01T18:39:20Z

@bors try

bors · 2019-09-01T18:39:31Z

⌛ Trying commit b0674d3 with merge 9d1bd60...

Attempt to make Box::default() construct in-place + Box::new_in_place() To start with, I'll say straight up this is more of a sketch of a possible API then something I'm sure we should merge. The first part here is the `Box::new_in_place()` API. Basically the idea is that we could provide a way to reliably create large boxed values in-place via return-value-optimization. This is something that often is *never* possible to optimize, even in theory, because allocation is fallible and if the creation of the value is itself fallible, the optimizer can't move the allocation to happen first because that would be an observable change. Unfortunately this doesn't quite work if the closure can't be inlined sufficiently due to optimizer limitations. But hopefully this at least shows a way forward that might be interesting to others. Secondly, I use this `Box::new_in_place()` API to optimize `Box::default()`. I'm not totally sure this is a good idea to actually merge, as currently this is kind of an observable change in behavior for things like large arrays that would otherwise blow up the stack; people might rely on it, and currently the optimization is unreliable. Finally, this of course this could be extended to `Rc` and `Arc` as well in much the same fashion.

bors · 2019-09-01T21:23:56Z

☀️ Try build successful - checks-azure
Build commit: 9d1bd60

Centril · 2019-09-01T21:25:46Z

@rust-timer build 9d1bd60

rust-timer · 2019-09-01T21:25:47Z

Success: Queued 9d1bd60 with parent e29faf0, comparison URL.

rust-timer · 2019-09-02T01:52:43Z

Finished benchmarking try commit 9d1bd60, comparison URL.

petertodd · 2019-09-02T12:53:04Z

Looks like this is on average very slightly slower than before.

I'll bet you this is a function of caching: by allocating first, you have to access allocation-related memory flushing both registers and cache, followed by doing the work to create the object. As most objects are fairly small anyway, having to reload registers and cache to do the work of creating it may be slightly more costly on average than the extra copy.

bjorn3 · 2019-09-02T16:22:01Z

box expr already allocate a box first and then evaluate expr. That means that Box::default would have already been subject to rvo if possible.

#![feature(box_syntax)]

fn main() {
    let _: Box<String> = box String::new();
}

// [...]
        _2 = Box(std::string::String);
        (*_2) = const std::string::String::new() -> [return: bb2, unwind: bb4];
// [...]

petertodd · 2019-09-05T01:48:16Z

box expr already allocate a box first and then evaluate expr. That means that Box::default would have already been subject to rvo if possible.

Ha, that's embarassing! Yeah, I wrote the initial version of this for Rc/Arc, and I was sure I'd double-checked the assembly for the Box version... But I guess not.

@Centril Just pushed another commit which adds Rc/Arc if you want to try that benchmarking thing again.

I suppose since the performance isn't significantly changed Box::new_in_place() still might be worth merging if only to provide a way to remove the old box syntax.

Centril · 2019-09-05T02:41:40Z

@Centril Just pushed another commit which adds Rc/Arc if you want to try that benchmarking thing again.

Sure; can't hurt.

@bors try

bors · 2019-09-05T02:41:52Z

⌛ Trying commit 9b57963 with merge a683cc0...

Attempt to make Box::default() construct in-place + Box::new_in_place() To start with, I'll say straight up this is more of a sketch of a possible API then something I'm sure we should merge. The first part here is the `Box::new_in_place()` API. Basically the idea is that we could provide a way to reliably create large boxed values in-place via return-value-optimization. This is something that often is *never* possible to optimize, even in theory, because allocation is fallible and if the creation of the value is itself fallible, the optimizer can't move the allocation to happen first because that would be an observable change. Unfortunately this doesn't quite work if the closure can't be inlined sufficiently due to optimizer limitations. But hopefully this at least shows a way forward that might be interesting to others. Secondly, I use this `Box::new_in_place()` API to optimize `Box::default()`. I'm not totally sure this is a good idea to actually merge, as currently this is kind of an observable change in behavior for things like large arrays that would otherwise blow up the stack; people might rely on it, and currently the optimization is unreliable. Finally, this of course this could be extended to `Rc` and `Arc` as well in much the same fashion.

bors · 2019-09-05T05:28:23Z

☀️ Try build successful - checks-azure
Build commit: a683cc0

Centril · 2019-09-05T06:25:34Z

@rust-timer build a683cc0

rust-timer · 2019-09-05T06:25:35Z

Success: Queued a683cc0 with parent f257c40, comparison URL.

rust-timer · 2019-09-05T08:18:31Z

Finished benchmarking try commit a683cc0, comparison URL.

petertodd · 2019-09-05T13:42:04Z

Heh, I wonder if that benchmark suit even has a single case of Rc::default() or Arc::default() being used?

Anyway, might be too rare to be worth optimizing.

Centril · 2019-09-05T15:40:28Z

@petertodd "The benchmark suite" is not those crates having Arc/Rc::default() but rather it is the performance of the compiler type checking, generating code, etc. for those crates. In other words, we are measuring compile time perf.

petertodd · 2019-09-10T18:17:10Z

@petertodd "The benchmark suite" is not those crates having Arc/Rc::default() but rather it is the performance of the compiler type checking, generating code, etc. for those crates. In other words, we are measuring compile time perf.

Ah ok, so basically my question boils down to whether or not the compiler uses Rc/Arc default()

Anyway, if you guys think this is worth merging, let me know and I'll fix up the comments appropriately.

Centril · 2019-09-10T23:25:33Z

Ah ok, so basically my question boils down to whether or not the compiler uses Rc/Arc default()

Basically yeah.

Anyway, if you guys think this is worth merging, let me know and I'll fix up the comments appropriately.

I'd drop the Box changes but potentially there's an improvement for Rc and Arc but it could also just be noise.

r? @nnethercote

nnethercote · 2019-09-11T09:21:21Z

The general comments I can make are as follows.

The performance improvement on the benchmarks is vanishingly small, right at the edge of what can be reliably detected.
The lack of certainty about whether the optimization is even working as expected is discouraging.
The commits introduce multiple uses of unsafe.

Based on those observations, my inclination is that these commits do not have enough of a benefit to warrant landing.

I am not comfortable reviewing the specifics of the code, because my understanding of things like MaybeUninit is almost zero. @pnkfelix might be a better reviewer, if you want to push forward.

r? @pnkfelix

pietroalbini · 2019-09-11T09:31:32Z

r? @pietroalbini

Just testing highfive.

pietroalbini · 2019-09-11T09:31:44Z

r? @pnkfelix

pietroalbini · 2019-09-11T09:33:57Z

Sorry y'all for the useless pings, trying to figure out why highfive didn't work earlier.

r? @nnethercote

pietroalbini · 2019-09-11T09:34:35Z

Ok, apparently it's working now? The weird thing is I can't find any errors that happened when the last two r? failed, everything is green...

r? @pnkfelix

JohnCSimon · 2019-09-21T01:52:21Z

ping from triage -
@pnkfelix @Centril @nnethercote
CC: @petertodd
Hi! This has sat idle for ten days. Is this ready for review?

Thanks

Centril · 2019-09-28T01:59:41Z

@pnkfelix Now that you're back, can you have a look at this please? // triage

JohnCSimon · 2019-10-05T20:02:50Z

Pinging again from triage
@pnkfelix ?
Thanks

JohnCSimon · 2019-10-12T05:25:21Z

This is still waiting on review
@pnkfelix @Centril
Thanks

pnkfelix · 2019-10-24T08:34:36Z

after reading @nnethercote 's evaluation above, I am closing this PR.

Some extra notes. I'll leave it up to the reader to decide how relevant they are.

There have been past efforts to generalize box (and placement in) to handle Rc/Arc (and other collection types like Vec); see e.g. Tracking issue for placement new #27779
I think the place we should be focusing on effort is on trying to eliminate unnecessary memcpy's in our generated code, in the compiler itself (and in the definition of the Rust ABI), rather than trying to fix it in the details of individual library types. See for example issue Consider aggressively avoiding generating memcpys for moves if we can avoid it #13707 and prototype PR [WIP] Implement a "place unification" MIR optimization (aka source/destination propagation aka NRVO). #47954

petertodd · 2019-10-31T17:09:41Z

@pnkfelix Seems reasonable. Thanks for the notes!

Attempt to make Box::default() construct in-place

b0674d3

rust-highfive assigned kennytm Sep 1, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 1, 2019

Centril reviewed Sep 1, 2019

View reviewed changes

Attempt to create Rc/Arc default values in place

9b57963

rust-highfive unassigned kennytm Sep 11, 2019

rust-highfive assigned pietroalbini Sep 11, 2019

rust-highfive assigned pnkfelix and unassigned pietroalbini Sep 11, 2019

rust-highfive assigned nnethercote and unassigned pnkfelix Sep 11, 2019

rust-highfive assigned pnkfelix and unassigned nnethercote Sep 11, 2019

pnkfelix closed this Oct 24, 2019

Attempt to make Box::default() construct in-place + Box::new_in_place() #64062

Attempt to make Box::default() construct in-place + Box::new_in_place() #64062

Conversation

petertodd commented Sep 1, 2019

rust-highfive commented Sep 1, 2019

Centril Sep 1, 2019

Choose a reason for hiding this comment

petertodd Sep 1, 2019 • edited Loading

Choose a reason for hiding this comment

Centril Sep 1, 2019

Choose a reason for hiding this comment

petertodd Sep 1, 2019

Choose a reason for hiding this comment

Centril commented Sep 1, 2019

bors commented Sep 1, 2019

bors commented Sep 1, 2019

Centril commented Sep 1, 2019

rust-timer commented Sep 1, 2019

rust-timer commented Sep 2, 2019

petertodd commented Sep 2, 2019

bjorn3 commented Sep 2, 2019 • edited Loading

petertodd commented Sep 5, 2019

Centril commented Sep 5, 2019

bors commented Sep 5, 2019

bors commented Sep 5, 2019

Centril commented Sep 5, 2019

rust-timer commented Sep 5, 2019

rust-timer commented Sep 5, 2019

petertodd commented Sep 5, 2019

Centril commented Sep 5, 2019

petertodd commented Sep 10, 2019

Centril commented Sep 10, 2019

nnethercote commented Sep 11, 2019 • edited Loading

pietroalbini commented Sep 11, 2019

pietroalbini commented Sep 11, 2019

pietroalbini commented Sep 11, 2019

pietroalbini commented Sep 11, 2019

JohnCSimon commented Sep 21, 2019

Centril commented Sep 28, 2019

JohnCSimon commented Oct 5, 2019

JohnCSimon commented Oct 12, 2019

pnkfelix commented Oct 24, 2019

petertodd commented Oct 31, 2019

petertodd Sep 1, 2019 •

edited

Loading

bjorn3 commented Sep 2, 2019 •

edited

Loading

nnethercote commented Sep 11, 2019 •

edited

Loading