Tracking Issue for array_zip

[usbalbin]

Feature gate: #![feature(array_zip)]

This is a tracking issue for the zip method on arrays.

Just like Iterator::zip combines two iterators into one, [T; N]::zip() returns a new array where every element is a tuple where the first element comes from the first array, and the second element comes from the second array. In other words, it zips two arrays together, into a single one.

let x = [1, 2, 3];
let y = [4, 5, 6];
let z = x.zip(y);
assert_eq!(z, [(1, 4), (2, 5), (3, 6)]);

Public API

pub fn zip<U>(self, rhs: [U; N]) -> [(T, U); N]

Steps / History

• Implementation: Added [T; N]::zip() #79451
• Stabilization PR

Unresolved Questions

@cramertj comment

[...] This seems like a pretty niche feature to me personally, but I can understand that there are situations when you might want it. I'm not sure if the libs team has a standard for "fill in the missing methods" PRs like this.

@yoshuawuyts comment

overlap with potential std::iter::Zip trait [...]

[leonardo-m]

Could you show two use cases?

[usbalbin]

Could you show two use cases?

What made me open the PR was for doing element wise operations on vector-like objects without the need for any heap allocations or unnecessary zeroing.

struct Vector([T; N]);

impl Add<Vector> for Vector {
    fn add(self, rhs: Vector) -> Vector {
        self.0.zip(rhs.0).map(|(lhs, rhs)| lhs + rhs)
    }
}

As for a second use case, well I will have to think about that one...

Maybe bit far-fetched but here goes:

const RES: usize = WIDTH * HEIGHT;
fn compute_pixels_for_screen_on_embedded_thing(
    background: [ImgColor; RES],
    object: [ImgColor; RES]
) -> [ScreenColor; RES] {
    background.zip(object).map(magic_color_mix)
}

(which I suppose is pretty much the same use case but with different names)

[cauebs]

Wouldn't this become obsolete after #65798?

[CryZe]

No because iterators don't encode the length in the type, so if you intend to go back to an array you'd have to unwrap.

[leonardo-m]

No because iterators don't encode the length in the type,

Can't we give arrays a different iterator that encodes the length too? :-)

[usbalbin]

No because iterators don't encode the length in the type,

Can't we give arrays a different iterator that encodes the length too? :-)

Wouldn't that be problematic in combination with methods like filter or skip etc?

[leonardo-m]

Yeah. We can't have a "filter" there. So only some iterators are possible. But "skip" is doable as long as the number of items to skip is given as a const generics value (and in my code the argument of "skip" is often a value known at compile-time).

[usbalbin]

No because iterators don't encode the length in the type,

Can't we give arrays a different iterator that encodes the length too? :-)

Would that be/work with #80470 or something completely different like new trait? Just asking since it seems like array IntoIter might be getting stabilized

[usbalbin]

@leonardo-m

Yeah. We can't have a "filter" there. So only some iterators are possible. But "skip" is doable as long as the number of items to skip is given as a const generics value (and in my code the argument of "skip" is often a value known at compile-time).

I have started doing some experimenting here. Atleast to me this feels like a much more reusable and flexible solution than adding all of those of methods directly to [T; N].

[MDoerner]

As somebody who has stumbled upon this unstable method while implementing some const generic vector operations, I would like to point out something regarding the presented use case. To actually efficiently lift traits like Add as component-wise operations, one would rather need a zip_with operation combining zip and map, maybe with some other name more consistent with rust conventions. Using the unstable array_map from #75243 on array_zip would potentially write the array twice, at least given the current implementation.

[usbalbin]

@MDoerner any thoughts on something like iter_fixed?

Edit:

There is an example here for vector addition

let x = [1, 2, 3];
let y = [4, 5, 6];

let z: [_; 3] = x
   .into_iter_fixed()
   .zip(y)
   .map(|(a, b)| a + b)
   .collect();

[linclelinkpart5]

This method has ended up being incredibly useful in my fixed-array audio DSP library, mainly in frame addition/multiplication. It allows me to do an element-wise binary operation without needing to use iterators (non-const? pah!) or having to create an incrementally-updated output array. The latter is especially nice in cases where the output type is expensive to instantiate: having to create N instances for a temporary output buffer that are immediately going to be overwritten is wasteful.

Something I'd love to suggest would be a corresponding unzip method, the inverse operation of zip. I imagine its signature looking like something along the lines of:

impl<T, U, const N: usize> [(T, U); N] {
    fn unzip(self) -> ([T; N], [U; N]) {
        // impl
    }
}

I'd be happy to work on an impl of unzip if others think it would be a useful method. I'd personally find quite a few good spots to use this in that DSP library.

[HKalbasi]

@usbalbin can we get rid of into_iter_fixed and make into_iter return a fixed iterator (which is an iterator as well)?

[usbalbin]

The problem is that regular Iterators shrink every time you call next() on them (unless they are infinite). So as far as I can tell, that makes it problematic (impossible?) to make a useful type that is both Iterator and IteratorFixed(I would be happy to be proven wrong :) ).

Note however that IteratorFixed does implement IntoIterator.

[HKalbasi]

hmm, yes it looks impossible. I was dreaming of an iterator with a Option<usize>, which next and similar methods are only for None (unknown sizes) but map and zip and ... can handle sized ones as well as unknowns. But it still needs two methods, one with None size for using next, and one with known size (or it can be generic but it will need explicit type annonations which is worse)

[vadixidav]

The problem is that regular Iterators shrink every time you call next() on them (unless they are infinite). So as far as I can tell, that makes it problematic (impossible?) to make a useful type that is both Iterator and IteratorFixed(I would be happy to be proven wrong :) ).

Note however that IteratorFixed does implement IntoIterator.

I think such a fixed iterator would have to implement a separate iterator trait, but that trait could have a method that converts it into a normal iterator. For instance, the fixed iterator could have methods that operate on it that transform it from one fixed length to another, and it could lazily evaluate to an array (no next() at all). However, the fixed iterator could have a method that converts it into a regular iterator. For instance:

// Note that an array would implement the fixed iterator trait (output equals the array itself).
let arr = [1, 2, 3];
// You could use the map method to change the array values.
let mapped = arr.map(|v| v*2);
// Then you could either turn the mapped fixed iterator into an array
let double_arr: [i32; 3] = mapped.array();
// or you could turn it into an iterator (note that the iterator values are of type `i32`, not `&i32`).
let sum: i32 = mapped.iter().sum();
// Note that it was used a second time here because the mapped array implements `Copy`.

[leonardo-m]

// Then you could either turn the mapped fixed iterator into an array
let double_arr: [i32; 3] = mapped.array();

Now that method needs to be named something like "lazy_map()"

[vadixidav]

@leonardo-m Yes, I suppose it is too late to create a lazy array trait with a map() method at this point.

[the8472]

In #103555 .zip().map() has been reported to be slow. #80094 (comment) mentions that kind of pattern as motivation.

Does anyone actually need the [(T, U); N] output? Or would a [T; N]::map_pairwise(other: [U; N]) -> [V; N] (or the iter_fixed crate) cover most needs?

[yoshuawuyts]

Potential concern: overlap with potential std::iter::Zip trait

I'm currently working on the futures-concurrency library as part of the Async WG. We're currently adding array::zip API which conflicts with the proposed API here. The API we're working on would work as a counterpart to our Merge trait, but providing a different output ordering:

//! Intended use if `Zip` were part of the stdlib.

use std::iter::{self, Zip};

let s1 = iter::repeat(0).take(5);
let s2 = iter::repeat(1).take(5);
let s3 = iter::repeat(2).take(5);

// zips N iterators together.
for [n1, n2, n3] in [s1, s2, s3].zip() {
    assert_eq!([n1, n2, n3], [0, 1, 2]);
}

// it works on tuples too!
for (n1, n2, n3) in (s1, s2, s3).zip() {
    assert_eq!((n1, n2, n3), (0, 1, 2));
}

In particular because I believe the plan is still to e.g. enable collect to work for arrays - which means what's provided by this API may be achievable through other means as well. In comparison: being able to zip arrays, tuples, and vecs through a shared API may be a lot harder, if not impossible to achieve through other means.

Conclusion

I'm not trying to say that just because we're experimenting with something, this API should be abandoned. But I'm raising this so that when a discussing pops up about potentially stabilizing this API, the fact that it would shut out (parts of ¹) the other design should be weighed into consideration.

Footnotes

1. It would certainly be possible to implement Zip just on tuples and vecs, but tbh that would be asymmetrical with how we're looking at implementing Merge and potentially Chain as well. array::zip as defined in this unstable tracking issue is the only API which would present a conflict, hence I'm raising this issue here. ↩

[the8472]

So, I'll repeat my previous question:

Is there anyone who's in favor of keeping zip as it is today or can we rip it out and then consider replacements separately?

[usbalbin]

@the8472 see my earlier comment, do you have any thoughts on that?

Edit: Just saw you mentioned it earlier.

[the8472]

Nominating this for removal since [(T, U); N] optimizes poorly and seems to be rarely what's really wanted.

[m-ou-se]

@rfcbot close

[rfcbot]

Team member @m-ou-se has proposed to close this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[scottmcm]

FWIW, this now at least codegens decently well for basic cases. For example, x.zip(y).map(|(x, y)| x - y) gives a simd subtraction, with no extra ceremony https://rust.godbolt.org/z/63fEGnKd6:

rust/tests/codegen/array-map.rs

Lines 21 to 27 in 669e751

	pub fn short_integer_zip_map(x: [u32; 8], y: [u32; 8]) -> [u32; 8] {
	// CHECK: %[[A:.+]] = load <8 x i32>
	// CHECK: %[[B:.+]] = load <8 x i32>
	// CHECK: sub <8 x i32> %[[A]], %[[B]]
	// CHECK: store <8 x i32>
	x.zip(y).map(|(x, y)| x - y)
	}

But I do also think that a zip_with is probably what I'd always actually want here, because eagerly getting tuples isn't particularly useful.

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to close, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.