docstring setindex

[matbesancon]

Even though it's not exported, it is nice to have the semantics available in a docstring, to use it elsewhere (see https://github.com/JuliaDiffEq/ArrayInterface.jl/pull/15/files)

[ChrisRackauckas]

is that intentional? Seems like odd behavior. I would assume it should boundscheck by default?

[matbesancon]

No idea, @vtjnash was the last to edit the line

[mbauman]

Was introduced in #20154 as a simple helper function. The behavior is very much intentional there, but perhaps we should see if we can add a @boundscheck block without sacrificing performance if we're going to move towards this being more documented.

[matbesancon]

Wouldn't it be considered as a breaking change then (replacing noop with BoundsError)?

[mbauman]

Not really as it was neither exported nor documented. That's why if we document it, it'd be good to make sure the semantics are what we want.

[tkf]

FYI StaticArrays.jl does bound check with setindex https://github.com/JuliaArrays/StaticArrays.jl/blob/2583967e282b21fd8f8a4e2fc306efc672b4d83d/src/deque.jl#L73-L86

[tkf]

I opened two issues in ArrayInterface.jl. But maybe this should be discussed in Base?

• Should setindex(x::AbstractArray, v, i) use mutate-or-widen strategy? · Issue #16 · JuliaDiffEq/ArrayInterface.jl
• Should setindex(x::AbstractArray, v, i) support arbitrary indices? · Issue #17 · JuliaDiffEq/ArrayInterface.jl

[matbesancon]

First take on @mbauman's idea:
https://gist.github.com/matbesancon/69fca8afdba2a65fc3a539cf18863a13

It seems slightly slower, but not all the time (which is curious)

[matbesancon]

@tkf for the moment only ArrayInterface.jl is using setindex on arrays, maybe this could be considered in Base

[matbesancon]

up here, I updated the gist above, there seems to be some significant performance hits when typeof(v) is not a bitstype (Array, String).
cc @mbauman

[matbesancon]

In any case, @inbounds removes the performance overhead (phew)

[matbesancon]

bump here, @mbauman?

[mbauman]

I don't think we should document it to be a no-op for out of bounds accesses. I think we should change the behavior to check bounds with @boundscheck before documenting it.

[matbesancon]

Thanks, I'll make the commits following the gist above

[matbesancon]

no idea why the tests are failing here, it seems unrelated to setindex

[mbauman]

No, I believe that's very much related. Looks like we've been relying on the ignore-out-of-bounds behavior in some internal implementations.

[matbesancon]

Sorry yes, it's a bit weird to have part of the CI running fine.
The error comes from selectdim:

@inline selectdim(A::AbstractArray, d::Integer, i) = _selectdim(A, d, i, setindex(map(Slice, axes(A)), i, d))
@noinline function _selectdim(A, d, i, idxs)
    d >= 1 || throw(ArgumentError("dimension must be ≥ 1, got $d"))
    nd = ndims(A)
    d > nd && (i == 1 || throw(BoundsError(A, (ntuple(k->Colon(),d-1)..., i))))
    return view(A, idxs...)
end

The good news is this should be fixable without breaking, but could lead to performance regression. Since we are errorring in setindex, the check should be before calling it, meaning we cannot @inline selectdim anymore as easily (there needs to be branching with error throwing here)

[mbauman]

I'd just copy the old implementation and call it something internally like _dropping_setindex or somesuch.

As far as why ~half the CI services report ✅ and the other half report ❌, it's because we've split the process in two parts. The "package" buildbots just build Julia whereas the "test" buildbots are the ones that actually run the test suite. There are other buildbots that specifically run a handful of other checks.

[matbesancon]

I'd just copy the old implementation and call it something internally like _dropping_setindex or somesuch.

The other possibility is using the intern _setindex implementation, just did that

[StefanKarpinski]

Any reason not to just merge this once CI passes?

[matbesancon]

this is changing some behavior if somebody was using the no-op as a feature

[mbauman]

I say we merge it — it's a (formerly) undocumented internal function, so if folks were relying on this somewhat obscure functionality it's on them.

[mbauman]

Thanks so much for the patience and endurance in pushing this one through, @matbesancon!

[matbesancon]

Thanks for the help along the way, my pleasure :)