Update mono to support shuffle for constant inputs

[tannergooding]

Mono wasn't accelerating shuffle for anything other than byte/sbyte on x64. This was in part because doing so required detecting inputs as constant which didn't exist when that support was first brought online.

However, I had since added basic support for constants last year in #81902. This PR adds support that mirrors most of the RyuJIT Shuffle importation logic and so we can now do the right thing. As a side effect, the general support for generating such constants for Create calls when all inputs are constants was also expanded which should have some indirect benefits in other places.

Shuffle having not being accelerated is very impactful to the entire BCL and I would expect this to have a broad impact across many core benchmarks including HexConverter, XxHash, TensorPrimitives, ReverseEndianness, Base64Decoder, Guid, Matrix4x4, Quaternion, IndexOfAny, ProbablisticMap, various SpanHelpers APIs and more.

[dotnet-policy-service]

Tagging subscribers to this area: @lambdageek, @steveisok
See info in area-owners.md if you want to be subscribed.

[tannergooding]

No diffs for x64 is expected above since the x64 changes are MonoLLVM only.

For arm64 it should be both MonoJIT and MonoLLVM

[EgorBo]

@EgorBot -arm64 -mono --envvars "MONO_VERBOSE_METHOD:ShuffleBench"

using System.Numerics;
using BenchmarkDotNet.Attributes;
using System.Runtime.Intrinsics;

public class StrengthReduction
{
    Vector128<int> srcVec = Vector128.Create(1, 2, 3, 4);
    Vector128<int> dstVec;

    [Benchmark]
    public void ShuffleBench() => 
        dstVec = Vector128.Shuffle(srcVec, Vector128.Create(2, 2, 2, 2));

    [Benchmark]
    public Quaternion ConcatenateBenchmark() => 
        Quaternion.Concatenate(Quaternion.Identity, Quaternion.Identity);
}

[EgorBot]

Benchmark results on Arm64

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Unknown processor
  Job-DMCBWN : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
  Job-EBPPKC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3048 ns	0.0015 ns	1.00
ShuffleBench	PR	0.2047 ns	0.0075 ns	0.01
ConcatenateBenchmark	Main	120.1574 ns	0.0091 ns	1.00
ConcatenateBenchmark	PR	NA	NA	?
Benchmarks with issues:
StrengthReduction.ConcatenateBenchmark: Job-EBPPKC(EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench, Toolchain=PR)

BDN_Artifacts.zip

[tannergooding]

Need to fix a MonoLLVM assert, but...

Before:

il_seq_point intr il: 0x0
 move R33 <- R32
 move R34 <- R32
 loadx_membase R35 <- [R32 + 0x10] [System.Runtime.Intrinsics.Vector128`1<int>]
 iconst R36 <- [2]
 iconst R37 <- [2]
 iconst R38 <- [2]
 iconst R39 <- [2]
 xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>]
 insert_i4 R41 <- R40 R36
 insert_i4 R42 <- R41 R37
 insert_i4 R43 <- R42 R38
 insert_i4 R44 <- R43 R39
 il_seq_point il: 0x10, nonempty-stack
 outarg_vtretaddr R46 <- R45
 move R47 <- R46
 outarg_vt R35
 outarg_vt R44
 xcall R45 <- [System.Runtime.Intrinsics.Vector128`1<int> System.Runtime.Intrinsics.Vector128:Shuffle (System.Runtime.Intrinsics.Vector128`1<int>,System.Runtime.Intrinsics.Vector128`1<int>)]
 il_seq_point il: 0x15, nonempty-stack
 compare_imm R32 [0]
 cond_exc_eq NullReferenceException
 storex_membase [R32 + 0x20] <- R45 [System.Runtime.Intrinsics.Vector128`1<int>]
 il_seq_point il: 0x1a
 nop

After:

il_seq_point intr il: 0x0
 move R33 <- R32
 move R34 <- R32
 loadx_membase R35 <- [R34 + 0x10] [System.Runtime.Intrinsics.Vector128`1<int>]
 iconst R36 <- [2]
 iconst R37 <- [2]
 iconst R38 <- [2]
 iconst R39 <- [2]
 xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>]
 xconst R41 <-
 il_seq_point il: 0x10, nonempty-stack
 xones R42 <-
 xop_ovr_x_x_x R43 <- R35 R42
 il_seq_point il: 0x15, nonempty-stack
 compare_imm R33 [0]
 cond_exc_eq NullReferenceException
 storex_membase [R33 + 0x20] <- R43 [System.Runtime.Intrinsics.Vector128`1<int>]
 il_seq_point il: 0x1a
 br [B1]

[tannergooding]

@matouskozak or @fanyang-mono, is there an easy way to remove MonoInst * that are known to be dead or is it perhaps handled elsewhere?

In a case like this, element will be dead because we're just folding it directly into the constant. In the case that ins is xzero or xones we will likewise be replacing it with an xconst that includes the underlying constant from element.

So in the example given above, the following IR is "dead":

iconst R36 <- [2]
iconst R37 <- [2]
iconst R38 <- [2]
iconst R39 <- [2]
xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>]

[matouskozak]

I haven't tried it but I think MONO_REMOVE_INS (or MONO_DELETE_INS) could work for this.

[tannergooding]

This should now be ready for review.

-- Please feel free to call out anywhere I messed up the styling. Tried to follow the conventions, but it's entirely possible I made some mistakes.

[EgorBo]

@EgorBot -arm64 -mono --envvars "MONO_VERBOSE_METHOD:ShuffleBench"

using System.Numerics;
using BenchmarkDotNet.Attributes;
using System.Runtime.Intrinsics;

public class StrengthReduction
{
    Vector128<int> srcVec = Vector128.Create(1, 2, 3, 4);
    Vector128<int> dstVec;

    [Benchmark]
    public void ShuffleBench() => 
        dstVec = Vector128.Shuffle(srcVec, Vector128.Create(2, 2, 2, 2));

    [Benchmark]
    public Quaternion ConcatenateBenchmark() => 
        Quaternion.Concatenate(Quaternion.Identity, Quaternion.Identity);
}

[EgorBot]

Benchmark results on Arm64

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Unknown processor
  Job-OKJKWC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
  Job-CZGEXJ : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3735 ns	0.0047 ns	1.00
ShuffleBench	PR	0.2321 ns	0.0052 ns	0.01
ConcatenateBenchmark	Main	118.2752 ns	0.0251 ns	1.00
ConcatenateBenchmark	PR	7.3540 ns	0.0022 ns	0.06

BDN_Artifacts.zip

[tannergooding]

Both of those numbers for MonoJIT are very good improvements:

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish)
Unknown processor
  Job-OKJKWC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
  Job-CZGEXJ : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase
EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3735 ns	0.0047 ns	1.00
ShuffleBench	PR	0.2321 ns	0.0052 ns	0.01
ConcatenateBenchmark	Main	118.2752 ns	0.0251 ns	1.00
ConcatenateBenchmark	PR	7.3540 ns	0.0022 ns	0.06

I had done some local testing for a few other scenarios and seen similar improvements to APIs using Shuffle. The IR from MONO_VERBOSE_METHOD generally looks good too, but I think there's a broader possibility to trim out some MonoInst in a later phase if they're unused. Such trimming I think is better suited for .NET 10, however.

[tannergooding]

For ShuffleBench the codegen is as below. There's still a few places that could be optimized further, but I don't think those are as prevalent for .NET 9 and are likely more involved/general-purpose.

Before

   0:	a9ba7bfd 	stp	x29, x30, [sp, #-96]!
   4:	910003fd 	mov	x29, sp
   8:	f9000bba 	str	x26, [x29, #16]
   c:	aa0003fa 	mov	x26, x0
  10:	d29c1000 	mov	x0, #0xe080                	// #57472
  14:	f2b6d620 	movk	x0, #0xb6b1, lsl #16
  18:	f2defa00 	movk	x0, #0xf7d0, lsl #32
  1c:	f9400011 	ldr	x17, [x0]
  20:	b4000051 	cbz	x17, 0x28
  24:	954b13d1 	bl	0x52c4f68
  28:	3dc00740 	ldr	q0, [x26, #16]
  2c:	3d800fa0 	str	q0, [x29, #48]
  30:	d2800044 	mov	x4, #0x2                   	// #2
  34:	d2800043 	mov	x3, #0x2                   	// #2
  38:	d2800042 	mov	x2, #0x2                   	// #2
  3c:	d2800041 	mov	x1, #0x2                   	// #2
  40:	4f00e400 	movi	v0.16b, #0x0
  44:	4e041c80 	mov	v0.s[0], w4
  48:	4e0c1c60 	mov	v0.s[1], w3
  4c:	4e141c40 	mov	v0.s[2], w2
  50:	4e1c1c20 	mov	v0.s[3], w1
  54:	3d8013a0 	str	q0, [x29, #64]
  58:	910083a0 	add	x0, x29, #0x20
  5c:	f9002ba0 	str	x0, [x29, #80]
  60:	f9401ba0 	ldr	x0, [x29, #48]
  64:	f9401fa1 	ldr	x1, [x29, #56]
  68:	f94023a2 	ldr	x2, [x29, #64]
  6c:	f94027a3 	ldr	x3, [x29, #72]
  70:	9400001c 	bl	0xe0
  74:	f9402bbe 	ldr	x30, [x29, #80]
  78:	a90007c0 	stp	x0, x1, [x30]
  7c:	eb1f035f 	cmp	x26, xzr
  80:	10000011 	adr	x17, 0x80
  84:	540000e0 	b.eq	0xa0  // b.none
  88:	3dc00ba0 	ldr	q0, [x29, #32]
  8c:	3d800b40 	str	q0, [x26, #32]
  90:	f9400bba 	ldr	x26, [x29, #16]
  94:	910003bf 	mov	sp, x29
  98:	a8c67bfd 	ldp	x29, x30, [sp], #96
  9c:	d65f03c0 	ret
  a0:	d28027c0 	mov	x0, #0x13e                 	// #318
  a4:	aa1103e1 	mov	x1, x17
  a8:	954b50ac 	bl	0x52d4358

After:

   0:	a9be7bfd 	stp	x29, x30, [sp, #-32]!
   4:	910003fd 	mov	x29, sp
   8:	f9000bba 	str	x26, [x29, #16]
   c:	aa0003fa 	mov	x26, x0
  10:	3dc00740 	ldr	q0, [x26, #16]
  14:	9c0001e1 	ldr	q1, 0x50
  18:	4e010000 	tbl	v0.16b, {v0.16b}, v1.16b
  1c:	eb1f035f 	cmp	x26, xzr
  20:	10000011 	adr	x17, 0x20
  24:	540000c0 	b.eq	0x3c  // b.none
  28:	3d800b40 	str	q0, [x26, #32]
  2c:	f9400bba 	ldr	x26, [x29, #16]
  30:	910003bf 	mov	sp, x29
  34:	a8c27bfd 	ldp	x29, x30, [sp], #32
  38:	d65f03c0 	ret
  3c:	d28027c0 	mov	x0, #0x13e                 	// #318
  40:	aa1103e1 	mov	x1, x17
  44:	94000007 	bl	0x60
  48:	b9006740 	str	w0, [x26, #100]
  4c:	b9806740 	ldrsw	x0, [x26, #100]
  50:	0b0a0908 	add	w8, w8, w10, lsl #2
  54:	0b0a0908 	add	w8, w8, w10, lsl #2
  58:	0b0a0908 	add	w8, w8, w10, lsl #2
  5c:	0b0a0908 	add	w8, w8, w10, lsl #2

[fanyang-mono]

LGTM! Just some minor format adjustment needed. Thanks for adding this for mono.

[kg]

I'm not sure whether interp does this kind of optimization, but jiterpreter does, just FYI. If it becomes important we could probably move this optimization from the jiterpreter up into the interpreter, but I think the only target where interpreter performance really matters right now is WASM.