ARM64-SVE: Fix conditional select for Zeroing predicates #102904 #105737

a74nh · 2024-07-31T07:29:22Z

Some SVE instructions use Pg/Z instead of Pg/M.
This means they cannot be wrapped in conditional select in the standard way - if they are then any inactive lanes will end up with zeros instead of the false value.

For example:
Sve.ConditionalSelect(mask, Sve.LoadVector(address), falseOp);

Cannot be converted into a single load:
LD1W zFalseVal, pMask/Z, xAddress

Instead two instructions are required:

LD1W zDest, pTrue/Z, xAddress
SEL zDest, pMask/M, zDest, zFalseVal

Fix by adding marking all relevant instructions and disabling the condition select optimisation for these instructions.

a74nh · 2024-07-31T07:37:48Z

Stress test results:

❯ ~/stress_tester.py $CORE_ROOT/corerun ./artifacts/tests/coreclr/linux.arm64.Checked/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/HardwareIntrinsics_Arm_ro.dll Sve_GatherVector
Starting test: /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root/corerun ./artifacts/tests/coreclr/linux.arm64.Checked/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/HardwareIntrinsics_Arm_ro.dll Sve_GatherVector
===================Running default===================
------------------- {} -------------------
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Bases_double_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_float_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_float_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_double_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_double_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVector_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorByteZeroExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16SignExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt16WithByteOffsetsSignExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32SignExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32WithByteOffsetsSignExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32WithByteOffsetsSignExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32WithByteOffsetsSignExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorInt32WithByteOffsetsSignExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorSByteSignExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16WithByteOffsetsZeroExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt16ZeroExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32WithByteOffsetsZeroExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Bases_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Bases_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_ulong_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorUInt32ZeroExtend_Indices_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_float_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_int_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_uint_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_float_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_int_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_uint_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_double_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_long_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_ulong_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_double_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_long_ulong() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_GatherVectorWithByteOffsets_ulong_ulong() : 2
===================Running jitstress===================
------------------- {'JitMinOpts': '1'} -------------------
------------------- {'JitStress': '1'} -------------------
------------------- {'JitStress': '2'} -------------------
------------------- {'JitStress': '1', 'TieredCompilation': '1'} -------------------
------------------- {'JitStress': '2', 'TieredCompilation': '1'} -------------------
------------------- {'TailcallStress': '1'} -------------------
------------------- {'ReadyToRun': '0'} -------------------
===================Running jitstressregs===================
------------------- {'JitStressRegs': '1'} -------------------
------------------- {'JitStressRegs': '2'} -------------------
------------------- {'JitStressRegs': '3'} -------------------
------------------- {'JitStressRegs': '4'} -------------------
------------------- {'JitStressRegs': '8'} -------------------
------------------- {'JitStressRegs': '0x10'} -------------------
------------------- {'JitStressRegs': '0x80'} -------------------
------------------- {'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStressRegs': '0x2000'} -------------------
===================Running jitstress2-jitstressregs===================
------------------- {'JitStress': '2', 'JitStressRegs': '1'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '2'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '3'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '4'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '8'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x10'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x80'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x2000'} -------------------

❯ ~/stress_tester.py $CORE_ROOT/corerun ./artifacts/tests/coreclr/linux.arm64.Checked/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/HardwareIntrinsics_Arm_ro.dll Sve_LoadVectorNonFaulting
Starting test: /home/alahay01/dotnet/runtime_sve/artifacts/tests/coreclr/linux.arm64.Checked/Tests/Core_Root/corerun ./artifacts/tests/coreclr/linux.arm64.Checked/JIT/HardwareIntrinsics/HardwareIntrinsics_Arm_ro/HardwareIntrinsics_Arm_ro.dll Sve_LoadVectorNonFaulting
===================Running default===================
------------------- {} -------------------
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_float() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_double() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_sbyte() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_short() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_int() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_long() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_byte() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_ushort() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_uint() : 2
Passed test: _Sve_ro::JIT.HardwareIntrinsics.Arm._Sve.Program.Sve_LoadVectorNonFaulting_ulong() : 2
===================Running jitstress===================
------------------- {'JitMinOpts': '1'} -------------------
------------------- {'JitStress': '1'} -------------------
------------------- {'JitStress': '2'} -------------------
------------------- {'JitStress': '1', 'TieredCompilation': '1'} -------------------
------------------- {'JitStress': '2', 'TieredCompilation': '1'} -------------------
------------------- {'TailcallStress': '1'} -------------------
------------------- {'ReadyToRun': '0'} -------------------
===================Running jitstressregs===================
------------------- {'JitStressRegs': '1'} -------------------
------------------- {'JitStressRegs': '2'} -------------------
------------------- {'JitStressRegs': '3'} -------------------
------------------- {'JitStressRegs': '4'} -------------------
------------------- {'JitStressRegs': '8'} -------------------
------------------- {'JitStressRegs': '0x10'} -------------------
------------------- {'JitStressRegs': '0x80'} -------------------
------------------- {'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStressRegs': '0x2000'} -------------------
===================Running jitstress2-jitstressregs===================
------------------- {'JitStress': '2', 'JitStressRegs': '1'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '2'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '3'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '4'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '8'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x10'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x80'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x1000'} -------------------
------------------- {'JitStress': '2', 'JitStressRegs': '0x2000'} -------------------

a74nh · 2024-07-31T07:50:51Z

@dotnet/arm64-contrib @kunalspathak

src/coreclr/jit/lowerarmarch.cpp

amanasifkhalid · 2024-08-05T15:08:26Z

ping @tannergooding
@a74nh looks like the recent refactor of hwintrinsiclistarm64sve.h created merge conflicts.

a74nh · 2024-08-05T16:18:54Z

@a74nh looks like the recent refactor of hwintrinsiclistarm64sve.h created merge conflicts.

Done! Thanks.

a74nh · 2024-08-12T14:43:48Z

ping @tannergooding

tannergooding · 2024-08-12T15:37:18Z

src/coreclr/jit/lowerarmarch.cpp

+        // If the nested op uses Pg/Z, then inactive lanes will result in zeros, so can only transform if
+        // op3 is all zeros.
+
+        if (nestedOp1->IsMaskAllBitsSet() &&
+            (!HWIntrinsicInfo::IsZeroingMaskedOperation(nestedOp2Id) || op3->IsVectorZero()))


Are there any instructions that only allow Pg/M?

Basically we have:

Pg/Z only

Pg/M -or- Pg/Z

So I'm wanting to discern if we also have:

Pg/M only

Yes, quite a few.
eg:
https://docsmirror.github.io/A64/2023-09/add_z_p_zz.html
ADD <Zdn>.<T>, <Pg>/M, <Zdn>.<T>, <Zm>.<T>

We likely need something that indicates that and avoids containment of zero for that case then, correct?

If it's <Pg>/M only then the optimisation will continue to work the same as it does in HEAD (because IsZeroingMaskedOperation() will be false)

tannergooding

LGTM assuming we don't have any Pg/M only instructions

dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 31, 2024

dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Jul 31, 2024

ARM64-SVE: Fix conditional select for Zeroing predicates

e4473d5

a74nh force-pushed the selectz_github branch from 4752d12 to e4473d5 Compare July 31, 2024 07:42

a74nh marked this pull request as ready for review July 31, 2024 07:50

a74nh requested a review from tannergooding July 31, 2024 07:51

a74nh changed the title ~~ARM64-SVE: Fix conditional select for Zeroing predicates~~ ARM64-SVE: Fix conditional select for Zeroing predicates (#102904) Jul 31, 2024

This was referenced Jul 31, 2024

MSBuild crashing in the build #92290

Open

TimeProviderTests.TestProviderTimer failed in CI #103459

Closed

msbuild crashes with "MSB0001: Internal MSBuild Error: must be valid" dotnet/dnceng#3304

Open

tannergooding reviewed Jul 31, 2024

View reviewed changes

src/coreclr/jit/lowerarmarch.cpp Outdated Show resolved Hide resolved

Allow if conditional select op3 is all zeros

b338154

build-analysis bot mentioned this pull request Jul 31, 2024

RestApiException`1: The response contained an invalid status code 500 Internal Server Error dotnet/dnceng#2298

Open

3 tasks

amanasifkhalid requested a review from tannergooding August 1, 2024 16:47

JulieLeeMSFT assigned a74nh Aug 1, 2024

JulieLeeMSFT added this to the 9.0.0 milestone Aug 1, 2024

a74nh changed the title ~~ARM64-SVE: Fix conditional select for Zeroing predicates (#102904)~~ ARM64-SVE: Fix conditional select for Zeroing predicates #102904 Aug 2, 2024

a74nh added 2 commits August 5, 2024 17:14

Merge main

21156d4

Remove tabs

158ff30

This was referenced Aug 5, 2024

[NativeAOT] Object synchronization method was called from an unsynchronized block of code. #104340

Closed

BinderTracingTest.ResolutionFlow times out #104670

Open

a74nh added 2 commits August 6, 2024 15:18

Merge main

d78532d

Remove newline

5c42f36

This was referenced Aug 6, 2024

System.IO.Net5Compat.Tests and System.IO.Tests suddenly exiting with error 137 #100558

Open

SIGKILL (OOM?) while running LibraryImportGenerator.Tests w/o actionable log messages or artifacts dotnet/dnceng#2496

Open

kunalspathak added the arm-sve Work related to arm64 SVE/SVE2 support label Aug 8, 2024

a74nh requested a review from TIHan August 12, 2024 14:44

tannergooding reviewed Aug 12, 2024

View reviewed changes

tannergooding approved these changes Aug 12, 2024

View reviewed changes

Merge main

e8967ec

This was referenced Aug 12, 2024

ARM64-SVE: Add GatherVectorWithByteOffsetFirstFaulting #106199

Merged

ARM64-SVE: Enable ConditionalSelect_FalseOp scenario in first-faulting test templates #106279

Closed

This was referenced Aug 12, 2024

Build failure: Static graph-based restore failed with exit code .* but did not log an error. #103526

Open

Build failure: Static graph-based restore failed with exit code .* but did not log an error. dotnet/dnceng#3139

Closed

a74nh and others added 2 commits August 13, 2024 10:22

Merge main

acaf565

Merge branch 'main' into selectz_github

737cb38

build-analysis bot mentioned this pull request Aug 13, 2024

Microsoft.Extensions.Hosting.Internal.HostTests.StartCanBeCancelled test failure #106135

Open

amanasifkhalid merged commit 329bff4 into dotnet:main Aug 13, 2024
115 of 117 checks passed

a74nh deleted the selectz_github branch August 13, 2024 15:08

build-analysis bot mentioned this pull request Aug 13, 2024

System.Threading.ThreadPools.Tests.ThreadPoolTests.IOCompletionPortCountConfigVarTest failure - time out, unexpected exit code #106206

Closed

github-actions bot locked and limited conversation to collaborators Sep 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARM64-SVE: Fix conditional select for Zeroing predicates #102904 #105737

ARM64-SVE: Fix conditional select for Zeroing predicates #102904 #105737

a74nh commented Jul 31, 2024 •

edited

Loading

a74nh commented Jul 31, 2024

a74nh commented Jul 31, 2024

amanasifkhalid commented Aug 5, 2024

a74nh commented Aug 5, 2024

a74nh commented Aug 12, 2024

tannergooding Aug 12, 2024

a74nh Aug 12, 2024

tannergooding Aug 12, 2024

a74nh Aug 12, 2024

tannergooding left a comment

ARM64-SVE: Fix conditional select for Zeroing predicates #102904 #105737

ARM64-SVE: Fix conditional select for Zeroing predicates #102904 #105737

Conversation

a74nh commented Jul 31, 2024 • edited Loading

a74nh commented Jul 31, 2024

a74nh commented Jul 31, 2024

amanasifkhalid commented Aug 5, 2024

a74nh commented Aug 5, 2024

a74nh commented Aug 12, 2024

tannergooding Aug 12, 2024

Choose a reason for hiding this comment

a74nh Aug 12, 2024

Choose a reason for hiding this comment

tannergooding Aug 12, 2024

Choose a reason for hiding this comment

a74nh Aug 12, 2024

Choose a reason for hiding this comment

tannergooding left a comment

Choose a reason for hiding this comment

a74nh commented Jul 31, 2024 •

edited

Loading