-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Perf] Linux/x64: 20 Regressions on 6/30/2023 4:39:46 PM #88375
Comments
Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*' PayloadsSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in Benchstone.BenchI.Fib
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.Fib*' PayloadsBenchstone.BenchI.Fib.TestHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository |
Most of the regressions are from this diff. Probably #88182. cc @jakobbotsch |
#88182 had no diffs. It's much more likely to be from #88151 and differences in spills. cc @tannergooding |
Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch Issue DetailsRun Information
Regressions in Burgers
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*' PayloadsBurgers.Test1Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Random
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*' PayloadsSystem.Tests.Perf_Random.NextSingle_unseededHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*' PayloadsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferentHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_EqualHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReferenceHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZeroHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferentHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferentHistogram
Description of detection logic
JIT DisasmsSystem.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrorsHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Numerics.Tests.Perf_BigInteger
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*' PayloadsSystem.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits)Histogram
Description of detection logic
JIT DisasmsSystem.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in PerfLabTests.LowLevelPerf
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*' PayloadsPerfLabTests.LowLevelPerf.GenericGenericMethodHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in Benchstone.BenchI.BubbleSort
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort*' PayloadsBenchstone.BenchI.BubbleSort.TestHistogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Memory.ReadOnlySpan
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*' PayloadsSystem.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*' PayloadsSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)Histogram
Description of detection logic
JIT DisasmsSystem.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Collections.TryGetValueFalse<String, String>
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<String, String>*' PayloadsSystem.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository Run Information
Regressions in System.Tests.Perf_Enum
ReproGeneral Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*' PayloadsSystem.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)Histogram
Description of detection logic
JIT DisasmsDocsProfiling workflow for dotnet/runtime repository
|
This is expected and within a reasonable amount for the APIs in question. We took a change to the LSRA heuristics which gave an overall size win, but which resulted in some size regressions in other APIs. As Jakob pointed out, a lot of this is just due to slightly different register selection resulting in different spillage and there's not much that can be done without changing how LSRA works more deeply. |
The regression in Burgers is pretty sizeable, did you look at that one in any detail? |
Run Information
Regressions in Burgers
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
Burgers.Test1
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Random
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Tests.Perf_Random.NextSingle_unseeded
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent
Histogram
Description of detection logic
JIT Disasms
System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Numerics.Tests.Perf_BigInteger
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits)
Histogram
Description of detection logic
JIT Disasms
System.Numerics.Tests.Perf_BigInteger.ToStringD(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in PerfLabTests.LowLevelPerf
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
PerfLabTests.LowLevelPerf.GenericGenericMethod
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in Benchstone.BenchI.BubbleSort
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
Benchstone.BenchI.BubbleSort.Test
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Memory.ReadOnlySpan
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)
Histogram
Description of detection logic
JIT Disasms
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Collections.TryGetValueFalse<String, String>
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Collections.TryGetValueFalse<String, String>.ImmutableDictionary(Size: 512)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
Regressions in System.Tests.Perf_Enum
Test Report
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
Baseline
Compare
System.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)
Histogram
Description of detection logic
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
The text was updated successfully, but these errors were encountered: