Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/x64: 77 Regressions on 1/12/2023 10:41:19 PM #11805

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 8 comments
Closed

[Perf] Linux/x64: 77 Regressions on 1/12/2023 10:41:19 PM #11805

performanceautofiler bot opened this issue Jan 17, 2023 · 8 comments
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivideBenchmark - Duration of single invocation 137.52 ns 239.78 ns 1.74 0.08 False
MinBenchmark - Duration of single invocation 138.20 ns 249.60 ns 1.81 0.08 False
EqualsBenchmark - Duration of single invocation 221.49 ns 264.10 ns 1.19 0.06 False
GreaterThanOrEqualBenchmark - Duration of single invocation 132.05 ns 239.12 ns 1.81 0.05 False
LessThanBenchmark - Duration of single invocation 126.35 ns 238.33 ns 1.89 0.07 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.DivideBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 239.78326904318521 > 145.7056279443874.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -62.92614700555615 (T) = (0 -242.04284857728683) / Math.Sqrt((4.857687266357598 / (32)) + (61.23464130930432 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.7465614152080653 = (138.58250071810536 - 242.04284857728683) / 138.58250071810536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 249.59609728108902 > 145.23576672508088.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -117.8389619591579 (T) = (0 -252.94972729768557) / Math.Sqrt((18.441501519774985 / (32)) + (8.645554871175577 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8094660703675952 = (139.79246775613677 - 252.94972729768557) / 139.79246775613677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.EqualsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 264.0960326799239 > 217.98523571550865.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -73.88132551651269 (T) = (0 -266.069189898891) / Math.Sqrt((12.473771179563865 / (32)) + (5.053481920047308 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.2716806403356636 = (209.22642168135965 - 266.069189898891) / 209.22642168135965 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.GreaterThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 239.12147916325986 > 137.30046854430077.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -179.82009526464637 (T) = (0 -241.67389329747115) / Math.Sqrt((3.680870663547209 / (32)) + (6.5878563805236245 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8442999651928513 = (131.03827894514993 - 241.67389329747115) / 131.03827894514993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;.LessThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 238.32637952665613 > 137.58252083018328.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -72.46186131677658 (T) = (0 -243.1522791240992) / Math.Sqrt((25.642278102067603 / (32)) + (41.307371445105794 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8754824597617049 = (129.64785560030973 - 243.1522791240992) / 129.64785560030973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsBenchmark - Duration of single invocation 373.60 ns 411.55 ns 1.10 0.09 False
LessThanBenchmark - Duration of single invocation 198.13 ns 299.26 ns 1.51 0.04 False
MinBenchmark - Duration of single invocation 220.94 ns 321.49 ns 1.46 0.07 False
DivideBenchmark - Duration of single invocation 199.07 ns 291.52 ns 1.46 0.07 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 411.5549619479343 > 389.04085974007484.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -25.26018713512362 (T) = (0 -419.66830106787876) / Math.Sqrt((12.176484333057259 / (32)) + (82.99798341138269 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.13094817836171163 = (371.07650827627583 - 419.66830106787876) / 371.07650827627583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.LessThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 299.25646168249426 > 208.27390206923155.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -116.13405537802774 (T) = (0 -301.3053771414485) / Math.Sqrt((16.644786779204516 / (32)) + (6.02804944391917 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.5066971831368365 = (199.9773946043704 - 301.3053771414485) / 199.9773946043704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 321.49277961980556 > 231.58282790293356.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -126.50709747610259 (T) = (0 -324.39502227602543) / Math.Sqrt((7.527218490332015 / (32)) + (10.737648641137612 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.4661760669908768 = (221.25243316909493 - 324.39502227602543) / 221.25243316909493 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;.DivideBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 291.52410503058996 > 211.02126801992614.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -88.63810620845122 (T) = (0 -296.53818230309395) / Math.Sqrt((11.778123558989323 / (32)) + (18.65011798423453 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.4703045997789252 = (201.6848633593892 - 296.53818230309395) / 201.6848633593892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivisionOperatorBenchmark - Duration of single invocation 122.62 ns 235.07 ns 1.92 0.09 False
AbsBenchmark - Duration of single invocation 111.57 ns 190.20 ns 1.70 0.05 False
EqualsStaticBenchmark - Duration of single invocation 117.11 ns 224.37 ns 1.92 0.11 False
MinBenchmark - Duration of single invocation 124.69 ns 229.11 ns 1.84 0.09 False
MaxBenchmark - Duration of single invocation 129.82 ns 229.45 ns 1.77 0.06 False
LessThanOrEqualBenchmark - Duration of single invocation 124.15 ns 222.72 ns 1.79 0.07 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.071794873311 > 129.41030711139453.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -62.38201284693511 (T) = (0 -239.42517436781742) / Math.Sqrt((1.225090363496794 / (32)) + (85.27223182806374 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.9375732676185546 = (123.56961069249871 - 239.42517436781742) / 123.56961069249871 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.AbsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.19876336898398 > 116.73973930577324.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -104.29993533027215 (T) = (0 -183.76243917138999) / Math.Sqrt((1.9219023664278065 / (32)) + (10.474039534398969 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.6469850356082629 = (111.57505089505747 - 183.76243917138999) / 111.57505089505747 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 224.37376745223264 > 123.07410269265753.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -80.53510204294945 (T) = (0 -228.09564914771036) / Math.Sqrt((5.203942379751406 / (32)) + (43.12616959570394 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.9421245677624438 = (117.44645679988663 - 228.09564914771036) / 117.44645679988663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.10634475664207 > 130.8793014919083.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -90.97902672906437 (T) = (0 -231.23745224985544) / Math.Sqrt((11.037228052914083 / (32)) + (25.09887851087782 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.841448345788302 = (125.57368376839561 - 231.23745224985544) / 125.57368376839561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.MaxBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.45101078332132 > 134.28677208947195.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -104.14810447626542 (T) = (0 -232.63094762798548) / Math.Sqrt((23.25037529433001 / (32)) + (6.866529153703084 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8115060634880699 = (128.4185310316062 - 232.63094762798548) / 128.4185310316062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 222.72043914535294 > 127.53829279041706.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -156.35278781574246 (T) = (0 -226.10621056185343) / Math.Sqrt((9.431726248155966 / (32)) + (3.948136337679055 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8700182086057203 = (120.91123472558992 - 226.10621056185343) / 120.91123472558992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DivisionOperatorBenchmark - Duration of single invocation 117.62 ns 230.75 ns 1.96 0.07 False
LessThanOrEqualBenchmark - Duration of single invocation 122.16 ns 222.97 ns 1.83 0.09 False
MinBenchmark - Duration of single invocation 123.65 ns 226.34 ns 1.83 0.11 False
EqualityOperatorBenchmark - Duration of single invocation 53.32 ns 59.80 ns 1.12 0.08 False
MaxBenchmark - Duration of single invocation 127.30 ns 229.60 ns 1.80 0.08 False
EqualsStaticBenchmark - Duration of single invocation 117.15 ns 225.40 ns 1.92 0.05 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.DivisionOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 230.74667845911947 > 125.55849542783423.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -120.79242488525908 (T) = (0 -234.11941651378766) / Math.Sqrt((7.255181747028935 / (32)) + (16.538182483799662 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.9465173388094534 = (120.27604986913802 - 234.11941651378766) / 120.27604986913802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.LessThanOrEqualBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 222.97179262854888 > 126.54190675764688.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -50.110757794026945 (T) = (0 -227.28064891482535) / Math.Sqrt((82.97765516612247 / (32)) + (44.591725480070686 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8561724566137753 = (122.4458687041691 - 227.28064891482535) / 122.4458687041691 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 226.34248667424427 > 129.90613279403058.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -120.4254201314741 (T) = (0 -230.8862695254785) / Math.Sqrt((18.31251062161677 / (32)) + (4.7577439416482985 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8364523752613432 = (125.72407138661647 - 230.8862695254785) / 125.72407138661647 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualityOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.80106621116119 > 55.955546119504085.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -25.11298672781257 (T) = (0 -62.34532957588713) / Math.Sqrt((0.6161789542041043 / (32)) + (2.7275020539171 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.1686500437900982 = (53.348160047718274 - 62.34532957588713) / 53.348160047718274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.MaxBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.59604148155793 > 133.45268282450328.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -83.52201940373402 (T) = (0 -234.06463737991461) / Math.Sqrt((11.969175289512231 / (32)) + (31.66015207250724 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8417260185061762 = (127.08982499458004 - 234.06463737991461) / 127.08982499458004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 225.39653108144662 > 123.65758433016761.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -116.29496690859527 (T) = (0 -227.73351088949) / Math.Sqrt((14.26079309043965 / (32)) + (11.078981972261525 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.9284008495542622 = (118.09448794949928 - 227.73351088949) / 118.09448794949928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 182.18 ns 300.22 ns 1.65 0.07 False
EqualsStaticBenchmark - Duration of single invocation 198.88 ns 301.21 ns 1.51 0.06 False
GreaterThanBenchmark - Duration of single invocation 199.35 ns 297.04 ns 1.49 0.08 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Float.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 300.2151498404622 > 192.9212923534773.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -97.80208074181492 (T) = (0 -291.30529762088145) / Math.Sqrt((23.97553212656678 / (32)) + (10.609602429121201 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.5716272340489081 = (185.35266589292013 - 291.30529762088145) / 185.35266589292013 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 301.2091968151218 > 208.84432604494964.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -111.76288184870711 (T) = (0 -300.7728918956396) / Math.Sqrt((16.027959624658095 / (32)) + (7.468285611063254 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.49762085672827344 = (200.83380285762905 - 300.7728918956396) / 200.83380285762905 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 297.0394915120924 > 209.10199838017616.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -103.4721340748139 (T) = (0 -300.9624220376325) / Math.Sqrt((16.514151049636794 / (32)) + (10.85753499003227 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.504130014037517 = (200.0906964350528 - 300.9624220376325) / 200.0906964350528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CallBlockCopy - Duration of single invocation 175.05 ns 235.92 ns 1.35 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.9167245464824 > 181.47427378807078.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -35.71515758775375 (T) = (0 -239.28315545450133) / Math.Sqrt((4.495517792202353 / (32)) + (83.48585873404905 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.3858780128400046 = (172.65816560878352 - 239.28315545450133) / 172.65816560878352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 2.70 μs 6.58 μs 2.44 0.03 False
GetString - Duration of single invocation 2.86 μs 6.62 μs 2.31 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 512, encName: "utf-8")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.580515703337362 > 2.8247133263483333.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -160.55700788617023 (T) = (0 -6667.036315695524) / Math.Sqrt((415.20396635140196 / (32)) + (14352.281708731032 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -1.4707694803709341 = (2698.364363273019 - 6667.036315695524) / 2698.364363273019 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Tests.Perf_Encoding.GetString(size: 512, encName: "utf-8")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.616170613519863 > 2.9846442547668057.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -433.4323366736403 (T) = (0 -6647.839346301339) / Math.Sqrt((889.4411035901543 / (32)) + (1233.9514832999405 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.3402059789199101 = (2840.7069318614244 - 6647.839346301339) / 2840.7069318614244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadToEnd - Duration of single invocation 340.82 ms 627.47 ms 1.84 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 627.4743334666667 > 355.03111879375.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -150.16914242334278 (T) = (0 -627168770.1857053) / Math.Sqrt((28927595637383.953 / (32)) + (68568286240382.21 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8424463942995691 = (340400009.5341346 - 627168770.1857053) / 340400009.5341346 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 62.78 ns 461.29 ns 7.35 0.15 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 461.293140818357 > 65.72868218566349.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -14.324578399714408 (T) = (0 -362.7279077775698) / Math.Sqrt((2.9714133402336684 / (32)) + (10986.586628369296 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -4.812523604848054 = (62.40454791014168 - 362.7279077775698) / 62.40454791014168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MultiplyOperatorBenchmark - Duration of single invocation 151.89 ns 258.64 ns 1.70 0.09 False
GreaterThanBenchmark - Duration of single invocation 176.69 ns 264.26 ns 1.50 0.05 False
EqualsStaticBenchmark - Duration of single invocation 179.06 ns 266.10 ns 1.49 0.10 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.MultiplyOperatorBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 258.6374761287969 > 160.32017867016958.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -78.56268290635943 (T) = (0 -261.3135639823522) / Math.Sqrt((4.624272963168113 / (32)) + (43.98510120167915 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.7089265781697557 = (152.91093679531662 - 261.3135639823522) / 152.91093679531662 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.GreaterThanBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 264.26459781429855 > 186.19863831153282.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -84.80742479270126 (T) = (0 -267.00825301606443) / Math.Sqrt((24.68951063154463 / (32)) + (7.4270938718275445 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.4888473067585182 = (179.33890990969937 - 267.00825301606443) / 179.33890990969937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 266.1033640822852 > 188.2437218437087.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -110.61892294830596 (T) = (0 -270.69163345414665) / Math.Sqrt((1.5371248491428882 / (32)) + (15.771201479715982 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.507629326160178 = (179.547869464425 - 270.69163345414665) / 179.547869464425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadAllBytes - Duration of single invocation 26.04 μs 28.39 μs 1.09 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_File*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_File.ReadAllBytes(size: 4096)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.394403771289532 > 27.405245808560444.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -16.316092222625148 (T) = (0 -28160.68233680431) / Math.Sqrt((241279.54508228318 / (32)) + (138056.21148729086 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.07158113151773715 = (26279.561582909588 - 28160.68233680431) / 26279.561582909588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 862.38 ns 3.43 μs 3.97 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'i', newChar: 'I')

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4270579117668807 > 907.6348953586129.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -191.9786859766817 (T) = (0 -3432.0835447267064) / Math.Sqrt((149.76101609890586 / (32)) + (4353.725276153444 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -2.9685955144758616 = (864.8106193256097 - 3432.0835447267064) / 864.8106193256097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 145.13 ns 161.43 ns 1.11 0.11 False
HashSet - Duration of single invocation 135.58 ns 157.53 ns 1.16 0.07 False
LinkedList - Duration of single invocation 16.71 ns 21.54 ns 1.29 0.57 False
ConcurrentDictionary - Duration of single invocation 352.60 ns 391.09 ns 1.11 0.06 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.Dictionary

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 161.4250312633209 > 154.12217724738156.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -15.704717616151504 (T) = (0 -163.43636045960588) / Math.Sqrt((15.0773302165141 / (32)) + (13.45502795994379 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.10685555306437317 = (147.65825586466624 - 163.43636045960588) / 147.65825586466624 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.HashSet

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.53140764832173 > 142.5106273003012.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -24.139077311043525 (T) = (0 -155.9166350018236) / Math.Sqrt((11.389468899244566 / (32)) + (6.3045283827269545 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.1373075306533733 = (137.09276585221488 - 155.9166350018236) / 137.09276585221488 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.LinkedList

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.543923986851848 > 17.13677647446255.
IsChangePoint: Marked as a change because one of 12/20/2022 6:23:25 PM, 1/11/2023 4:46:06 PM, 1/12/2023 10:41:19 PM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -6.728065958130929 (T) = (0 -19.249349728925363) / Math.Sqrt((9.17555696385456 / (33)) + (1.5827470346231742 / (24))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (24) - 2, .025) and -0.2578589203311004 = (15.30326606409759 - 19.249349728925363) / 15.30326606409759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CtorDefaultSize&lt;String&gt;.ConcurrentDictionary

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 391.0930863633426 > 369.0665727348711.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -14.91802998961117 (T) = (0 -386.1249031470745) / Math.Sqrt((54.545897194718556 / (32)) + (58.321612172127 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.0841649341594824 = (356.14959586054545 - 386.1249031470745) / 356.14959586054545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformByMatrix4x4Benchmark - Duration of single invocation 130.39 ns 326.31 ns 2.50 0.12 False
TransformNormalByMatrix4x4Benchmark - Duration of single invocation 120.53 ns 320.19 ns 2.66 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 326.3069783349394 > 139.70656907169382.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -163.19932253318726 (T) = (0 -327.45680457555557) / Math.Sqrt((5.357463777090845 / (32)) + (31.217289929388826 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.457484732456532 = (133.2487645806148 - 327.45680457555557) / 133.2487645806148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 320.1913533820456 > 129.058919617134.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -214.48268566887862 (T) = (0 -315.9637704032151) / Math.Sqrt((7.955238697586974 / (32)) + (13.858751870411504 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.5528074730243233 = (123.77109270558947 - 315.9637704032151) / 123.77109270558947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation 119.80 ns 173.42 ns 1.45 0.10 False
MultiplyByMatrixOperatorBenchmark - Duration of single invocation 167.15 ns 409.18 ns 2.45 0.07 False
CreateFromScalars - Duration of single invocation 46.25 ns 108.59 ns 2.35 0.23 False
LerpBenchmark - Duration of single invocation 171.31 ns 446.26 ns 2.60 0.09 False
SubtractBenchmark - Duration of single invocation 140.94 ns 329.18 ns 2.34 0.07 False
NegateBenchmark - Duration of single invocation 117.98 ns 209.27 ns 1.77 0.11 False
MultiplyByScalarBenchmark - Duration of single invocation 114.07 ns 227.75 ns 2.00 0.09 False
CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation 94.93 ns 162.60 ns 1.71 0.15 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 173.42288163789695 > 127.09581515273516.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -35.53261131024519 (T) = (0 -158.74219279844814) / Math.Sqrt((17.097384977621036 / (32)) + (13.333562554005686 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.3049362736097586 = (121.64746739649604 - 158.74219279844814) / 121.64746739649604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 409.18190934234207 > 175.7779685452694.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -155.85981928652723 (T) = (0 -406.52629134066643) / Math.Sqrt((3.825988998942288 / (32)) + (55.463015002721306 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.4168601899853992 = (168.20430615935726 - 406.52629134066643) / 168.20430615935726 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateFromScalars

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.58557701756773 > 50.253721947413524.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -67.53809153683437 (T) = (0 -100.63858300784035) / Math.Sqrt((1.9894937876806784 / (32)) + (13.137095756453078 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.0595327243797437 = (48.86476520451941 - 100.63858300784035) / 48.86476520451941 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 446.2600237919245 > 180.04260958157775.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -190.05132651532028 (T) = (0 -452.18451217119565) / Math.Sqrt((5.62255435477078 / (32)) + (50.38203004444409 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.6463514066330645 = (170.87092479018386 - 452.18451217119565) / 170.87092479018386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 329.17875070015117 > 149.48420457806736.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -109.3756254530465 (T) = (0 -312.7667976937235) / Math.Sqrt((2.8211821708609923 / (32)) + (58.716436688779574 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.202148875922534 = (142.0279987031748 - 312.7667976937235) / 142.0279987031748 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 209.2740377873595 > 121.32104962189658.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -37.67761359939223 (T) = (0 -195.84711724641983) / Math.Sqrt((2.647743151227947 / (32)) + (110.79125967740102 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.6913537056018615 = (115.79311683757385 - 195.84711724641983) / 115.79311683757385 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 227.75494537989195 > 122.94983179766487.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -48.37546925384439 (T) = (0 -219.8593921647833) / Math.Sqrt((3.8959895711034602 / (32)) + (107.73599979489343 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.8627867896340567 = (118.02713729142052 - 219.8593921647833) / 118.02713729142052 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.60417573227016 > 100.73701888304164.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -48.587018894789736 (T) = (0 -163.21247375009708) / Math.Sqrt((46.33634067799166 / (32)) + (9.284165103160545 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.6709540876175971 = (97.67621681503002 - 163.21247375009708) / 97.67621681503002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 974.81 ns 1.06 μs 1.09 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.064843368450429 > 1019.4330515320617.
IsChangePoint: Marked as a change because one of 12/7/2022 5:17:51 AM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -30.713738717548964 (T) = (0 -1057.0501668549161) / Math.Sqrt((69.4301712600632 / (32)) + (124.37985655251548 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.08420674431155552 = (974.9525839060491 - 1057.0501668549161) / 974.9525839060491 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Ctor_Span - Duration of single invocation 372.98 ns 602.34 ns 1.61 0.07 False
Ctor_Bytes - Duration of single invocation 392.41 ns 618.00 ns 1.57 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

Histogram

System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 602.337381905501 > 392.14996404141766.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -76.09146415551993 (T) = (0 -602.4141358553644) / Math.Sqrt((113.48742337330344 / (32)) + (127.46633409255799 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.6023650674717953 = (375.95311336002277 - 602.4141358553644) / 375.95311336002277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 617.9997272996328 > 414.03709896529136.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -65.37161430619418 (T) = (0 -621.0374014104322) / Math.Sqrt((242.2161096349737 / (32)) + (94.58175643814315 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.5495814603565411 = (400.7775114110739 - 621.0374014104322) / 400.7775114110739 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinarySearch - Duration of single invocation 346.77 ns 459.03 ns 1.32 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.BinarySearch(Size: 33)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 459.02940376231646 > 366.0539344933922.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -38.95045786771885 (T) = (0 -456.13097702057513) / Math.Sqrt((153.6094627568783 / (32)) + (54.705203552598825 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.29156151668342384 = (353.1624093228367 - 456.13097702057513) / 353.1624093228367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformByMatrix4x4Benchmark - Duration of single invocation 63.72 ns 266.79 ns 4.19 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 266.79173823193935 > 67.09666065337795.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -272.4932940073264 (T) = (0 -268.4090379147886) / Math.Sqrt((3.4582275217135146 / (32)) + (11.418125895039752 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -3.218766033515023 = (63.62264126108779 - 268.4090379147886) / 63.62264126108779 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsAllBenchmark - Duration of single invocation 70.23 ns 80.77 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsAllBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.76920803274348 > 73.71180868047378.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -35.05888536444951 (T) = (0 -83.54061982594777) / Math.Sqrt((0.8538492088171243 / (32)) + (2.7639446807256802 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.18408808837873566 = (70.55270688545846 - 83.54061982594777) / 70.55270688545846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_capacity - Duration of single invocation 158.05 ns 190.05 ns 1.20 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ctor_capacity(length: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 190.04895177687743 > 167.6519154152612.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -27.554446439141657 (T) = (0 -193.14143446964118) / Math.Sqrt((4.485303874765246 / (32)) + (33.45900187444263 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.2098811324736881 = (159.6367025533738 - 193.14143446964118) / 159.6367025533738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateScaleFromScalarXYZBenchmark - Duration of single invocation 145.43 ns 267.37 ns 1.84 0.20 False
CreateRotationXBenchmark - Duration of single invocation 107.09 ns 243.35 ns 2.27 0.21 False
GetDeterminantBenchmark - Duration of single invocation 197.20 ns 410.85 ns 2.08 0.11 False
AddOperatorBenchmark - Duration of single invocation 219.95 ns 719.68 ns 3.27 0.06 False
CreateRotationYWithCenterBenchmark - Duration of single invocation 155.60 ns 251.94 ns 1.62 0.14 False
CreateRotationYBenchmark - Duration of single invocation 109.01 ns 236.09 ns 2.17 0.23 False
MultiplyByScalarBenchmark - Duration of single invocation 195.24 ns 534.86 ns 2.74 0.08 False
CreateRotationZBenchmark - Duration of single invocation 109.71 ns 239.72 ns 2.19 0.25 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 267.36587082778345 > 144.3023931519775.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -110.97147532115069 (T) = (0 -268.16223103614783) / Math.Sqrt((21.69942680782975 / (32)) + (16.753374910050542 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.9249454322106296 = (139.30900406261762 - 268.16223103614783) / 139.30900406261762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 243.35231587192834 > 114.27841643597974.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -101.0812094647465 (T) = (0 -241.8908637977231) / Math.Sqrt((14.80055913975751 / (32)) + (30.353733597881725 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.1791051489409607 = (111.00467727098044 - 241.8908637977231) / 111.00467727098044 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.GetDeterminantBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.8458555198496 > 196.29548674890108.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -115.50512929862703 (T) = (0 -396.9250894059169) / Math.Sqrt((40.29596166309849 / (32)) + (49.11178192313738 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.094162150107908 = (189.53885179591472 - 396.9250894059169) / 189.53885179591472 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 719.6838980270178 > 232.22943105939828.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -149.69867245877913 (T) = (0 -734.2416855472725) / Math.Sqrt((15.424123114604344 / (32)) + (279.93189998823703 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -2.297783120206161 = (222.64705069549004 - 734.2416855472725) / 222.64705069549004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 251.93841305086394 > 161.27688612864682.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -84.44705902753253 (T) = (0 -254.653851198698) / Math.Sqrt((21.121659291438494 / (32)) + (16.92172559745749 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.6218782857398675 = (157.01169035784343 - 254.653851198698) / 157.01169035784343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 236.08627881429695 > 114.75020848260911.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -121.43011748478423 (T) = (0 -240.3165761998661) / Math.Sqrt((10.687728007412419 / (32)) + (20.098638211342244 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.1692735344537966 = (110.78205324639966 - 240.3165761998661) / 110.78205324639966 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 534.8553787309497 > 207.64116786938504.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -166.4069884594007 (T) = (0 -516.1062295821789) / Math.Sqrt((8.895914185326962 / (32)) + (80.84860041091387 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -1.6022553806511344 = (198.33035351551055 - 516.1062295821789) / 198.33035351551055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 239.72272926803117 > 115.09511511238712.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -51.85583084586204 (T) = (0 -243.24853603970513) / Math.Sqrt((14.840358901929662 / (32)) + (150.27509057624493 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -1.1855513918389795 = (111.29847458541322 - 243.24853603970513) / 111.29847458541322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GreaterThanBenchmark - Duration of single invocation 176.32 ns 265.76 ns 1.51 0.05 False
AbsBenchmark - Duration of single invocation 156.75 ns 222.37 ns 1.42 0.03 False
EqualsStaticBenchmark - Duration of single invocation 180.45 ns 268.00 ns 1.49 0.10 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 265.7602593931529 > 184.8589586142774.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -119.77649255006283 (T) = (0 -267.05939139057506) / Math.Sqrt((5.49027875456164 / (32)) + (9.844738211561248 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.5088196785712943 = (176.9988787814952 - 267.05939139057506) / 176.9988787814952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.AbsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 222.36859064214894 > 164.47417709676216.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -89.52962696700992 (T) = (0 -225.24217191587266) / Math.Sqrt((3.6952160095481927 / (32)) + (11.134212379272874 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.433808196655224 = (157.09365620960722 - 225.24217191587266) / 157.09365620960722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;.EqualsStaticBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 268.0000781534092 > 189.04940267362406.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -74.95226707914861 (T) = (0 -270.3573638693804) / Math.Sqrt((30.168980444833878 / (32)) + (11.75209628078112 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.49149419487452606 = (181.266118767646 - 270.3573638693804) / 181.266118767646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntegerFormatting - Duration of single invocation 8.03 ms 55.84 ms 6.95 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.IntegerFormatting

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.83857141071428 > 8.376715626171874.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -18.407710898013303 (T) = (0 -45954336.96497852) / Math.Sqrt((17908333562.626812 / (32)) + (102706651653661.34 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -4.837581655901876 = (7872153.1746828845 - 45954336.96497852) / 7872153.1746828845 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EqualsStaticBenchmark - Duration of single invocation 180.38 ns 267.85 ns 1.48 0.08 False
DivisionOperatorBenchmark - Duration of single invocation 160.72 ns 269.09 ns 1.67 0.12 False
MinBenchmark - Duration of single invocation 197.45 ns 277.70 ns 1.41 0.11 False
MaxBenchmark - Duration of single invocation 197.13 ns 280.53 ns 1.42 0.09 False
AbsBenchmark - Duration of single invocation 156.94 ns 231.49 ns 1.48 0.07 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 267.8464787207102 > 189.30838863569278.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -54.52999793368553 (T) = (0 -272.5178528930313) / Math.Sqrt((16.14095281918915 / (32)) + (58.52928616074567 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.5095257908936266 = (180.53209460681228 - 272.5178528930313) / 180.53209460681228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.DivisionOperatorBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 269.0857772013622 > 169.3866342468249.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -128.88972094601 (T) = (0 -271.9962497523591) / Math.Sqrt((9.127640855364426 / (32)) + (10.753445482208 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.6688790512869838 = (162.98140332135196 - 271.9962497523591) / 162.98140332135196 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 277.7014463042736 > 205.80382303711036.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -31.818362639200934 (T) = (0 -283.7185341379664) / Math.Sqrt((23.87567886925905 / (32)) + (167.37766099102762 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.4407624878544253 = (196.92248828637835 - 283.7185341379664) / 196.92248828637835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.5318714797 > 207.565661080195.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -59.49399123408915 (T) = (0 -284.46129238615094) / Math.Sqrt((18.494713271743613 / (32)) + (37.08916289802371 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.4291684491073091 = (199.0397231087993 - 284.46129238615094) / 199.0397231087993 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AbsBenchmark

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.48831444817145 > 164.87263981939986.
IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -74.31454381602784 (T) = (0 -226.61051077305208) / Math.Sqrt((12.621396242054935 / (32)) + (11.550198518345764 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.4357215685109841 = (157.8373660626095 - 226.61051077305208) / 157.8373660626095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 46.68 μs 54.10 μs 1.16 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty(Mode: Reflection)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.09580430002883 > 49.14768527974597.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -91.5573699210482 (T) = (0 -54289.101108045885) / Math.Sqrt((127631.33999347566 / (32)) + (71438.07230897676 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.16216927129230416 = (46713.591943175124 - 54289.101108045885) / 46713.591943175124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 29.84 ns 456.51 ns 15.30 0.34 False
TryFormat - Duration of single invocation 500.38 ns 581.96 ns 1.16 0.11 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 456.51355564419356 > 32.222670221802574.
IsChangePoint: Marked as a change because one of 12/1/2022 11:48:13 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -15.496562642494746 (T) = (0 -356.07923304201773) / Math.Sqrt((2.2597909025139082 / (32)) + (11041.01547851025 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -10.717333585919212 = (30.38910093589212 - 356.07923304201773) / 30.38910093589212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 581.9630637869761 > 526.1707268552511.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 7:15:22 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -11.598197245430944 (T) = (0 -580.6839219606552) / Math.Sqrt((3992.047375280082 / (46)) + (98.59364337648204 / (10))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (10) - 2, .025) and -0.2443262714192188 = (466.6653234752932 - 580.6839219606552) / 466.6653234752932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 3f5210dca709def1e1253916e4244c82b2177e9c
Compare 414018530f0788c418d92ff817bb5fd6932324f6
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 3.21 ms 4.03 ms 1.25 0.11 False
ReadLine - Duration of single invocation 1.55 ms 1.86 ms 1.20 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.030299278846154 > 3.370756440330529.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -82.41708883495151 (T) = (0 -4073696.9219672615) / Math.Sqrt((467730570.64154166 / (32)) + (2294262739.030274 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.2637052758118376 = (3223613.131907051 - 4073696.9219672615) / 3223613.131907051 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8625932485119048 > 1.6364569728870193.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/8/2022 9:11:55 AM, 1/12/2023 3:42:01 AM, 1/17/2023 11:08:46 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 11:08:46 AM.
IsRegressionStdDev: Marked as regression because -40.31308788040016 (T) = (0 -1871995.027871871) / Math.Sqrt((494518797.81424004 / (32)) + (1003406756.2951909 / (25))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (25) - 2, .025) and -0.1912715691914886 = (1571425.9252761204 - 1871995.027871871) / 1571425.9252761204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@lewing
Copy link
Member

lewing commented Jan 18, 2023

some of this is dotnet/runtime#80668 on the interpreter

@lewing
Copy link
Member

lewing commented Jan 18, 2023

dotnet/runtime#80762

@lewing
Copy link
Member

lewing commented Jan 18, 2023

@DrewScoggins this one seems like it spans the switch and the data is confusing

@kotlarmilos kotlarmilos mentioned this issue Jan 18, 2023
Closed
@lewing
Copy link
Member

lewing commented Feb 1, 2023

This was also dotnet/runtime#76642

@DrewScoggins DrewScoggins mentioned this issue Sep 5, 2023
Closed
@radekdoulik
Copy link
Member

This is fixed by dotnet/runtime#81417

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-wasm arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lewing @radekdoulik