Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Regressions in Burgers.Test3 #80129

Closed
performanceautofiler bot opened this issue Jan 3, 2023 · 11 comments
Closed

Regressions in Burgers.Test3 #80129

performanceautofiler bot opened this issue Jan 3, 2023 · 11 comments
Assignees
@EgorBo @tannergooding
Labels
arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
8.0.0

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 50.00 ns 57.82 ns 1.16 0.05 False
EncodeUtf8 - Duration of single invocation 71.39 ns 81.55 ns 1.14 0.04 False
EncodeUtf8 - Duration of single invocation 46.24 ns 54.00 ns 1.17 0.04 False
EncodeUtf8 - Duration of single invocation 46.55 ns 54.03 ns 1.16 0.04 False
EncodeUtf16 - Duration of single invocation 54.81 ns 62.79 ns 1.15 0.05 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.81583277258387 > 52.417481129298075.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -25.145786383702696 (T) = (0 -57.933217934322485) / Math.Sqrt((2.3623964474246115 / (29)) + (0.1613397566644288 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.15254773986311304 = (50.265352080949945 - 57.933217934322485) / 50.265352080949945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.54861659265462 > 75.32232075944864.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -114.0821565588086 (T) = (0 -81.65124012302711) / Math.Sqrt((0.06638959609032156 / (29)) + (0.07100614020406576 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1362020840470246 = (71.86330783006007 - 81.65124012302711) / 71.86330783006007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no <escaping /> required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.00301563830678 > 48.5118806196726.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -105.69282709382192 (T) = (0 -54.18273969123065) / Math.Sqrt((0.08034202410386392 / (29)) + (0.04128019098655086 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1730450971989896 = (46.1898181243064 - 54.18273969123065) / 46.1898181243064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.02755372264943 > 48.69273409220534.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -170.4406772588238 (T) = (0 -54.13547871090059) / Math.Sqrt((0.023406243279603695 / (29)) + (0.018393288510694467 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.16958389434541218 = (46.28610138411568 - 54.13547871090059) / 46.28610138411568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.78543086769447 > 57.4718640789619.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -49.258615720152235 (T) = (0 -62.81305937612173) / Math.Sqrt((0.7287449900611545 / (29)) + (0.0011130533439879145 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14221852718437203 = (54.99215595019211 - 62.81305937612173) / 54.99215595019211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScalarValueTypeObj - Duration of single invocation 167.08 μs 250.53 μs 1.50 0.02 False
IFooObjIsIFooInterAlia - Duration of single invocation 417.88 μs 445.61 μs 1.07 0.05 False
ObjObjrefValueType - Duration of single invocation 832.79 μs 947.79 μs 1.14 0.12 False
ObjrefValueTypeObj - Duration of single invocation 779.82 μs 946.65 μs 1.21 0.08 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.52506720430105 > 177.88717126125206.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -157.34889274663954 (T) = (0 -250698.64349911804) / Math.Sqrt((7772358.433191751 / (29)) + (10568.652988295244 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.4823308478022079 = (169124.6214506153 - 250698.64349911804) / 169124.6214506153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 445.60746527777786 > 438.70099191337715.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.460168000145664 (T) = (0 -445711.48518582113) / Math.Sqrt((51528625.10230869 / (29)) + (59427.090821034675 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.061874146213374455 = (419740.4059372016 - 445711.48518582113) / 419740.4059372016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjObjrefValueType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 947.7931843891404 > 874.5227331098178.
IsChangePoint: Marked as a change because one of 12/6/2022 6:34:01 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.770505381122266 (T) = (0 -937213.1517214874) / Math.Sqrt((353415415.0537492 / (17)) + (188433114.7229348 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1186275434229421 = (837824.1330028974 - 937213.1517214874) / 837824.1330028974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 946.6478789592763 > 818.8279194078948.
IsChangePoint: Marked as a change because one of 12/9/2022 7:57:12 PM, 12/10/2022 11:25:25 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -26.85181093902272 (T) = (0 -925516.8672099016) / Math.Sqrt((32723.634549815342 / (17)) + (766296265.8028983 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1869612719955682 = (779736.3646531487 - 925516.8672099016) / 779736.3646531487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 111.86 ns 118.54 ns 1.06 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.54351975679164 > 118.10710017101991.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/14/2022 1:59:26 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -43.09754038158343 (T) = (0 -119.18940387461092) / Math.Sqrt((0.29301881287536213 / (29)) + (0.18750842099727472 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.05868011221014512 = (112.58301964866999 - 119.18940387461092) / 112.58301964866999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereLast_LastElementMatches - Duration of single invocation 650.22 ns 724.73 ns 1.11 0.05 False
WhereFirst_LastElementMatches - Duration of single invocation 639.30 ns 702.08 ns 1.10 0.14 False
WhereSelect - Duration of single invocation 838.32 ns 894.79 ns 1.07 0.05 False
Min - Duration of single invocation 440.62 ns 468.21 ns 1.06 0.07 False
Where - Duration of single invocation 689.69 ns 736.30 ns 1.07 0.04 False
WhereSelect - Duration of single invocation 1.25 μs 1.34 μs 1.07 0.11 False
Repeat - Duration of single invocation 382.25 ns 410.60 ns 1.07 0.05 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 724.7254658714982 > 682.5974208911517.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.36085607764159 (T) = (0 -716.0102863111845) / Math.Sqrt((183.58905449092907 / (29)) + (70.05995345922597 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0944921946076017 = (654.194054410675 - 716.0102863111845) / 654.194054410675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 702.0825666457229 > 671.7574198262187.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.88758054547226 (T) = (0 -698.0520999185754) / Math.Sqrt((221.8120470681958 / (29)) + (3.2052738842730317 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.08218407656808165 = (645.0400768530067 - 698.0520999185754) / 645.0400768530067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 894.7945473488121 > 881.9290342422098.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.710372271437192 (T) = (0 -899.8308587803176) / Math.Sqrt((265.31080182033116 / (29)) + (150.9514640344226 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06730533284694304 = (843.0866323698561 - 899.8308587803176) / 843.0866323698561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 468.213976760022 > 462.9503218579129.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 8:57:30 PM, 11/10/2022 9:13:35 AM, 12/3/2022 7:24:45 AM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 7:19:51 AM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.29721173302842 (T) = (0 -467.92709626144557) / Math.Sqrt((47.436089684565395 / (28)) + (0.049884560962766014 / (15))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (15) - 2, .025) and -0.05678094445247723 = (442.7853271936888 - 467.92709626144557) / 442.7853271936888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 736.2994828147459 > 716.8184132859758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -20.26576807083207 (T) = (0 -738.2921684769659) / Math.Sqrt((189.62581591898518 / (29)) + (3.4337103742602078 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.07700104364540426 = (685.5073844478501 - 738.2921684769659) / 685.5073844478501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3438325908230024 > 1.3145043872684157.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/5/2022 3:14:07 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -9.562000836067062 (T) = (0 -1397.2041165898495) / Math.Sqrt((979.4742107775817 / (29)) + (1714.605452420122 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09354748590304915 = (1277.6803335943307 - 1397.2041165898495) / 1277.6803335943307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Repeat

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.59924185730245 > 401.7604620467154.
IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.516430850153665 (T) = (0 -410.7379330275285) / Math.Sqrt((58.546976033624375 / (29)) + (0.04626765149313623 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06849744333201213 = (384.40703399970675 - 410.7379330275285) / 384.40703399970675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 51.38 ns 57.43 ns 1.12 0.05 False
Subtract - Duration of single invocation 53.29 ns 58.68 ns 1.10 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.42732197836272 > 53.48368915016196.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -60.00067824227273 (T) = (0 -56.74103679987769) / Math.Sqrt((0.12083809240293349 / (29)) + (0.08004781347707045 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.11748385839528891 = (50.775710426241 - 56.74103679987769) / 50.775710426241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.680051163526294 > 55.93998442951785.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -51.453698525063 (T) = (0 -58.539282567362946) / Math.Sqrt((0.2313144713726593 / (29)) + (0.0291126567627274 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09666085854620245 = (53.37956772248262 - 58.539282567362946) / 53.37956772248262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ChangeType - Duration of single invocation 23.38 ns 26.97 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.96547629232403 > 24.831430125103818.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.40126556261769 (T) = (0 -26.82030325896489) / Math.Sqrt((0.140839807192652 / (29)) + (0.24387579328379141 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12103361363189857 = (23.924620040672195 - 26.82030325896489) / 23.924620040672195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 1.74 μs 2.16 μs 1.24 0.04 False
ImmutableQueue - Duration of single invocation 3.73 μs 4.02 μs 1.08 0.10 False
ConcurrentStack - Duration of single invocation 3.99 μs 4.88 μs 1.22 0.41 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.158741014574434 > 1.8229782976610758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -564.4070078001165 (T) = (0 -2159.2521116215617) / Math.Sqrt((15.50447013384292 / (29)) + (0.3182434813910868 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.2424618037325781 = (1737.8820863021954 - 2159.2521116215617) / 1737.8820863021954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.01852280644236 > 3.919239245106011.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/13/2022 5:34:08 PM, 12/10/2022 11:25:25 PM, 12/12/2022 10:47:19 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -6.4175585805493265 (T) = (0 -4093.2681310405783) / Math.Sqrt((1537.6210689605634 / (29)) + (39587.399157815475 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0918804722671955 = (3748.824376848924 - 4093.2681310405783) / 3748.824376848924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.877410345631326 > 3.969456488386927.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -8.744637020481978 (T) = (0 -4509.761731346228) / Math.Sqrt((36053.47192347738 / (29)) + (73981.46956850182 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1857650776947188 = (3803.250590002018 - 4509.761731346228) / 3803.250590002018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DelegateInvoke - Duration of single invocation 390.20 μs 445.91 μs 1.14 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.DelegatePerf.DelegateInvoke

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 445.91466435185185 > 409.46105182926823.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -1184.8483064031595 (T) = (0 -445594.94825280533) / Math.Sqrt((32956.49445274069 / (29)) + (14951.814715657889 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14265408250665315 = (389964.86782360123 - 445594.94825280533) / 389964.86782360123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerate - Duration of single invocation 524.32 ns 608.51 ns 1.16 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<String, String>.Enumerate(Size: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 608.5052246582078 > 550.6445918221261.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.614412808755247 (T) = (0 -607.910203011457) / Math.Sqrt((456.19465233773826 / (29)) + (0.10619477266546123 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14679158290872527 = (530.0964988507781 - 607.910203011457) / 530.0964988507781 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 138.51 ns 151.69 ns 1.10 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.68841140292508 > 145.3847891491265.
IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 10/25/2022 2:34:53 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -17.19404587379948 (T) = (0 -151.6984421270046) / Math.Sqrt((13.84884994997729 / (29)) + (0.06402211627913067 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0854226874001251 = (139.75978564660608 - 151.6984421270046) / 139.75978564660608 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NextBytes_span - Duration of single invocation 7.64 μs 8.30 μs 1.09 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.NextBytes_span

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.300104372248368 > 8.021303696337387.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -30.28649369282202 (T) = (0 -8320.84352871568) / Math.Sqrt((12.697376517735384 / (29)) + (7081.456866794183 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.08920214605039334 = (7639.393255777432 - 8320.84352871568) / 7639.393255777432 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString_Unspecified - Duration of single invocation 63.63 ns 68.76 ns 1.08 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.7576462057019 > 66.77276950931972.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/10/2022 12:18:51 PM, 11/11/2022 4:50:14 AM, 11/18/2022 6:07:58 AM, 11/18/2022 4:41:14 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.471349058018443 (T) = (0 -69.11790268009089) / Math.Sqrt((0.8973046920664801 / (29)) + (1.5997202988179553 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.07383360330285414 = (64.36556135652752 - 69.11790268009089) / 64.36556135652752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticMethod4_arrayNotCached_int_string_struct_class - Duration of single invocation 67.58 ns 76.60 ns 1.13 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 76.60054268292681 > 68.98119238467027.
IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 11/14/2022 7:25:10 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -10.345878855024381 (T) = (0 -75.37194714527784) / Math.Sqrt((4.719203943500688 / (29)) + (13.791967035520202 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.17241995378057579 = (64.28749945975764 - 75.37194714527784) / 64.28749945975764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Drawing.Tests.Perf_Color

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHue - Duration of single invocation 1.04 μs 1.13 μs 1.08 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Drawing.Tests.Perf_Color*'

Payloads

Baseline
Compare

Histogram

System.Drawing.Tests.Perf_Color.GetHue

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1256238501421643 > 1.0897112237127544.
IsChangePoint: Marked as a change because one of 10/17/2022 3:19:47 PM, 11/1/2022 6:58:35 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -39.143592185212114 (T) = (0 -1130.1844274585353) / Math.Sqrt((75.70874992329493 / (29)) + (38.29256379423281 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.08704992654716272 = (1039.680331011458 - 1130.1844274585353) / 1039.680331011458 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ExpM1 - Duration of single invocation 19.49 μs 20.89 μs 1.07 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ExpM1

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.894267491087344 > 20.471175137576733.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/18/2022 8:48:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -802.1892828940311 (T) = (0 -20898.10669569401) / Math.Sqrt((32.2052942984135 / (29)) + (26.879316923253068 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.07160798712229503 = (19501.633943410554 - 20898.10669569401) / 19501.633943410554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IntObj - Duration of single invocation 278.52 μs 306.38 μs 1.10 0.02 False
ObjrefValueTypeObj - Duration of single invocation 278.51 μs 306.37 μs 1.10 0.02 False
ScalarValueTypeObj - Duration of single invocation 278.51 μs 306.19 μs 1.10 0.02 False
FooObjIsFoo - Duration of single invocation 278.49 μs 306.28 μs 1.10 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.IntObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.3819852941176 > 292.4515925480769.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -915.6836759050922 (T) = (0 -306360.611651712) / Math.Sqrt((6969.429867093914 / (29)) + (9579.520870962968 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09996877329595574 = (278517.55348811444 - 306360.611651712) / 278517.55348811444 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.3671218487395 > 292.43891601562507.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -1334.9013343481704 (T) = (0 -306356.9647314054) / Math.Sqrt((5754.71412522813 / (29)) + (3318.846632070658 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1000272628722021 = (278499.4291246007 - 306356.9647314054) / 278499.4291246007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.ScalarValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.19109162895927 > 292.3997861328125.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -882.3708241224867 (T) = (0 -306350.40921176167) / Math.Sqrt((4891.616397892133 / (29)) + (11581.599517814488 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09998494050167814 = (278504.1848591511 - 306350.40921176167) / 278504.1848591511 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.FooObjIsFoo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.281658496732 > 292.44601171875.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -813.7406631600054 (T) = (0 -306453.17744405457) / Math.Sqrt((9283.757741628731 / (29)) + (11989.231875643183 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.10020568119582378 = (278541.7151372712 - 306453.17744405457) / 278541.7151372712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyFourValues - Duration of single invocation 28.67 ns 41.51 ns 1.45 0.20 False
IndexOfAnyFourValues - Duration of single invocation 5.79 ns 8.53 ns 1.47 0.41 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.513885017197865 > 30.243886932326742.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.38825090619667 (T) = (0 -43.864804577908885) / Math.Sqrt((7.7845540867263825 / (28)) + (14.405453527665015 / (14))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (14) - 2, .025) and -0.4768224193316467 = (29.70215240757275 - 43.864804577908885) / 29.70215240757275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Int32&gt;.IndexOfAnyFourValues(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.525407489243046 > 6.061748627984955.
IsChangePoint: Marked as a change because one of 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.909321706731888 (T) = (0 -7.4710059171358605) / Math.Sqrt((0.0004887761268551399 / (17)) + (0.45182490088209476 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.2952899950635773 = (5.767824923845843 - 7.4710059171358605) / 5.767824923845843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateActivityEventsSmall - Duration of single invocation 77.35 ns 92.58 ns 1.20 0.26 False
EnumerateActivityLinksSmall - Duration of single invocation 86.31 ns 101.85 ns 1.18 0.22 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Activity.EnumerateActivityEventsSmall

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.57609554079114 > 83.0650220817569.
IsChangePoint: Marked as a change because one of 10/21/2022 10:18:15 PM, 10/24/2022 10:11:47 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -11.642403235089883 (T) = (0 -92.72757625447191) / Math.Sqrt((3.0561544873751516 / (29)) + (14.5321968980091 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.15507540032226225 = (80.27837509880413 - 92.72757625447191) / 80.27837509880413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.EnumerateActivityLinksSmall

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.84532153638516 > 94.83737126178406.
IsChangePoint: Marked as a change because one of 10/20/2022 10:19:58 PM, 11/11/2022 5:53:56 PM, 11/16/2022 6:51:59 PM, 11/24/2022 9:42:44 PM, 12/14/2022 8:02:51 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.127517654173841 (T) = (0 -101.09627406745555) / Math.Sqrt((6.3495851652531785 / (29)) + (8.199022547078057 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12057788480680524 = (90.21798077416564 - 101.09627406745555) / 90.21798077416564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticDelegate - Duration of single invocation 250.81 μs 278.70 μs 1.11 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.StaticDelegate

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.69705513784464 > 263.1544861111111.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -778.18402941027 (T) = (0 -278546.6876641008) / Math.Sqrt((6936.418184409841 / (29)) + (14652.027948131263 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.11132861640903964 = (250642.9543442782 - 278546.6876641008) / 250642.9543442782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteString - Duration of single invocation 370.13 ms 420.89 ms 1.14 0.03 False
WriteCharArray - Duration of single invocation 371.45 ms 420.96 ms 1.13 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 420.8929533333333 > 388.7498175.
IsChangePoint: Marked as a change because one of 10/13/2022 10:51:15 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -140.59382262663075 (T) = (0 -421022593.4013605) / Math.Sqrt((3501407790497.518 / (29)) + (90430461596.43338 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.13519853237389653 = (370880142.45484394 - 421022593.4013605) / 370880142.45484394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 420.9648923076923 > 389.435431.
IsChangePoint: Marked as a change because one of 10/13/2022 10:51:15 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 12/15/2022 2:06:03 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -193.71303034909323 (T) = (0 -421110909.0423862) / Math.Sqrt((1759211943826.5837 / (29)) + (111389476760.60367 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.13700861839024936 = (370367385.2873563 - 421110909.0423862) / 370367385.2873563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReplaceWords - Duration of single invocation 2.72 μs 2.86 μs 1.05 0.02 False
MatchesBoundary - Duration of single invocation 49.52 μs 60.07 μs 1.21 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.858219532977156 > 2.8513345182137386.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -22.95461114395352 (T) = (0 -2866.69228540635) / Math.Sqrt((801.9140944236088 / (29)) + (123.63184172137873 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.05082345991471345 = (2728.0436674291755 - 2866.69228540635) / 2728.0436674291755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 60.07426871500147 > 52.35098886402958.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -106.38656114853704 (T) = (0 -59850.63674019208) / Math.Sqrt((240780.219077881 / (29)) + (6489.400790232217 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.19965621838789474 = (49889.823286724364 - 59850.63674019208) / 49889.823286724364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlyMemorySpanStartLength - Duration of single invocation 7.22 ns 8.49 ns 1.17 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Slice&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<String>.ReadOnlyMemorySpanStartLength

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.488207657768557 > 7.617953636249234.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/2/2022 9:35:40 AM, 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/6/2022 2:12:43 AM, 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -43.13239262955271 (T) = (0 -8.303743825754044) / Math.Sqrt((0.0008586015356861508 / (29)) + (0.007640737742427803 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14232993672773261 = (7.269129135791169 - 8.303743825754044) / 7.269129135791169 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test3 - Duration of single invocation 88.62 ms 125.75 ms 1.42 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*'

Payloads

Baseline
Compare

Histogram

Burgers.Test3

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 125.75220357142857 > 93.047091375.
IsChangePoint: Marked as a change because one of 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -3162.5236958158384 (T) = (0 -125807453.75) / Math.Sqrt((701399090.4432294 / (29)) + (1598157324.646178 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.4197814220177448 = (88610438.05687128 - 125807453.75) / 88610438.05687128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_IsUpper - Duration of single invocation 16.16 ns 24.07 ns 1.49 0.09 False
Char_IsLower - Duration of single invocation 21.12 ns 28.25 ns 1.34 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.074445806073697 > 16.960615991014375.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 9:07:41 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -16.53939400685506 (T) = (0 -23.592951803458664) / Math.Sqrt((4.3408645492760085 / (29)) + (0.25605623759311996 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.40314571023334517 = (16.814327714785318 - 23.592951803458664) / 16.814327714785318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.250520496244214 > 21.99604247757582.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -38.26346394386054 (T) = (0 -28.47125913612601) / Math.Sqrt((0.10568579480154006 / (29)) + (0.4814707194432893 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.35520038361033623 = (21.008892471146478 - 28.47125913612601) / 21.008892471146478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Xml.Linq.Perf_XElementList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Enumerator - Duration of single invocation 229.39 ns 258.08 ns 1.13 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElementList*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XElementList.Enumerator

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 258.0780139524235 > 244.97457309211714.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 8:10:07 PM, 11/2/2022 9:35:40 AM, 11/18/2022 8:48:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -11.836268031423733 (T) = (0 -263.13598290765754) / Math.Sqrt((91.18152051745697 / (29)) + (43.394180034010205 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12663094131856398 = (233.56005348095061 - 263.13598290765754) / 233.56005348095061 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateUsingIndexer - Duration of single invocation 1.43 μs 1.51 μs 1.06 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5133829355454222 > 1.502590079561976.
IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.479430838535972 (T) = (0 -1513.2899954241052) / Math.Sqrt((448.1809708191684 / (29)) + (0.25646742167918807 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.05333412941043472 = (1436.6666313861053 - 1513.2899954241052) / 1436.6666313861053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added RunKind=micro untriaged New issue has not been triaged by the area owner labels Jan 3, 2023
@EgorBo EgorBo removed RunKind=micro untriaged New issue has not been triaged by the area owner labels Jan 3, 2023
@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Jan 3, 2023
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jan 3, 2023
@EgorBo
Copy link
Member

EgorBo commented Jan 3, 2023

Most of these look like a noise, but some e.g. Burgers.Test3 is likely regressed by #79720

@EgorBo EgorBo added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark os-windows arch-x64 and removed untriaged New issue has not been triaged by the area owner labels Jan 3, 2023
@EgorBo EgorBo mentioned this issue Jan 3, 2023
Closed
@jeffschwMSFT jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jan 4, 2023
@ghost
Copy link

ghost commented Jan 4, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch, @kunalspathak
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 50.00 ns 57.82 ns 1.16 0.05 False
EncodeUtf8 - Duration of single invocation 71.39 ns 81.55 ns 1.14 0.04 False
EncodeUtf8 - Duration of single invocation 46.24 ns 54.00 ns 1.17 0.04 False
EncodeUtf8 - Duration of single invocation 46.55 ns 54.03 ns 1.16 0.04 False
EncodeUtf16 - Duration of single invocation 54.81 ns 62.79 ns 1.15 0.05 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,no escaping required,512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.81583277258387 > 52.417481129298075.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -25.145786383702696 (T) = (0 -57.933217934322485) / Math.Sqrt((2.3623964474246115 / (29)) + (0.1613397566644288 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.15254773986311304 = (50.265352080949945 - 57.933217934322485) / 50.265352080949945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,©2020,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 81.54861659265462 > 75.32232075944864.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -114.0821565588086 (T) = (0 -81.65124012302711) / Math.Sqrt((0.06638959609032156 / (29)) + (0.07100614020406576 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1362020840470246 = (71.86330783006007 - 81.65124012302711) / 71.86330783006007 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.00301563830678 > 48.5118806196726.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -105.69282709382192 (T) = (0 -54.18273969123065) / Math.Sqrt((0.08034202410386392 / (29)) + (0.04128019098655086 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1730450971989896 = (46.1898181243064 - 54.18273969123065) / 46.1898181243064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.02755372264943 > 48.69273409220534.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -170.4406772588238 (T) = (0 -54.13547871090059) / Math.Sqrt((0.023406243279603695 / (29)) + (0.018393288510694467 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.16958389434541218 = (46.28610138411568 - 54.13547871090059) / 46.28610138411568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no &lt;escaping /&gt; required,512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.78543086769447 > 57.4718640789619.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/22/2022 2:20:42 AM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -49.258615720152235 (T) = (0 -62.81305937612173) / Math.Sqrt((0.7287449900611545 / (29)) + (0.0011130533439879145 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.14221852718437203 = (54.99215595019211 - 62.81305937612173) / 54.99215595019211 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScalarValueTypeObj - Duration of single invocation 167.08 μs 250.53 μs 1.50 0.02 False
IFooObjIsIFooInterAlia - Duration of single invocation 417.88 μs 445.61 μs 1.07 0.05 False
ObjObjrefValueType - Duration of single invocation 832.79 μs 947.79 μs 1.14 0.12 False
ObjrefValueTypeObj - Duration of single invocation 779.82 μs 946.65 μs 1.21 0.08 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 250.52506720430105 > 177.88717126125206.
IsChangePoint: Marked as a change because one of 12/10/2022 11:25:25 PM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -157.34889274663954 (T) = (0 -250698.64349911804) / Math.Sqrt((7772358.433191751 / (29)) + (10568.652988295244 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.4823308478022079 = (169124.6214506153 - 250698.64349911804) / 169124.6214506153 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.IFooObjIsIFooInterAlia

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 445.60746527777786 > 438.70099191337715.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/14/2022 5:20:21 PM, 12/17/2022 10:44:50 PM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.460168000145664 (T) = (0 -445711.48518582113) / Math.Sqrt((51528625.10230869 / (29)) + (59427.090821034675 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.061874146213374455 = (419740.4059372016 - 445711.48518582113) / 419740.4059372016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjObjrefValueType

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 947.7931843891404 > 874.5227331098178.
IsChangePoint: Marked as a change because one of 12/6/2022 6:34:01 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.770505381122266 (T) = (0 -937213.1517214874) / Math.Sqrt((353415415.0537492 / (17)) + (188433114.7229348 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1186275434229421 = (837824.1330028974 - 937213.1517214874) / 837824.1330028974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf2.CastingPerf.ObjrefValueTypeObj

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 946.6478789592763 > 818.8279194078948.
IsChangePoint: Marked as a change because one of 12/9/2022 7:57:12 PM, 12/10/2022 11:25:25 PM, 12/23/2022 9:40:21 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -26.85181093902272 (T) = (0 -925516.8672099016) / Math.Sqrt((32723.634549815342 / (17)) + (766296265.8028983 / (26))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (26) - 2, .025) and -0.1869612719955682 = (779736.3646531487 - 925516.8672099016) / 779736.3646531487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Utf8JsonReaderCommentParsing - Duration of single invocation 111.86 ns 118.54 ns 1.06 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 118.54351975679164 > 118.10710017101991.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 10/31/2022 10:23:25 PM, 11/7/2022 4:10:19 PM, 11/14/2022 1:59:26 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -43.09754038158343 (T) = (0 -119.18940387461092) / Math.Sqrt((0.29301881287536213 / (29)) + (0.18750842099727472 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.05868011221014512 = (112.58301964866999 - 119.18940387461092) / 112.58301964866999 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereLast_LastElementMatches - Duration of single invocation 650.22 ns 724.73 ns 1.11 0.05 False
WhereFirst_LastElementMatches - Duration of single invocation 639.30 ns 702.08 ns 1.10 0.14 False
WhereSelect - Duration of single invocation 838.32 ns 894.79 ns 1.07 0.05 False
Min - Duration of single invocation 440.62 ns 468.21 ns 1.06 0.07 False
Where - Duration of single invocation 689.69 ns 736.30 ns 1.07 0.04 False
WhereSelect - Duration of single invocation 1.25 μs 1.34 μs 1.07 0.11 False
Repeat - Duration of single invocation 382.25 ns 410.60 ns 1.07 0.05 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.WhereLast_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 724.7254658714982 > 682.5974208911517.
IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.36085607764159 (T) = (0 -716.0102863111845) / Math.Sqrt((183.58905449092907 / (29)) + (70.05995345922597 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0944921946076017 = (654.194054410675 - 716.0102863111845) / 654.194054410675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 702.0825666457229 > 671.7574198262187.
IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.88758054547226 (T) = (0 -698.0520999185754) / Math.Sqrt((221.8120470681958 / (29)) + (3.2052738842730317 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.08218407656808165 = (645.0400768530067 - 698.0520999185754) / 645.0400768530067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: Array)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 894.7945473488121 > 881.9290342422098.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/21/2022 9:23:17 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -12.710372271437192 (T) = (0 -899.8308587803176) / Math.Sqrt((265.31080182033116 / (29)) + (150.9514640344226 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06730533284694304 = (843.0866323698561 - 899.8308587803176) / 843.0866323698561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 468.213976760022 > 462.9503218579129.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 11/7/2022 8:57:30 PM, 11/10/2022 9:13:35 AM, 12/3/2022 7:24:45 AM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 7:19:51 AM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.29721173302842 (T) = (0 -467.92709626144557) / Math.Sqrt((47.436089684565395 / (28)) + (0.049884560962766014 / (15))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (15) - 2, .025) and -0.05678094445247723 = (442.7853271936888 - 467.92709626144557) / 442.7853271936888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Where(input: List)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 736.2994828147459 > 716.8184132859758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/18/2022 7:31:41 AM, 12/3/2022 7:24:45 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -20.26576807083207 (T) = (0 -738.2921684769659) / Math.Sqrt((189.62581591898518 / (29)) + (3.4337103742602078 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.07700104364540426 = (685.5073844478501 - 738.2921684769659) / 685.5073844478501 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.WhereSelect(input: IEnumerable)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3438325908230024 > 1.3145043872684157.
IsChangePoint: Marked as a change because one of 11/18/2022 7:31:41 AM, 12/5/2022 3:14:07 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -9.562000836067062 (T) = (0 -1397.2041165898495) / Math.Sqrt((979.4742107775817 / (29)) + (1714.605452420122 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09354748590304915 = (1277.6803335943307 - 1397.2041165898495) / 1277.6803335943307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Repeat

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 410.59924185730245 > 401.7604620467154.
IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 12/10/2022 11:25:25 PM, 12/13/2022 4:26:35 PM, 12/17/2022 10:44:50 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -18.516430850153665 (T) = (0 -410.7379330275285) / Math.Sqrt((58.546976033624375 / (29)) + (0.04626765149313623 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.06849744333201213 = (384.40703399970675 - 410.7379330275285) / 384.40703399970675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 51.38 ns 57.43 ns 1.12 0.05 False
Subtract - Duration of single invocation 53.29 ns 58.68 ns 1.10 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 57.42732197836272 > 53.48368915016196.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/2/2022 9:35:40 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/10/2022 11:25:25 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -60.00067824227273 (T) = (0 -56.74103679987769) / Math.Sqrt((0.12083809240293349 / (29)) + (0.08004781347707045 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.11748385839528891 = (50.775710426241 - 56.74103679987769) / 50.775710426241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.680051163526294 > 55.93998442951785.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/26/2022 7:11:30 AM, 11/7/2022 4:10:19 PM, 11/18/2022 8:48:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -51.453698525063 (T) = (0 -58.539282567362946) / Math.Sqrt((0.2313144713726593 / (29)) + (0.0291126567627274 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.09666085854620245 = (53.37956772248262 - 58.539282567362946) / 53.37956772248262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ChangeType - Duration of single invocation 23.38 ns 26.97 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ChangeType

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.96547629232403 > 24.831430125103818.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -19.40126556261769 (T) = (0 -26.82030325896489) / Math.Sqrt((0.140839807192652 / (29)) + (0.24387579328379141 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.12103361363189857 = (23.924620040672195 - 26.82030325896489) / 23.924620040672195 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8c58fc2347820ce48e09605d8adddb993df9ebb5
Compare 1d15f2140f7eb30a976c66290491ec89cd628da0
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 1.74 μs 2.16 μs 1.24 0.04 False
ImmutableQueue - Duration of single invocation 3.73 μs 4.02 μs 1.08 0.10 False
ConcurrentStack - Duration of single invocation 3.99 μs 4.88 μs 1.22 0.41 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.List(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.158741014574434 > 1.8229782976610758.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/3/2022 7:24:45 AM, 12/17/2022 10:44:50 PM, 12/20/2022 1:34:12 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -564.4070078001165 (T) = (0 -2159.2521116215617) / Math.Sqrt((15.50447013384292 / (29)) + (0.3182434813910868 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.2424618037325781 = (1737.8820863021954 - 2159.2521116215617) / 1737.8820863021954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ImmutableQueue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.01852280644236 > 3.919239245106011.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/13/2022 5:34:08 PM, 12/10/2022 11:25:25 PM, 12/12/2022 10:47:19 AM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -6.4175585805493265 (T) = (0 -4093.2681310405783) / Math.Sqrt((1537.6210689605634 / (29)) + (39587.399157815475 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.0918804722671955 = (3748.824376848924 - 4093.2681310405783) / 3748.824376848924 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateForEach&lt;String&gt;.ConcurrentStack(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.877410345631326 > 3.969456488386927.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 12/28/2022 5:13:50 PM, 1/3/2023 6:29:42 AM falls between 12/23/2022 9:40:21 PM and 1/3/2023 6:29:42 AM.
IsRegressionStdDev: Marked as regression because -8.744637020481978 (T) = (0 -4509.761731346228) / Math.Sqrt((36053.47192347738 / (29)) + (73981.46956850182 / (14))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (14) - 2, .025) and -0.1857650776947188 = (3803.250590002018 - 4509.761731346228) / 3803.250590002018 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: EgorBo
Labels:

os-windows, tenet-performance, tenet-performance-benchmarks, arch-x64, area-CodeGen-coreclr
Milestone: -

@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Jan 4, 2023
@EgorBo
Copy link
Member

EgorBo commented Jan 5, 2023

Same on arm:
dotnet/perf-autofiling-issues#11344
dotnet/perf-autofiling-issues#11339
dotnet/perf-autofiling-issues#11349

(only Burgers.Test3 popped up)

@tannergooding
Copy link
Member

Working on getting a disasm diff right now.

@dakersnar dakersnar mentioned this issue Jan 5, 2023
Closed
@dakersnar dakersnar changed the title [Perf] Windows/x64: 50 Regressions on 12/28/2022 6:20:56 PM Regressions in Burgers.Test3 Jan 5, 2023
@tannergooding
Copy link
Member

tannergooding commented Jan 5, 2023 •
edited
Loading

Before 
; Assembly listing for method Burgers:GetCalculated3(int,int,double,double,double,double[]):double[]
; Emitting BLENDED_CODE for X64 CPU with AVX - Windows
; optimized code
; rsp based frame
; fully interruptible
; No PGO data
; Final local variable assignments
;
;  V00 arg0         [V00,T15] (  4,  7   )     int  ->  rdi         single-def
;  V01 arg1         [V01,T10] (  8, 11   )     int  ->  rsi         single-def
;  V02 arg2         [V02,T30] (  6, 12   )  double  ->  mm6         single-def
;  V03 arg3         [V03,T31] (  6, 12   )  double  ->  mm7         single-def
;  V04 arg4         [V04,T38] (  1,  1   )  double  ->  [rsp+E0H]   single-def
;  V05 arg5         [V05,T18] (  2,  2   )     ref  ->  rbx         class-hnd single-def
;  V06 loc0         [V06,T17] (  4,  4   )     int  ->  rbp         single-def
;  V07 loc1         [V07,T04] (  8, 53   )     ref  ->  registers   class-hnd
;  V08 loc2         [V08,T02] ( 16, 91   )     ref  ->  registers   class-hnd
;  V09 loc3         [V09,T34] (  4, 10   )  double  ->  mm0         single-def
;  V10 loc4         [V10,T11] (  4, 13   )     int  ->  rdx        
;  V11 loc5         [V11,T16] (  2,  8   )     ref  ->  r14         class-hnd
;  V12 loc6         [V12,T00] ( 12,180   )     int  ->   r8        
;  V13 loc7         [V13,T19] (  5, 80   )  simd32  ->  mm4         ld-addr-op
;  V14 loc8         [V14,T20] (  3, 48   )  simd32  ->  mm5         ld-addr-op
;  V15 loc9         [V15,T21] (  2, 32   )  simd32  ->  mm9         ld-addr-op
;  V16 loc10        [V16,T22] (  2, 32   )  simd32  ->  mm4         ld-addr-op
;  V17 OutArgs      [V17    ] (  1,  1   )  lclBlk (32) [rsp+00H]   "OutgoingArgSpace"
;  V18 tmp1         [V18,T28] (  2, 16   )  double  ->  mm4         "Strict ordering of exceptions for Array store"
;  V19 tmp2         [V19,T29] (  2, 16   )  double  ->  mm4         "Strict ordering of exceptions for Array store"
;  V20 cse0         [V20,T25] (  2, 17   )  simd32  ->  mm1         "CSE - moderate"
;  V21 cse1         [V21,T23] (  5, 20   )  double  ->  mm4         "CSE - moderate"
;  V22 cse2         [V22,T32] (  3, 12   )  double  ->  mm9         "CSE - moderate"
;  V23 cse3         [V23,T33] (  3, 12   )  double  ->  mm9         "CSE - moderate"
;  V24 cse4         [V24,T24] (  5, 20   )  double  ->  mm4         "CSE - moderate"
;  V25 cse5         [V25,T12] (  4, 13   )    long  ->  rcx         "CSE - moderate"
;  V26 cse6         [V26,T01] ( 12,120   )     int  ->   r9         "CSE - aggressive"
;  V27 cse7         [V27,T14] (  4, 10   )     int  ->  rax         "CSE - moderate"
;  V28 cse8         [V28,T09] (  3, 21   )     int  ->  rbp         "CSE - moderate"
;  V29 cse9         [V29,T26] (  2, 17   )  simd32  ->  mm2         "CSE - moderate"
;  V30 cse10        [V30,T03] (  6, 60   )     int  ->  r10         "CSE - aggressive"
;  V31 cse11        [V31,T05] (  3, 48   )     int  ->  r10         "CSE - aggressive"
;  V32 cse12        [V32,T06] (  3, 48   )     int  ->  r10         "CSE - aggressive"
;  V33 cse13        [V33,T07] (  3, 48   )     int  ->  r11         "CSE - aggressive"
;  V34 cse14        [V34,T08] (  3, 48   )     int  ->  rbx         "CSE - aggressive"
;  V35 cse15        [V35,T13] (  3, 12   )     int  ->   r8         "CSE - moderate"
;  V36 cse16        [V36,T35] (  4, 10   )  double  ->  mm8         "CSE - moderate"
;  V37 cse17        [V37,T27] (  2, 17   )  simd32  ->  mm3         "CSE - moderate"
;  V38 rat0         [V38,T36] (  2,  4   )  double  ->  mm0         "argument with side effect"
;  V39 rat1         [V39,T37] (  2,  4   )  double  ->  mm1         "argument with side effect"
;
; Lcl frame size = 136

G_M52065_IG01:
       push     r15
       push     r14
       push     rdi
       push     rsi
       push     rbp
       push     rbx
       sub      rsp, 136
       vzeroupper 
       vmovaps  xmmword ptr [rsp+70H], xmm6
       vmovaps  xmmword ptr [rsp+60H], xmm7
       vmovaps  xmmword ptr [rsp+50H], xmm8
       vmovaps  xmmword ptr [rsp+40H], xmm9
       vmovaps  xmmword ptr [rsp+30H], xmm10
       vmovaps  xmmword ptr [rsp+20H], xmm11
       mov      edi, ecx
       mov      esi, edx
       vmovaps  xmm6, xmm2
       vmovaps  xmm7, xmm3
       mov      rbx, gword ptr [rsp+E8H]
						;; size=74 bbWeight=1    PerfScore 21.25
G_M52065_IG02:
       mov      edx, esi
       sar      edx, 31
       and      edx, 3
       add      edx, esi
       and      edx, -4
       mov      ecx, esi
       sub      ecx, edx
       mov      edx, ecx
       neg      edx
       lea      ebp, [rdx+rsi+04H]
       movsxd   rdx, ebp
       mov      rcx, 0xD1FFAB1E      ; double[]
       call     CORINFO_HELP_NEWARR_1_VC
       mov      r14, rax
       movsxd   rdx, ebp
       mov      rcx, 0xD1FFAB1E      ; double[]
       call     CORINFO_HELP_NEWARR_1_VC
       mov      r15, rax
       mov      r8d, dword ptr [rbx+08H]
       mov      rcx, rbx
       mov      rdx, r15
       call     [System.Array:Copy(System.Array,System.Array,int)]
       vmulsd   xmm0, xmm7, qword ptr [rsp+E0H]
       vdivsd   xmm0, xmm0, xmm6
       vmovsd   xmm8, qword ptr [reloc @RWD00]
       vmovaps  xmm1, xmm8
       call     System.Math:Pow(double,double):double
       xor      edx, edx
       test     edi, edi
       jle      G_M52065_IG07
       add      ebp, -3
       vdivsd   xmm1, xmm7, xmm6
       vbroadcastsd ymm1, ymm1
       vmovupd  ymm2, ymmword ptr[reloc @RWD32]
       vbroadcastsd ymm3, ymm0
       lea      eax, [rsi-01H]
       mov      ecx, eax
						;; size=154 bbWeight=1    PerfScore 56.25
G_M52065_IG03:
       mov      r8d, 1
       cmp      ebp, 1
       jle      G_M52065_IG05
       mov      r9d, dword ptr [r15+08H]
       align    [0 bytes for IG04]
						;; size=19 bbWeight=4    PerfScore 14.00
G_M52065_IG04:
       cmp      r8d, r9d
       jae      G_M52065_IG10
       lea      r11d, [r8+03H]
       cmp      r11d, r9d
       jae      G_M52065_IG10
       vmovupd  ymm4, ymmword ptr[r15+8*r8+10H]
       lea      r10d, [r8-01H]
       cmp      r10d, r9d
       jae      G_M52065_IG10
       lea      ebx, [r8+02H]
       cmp      ebx, r9d
       jae      G_M52065_IG10
       vmovupd  ymm5, ymmword ptr[r15+8*r10+10H]
       lea      r10d, [r8+01H]
       cmp      r10d, r9d
       jae      G_M52065_IG10
       lea      ebx, [r8+04H]
       cmp      ebx, r9d
       jae      G_M52065_IG10
       vmovupd  ymm9, ymmword ptr[r15+8*r10+10H]
       vmulpd   ymm10, ymm4, ymm1
       vsubpd   ymm11, ymm4, ymm5
       vmulpd   ymm10, ymm10, ymm11
       vsubpd   ymm10, ymm4, ymm10
       vmulpd   ymm4, ymm4, ymm2
       vsubpd   ymm4, ymm9, ymm4
       vaddpd   ymm4, ymm4, ymm5
       vmulpd   ymm4, ymm3, ymm4
       vaddpd   ymm4, ymm10, ymm4
       mov      r10d, dword ptr [r14+08H]
       cmp      r8d, r10d
       jae      G_M52065_IG10
       cmp      r11d, r10d
       jae      G_M52065_IG11
       mov      r8d, r8d
       vmovupd  ymmword ptr[r14+8*r8+10H], ymm4
       mov      r8d, ebx
       cmp      r8d, ebp
       jl       G_M52065_IG04
						;; size=177 bbWeight=16    PerfScore 964.00
G_M52065_IG05:
       mov      r9d, dword ptr [r15+08H]
       test     r9d, r9d
       je       G_M52065_IG12
       vmovsd   xmm4, qword ptr [r15+10H]
       vmulsd   xmm5, xmm4, xmm7
       vdivsd   xmm5, xmm5, xmm6
       cmp      eax, r9d
       jae      G_M52065_IG12
       vmovsd   xmm9, qword ptr [r15+8*rcx+10H]
       vsubsd   xmm10, xmm4, xmm9
       vmulsd   xmm5, xmm5, xmm10
       vsubsd   xmm5, xmm4, xmm5
       cmp      r9d, 1
       jbe      G_M52065_IG12
       vmovsd   xmm10, qword ptr [r15+18H]
       vmulsd   xmm4, xmm4, xmm8
       vsubsd   xmm4, xmm10, xmm4
       vaddsd   xmm4, xmm4, xmm9
       vmulsd   xmm4, xmm4, xmm0
       vaddsd   xmm4, xmm5, xmm4
       mov      r10d, dword ptr [r14+08H]
       test     r10d, r10d
       je       G_M52065_IG12
       vmovsd   qword ptr [r14+10H], xmm4
       vmovsd   xmm4, qword ptr [r15+8*rcx+10H]
       vmulsd   xmm5, xmm4, xmm7
       vdivsd   xmm5, xmm5, xmm6
       lea      r8d, [rsi-02H]
       cmp      r8d, r9d
       jae      G_M52065_IG12
       mov      r9d, r8d
       vmovsd   xmm9, qword ptr [r15+8*r9+10H]
       vsubsd   xmm10, xmm4, xmm9
       vmulsd   xmm5, xmm5, xmm10
       vsubsd   xmm5, xmm4, xmm5
       vmovsd   xmm10, qword ptr [r15+10H]
       vmulsd   xmm4, xmm4, xmm8
       vsubsd   xmm4, xmm10, xmm4
       vaddsd   xmm4, xmm4, xmm9
       vmulsd   xmm4, xmm4, xmm0
       vaddsd   xmm4, xmm5, xmm4
       cmp      eax, r10d
       jae      SHORT G_M52065_IG12
       vmovsd   qword ptr [r14+8*rcx+10H], xmm4
       inc      edx
       cmp      edx, edi
       jl       SHORT G_M52065_IG09
						;; size=212 bbWeight=4    PerfScore 479.00
G_M52065_IG06:
       mov      r15, r14
						;; size=3 bbWeight=2    PerfScore 0.50
G_M52065_IG07:
       mov      rax, r15
						;; size=3 bbWeight=1    PerfScore 0.25
G_M52065_IG08:
       vmovaps  xmm6, xmmword ptr [rsp+70H]
       vmovaps  xmm7, xmmword ptr [rsp+60H]
       vmovaps  xmm8, xmmword ptr [rsp+50H]
       vmovaps  xmm9, xmmword ptr [rsp+40H]
       vmovaps  xmm10, xmmword ptr [rsp+30H]
       vmovaps  xmm11, xmmword ptr [rsp+20H]
       vzeroupper 
       add      rsp, 136
       pop      rbx
       pop      rbp
       pop      rsi
       pop      rdi
       pop      r14
       pop      r15
       ret      
						;; size=55 bbWeight=1    PerfScore 29.25
G_M52065_IG09:
       xchg     r14, r15
       jmp      G_M52065_IG03
						;; size=8 bbWeight=2    PerfScore 6.00
G_M52065_IG10:
       call     CORINFO_HELP_THROW_ARGUMENTOUTOFRANGEEXCEPTION
       int3     
						;; size=6 bbWeight=0    PerfScore 0.00
G_M52065_IG11:
       call     CORINFO_HELP_THROW_ARGUMENTEXCEPTION
       int3     
						;; size=6 bbWeight=0    PerfScore 0.00
G_M52065_IG12:
       call     CORINFO_HELP_RNGCHKFAIL
       int3     
						;; size=6 bbWeight=0    PerfScore 0.00
RWD00  	dq	4000000000000000h	;            2
RWD08  	dd	00000000h, 00000000h, 00000000h, 00000000h, 00000000h, 00000000h
RWD32  	dq	4000000000000000h, 4000000000000000h, 4000000000000000h, 4000000000000000h


; Total bytes of code 723, prolog size 74, PerfScore 1642.80, instruction count 172, allocated bytes for code 723 (MethodHash=2f54349e) for method Burgers:GetCalculated3(int,int,double,double,double,double[]):double[]
; ============================================================

Unwind Info:
  >> Start offset   : 0x000000 (not in unwind data)
  >>   End offset   : 0xd1ffab1e (not in unwind data)
  Version           : 1
  Flags             : 0x00
  SizeOfProlog      : 0x36
  CountOfUnwindCodes: 20
  FrameRegister     : none (0)
  FrameOffset       : N/A (no FrameRegister) (Value=0)
  UnwindCodes       :
    CodeOffset: 0x36 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM11 (11)
      Scaled Small Offset: 2 * 16 = 32 = 0x00020
    CodeOffset: 0x30 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM10 (10)
      Scaled Small Offset: 3 * 16 = 48 = 0x00030
    CodeOffset: 0x2A UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM9 (9)
      Scaled Small Offset: 4 * 16 = 64 = 0x00040
    CodeOffset: 0x24 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM8 (8)
      Scaled Small Offset: 5 * 16 = 80 = 0x00050
    CodeOffset: 0x1E UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM7 (7)
      Scaled Small Offset: 6 * 16 = 96 = 0x00060
    CodeOffset: 0x18 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM6 (6)
      Scaled Small Offset: 7 * 16 = 112 = 0x00070
    CodeOffset: 0x0F UnwindOp: UWOP_ALLOC_LARGE (1)     OpInfo: 0 - Scaled small  
      Size: 17 * 8 = 136 = 0x00088
    CodeOffset: 0x08 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rbx (3)
    CodeOffset: 0x07 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rbp (5)
    CodeOffset: 0x06 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rsi (6)
    CodeOffset: 0x05 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rdi (7)
    CodeOffset: 0x04 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: r14 (14)
    CodeOffset: 0x02 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: r15 (15)
After 
; Assembly listing for method Burgers:GetCalculated3(int,int,double,double,double,double[]):double[]
; Emitting BLENDED_CODE for X64 CPU with AVX - Windows
; optimized code
; rsp based frame
; fully interruptible
; No PGO data
; 0 inlinees with PGO data; 4 single block inlinees; 4 inlinees without PGO data
; Final local variable assignments
;
;  V00 arg0         [V00,T13] (  4,  7   )     int  ->  rdi         single-def
;  V01 arg1         [V01,T08] (  8, 14   )     int  ->  rsi         single-def
;  V02 arg2         [V02,T20] (  6, 27   )  double  ->  mm6         single-def
;  V03 arg3         [V03,T21] (  6, 27   )  double  ->  mm7         single-def
;  V04 arg4         [V04,T32] (  1,  1   )  double  ->  [rsp+B0H]   single-def
;  V05 arg5         [V05,T16] (  2,  2   )     ref  ->  rbx         class-hnd single-def
;  V06 loc0         [V06,T15] (  4,  4   )     int  ->  rbp         single-def
;  V07 loc1         [V07,T06] (  8, 53   )     ref  ->  registers   class-hnd
;  V08 loc2         [V08,T04] ( 16,103   )     ref  ->  registers   class-hnd
;  V09 loc3         [V09,T22] (  4, 25   )  double  ->  mm0         single-def
;  V10 loc4         [V10,T11] (  4, 13   )     int  ->  rdx        
;  V11 loc5         [V11,T14] (  2,  8   )     ref  ->  r14         class-hnd
;  V12 loc6         [V12,T00] ( 13,196   )     int  ->  rax        
;  V13 loc7         [V13,T17] (  5, 80   )  simd32  ->  mm1         ld-addr-op
;  V14 loc8         [V14,T18] (  3, 48   )  simd32  ->  mm2         ld-addr-op
;* V15 loc9         [V15    ] (  0,  0   )  simd32  ->  zero-ref    ld-addr-op
;  V16 loc10        [V16,T19] (  2, 32   )  simd32  ->  mm1         ld-addr-op
;  V17 OutArgs      [V17    ] (  1,  1   )  lclBlk (32) [rsp+00H]   "OutgoingArgSpace"
;  V18 tmp1         [V18,T25] (  2, 16   )  double  ->  mm1         "Strict ordering of exceptions for Array store"
;  V19 tmp2         [V19,T26] (  2, 16   )  double  ->  mm1         "Strict ordering of exceptions for Array store"
;* V20 tmp3         [V20    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;  V21 tmp4         [V21,T01] (  5,160   )     int  ->   r8         "Inlining Arg"
;* V22 tmp5         [V22    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;  V23 tmp6         [V23,T02] (  5,160   )     int  ->   r8         "Inlining Arg"
;* V24 tmp7         [V24    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V25 tmp8         [V25    ] (  0,  0   )   byref  ->  zero-ref    "Inlining Arg"
;* V26 tmp9         [V26    ] (  0,  0   )  simd32  ->  zero-ref    "Inlining Arg"
;  V27 cse0         [V27,T23] (  5, 20   )  double  ->  mm1         "CSE - moderate"
;  V28 cse1         [V28,T27] (  3, 12   )  double  ->  mm3         "CSE - moderate"
;  V29 cse2         [V29,T28] (  3, 12   )  double  ->  mm3         "CSE - moderate"
;  V30 cse3         [V30,T24] (  5, 20   )  double  ->  mm1         "CSE - moderate"
;  V31 cse4         [V31,T09] (  4, 16   )    long  ->   r8         "CSE - moderate"
;  V32 cse5         [V32,T03] ( 12,132   )     int  ->  rcx         "CSE - aggressive"
;  V33 cse6         [V33,T10] (  4, 16   )     int  ->  rax         "CSE - moderate"
;  V34 cse7         [V34,T05] (  6, 60   )     int  ->  registers   "CSE - aggressive"
;  V35 cse8         [V35,T07] (  3, 21   )     int  ->  rbp         "CSE - moderate"
;  V36 cse9         [V36,T12] (  3, 12   )     int  ->  r10         "CSE - moderate"
;  V37 cse10        [V37,T29] (  4, 10   )  double  ->  mm8         "CSE - moderate"
;  V38 rat0         [V38,T30] (  2,  4   )  double  ->  mm0         "argument with side effect"
;  V39 rat1         [V39,T31] (  2,  4   )  double  ->  mm1         "argument with side effect"
;
; Lcl frame size = 88

G_M52065_IG01:
       push     r15
       push     r14
       push     rdi
       push     rsi
       push     rbp
       push     rbx
       sub      rsp, 88
       vzeroupper 
       vmovaps  xmmword ptr [rsp+40H], xmm6
       vmovaps  xmmword ptr [rsp+30H], xmm7
       vmovaps  xmmword ptr [rsp+20H], xmm8
       mov      edi, ecx
       mov      esi, edx
       vmovaps  xmm6, xmm2
       vmovaps  xmm7, xmm3
       mov      rbx, gword ptr [rsp+B8H]
						;; size=53 bbWeight=1    PerfScore 15.25
G_M52065_IG02:
       mov      edx, esi
       sar      edx, 31
       and      edx, 3
       add      edx, esi
       and      edx, -4
       mov      ecx, esi
       sub      ecx, edx
       mov      edx, ecx
       neg      edx
       lea      ebp, [rdx+rsi+04H]
       movsxd   rdx, ebp
       mov      rcx, 0xD1FFAB1E      ; double[]
       call     CORINFO_HELP_NEWARR_1_VC
       mov      r14, rax
       movsxd   rdx, ebp
       mov      rcx, 0xD1FFAB1E      ; double[]
       call     CORINFO_HELP_NEWARR_1_VC
       mov      r15, rax
       mov      r8d, dword ptr [rbx+08H]
       mov      rcx, rbx
       mov      rdx, r15
       call     [System.Array:Copy(System.Array,System.Array,int)]
       vmulsd   xmm0, xmm7, qword ptr [rsp+B0H]
       vdivsd   xmm0, xmm0, xmm6
       vmovsd   xmm8, qword ptr [reloc @RWD00]
       vmovaps  xmm1, xmm8
       call     System.Math:Pow(double,double):double
       xor      edx, edx
       test     edi, edi
       jle      G_M52065_IG08
       add      ebp, -3
						;; size=127 bbWeight=1    PerfScore 35.50
G_M52065_IG03:
       mov      eax, 1
       cmp      ebp, 1
       jle      G_M52065_IG06
       align    [0 bytes for IG04]
						;; size=14 bbWeight=4    PerfScore 6.00
G_M52065_IG04:
       test     eax, eax
       jl       G_M52065_IG11
       mov      ecx, dword ptr [r15+08H]
       mov      r8d, ecx
       sub      r8d, eax
       cmp      r8d, 4
       jl       G_M52065_IG11
       cmp      eax, ecx
       jae      G_M52065_IG14
       mov      r8d, eax
       vmovupd  ymm1, ymmword ptr[r15+8*r8+10H]
       lea      r8d, [rax-01H]
       test     r8d, r8d
       jl       G_M52065_IG11
       mov      r9d, ecx
       sub      r9d, r8d
       cmp      r9d, 4
       jl       G_M52065_IG11
       cmp      r8d, ecx
       jae      G_M52065_IG14
       mov      r8d, r8d
       vmovupd  ymm2, ymmword ptr[r15+8*r8+10H]
       lea      r8d, [rax+01H]
       test     r8d, r8d
       jl       G_M52065_IG11
       mov      r9d, ecx
       sub      r9d, r8d
       cmp      r9d, 4
       jl       G_M52065_IG11
       vdivsd   xmm3, xmm7, xmm6
       vbroadcastsd ymm3, ymm3
       vmulpd   ymm3, ymm1, ymm3
       vsubpd   ymm4, ymm1, ymm2
       vmulpd   ymm3, ymm3, ymm4
       vsubpd   ymm3, ymm1, ymm3
       cmp      r8d, ecx
       jae      G_M52065_IG14
       mov      ecx, r8d
       vmovupd  ymm4, ymmword ptr[r15+8*rcx+10H]
       vmulpd   ymm1, ymm1, ymmword ptr[reloc @RWD32]
       vsubpd   ymm1, ymm4, ymm1
       vaddpd   ymm1, ymm1, ymm2
       vbroadcastsd ymm2, ymm0
       vmulpd   ymm1, ymm2, ymm1
       vaddpd   ymm1, ymm3, ymm1
       mov      ecx, dword ptr [r14+08H]
       cmp      ecx, eax
       jbe      G_M52065_IG12
       sub      ecx, eax
       cmp      ecx, 4
       jl       G_M52065_IG13
       mov      ecx, eax
						;; size=221 bbWeight=16    PerfScore 1304.00
G_M52065_IG05:
       vmovupd  ymmword ptr[r14+8*rcx+10H], ymm1
       add      eax, 4
       cmp      eax, ebp
       jl       G_M52065_IG04
						;; size=18 bbWeight=16    PerfScore 56.00
G_M52065_IG06:
       mov      ecx, dword ptr [r15+08H]
       test     ecx, ecx
       je       G_M52065_IG14
       vmovsd   xmm1, qword ptr [r15+10H]
       vmulsd   xmm2, xmm1, xmm7
       vdivsd   xmm2, xmm2, xmm6
       lea      eax, [rsi-01H]
       cmp      eax, ecx
       jae      G_M52065_IG14
       mov      r8d, eax
       vmovsd   xmm3, qword ptr [r15+8*r8+10H]
       vsubsd   xmm4, xmm1, xmm3
       vmulsd   xmm2, xmm2, xmm4
       vsubsd   xmm2, xmm1, xmm2
       cmp      ecx, 1
       jbe      G_M52065_IG14
       vmovsd   xmm4, qword ptr [r15+18H]
       vmulsd   xmm1, xmm1, xmm8
       vsubsd   xmm1, xmm4, xmm1
       vaddsd   xmm1, xmm1, xmm3
       vmulsd   xmm1, xmm1, xmm0
       vaddsd   xmm1, xmm2, xmm1
       mov      r9d, dword ptr [r14+08H]
       test     r9d, r9d
       je       G_M52065_IG14
       vmovsd   qword ptr [r14+10H], xmm1
       vmovsd   xmm1, qword ptr [r15+8*r8+10H]
       vmulsd   xmm2, xmm1, xmm7
       vdivsd   xmm2, xmm2, xmm6
       lea      r10d, [rsi-02H]
       cmp      r10d, ecx
       jae      G_M52065_IG14
       mov      ecx, r10d
       vmovsd   xmm3, qword ptr [r15+8*rcx+10H]
       vsubsd   xmm4, xmm1, xmm3
       vmulsd   xmm2, xmm2, xmm4
       vsubsd   xmm2, xmm1, xmm2
       vmovsd   xmm4, qword ptr [r15+10H]
       vmulsd   xmm1, xmm1, xmm8
       vsubsd   xmm1, xmm4, xmm1
       vaddsd   xmm1, xmm1, xmm3
       vmulsd   xmm1, xmm1, xmm0
       vaddsd   xmm1, xmm2, xmm1
       cmp      eax, r9d
       jae      SHORT G_M52065_IG14
       vmovsd   qword ptr [r14+8*r8+10H], xmm1
       inc      edx
       cmp      edx, edi
       jl       SHORT G_M52065_IG10
						;; size=209 bbWeight=4    PerfScore 482.00
G_M52065_IG07:
       mov      r15, r14
						;; size=3 bbWeight=2    PerfScore 0.50
G_M52065_IG08:
       mov      rax, r15
						;; size=3 bbWeight=1    PerfScore 0.25
G_M52065_IG09:
       vmovaps  xmm6, xmmword ptr [rsp+40H]
       vmovaps  xmm7, xmmword ptr [rsp+30H]
       vmovaps  xmm8, xmmword ptr [rsp+20H]
       vzeroupper 
       add      rsp, 88
       pop      rbx
       pop      rbp
       pop      rsi
       pop      rdi
       pop      r14
       pop      r15
       ret      
						;; size=34 bbWeight=1    PerfScore 17.25
G_M52065_IG10:
       xchg     r14, r15
       jmp      G_M52065_IG03
						;; size=8 bbWeight=2    PerfScore 6.00
G_M52065_IG11:
       call     [System.ThrowHelper:ThrowArgumentOutOfRange_IndexMustBeLessOrEqualException()]
       int3     
						;; size=7 bbWeight=0    PerfScore 0.00
G_M52065_IG12:
       call     [System.ThrowHelper:ThrowStartIndexArgumentOutOfRange_ArgumentOutOfRange_IndexMustBeLess()]
       int3     
						;; size=7 bbWeight=0    PerfScore 0.00
G_M52065_IG13:
       call     [System.ThrowHelper:ThrowArgumentException_DestinationTooShort()]
       int3     
						;; size=7 bbWeight=0    PerfScore 0.00
G_M52065_IG14:
       call     CORINFO_HELP_RNGCHKFAIL
       int3     
						;; size=6 bbWeight=0    PerfScore 0.00
RWD00  	dq	4000000000000000h	;            2
RWD08  	dd	00000000h, 00000000h, 00000000h, 00000000h, 00000000h, 00000000h
RWD32  	dq	4000000000000000h, 4000000000000000h, 4000000000000000h, 4000000000000000h


; Total bytes of code 717, prolog size 53, PerfScore 1994.45, instruction count 180, allocated bytes for code 717 (MethodHash=2f54349e) for method Burgers:GetCalculated3(int,int,double,double,double,double[]):double[]
; ============================================================

Unwind Info:
  >> Start offset   : 0x000000 (not in unwind data)
  >>   End offset   : 0xd1ffab1e (not in unwind data)
  Version           : 1
  Flags             : 0x00
  SizeOfProlog      : 0x21
  CountOfUnwindCodes: 13
  FrameRegister     : none (0)
  FrameOffset       : N/A (no FrameRegister) (Value=0)
  UnwindCodes       :
    CodeOffset: 0x21 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM8 (8)
      Scaled Small Offset: 2 * 16 = 32 = 0x00020
    CodeOffset: 0x1B UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM7 (7)
      Scaled Small Offset: 3 * 16 = 48 = 0x00030
    CodeOffset: 0x15 UnwindOp: UWOP_SAVE_XMM128 (8)     OpInfo: XMM6 (6)
      Scaled Small Offset: 4 * 16 = 64 = 0x00040
    CodeOffset: 0x0C UnwindOp: UWOP_ALLOC_SMALL (2)     OpInfo: 10 * 8 + 8 = 88 = 0x58
    CodeOffset: 0x08 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rbx (3)
    CodeOffset: 0x07 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rbp (5)
    CodeOffset: 0x06 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rsi (6)
    CodeOffset: 0x05 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: rdi (7)
    CodeOffset: 0x04 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: r14 (14)
    CodeOffset: 0x02 UnwindOp: UWOP_PUSH_NONVOL (0)     OpInfo: r15 (15)

@tannergooding
Copy link
Member

tannergooding commented Jan 5, 2023 •
edited
Loading

We have some significant diffs that result in 6 bytes less of codegen.
We have 4 inlinees, where we had 0 before, which are due to the Vector<T>.CopyTo method no longer being intrinsic and instead being inlined.
We have a significantly smaller local frame size (88 vs 136) and less total register spills.

It looks like the main cost difference is we have two hoisted broadcasts in the "before" case where-as we don't in the new one. This is a hoist of Create(dt / dx) and Create(factor) (both of which are used in multipliers).

We have the same number of arg#, loc#, and rat#. However, we have 7 fewer cse# in favor 7 new tmp# due to the inlining args.

@tannergooding
Copy link
Member

tannergooding commented Jan 5, 2023 •
edited
Loading

This basically comes down to the following "minimal" repro:

private static (int, int) Load(int[] array, int index)
{
    if ((index < 0) || ((array.Length - index) < 2))
    {
        throw new ArgumentOutOfRangeException();
    }

    return (array[index + 0], array[index + 1]);
}

When this was intrinsic, we created a COMMA BOUNDS_CHECK, LCL_VAR node. With it being inlined, we create a more complex sequence of IR involving two JTRUE and the actual comparisons.

Given such a pattern is fairly common/typical, it would likely be beneficial for us to either recognize/handle such sequences or even optimize them down to BOUNDS_CHECK where feasible.

@tannergooding tannergooding mentioned this issue Jan 5, 2023
Open
@tannergooding
Copy link
Member

Going to close this issue as "by design" given the other improvements and the relatively simple workaround users have (they can use Vector128.Load/StoreUnsafe for example).

I've opened #80256 to track the broader issue around manual bounds checks not getting recognized/optimized.

@ghost ghost locked as resolved and limited conversation to collaborators Feb 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@EgorBo EgorBo

@tannergooding tannergooding

Labels
arch-x64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-windows tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
8.0.0
Development

No branches or pull requests
4 participants
@EgorBo @jeffschwMSFT @tannergooding @JulieLeeMSFT