Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System.Text.RegularExpressions.Tests JitMinOpts only timeout #10680

Closed
BruceForstall opened this issue Jul 13, 2018 · 19 comments
Closed

System.Text.RegularExpressions.Tests JitMinOpts only timeout #10680

BruceForstall opened this issue Jul 13, 2018 · 19 comments
Assignees
@stephentoub
Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitUntriaged CLR JIT issues needing additional triage test-bug Problem in test source code (most likely)
Milestone
Future

Comments

@BruceForstall
Copy link
Member 

----- start 19:14:03.62 ===============  To repro directly: ===================================================== 
pushd C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64\
set COMPlus_TieredCompilation=0 
call C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm64\dotnet.exe xunit.console.netcore.exe System.Text.RegularExpressions.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
popd
===========================================================================================================

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>set COMPlus_TieredCompilation=0  

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>set COMPlus_JITMinOpts=1 

C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\tests\System.Text.RegularExpressions.Tests\netcoreapp-Windows_NT-Release-arm64>call C:\Users\robox\j\workspace\arm64_cross_c---f75df612\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm64\dotnet.exe xunit.console.netcore.exe System.Text.RegularExpressions.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
xUnit.net console test runner (64-bit .NET Core)
Copyright (C) 2014 Outercurve Foundation.

Discovering: System.Text.RegularExpressions.Tests
Discovered:  System.Text.RegularExpressions.Tests
Starting:    System.Text.RegularExpressions.Tests
   System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix [FAIL]
      Timed out after 60000ms waiting for remote process 5512
      Expected: True
      Actual:   False
      Stack Trace:
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\CoreFx.Private.TestUtilities\src\System\Diagnostics\RemoteExecutorTestBase.cs(308,0): at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose(Boolean disposing)
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\CoreFx.Private.TestUtilities\src\System\Diagnostics\RemoteExecutorTestBase.cs(295,0): at System.Diagnostics.RemoteExecutorTestBase.RemoteInvokeHandle.Dispose()
         D:\j\workspace\arm64_cross_c---101bec61\_\fx\src\System.Text.RegularExpressions\tests\Regex.Match.Tests.cs(795,0): at System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix()
Finished:    System.Text.RegularExpressions.Tests

=== TEST EXECUTION SUMMARY ===
   System.Text.RegularExpressions.Tests  Total: 1498, Errors: 0, Failed: 1, Skipped: 0, Time: 197.040s
----- end 19:17:36.82 ----- exit code 1 ----------------------------------------------------------

Related: https://github.com/dotnet/coreclr/issues/17754

category:testing
theme:minopts
skill-level:intermediate
cost:medium
@kouvel
Copy link
Member

kouvel commented Jul 31, 2018 •
edited
Loading

This is also failing on x64 on several OSes in the coreclr repo's corefx test runs when tiering is enabled (equivalent to minopts for this test). Not sure if it's related to the arm64 failure, but it's not clear to me what the problem is on x64. Rough numbers from my machine:

corefx repo all regex tests -

  • default
    • release: 2700 ms
    • debug: 2800 ms
  • minopts
    • release: 6100 ms
    • debug: 3000 ms

just the Match_ExcessPrefix test (times in ms for each regex in test) -

  • default
    • release coreclr + release corefx: 60, 60, 1000
    • debug coreclr + debug corefx: 840, 840, 840
  • minopts
    • release coreclr + release corefx: 40, 40, 4600
    • debug coreclr + debug corefx: 840, 840, 840

I don't see any numbers close to 60 seconds and yet it's timing out. I noticed that if I add RegexOptions.Compiled, the last regex in the test is taking ~60000 ms to complete with release coreclr and release corefx (and 1000 ms with debug coreclr and debug corefx), but that option is not specified in the test.

@kouvel kouvel changed the title [arm64] System.Text.RegularExpressions.Tests JitMinOpts only timeout System.Text.RegularExpressions.Tests JitMinOpts only timeout Jul 31, 2018
kouvel referenced this issue in kouvel/corefx Jul 31, 2018
@kouvel
Disable a regex test that is failing with minopts
fe7c303 
See https://github.com/dotnet/coreclr/issues/18912. It looks like corefx test runs are being triggered from PRs in coreclr, disabling because this would always fail once tiering is enabled.
@kouvel
Copy link
Member

kouvel commented Jul 31, 2018

PR to temporarily disable test: dotnet/corefx#31500

@echesakov
Copy link
Contributor

echesakov commented Aug 1, 2018 •
edited
Loading

I investigated this timeout failure and I believe this is due to the fact that under debug and minopts JIT does not expand CORINFO_INTRINSIC_StringGetChar and instead emit call to COMString::GetCharAt helper in System.Text.RegularExpressions.RegexBoyerMoore:.ctor(ref,bool,bool,ref):this.

I did simple experiment and for optimized code I disabled ONLY expansion of CORINFO_INTRINSIC_StringGetChar in importer which brings up running time for System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix from 2s up to 550s (which is roughly the same time needed for the test to finish under JITMinOpts).

Another experiment showed that setting JitMinOptsName=System.Text.RegularExpressions.RegexBoyerMoore:.ctor(ref,bool,bool,ref):this is enough for this test to fail due to timeout out.

@dotnet/jit-contrib Is this supposed to be this way? This call to COMString::GetCharAt happens in a loop and according to PerfView takes 99.4 Inc %

I also attached JitDisasm in case if someone is interested to see what's going on.

optimized code dasm.txt
optimized code - forbid CORINFO_INTRINSIC_StringGetChar expansion dasm.txt
minopts dasm.txt

@BruceForstall
Copy link
Member Author

The JIT wants to leave calls as calls, and not do the intrinsic optimization, during MinOpts. This seems completely reasonable. The test does pass, it's just super slow. That would be expected. Seems like we just need a way to bump the timeout of this test, significantly, under Minopts/debuggable code.

@AndyAyersMS
Copy link
Member

AndyAyersMS commented Aug 7, 2018 •
edited
Loading

This is perhaps and example where Tier0 should diverge from minopts?

kouvel referenced this issue in kouvel/coreclr Aug 14, 2018
@kouvel
Disable a regex test that is failing with minopts
f86233b 
See dotnet/coreclr#18912. This test would always fail once tiered compilation is enabled by default in CoreFX test runs in the CoreCLR repo.
@kouvel
Copy link
Member

kouvel commented Aug 14, 2018

Is that an issue even when RegexOptions.Compiled is not specified? The test doesn't specify RegexOptions.Compiled and locally the test was running fairly quickly with debug coreclr and debug corefx. Only when I change the test to use RegexOptions.Compiled it was very slow, any idea why it would be failing in the CI?

kouvel referenced this issue in kouvel/corefx Aug 14, 2018
@kouvel
Disable a regex test that is failing with minopts
4b4a95e 
See https://github.com/dotnet/coreclr/issues/18912. It looks like corefx test runs are being triggered from PRs in coreclr, disabling because this would always fail once tiering is enabled.
kouvel referenced this issue in kouvel/coreclr Aug 14, 2018
@kouvel
Disable a regex test that is failing with minopts
ee4144f 
See dotnet/coreclr#18912. This test would always fail once tiered compilation is enabled by default in CoreFX test runs in the CoreCLR repo.
kouvel referenced this issue in dotnet/coreclr Aug 16, 2018
@kouvel
Disable a regex test that is failing with minopts (#19488)
fc44943 
See dotnet/coreclr#18912. This test would always fail once tiered compilation is enabled by default in CoreFX test runs in the CoreCLR repo.
@kouvel
Copy link
Member

kouvel commented Aug 16, 2018

The test was disabled for the coreclr repo's "CoreFX Tests" job in dotnet/coreclr#19488, once this is fixed please re-enable the test.

@danmoseley
Copy link
Member

@ViktorHofer fyi.

@msftgits msftgits transferred this issue from dotnet/coreclr Jan 31, 2020
@msftgits msftgits added this to the Future milestone Jan 31, 2020
@stephentoub
Copy link
Member

FYI, this test has been changed since this issue was opened.

@BruceForstall BruceForstall added the JitUntriaged CLR JIT issues needing additional triage label Oct 28, 2020
@am11
Copy link
Member

am11 commented Sep 23, 2021

With engine SourceGenerated, same test has started to fail:

windows x64 - https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-1846899be12742d186/System.Text.RegularExpressions.Tests/1/console.a2979f90.log

    System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated) [FAIL]
      Half-way through waiting for remote process.
      Memory load: 11
      ...
      Timed out at 9/22/2021 11:53:49 PM after 60000ms waiting for remote process.
      Wrote mini dump to: C:\h\w\99FC08C0\w\B18309D5\uploads\5588.fckw5zf0.h4d.dmp
      	Process ID: 5588
      	Handle: 1984
      	Name: dotnet
      	MainModule: C:\h\w\99FC08C0\p\dotnet.exe
      	StartTime: 9/22/2021 11:52:49 PM
      	TotalProcessorTime: 00:01:02.6718750

linux musl arm64 - https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-ae9baf50200a4cdebf/System.Text.RegularExpressions.Tests/1/console.28be79eb.log

    System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated) [FAIL]
      Half-way through waiting for remote process.
      Timed out at 9/23/2021 12:13:52 AM after 60000ms waiting for remote process.
      	Process ID: 259
      	Handle: 3160
      	Name: dotnet
      	MainModule: /root/helix/work/correlation/dotnet
      	StartTime: 9/23/2021 12:12:51 AM
      	TotalProcessorTime: 00:01:03.2400000

@stephentoub
Copy link
Member

With engine SourceGenerated, same test has started to fail:

I'd expect this to behave similarly to RegexOptions.Compiled, but with even more expense, as it's also running Roslyn to compile C#.

You're seeing this failure only in JitMinOpts, or in general?

@VincentBu
Copy link
Contributor

Failed again in: runtime 20210922.59

Failed test:

net7.0-windows-Release-x64-CoreCLR_checked-Windows.10.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated)

net7.0-windows-Release-x86-CoreCLR_checked-Windows.10.Amd64.Open

- System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(engine: SourceGenerated)

Error message:

Half-way through waiting for remote process.
Memory load: 12

Image Name                     PID Services

========================= ======== ============================================
svchost.exe                    772 BrokerInfrastructure, DcomLaunch, LSM,

PlugPlay, Power, SystemEventsBroker

svchost.exe                    836 RpcEptMapper, RpcSs

svchost.exe                    944 Appinfo, CertPropSvc, gpsvc, iphlpsvc,

lfsvc, ProfSvc, sacsvr, Schedule, SENS,

SessionEnv, ShellHWDetection, Themes,

UserManager, Winmgmt, WpnService, wuauserv

svchost.exe                    952 TermService

svchost.exe                   1012 Dhcp, EventLog, lmhosts, TimeBrokerSvc,

vmictimesync

svchost.exe                    444 NcbService, PcaSvc, TrkWks, UALSVC,

UmRdpService, vmickvpexchange,

vmicshutdown, WdiSystemHost

svchost.exe                     84 CDPSvc, EventSystem, FontCache, netprofm,

nsi, W32Time, WinHttpAutoProxySvc

svchost.exe                   1140 CryptSvc, Dnscache, LanmanWorkstation,

NlaSvc, WinRM

svchost.exe                   1180 vmicheartbeat

svchost.exe                   1276 BFE, CoreMessagingRegistrar, DPS, MpsSvc,

pla

svchost.exe                   1484 Wcmsvc

svchost.exe                   2028 LanmanServer

svchost.exe                   1320 DiagTrack

svchost.exe                   1512 StateRepository, tiledatamodelsvc

svchost.exe                   1696 AppHostSvc

svchost.exe                   1904 W3SVC, WAS

svchost.exe                   3152 CDPUserSvc_43fee, OneSyncSvc_43fee

Timed out at 9/22/2021 9:32:11 PM after 60000ms waiting for remote process.
Wrote mini dump to: C:\h\w\B8C009E0\w\C4A90A8F\uploads\4664.i4er4p0z.55v.dmp
Process ID: 4664
Handle: 1496
Name: dotnet
MainModule: C:\h\w\B8C009E0\p\dotnet.exe
StartTime: 9/22/2021 9:31:11 PM
TotalProcessorTime: 00:01:02.5312500
Threads:
Thread #1 (OS 0x39C)   [MTA]
[HelperMethodFrame_1OBJ] (System.Threading.Monitor.ObjWait)
System.Threading.Monitor.Wait(System.Object, Int32)
System.Threading.ManualResetEventSlim.Wait(Int32, System.Threading.CancellationToken)
System.Threading.Tasks.Task.SpinThenBlockingWait(Int32, System.Threading.CancellationToken)
System.Threading.Tasks.Task.InternalWaitCore(Int32, System.Threading.CancellationToken)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.WaitForWorkers()
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethodBodies(Microsoft.CodeAnalysis.CSharp.CSharpCompilation, Microsoft.CodeAnalysis.CSharp.Emit.PEModuleBuilder, Boolean, Boolean, Boolean, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, System.Predicate1<Microsoft.CodeAnalysis.CSharp.Symbol>, System.Threading.CancellationToken) 			Microsoft.CodeAnalysis.CSharp.CSharpCompilation.CompileMethods(Microsoft.CodeAnalysis.Emit.CommonPEModuleBuilder, Boolean, Boolean, Boolean, Microsoft.CodeAnalysis.DiagnosticBag, System.Predicate1<Microsoft.CodeAnalysis.Symbols.ISymbolInternal>, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, Microsoft.CodeAnalysis.RebuildData, Microsoft.CodeAnalysis.CodeGen.CompilationTestData, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, System.IO.Stream, Microsoft.CodeAnalysis.RebuildData, System.Threading.CancellationToken)
Microsoft.CodeAnalysis.Compilation.Emit(System.IO.Stream, System.IO.Stream, System.IO.Stream, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.ResourceDescription>, Microsoft.CodeAnalysis.Emit.EmitOptions, Microsoft.CodeAnalysis.IMethodSymbol, System.IO.Stream, System.Collections.Generic.IEnumerable1<Microsoft.CodeAnalysis.EmbeddedText>, System.IO.Stream, System.Threading.CancellationToken)
System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6.MoveNext()
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6, System.Text.RegularExpressions.Tests]](<SourceGenRegexAsync>d__6 ByRef)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder1[[System.__Canon, System.Private.CoreLib]].Start[[System.Text.RegularExpressions.Tests.RegexGeneratorHelper+<SourceGenRegexAsync>d__6, System.Text.RegularExpressions.Tests]](<SourceGenRegexAsync>d__6 ByRef) 			System.Text.RegularExpressions.Tests.RegexGeneratorHelper.SourceGenRegexAsync(System.String, System.Nullable1<System.Text.RegularExpressions.RegexOptions>, System.Nullable1<System.TimeSpan>, System.Threading.CancellationToken) 			System.Text.RegularExpressions.Tests.RegexHelpers+<GetRegexAsync>d__8.MoveNext() 			System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexHelpers+<GetRegexAsync>d__8, System.Text.RegularExpressions.Tests]](<GetRegexAsync>d__8 ByRef) 			System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d.MoveNext() 			System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d, System.Text.RegularExpressions.Tests]](<<Match_ExcessPrefix>b__29_0>d ByRef) 			System.Runtime.CompilerServices.AsyncTaskMethodBuilder.Start[[System.Text.RegularExpressions.Tests.RegexMatchTests+<>c+<<Match_ExcessPrefix>b__29_0>d, System.Text.RegularExpressions.Tests]](<<Match_ExcessPrefix>b__29_0>d ByRef) 			System.Text.RegularExpressions.Tests.RegexMatchTests+<>c.<Match_ExcessPrefix>b__29_0(System.String) 			[DebuggerU2MCatchHandlerFrame] 			[HelperMethodFrame_PROTECTOBJ] (System.RuntimeMethodHandle.InvokeMethod) 			System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo) 			System.Reflection.MethodBase.Invoke(System.Object, System.Object[]) 			Microsoft.DotNet.RemoteExecutor.Program.Main(System.String[]) 		Thread #2 (OS 0xAE0) [Finalizer] [Background] [MTA] 			[DebuggerU2MCatchHandlerFrame] 		Thread #3 (OS 0x1540) [Background] [MTA] 			[DebuggerU2MCatchHandlerFrame] 		Thread #5 (OS 0x115C) [Thread pool worker] [Background] [MTA] 			[DebuggerU2MCatchHandlerFrame] 		Thread #8 (OS 0x146C) [Thread pool worker] [Background] [MTA] 			[InlinedCallFrame] 			[InlinedCallFrame] 			Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) 			System.Threading.LowLevelLifoSemaphore.WaitCore(Int32) 			System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) 			System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) 			System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() 			System.Threading.Thread.StartCallback() 			[DebuggerU2MCatchHandlerFrame] 		Thread #9 (OS 0x1320) [Thread pool worker] [Background] [MTA] 			[InlinedCallFrame] 			[InlinedCallFrame] 			Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32) 			System.Threading.LowLevelLifoSemaphore.WaitCore(Int32) 			System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32) 			System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean) 			System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart() 			System.Threading.Thread.StartCallback() 			[DebuggerU2MCatchHandlerFrame] 		Thread #10 (OS 0xE9C) [Thread pool worker] [Background] [MTA] 			[PrestubMethodFrame] (Microsoft.CodeAnalysis.MetadataDecoder<System.__Canon,System.__Canon,System.__Canon,System.__Canon,System.__Canon>.DecodeTypeOrThrow) 			Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetTypeOfTypeSpec(System.Reflection.Metadata.TypeSpecificationHandle)
Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetTypeOfToken(System.Reflection.Metadata.EntityHandle, Boolean ByRef) 			Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].EnqueueTypeDefInterfacesAndBaseTypeOrThrow(System.Collections.Generic.Queue1<System.Reflection.Metadata.TypeDefinitionHandle>, System.Collections.Generic.Queue1<System.__Canon>, System.Reflection.Metadata.TypeDefinitionHandle)
Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].FindMethodSymbolInSuperType(System.Reflection.Metadata.TypeDefinitionHandle, System.Reflection.Metadata.MethodDefinitionHandle) 			Microsoft.CodeAnalysis.MetadataDecoder5[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].GetExplicitlyOverriddenMethods(System.Reflection.Metadata.TypeDefinitionHandle, System.Reflection.Metadata.MethodDefinitionHandle, System.__Canon)
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.get_ExplicitInterfaceImplementations()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.ComputeMethodKind()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PEMethodSymbol.get_MethodKind()
Microsoft.CodeAnalysis.CSharp.Symbols.SymbolExtensions.IsParameterlessConstructor(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol)
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.LoadMembers()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.EnsureAllMembersAreLoaded()
Microsoft.CodeAnalysis.CSharp.Symbols.Metadata.PE.PENamedTypeSymbol.GetSimpleNonTypeMembers(System.String)
Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol.GetOperators(System.String)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.<ComputeApplicableUserDefinedImplicitConversionSet>g__addCandidatesFromType|133_0(Microsoft.CodeAnalysis.CSharp.Symbols.TypeParameterSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol, Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.UserDefinedConversionAnalysis>, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.ComputeApplicableUserDefinedImplicitConversionSet(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol>, Microsoft.CodeAnalysis.PooledObjects.ArrayBuilder1<Microsoft.CodeAnalysis.CSharp.UserDefinedConversionAnalysis>, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean) 			Microsoft.CodeAnalysis.CSharp.ConversionsBase.AnalyzeImplicitUserDefinedConversions(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.GetImplicitUserDefinedConversion(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef) 			Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyImplicitConversionFromExpression(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef)
Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyConversionFromExpressionForCast(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef) 			Microsoft.CodeAnalysis.CSharp.ConversionsBase.ClassifyConversionFromExpression(Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeSymbol, Microsoft.CodeAnalysis.CompoundUseSiteInfo1<Microsoft.CodeAnalysis.CSharp.Symbols.AssemblySymbol> ByRef, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindCastCore(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BoundExpression, Microsoft.CodeAnalysis.CSharp.Symbols.TypeWithAnnotations, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindCast(Microsoft.CodeAnalysis.CSharp.Syntax.CastExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindValue(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, BindValueKind)
Microsoft.CodeAnalysis.CSharp.Binder.BindAssignment(Microsoft.CodeAnalysis.CSharp.Syntax.AssignmentExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionInternal(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpression(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindValue(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, BindValueKind)
Microsoft.CodeAnalysis.CSharp.Binder.BindRValueWithoutTargetType(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionStatement(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode, Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionSyntax, Boolean, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindExpressionStatement(Microsoft.CodeAnalysis.CSharp.Syntax.ExpressionStatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindBlockParts(Microsoft.CodeAnalysis.CSharp.Syntax.BlockSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindBlock(Microsoft.CodeAnalysis.CSharp.Syntax.BlockSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindStatement(Microsoft.CodeAnalysis.CSharp.Syntax.StatementSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag)
Microsoft.CodeAnalysis.CSharp.Binder.BindConstructorBody(Microsoft.CodeAnalysis.CSharp.Syntax.ConstructorDeclarationSyntax, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.Binder.BindMethodBody(Microsoft.CodeAnalysis.CSharp.CSharpSyntaxNode, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, Boolean)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.BindMethodBody(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol, Microsoft.CodeAnalysis.CSharp.TypeCompilationState, Microsoft.CodeAnalysis.CSharp.BindingDiagnosticBag, VariableState, Boolean, Microsoft.CodeAnalysis.CSharp.ImportChain ByRef, Boolean ByRef, InitialState ByRef)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileMethod(Microsoft.CodeAnalysis.CSharp.Symbols.MethodSymbol, Int32, ProcessedFieldInitializers ByRef, Microsoft.CodeAnalysis.CSharp.SynthesizedSubmissionFields, Microsoft.CodeAnalysis.CSharp.TypeCompilationState)
Microsoft.CodeAnalysis.CSharp.MethodCompiler.CompileNamedType(Microsoft.CodeAnalysis.CSharp.Symbols.NamedTypeSymbol)
Microsoft.CodeAnalysis.CSharp.MethodCompiler+<>c__DisplayClass23_0.<CompileNamedTypeAsync>b__0()
Roslyn.Utilities.UICultureUtilities+<>c__DisplayClass5_0.<WithCurrentUICulture>b__0()
System.Threading.Tasks.Task.InnerInvoke()
System.Threading.Tasks.Task+<>c.<.cctor>b__271_0(System.Object)
System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(System.Threading.Thread, System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
System.Threading.ThreadPoolWorkQueue.Dispatch()
System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
System.Threading.Thread.StartCallback()
[DebuggerU2MCatchHandlerFrame]
Thread #11 (OS 0x48C) [Thread pool worker] [Background] [MTA]
[InlinedCallFrame]
[InlinedCallFrame]
Interop+Kernel32.GetQueuedCompletionStatus(IntPtr, Int32 ByRef, UIntPtr ByRef, IntPtr ByRef, Int32)
System.Threading.LowLevelLifoSemaphore.WaitCore(Int32)
System.Threading.LowLevelLifoSemaphore.WaitForSignal(Int32)
System.Threading.LowLevelLifoSemaphore.Wait(Int32, Boolean)
System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
System.Threading.Thread.StartCallback()
[DebuggerU2MCatchHandlerFrame]


Stack trace
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 224
   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose() in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 57
   at System.Text.RegularExpressions.Tests.RegexMatchTests.Match_ExcessPrefix(RegexEngine engine) in /_/src/libraries/System.Text.RegularExpressions/tests/Regex.Match.Tests.cs:line 1095

@am11
Copy link
Member

am11 commented Sep 23, 2021

You're seeing this failure only in JitMinOpts, or in general?

In general. I noticed this in a few recent outerloop runs on main, also failing in some PRs.

@stephentoub
Copy link
Member

Yeah, it looks like running Roslyn on a checked runtime is just too slow.

@stephentoub stephentoub self-assigned this Sep 23, 2021
@stephentoub
Copy link
Member

Actually, this test is already disabled against both checked and min opts:

runtime/src/libraries/System.Text.RegularExpressions/tests/Regex.Match.Tests.cs

Lines 1091 to 1094 in 00aa651

[SkipOnCoreClr("Long running tests: https://github.com/dotnet/runtime/issues/10680", RuntimeConfiguration.Checked, RuntimeTestModes.JitMinOpts)]
[MemberData(nameof(RegexHelpers.AvailableEngines_MemberData), MemberType = typeof(RegexHelpers))]
public void Match_ExcessPrefix(RegexEngine engine)
{

But clearly from the output above, it's running on checked.

@safern, any known issue here?

@am11
Copy link
Member

am11 commented Sep 23, 2021

Ah, perhaps SkipOnTargetFramework followed by SkipOnCoreClr is not being handled properly?

Looks like it also failed a few times on arm64-musl release (e.g. https://github.com/dotnet/runtime/runs/3683859860) and windows x64 debug.

@am11
Copy link
Member

am11 commented Sep 24, 2021

Ops, I missed that RuntimeConfiguration == checked, LibrariesConfiguration == Release.
Double checked in runfo results: #59541, they are all failing on checked.

@am11
Copy link
Member

am11 commented Sep 24, 2021 •
edited
Loading

In SkipOnCoreClrDiscoverer, we use stressMode && runtimeConfiguration: https://github.com/dotnet/arcade/blob/8f58bb36250772fc0a4a0305daf307292a75a735/src/Microsoft.DotNet.XUnitExtensions/src/Discoverers/SkipOnCoreClrDiscoverer.cs#L50. However, in this case we expect it to perform stressMode || runtimeConfiguration, but I can see why we would need to keep && in other cases.

I have tested locally on macOS, it does not skip this test with RuntimeConfiguration=checked, and commenting /* , RuntimeTestModes.JitMinOpts */ part does skip it.

@am11 am11 mentioned this issue Sep 24, 2021
Merged
@radical radical mentioned this issue Sep 27, 2021
Merged
@AndyAyersMS AndyAyersMS mentioned this issue Sep 27, 2021
Closed
@radical radical mentioned this issue Sep 27, 2021
Merged
@ericstj ericstj mentioned this issue Sep 28, 2021
Merged
@stephentoub
Copy link
Member

This was fixed.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 29, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@stephentoub stephentoub

Labels
area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitUntriaged CLR JIT issues needing additional triage test-bug Problem in test source code (most likely)
Projects
None yet
Milestone
Future
Development

No branches or pull requests
9 participants
@stephentoub @am11 @echesakov @danmoseley @msftgits @AndyAyersMS @BruceForstall @kouvel @VincentBu