Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hundreds of tests failed with Assert failure: Consistency check failed: AV in clr at this callstack #52738

Closed
VincentBu opened this issue May 14, 2021 · 4 comments · Fixed by #52745
Closed

Hundreds of tests failed with Assert failure: Consistency check failed: AV in clr at this callstack #52738

VincentBu opened this issue May 14, 2021 · 4 comments · Fixed by #52745
Assignees
@AaronRobinsonMSFT
Labels
area-Interop-coreclr GCStress
Milestone
6.0.0

Comments

@VincentBu
Copy link
Contributor

Run: runtime-coreclr gcstress0x3-gcstress0xc 20210513.1

One of Error messages:

Assert failure(PID 4360 [0x00001108], Thread: 10196 [0x27d4]): Consistency check failed: AV in clr at this callstack:
CORECLR! WKS::gc_heap::plan_phase + 0x1917 (0x00007ffedd3268b3) CORECLR! WKS::gc_heap::gc1 + 0x276 (0x00007ffedd31ab26)
CORECLR! WKS::gc_heap::garbage_collect + 0x4E2 (0x00007ffedd31a7fe) CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x292 (0x00007ffedd305aae)
CORECLR! WKS::GCHeap::GarbageCollect + 0x11E (0x00007ffedd3057ce) CORECLR! WKS::GCHeap::StressHeap + 0x4F3 (0x00007ffedd3082d3)
CORECLR! Thread::PerformPreemptiveGC + 0x339 (0x00007ffedd191821) CORECLR! Thread::RareEnablePreemptiveGC + 0x13F (0x00007ffedd192033)
CORECLR! Thread::EnablePreemptiveGC + 0x11A (0x00007ffedcc7e4c6) CORECLR! GCHolderBase::EnterInternalPreemp + 0x183 (0x00007ffedcc7fe47)
CORECLR! GCPreemp::GCPreemp + 0x29 (0x00007ffedcc62795) CORECLR! Thread::SetApartment + 0x3E1 (0x00007ffedcef42ad)
CORECLR! Thread::FinishInitialization + 0x10B (0x00007ffedceec367) CORECLR! Thread::HasStarted + 0x209 (0x00007ffedceeef05)
CORECLR! ThreadNative::KickOffThread + 0xD6 (0x00007ffedcf82f86) KERNEL32! BaseThreadInitThunk + 0x14 (0x00007fff098984d4)
NTDLL! RtlUserThreadStart + 0x21 (0x00007fff`0a951781)
.AV on tid=0x27d4 (10196), cxr=000000CAC75FE580, exr=000000CAC75FEA70
FAILED: false

CORECLR! CHECK::Trigger + 0x2C9 (0x00007ffedd36e1e9) CORECLR! CLRVectoredExceptionHandlerPhase3 + 0x2B6 (0x00007ffedcd905c2)
CORECLR! CLRVectoredExceptionHandlerPhase2 + 0x97 (0x00007ffedcd9007b) CORECLR! CLRVectoredExceptionHandler + 0x215 (0x00007ffedcd8ffc1)
CORECLR! CLRVectoredExceptionHandlerShim + 0x1AA (0x00007ffedcd9084a) NTDLL! RtlInitializeCriticalSection + 0x190 (0x00007fff0a966b30)
NTDLL! RtlImageNtHeaderEx + 0x17B (0x00007fff0a9346bb) NTDLL! KiUserExceptionDispatcher + 0x3A (0x00007fff0a9a987a)
CORECLR! WKS::gc_heap::plan_phase + 0x1917 (0x00007ffedd3268b3) CORECLR! WKS::gc_heap::gc1 + 0x276 (0x00007ffedd31ab26)
File: D:\workspace_work\1\s\src\coreclr\vm\excep.cpp Line: 7502
Image: C:\h\w\9E41087C\p\corerun.exe

Return code:      1
Raw output file:      C:\h\w\9E41087C\w\B3810975\e\baseservices\exceptions\Reports\baseservices.exceptions\simple\ParallelCrash\ParallelCrash.output.txt
Raw output:
BEGIN EXECUTION
"C:\h\w\9E41087C\p\corerun.exe"  ParallelCrash.dll
Expected: -2146232797
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\9E41087C\p
C:\h\w\9E41087C\w\B3810975\e\baseservices\exceptions\simple\ParallelCrash\ParallelCrash.cmd
Expected: True
Actual:   False


Stack trace
   at baseservices_exceptions._simple_ParallelCrash_ParallelCrash_._simple_ParallelCrash_ParallelCrash_cmd()
@dotnet-issue-labeler dotnet-issue-labeler bot added area-GC-coreclr untriaged New issue has not been triaged by the area owner labels May 14, 2021
@ghost
Copy link

ghost commented May 14, 2021

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

Run: runtime-coreclr gcstress0x3-gcstress0xc 20210513.1

One of Error messages:

Assert failure(PID 4360 [0x00001108], Thread: 10196 [0x27d4]): Consistency check failed: AV in clr at this callstack:
CORECLR! WKS::gc_heap::plan_phase + 0x1917 (0x00007ffedd3268b3) CORECLR! WKS::gc_heap::gc1 + 0x276 (0x00007ffedd31ab26)
CORECLR! WKS::gc_heap::garbage_collect + 0x4E2 (0x00007ffedd31a7fe) CORECLR! WKS::GCHeap::GarbageCollectGeneration + 0x292 (0x00007ffedd305aae)
CORECLR! WKS::GCHeap::GarbageCollect + 0x11E (0x00007ffedd3057ce) CORECLR! WKS::GCHeap::StressHeap + 0x4F3 (0x00007ffedd3082d3)
CORECLR! Thread::PerformPreemptiveGC + 0x339 (0x00007ffedd191821) CORECLR! Thread::RareEnablePreemptiveGC + 0x13F (0x00007ffedd192033)
CORECLR! Thread::EnablePreemptiveGC + 0x11A (0x00007ffedcc7e4c6) CORECLR! GCHolderBase::EnterInternalPreemp + 0x183 (0x00007ffedcc7fe47)
CORECLR! GCPreemp::GCPreemp + 0x29 (0x00007ffedcc62795) CORECLR! Thread::SetApartment + 0x3E1 (0x00007ffedcef42ad)
CORECLR! Thread::FinishInitialization + 0x10B (0x00007ffedceec367) CORECLR! Thread::HasStarted + 0x209 (0x00007ffedceeef05)
CORECLR! ThreadNative::KickOffThread + 0xD6 (0x00007ffedcf82f86) KERNEL32! BaseThreadInitThunk + 0x14 (0x00007fff098984d4)
NTDLL! RtlUserThreadStart + 0x21 (0x00007fff`0a951781)
.AV on tid=0x27d4 (10196), cxr=000000CAC75FE580, exr=000000CAC75FEA70
FAILED: false

CORECLR! CHECK::Trigger + 0x2C9 (0x00007ffedd36e1e9) CORECLR! CLRVectoredExceptionHandlerPhase3 + 0x2B6 (0x00007ffedcd905c2)
CORECLR! CLRVectoredExceptionHandlerPhase2 + 0x97 (0x00007ffedcd9007b) CORECLR! CLRVectoredExceptionHandler + 0x215 (0x00007ffedcd8ffc1)
CORECLR! CLRVectoredExceptionHandlerShim + 0x1AA (0x00007ffedcd9084a) NTDLL! RtlInitializeCriticalSection + 0x190 (0x00007fff0a966b30)
NTDLL! RtlImageNtHeaderEx + 0x17B (0x00007fff0a9346bb) NTDLL! KiUserExceptionDispatcher + 0x3A (0x00007fff0a9a987a)
CORECLR! WKS::gc_heap::plan_phase + 0x1917 (0x00007ffedd3268b3) CORECLR! WKS::gc_heap::gc1 + 0x276 (0x00007ffedd31ab26)
File: D:\workspace_work\1\s\src\coreclr\vm\excep.cpp Line: 7502
Image: C:\h\w\9E41087C\p\corerun.exe

Return code:      1
Raw output file:      C:\h\w\9E41087C\w\B3810975\e\baseservices\exceptions\Reports\baseservices.exceptions\simple\ParallelCrash\ParallelCrash.output.txt
Raw output:
BEGIN EXECUTION
"C:\h\w\9E41087C\p\corerun.exe"  ParallelCrash.dll
Expected: -2146232797
Actual: -1073740286
END EXECUTION - FAILED
FAILED
Test Harness Exitcode is : 1
To run the test:

set CORE_ROOT=C:\h\w\9E41087C\p
C:\h\w\9E41087C\w\B3810975\e\baseservices\exceptions\simple\ParallelCrash\ParallelCrash.cmd
Expected: True
Actual:   False


Stack trace
   at baseservices_exceptions._simple_ParallelCrash_ParallelCrash_._simple_ParallelCrash_ParallelCrash_cmd()
Author: VincentBu
Assignees: -
Labels:

GCStress, area-GC-coreclr, untriaged
Milestone: -

@jkotas
Copy link
Member

jkotas commented May 14, 2021

@AaronRobinsonMSFT Looks like regression introduced by #52023

@AaronRobinsonMSFT
Copy link
Member

@jkotas Yes. This makes me very sad. The order of operations in HasStarted() was changed slightly and it wasn't obvious that the order would cause this sort of problem. I will see how much effort it takes to fix it up or just revert the changes if they require a lot of churn.

@AaronRobinsonMSFT AaronRobinsonMSFT removed the untriaged New issue has not been triaged by the area owner label May 14, 2021
@AaronRobinsonMSFT AaronRobinsonMSFT added this to the 6.0.0 milestone May 14, 2021
@AaronRobinsonMSFT AaronRobinsonMSFT self-assigned this May 14, 2021
@jkotas
Copy link
Member

jkotas commented May 14, 2021

The refactoring that introduced this issue was not really needed for the final version of #52023. It may be be easiest to revert it.

@AaronRobinsonMSFT AaronRobinsonMSFT mentioned this issue May 14, 2021
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label May 14, 2021
@ghost ghost closed this as completed in #52745 May 14, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 14, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Jun 13, 2021
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@AaronRobinsonMSFT AaronRobinsonMSFT

Labels
area-Interop-coreclr GCStress
Projects
None yet
Milestone
6.0.0
Development

Successfully merging a pull request may close this issue.

Prepare the COM apartment and context at a safe time AaronRobinsonMSFT/runtime
3 participants
@jkotas @AaronRobinsonMSFT @VincentBu