Pull Requests CI Tests Holes Tracker

[marek-safar]

This issue tracks rolling build (post PR merge) build and test failures caused by insufficient tests coverage during pull request CI build.

Broad impact:

• [wasm][aot] Almost all library tests failing on Windows with RuntimeError: null function or function signature mismatch #61721
  • Source (possibly) [mono][llvmonly] Optimize virtual calls/rgctx fetches in gsharedvt methods. #60491
• [wasm] Linux/AOT tests(9) broken #62362
  - Source [mono][wasm] Fix a windows cross compiler issue. #62339
• Constructing MD arrays in R2R code does not work on OSX/ARM64 #62747 (broke System.Console on OSX ARM64)
  - Source PE loader/image cleanups. No-copy mapping of R2R PEs on Windows. #61938
• [wasm] Update path to regex tests project, to be excluded for AOT #66044
  - Source Add regex "unit tests" test project #65944
• [wasm][aot] System.Text.RegularExpressions.Unit.Tests failing due to Maximum call stack size exceeded #66118
  - Source Avoid RegexCode/RegexWriter for all engines other than RegexInterpreter #65986
• [wasm] ILLinker update broke wasm builds #67538
  - Update PR broke wasm rolling builds in runtime-extra-platforms

Local impact:

• [Android][x64] Java errors in System.Security.Cryptography.Tests #62081
  • Source unknown
• Mono ARM64 JIT inconsistent with sign-extension #60649
  • Source Re-enable running DllImportGenerator.Tests suite on Mono. #60259
• [wasm] Loader/classloader/explicitlayout/NestedStructs/case04/case04.sh failing with FAIL: object and non-object field overlap was not detected #62303
  • Source [Mono] Fix support for nested structs with explicit layout #61467
• Test failure Loader/classloader/explicitlayout/NestedStructs/case05/case05.sh #62311
  • Source [mono] Fix StackTrace from a dim and Vtable offsets for static interface method #60770
• Test failure Loader/classloader/DictionaryExpansion/DictionaryExpansion/DictionaryExpansion.sh #61299
  • Source Use the Xunit assert library instead of CoreCLRTestLibrary for most asserts #61226
• jit/intrinsics/createspan_il/createspan_il.sh #62285
  • Source RuntimeHelpers.CreateSpan<T> #61079
• Mono jit/hardwareintrinsics/x86/x86base/pause tests failing #61693
  • Source Adding support for the X86Base.Pause intrinsic on Mono #61707
• [iOS/tvOS] Test Failure System.IO.Compression.Tests.ZipFile_Unix.CanZipNamedPipe  #62341
  • Source Add test that ensures that it's possible to zip a named pipe on Unix #62240
• Test failure Interop\\COM\\Activator\\ActivatorBuiltInComDisabled\\ActivatorBuiltInComDisabled.cmd #62067
  • Source Use reflection or a compile-time only shim assembly to reference unexposed corelib types. #61802
• System.Memory.Tests failing in checked ARM64 #61084
  • Source JIT: "Scaled" addressing mode on ARM #60808

@SamMonoRT @lewing @steveisok

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

General tracker for issues that were caused by insufficient tests run during CI build

• [wasm][aot] Almost all library tests failing on Windows with RuntimeError: null function or function signature mismatch #61721
• [Android][x64] Java errors in System.Security.Cryptography.Tests #62081

@SamMonoRT @lewing @steveisok

Author:	marek-safar
Assignees:	-
Labels:	area-Infrastructure, untriaged, tracking
Milestone:	-

[jkotas]

cc @agocke

[jkotas]

• System.Memory.Tests failing in checked ARM64 #61084

[danmoseley]

What sorts of issues are we tracking here -- in principle any outerloop failure is a CI test hole. I assume this is about inner loop failures in configurations that didn't run before commit?

BTW it is not well defined whether "CI" means before or after commit so I avoid the term.

[marek-safar]

What sorts of issues are we tracking here

Update the description to be clearer.

in principle any outerloop failure is a CI test hole

Yes and I would include them here if they ever show up as a problem.