NUnit does not work without .NET Core

[adampraszywka]

Since 3.16.0 NUnit.Console doesn't start without "dotnet.exe" available.
Installation of any .NET Core SDK works as a workaround.

& "C:\Program Files (x86)\NUnit.org\nunit-console\tools\nunit3-console.exe" SeleniumTestProject.dll --where "cat == X" 
NUnit Console 3.16.0 (Release) 
Copyright (c) 2022 Charlie Poole, Rob Prouse 
Runtime Environment 
   OS Version: Microsoft Windows NT 6.2.9200.0 
   Runtime: .NET Framework CLR v4.0.30319.42000 
Test Files 
    SeleniumTestProject.dll 
Test Filters 
    Where: cat == X
System.TypeInitializationException : The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception. 
  ----> System.ComponentModel.Win32Exception : The system cannot find the file specified 
--TypeInitializationException 
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception. 
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package) 
   at NUnit.Engine.Services.RuntimeFrameworkService.ApplyImageData(TestPackage package) 
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package) 
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner() 
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter) 
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter) 
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter) 
   at NUnit.ConsoleRunner.Program.Main(String[] args) 
-- 
Win32Exception 
The system cannot find the file specified 
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo) 
   at NUnit.Engine.Internal.RuntimeFrameworks.NetCoreFrameworkLocator.<GetRuntimeList>d__3.MoveNext() 
   at NUnit.Engine.Internal.RuntimeFrameworks.NetCoreFrameworkLocator.<FindDotNetCoreFrameworks>d__1.MoveNext() 
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection) 
   at NUnit.Engine.RuntimeFramework.FindAvailableFrameworks() 
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks() 
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor() 

[CharliePoole]

@Prakson This would be a critical bug if confirmed. I'll may to set up a new machine to test this but I'll try just inspecting the code first.

What runtimes were installed on the machine displaying this problem?

[adampraszywka]

No development runtimes or Visual Studio at all.
I discovered it during debugging CI/CD process that stopped working after NUnit.Console 3.16 update.

Clean installation of Windows Server 2016 Datacenter with few Visual C++ Redistributable packages.

[CharliePoole]

OK... if I can't get a copy of that OS, I'll try one I have in a VM by itself.

[CharliePoole]

Try running this in process. We might get some additional trace info that way.

[CharliePoole]

@Prakson Looking at your exception stack trace, I see that the inner Win32Exception references RuntimeFramework methods, which no longer exist. As best I can tell, the go back to version 3.10 of the engine. Is something in your own code referencing an older version of the engine?

[CharliePoole]

As an experiment, I modified the code so that the engine doesn't find any .NET Core installs. It was still able to run .NET Framework tests without error. In combination with the apparent usage of an older version of the engine, this leads me to suspect that this isn't a current bug. I'll leave the issue open for the moment so we can try to figure out a solution for you.

[adampraszywka]

I reproduced the same issue outside of corporate network.
As test env I used brand new Azure VM with Windows Server 2016 Datacenter.

I downloaded nunit-conolse attached in the latest release as zip archive (NUnit.Console-3.16.0.zip), unpacked it and started from powershell. "test" file does not exist. App had crashed before it touched it. So no references to older framework here.

PS C:\Users\prakson\Downloads\NUnit.Console-3.16.0\bin> .\nunit3-console.exe  test
NUnit Console 3.16.0 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Monday, December 5, 2022 7:45:08 AM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    test

System.TypeInitializationException : The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
  ----> System.ComponentModel.Win32Exception : The system cannot find the file specified

--TypeInitializationException
The type initializer for 'NUnit.Engine.Services.RuntimeFrameworkService' threw an exception.
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFrameworkInner(TestPackage package) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Services\RuntimeFrameworkService.cs:line 105
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFrameworkInner(TestPackage package) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Services\RuntimeFrameworkService.cs:line 99
   at NUnit.Engine.Services.RuntimeFrameworkService.SelectRuntimeFramework(TestPackage package) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Services\RuntimeFrameworkService.cs:line 93
   at NUnit.Engine.Runners.MasterTestRunner.GetEngineRunner() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Runners\MasterTestRunner.cs:line 258
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Runners\MasterTestRunner.cs:line 472
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Runners\MasterTestRunner.cs:line 150
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitConsole\nunit3-console\ConsoleRunner.cs:line 197
   at NUnit.ConsoleRunner.Program.Main(String[] args) in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitConsole\nunit3-console\Program.cs:line 111
--
Win32Exception
The system cannot find the file specified
   at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
   at NUnit.Engine.Internal.RuntimeFrameworks.NetCoreFrameworkLocator.<GetRuntimeList>d__3.MoveNext() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine.core\Internal\RuntimeFrameworks\NetCoreFrameworkLocator.cs:line 80
   at NUnit.Engine.Internal.RuntimeFrameworks.NetCoreFrameworkLocator.<FindDotNetCoreFrameworks>d__1.MoveNext() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine.core\Internal\RuntimeFrameworks\NetCoreFrameworkLocator.cs:line 36
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at NUnit.Engine.RuntimeFramework.FindAvailableFrameworks() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine.core\RuntimeFramework.cs:line 563
   at NUnit.Engine.RuntimeFramework.get_AvailableFrameworks() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine.core\RuntimeFramework.cs:line 323
   at NUnit.Engine.Services.RuntimeFrameworkService..cctor() in C:\Users\charlie\dev\NUnit\nunit-console\src\NUnitEngine\nunit.engine\Services\RuntimeFrameworkService.cs:line 21

I checked source code attached to the 3.16.0 release. It doesn't contain changes introduced in #1170. That's reason it calls RuntimeFramework methods.

[CharliePoole]

I see you're right. I must have been looking at main branch, which is where I'm building version 2.0, rather than version3 branch, which is what we're concerned about. I can see that the code doesn't assume any particular runtime is installed but does call dotnet --list-runtimes without checking if dotnet is installed!

[CharliePoole]

@Prakson Give this build a try... https://ci.appveyor.com/project/CharliePoole/nunit-console/builds/45576209/artifacts

[adampraszywka]

Seems like fix works.

PS C:\Users\prakson\Downloads\NUnit.Console-3.16.1-ci00003-issue-1277\bin> .\nunit3-console.exe .\nunit.engine.api.dll
NUnit Console 3.16.1-ci00003-issue-1277 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Monday, December 5, 2022 10:37:46 AM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    .\nunit.engine.api.dll


Errors, Failures and Warnings

1) Invalid : C:\Users\prakson\Downloads\NUnit.Console-3.16.1-ci00003-issue-1277\bin\nunit.engine.api.dll
No suitable tests found in 'C:\Users\prakson\Downloads\NUnit.Console-3.16.1-ci00003-issue-1277\bin\nunit.engine.api.dll'.
Either assembly contains no tests or proper test driver has not been found.

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2022-12-05 10:37:46Z
    End time: 2022-12-05 10:37:47Z
    Duration: 0.437 seconds

Results (nunit3) saved as TestResult.xml
PS C:\Users\prakson\Downloads\NUnit.Console-3.16.1-ci00003-issue-1277\bin>

[CharliePoole]

Great! I'll do a PR and merge it. Sorry for my initial confusion.

[CharliePoole]

@Prakson The merged fix (including other fixes) is available on our myget feed as version 3.6.1-dev00006. I'd appreciate it if you could try that one as well.

[CharliePoole]

This issue has been resolved in version 3.16.1

The release is available on:
GitHub.
NuGet packages are also available NuGet.org and
Chocolatey Packages may be found at Chocolatey.org