Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fail dotnet test if no tests are found #706

Merged
merged 4 commits into from
Aug 8, 2023
Merged

Fail dotnet test if no tests are found #706

merged 4 commits into from
Aug 8, 2023

Conversation

edumserrano
Copy link
Owner

@edumserrano edumserrano commented Aug 8, 2023 •
edited
Loading

As a follow up to the reasoning explained in #704, this updates the workflow so that the dotnet test command fails if it doesn't find any test.

See:

Before this PR the dotnet test output would look like:

Test run for /home/runner/work/dotnet-sdk-extensions/dotnet-sdk-extensions/tests/DotNet.Sdk.Extensions.Testing.Tests/bin/Release/netcoreapp3.1/DotNet.Sdk.Extensions.Testing.Tests.dll (.NETCoreApp,Version=v3.1)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (x64)
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
No test is available in /home/runner/work/dotnet-sdk-extensions/dotnet-sdk-extensions/tests/DotNet.Sdk.Extensions.Testing.Tests/bin/Release/netcoreapp3.1/DotNet.Sdk.Extensions.Testing.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

but it wouldn't fail the dotnet test command.

With this change the output is still the same but the command now fails and the workflow is aborted.

@github-actions
Copy link

github-actions bot commented Aug 8, 2023 •
edited
Loading

Test runs on ubuntu-latest for commit 4b38b2d

✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v5.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:17:08 - 2023-08-08 12:17:12
Run Duration: 3s 485ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v5.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 5.0.17) [xUnit.net 00:00:00.68]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.88]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.89]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.71]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v5.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:17:13 - 2023-08-08 12:17:24
Run Duration: 11s 289ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v5.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 5.0.17) [xUnit.net 00:00:00.81]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.91]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.91]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:10.61]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v6.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:17:26 - 2023-08-08 12:17:33
Run Duration: 6s 6ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v6.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.20) [xUnit.net 00:00:00.48]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:01.29]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:01.29]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:05.22]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v6.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:17:33 - 2023-08-08 12:17:45
Run Duration: 11s 795ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v6.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.20) [xUnit.net 00:00:00.12]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.24]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.24]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:11.11]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v7.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:17:48 - 2023-08-08 12:17:53
Run Duration: 5s 479ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v7.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 7.0.9) [xUnit.net 00:00:00.30]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:01.08]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:01.08]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:04.80]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v7.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:17:54 - 2023-08-08 12:18:06
Run Duration: 11s 589ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v7.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 7.0.9) [xUnit.net 00:00:00.13]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.23]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.24]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:10.96]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v3.1 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:16:50 - 2023-08-08 12:16:54
Run Duration: 3s 916ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v3.1
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET Core 3.1.32) [xUnit.net 00:00:00.67]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.88]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.88]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.74]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v3.1 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:16:54 - 2023-08-08 12:17:06
Run Duration: 11s 214ms

GitHub Runner OS: ubuntu-latest
Operating System: Linux 5.15.0-1042-azure #49-Ubuntu SMP Tue Jul 11 17:28:46 UTC 2023
Framework: .NETCoreApp,Version=v3.1
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET Core 3.1.32) [xUnit.net 00:00:00.78]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.89]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.89]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:10.49]   Finished:    DotNet.Sdk.Extensions.Tests
----

@github-actions
Copy link

github-actions bot commented Aug 8, 2023 •
edited
Loading

Test runs on windows-latest for commit 4b38b2d

✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v5.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:19:45 - 2023-08-08 12:19:49
Run Duration: 4s 120ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v5.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 5.0.17) [xUnit.net 00:00:00.73]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.92]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:00.92]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:03.27]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v5.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:19:50 - 2023-08-08 12:20:02
Run Duration: 12s 370ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v5.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET 5.0.17) [xUnit.net 00:00:00.70]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.81]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.81]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:11.62]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v6.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:20:05 - 2023-08-08 12:20:41
Run Duration: 35s 491ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v6.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.20) [xUnit.net 00:00:01.56]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:04.00]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:04.00]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:11.70]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v6.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:20:42 - 2023-08-08 12:20:52
Run Duration: 10s 471ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v6.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 6.0.20) [xUnit.net 00:00:00.24]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.34]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.35]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:09.85]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v7.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:20:55 - 2023-08-08 12:21:04
Run Duration: 8s 700ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v7.0
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 7.0.9) [xUnit.net 00:00:00.71]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.48]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.48]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:07.93]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v7.0 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:21:04 - 2023-08-08 12:21:15
Run Duration: 10s 942ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v7.0
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.0.1+5ebf84cd75 (64-bit .NET 7.0.9) [xUnit.net 00:00:00.19]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.31]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.32]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:10.36]   Finished:    DotNet.Sdk.Extensions.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Testing.Tests.dll on .NETCoreApp,Version=v3.1 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 149

Date: 2023-08-08 12:19:19 - 2023-08-08 12:19:26
Run Duration: 6s 840ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v3.1
Assembly: DotNet.Sdk.Extensions.Testing.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
149 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET Core 3.1.32) [xUnit.net 00:00:02.29]   Discovering: DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.49]   Discovered:  DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:02.49]   Starting:    DotNet.Sdk.Extensions.Testing.Tests [xUnit.net 00:00:04.04]   Finished:    DotNet.Sdk.Extensions.Testing.Tests
----
✔️ Pass - DotNet.Sdk.Extensions.Tests.dll on .NETCoreApp,Version=v3.1 ---- ## Run Summary

Overall Result: ✔️ Pass
Pass Rate: 100%
Total Tests: 157

Date: 2023-08-08 12:19:29 - 2023-08-08 12:19:41
Run Duration: 12s 454ms

GitHub Runner OS: windows-latest
Operating System: Microsoft Windows 10.0.20348
Framework: .NETCoreApp,Version=v3.1
Assembly: DotNet.Sdk.Extensions.Tests.dll

✔️ Passed ❌ Failed ⚠️ Skipped
157 0 0
100% 0% 0%
## Run Messages
Informational 
 [xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.4.5+1caef2f33e (64-bit .NET Core 3.1.32) [xUnit.net 00:00:00.77]   Discovering: DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.87]   Discovered:  DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:00.88]   Starting:    DotNet.Sdk.Extensions.Tests [xUnit.net 00:00:11.70]   Finished:    DotNet.Sdk.Extensions.Tests
----

@codecov
Copy link

codecov bot commented Aug 8, 2023 •
edited
Loading

Codecov Report

Merging #706 (4b38b2d) into main (ea0cf9e) will not change coverage.
Report is 2 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #706   +/-   ##
=======================================
  Coverage   86.79%   86.79%           
=======================================
  Files          77       77           
  Lines        1409     1409           
  Branches      219      219           
=======================================
  Hits         1223     1223           
  Misses        100      100           
  Partials       86       86

see 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@edumserrano edumserrano changed the title Fail dotnet test if no test is found Fail dotnet test if no tests are found Aug 8, 2023
@edumserrano edumserrano merged commit 325e8dd into main Aug 8, 2023
9 checks passed
@edumserrano edumserrano deleted the test-zero branch August 8, 2023 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@edumserrano