-
Notifications
You must be signed in to change notification settings - Fork 319
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
VSTS build release agent unable to run Coded UI tests when using the Nuget VsTest platform #1353
Comments
@mjethwa Currently Coded UI and UWP tests not support with VSTest platform nuget package. |
We seem to have not called out the limitations in the package on nuget. Filing a bug to get the package description updated. Closing this issue out as adding Coded UI support is already being tracked as a story on our backlog. |
Fixed in latest preview package https://www.nuget.org/packages/Microsoft.TestPlatform/15.7.0-preview-20180307-01 |
I am still getting this error in my Release when using the latest preview of the Microsoft.TestPlatform Nuget package.
|
I'm having issues running Coded UI tests using the Nuget VsTest platform on a VSTS build agent, I don't think this is a licensing restriction as I have managed to get it working by independently installing a Visual Studio Test agent (https://docs.microsoft.com/en-us/visualstudio/test/lab-management/install-configure-test-agents) on the machine and then using the vstest.console.exe in it's path to run tests.
Steps to reproduce
In a VSTS agent running on Windows run the following steps on the agent phase:
Expected behavior
Should be able to run Coded UI tests
Actual behavior
Unable to find test framework dlls required to run the tests - The step also ignores the test steps and reports success
Diagnostic logs
Logs from the Visual Studio Test step:
2017-12-29T13:56:17.0872849Z ##[section]Starting: VsTest - testAssemblies
2017-12-29T13:56:17.0872849Z ==============================================================================
2017-12-29T13:56:17.0872849Z Task : Visual Studio Test
2017-12-29T13:56:17.0872849Z Description : Run tests with Visual Studio test runner
2017-12-29T13:56:17.0872849Z Version : 2.3.4
2017-12-29T13:56:17.0872849Z Author : Microsoft Corporation
2017-12-29T13:56:17.0872849Z Help : More Information
2017-12-29T13:56:17.0872849Z ==============================================================================
2017-12-29T13:56:21.5800849Z Run the tests locally using vstest.console.exe
2017-12-29T13:56:21.5800849Z ========================================================
2017-12-29T13:56:21.5800849Z Test selector : Test assemblies
2017-12-29T13:56:21.5800849Z Test assemblies : *test.dll,!*TestAdapter.dll,!*\obj**
2017-12-29T13:56:21.5800849Z Test filter criteria : null
2017-12-29T13:56:21.5800849Z Search folder : E:\Temp\VsTests\Acme
2017-12-29T13:56:21.5800849Z Run settings file : E:\VstsAgent_work\r1\a
2017-12-29T13:56:21.5800849Z Run in parallel : false
2017-12-29T13:56:21.5800849Z Run in isolation : false
2017-12-29T13:56:21.5800849Z Path to custom adapters : null
2017-12-29T13:56:21.5800849Z Other console options : null
2017-12-29T13:56:21.5800849Z Code coverage enabled : false
2017-12-29T13:56:21.5800849Z Rerun failed tests: false
2017-12-29T13:56:23.4208849Z VisualStudio version selected for test execution : toolsInstaller
2017-12-29T13:56:24.2632849Z ========================================================
2017-12-29T13:56:25.1212849Z [command]E:\VstsAgent_work_tool\VsTest\15.6.0-preview-20171211-02\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe @e:\VstsAgent_work_temp\0cab39c1-eca0-11e7-9a65-c50660987fc7.txt
2017-12-29T13:56:25.1212849Z ##[warning]Windows Error Reporting DontShowUI not set, if the windows error dialog pops-up in the middle of UI test execution than the test will hang
2017-12-29T13:56:25.1212849Z Microsoft (R) Test Execution Command Line Tool Version 15.6.0-preview-20171211-02
2017-12-29T13:56:25.1212849Z Copyright (c) Microsoft Corporation. All rights reserved.
2017-12-29T13:56:25.1212849Z
2017-12-29T13:56:25.1992849Z vstest.console.exe
2017-12-29T13:56:25.1992849Z "E:\Temp\VsTests\Acme\CodedUITestProject1.dll"
2017-12-29T13:56:25.1992849Z "E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll"
2017-12-29T13:56:25.1992849Z /logger:"trx"
2017-12-29T13:56:30.1444849Z Starting test execution, please wait...
2017-12-29T13:56:34.4812849Z Unable to load types from the test source 'E:\Temp\VsTests\Acme\CodedUITestProject1.dll'. Some or all of the tests in this source may not be discovered. If you are running unit tests for Windows Store apps and referencing custom WinMD type then please visit http://go.microsoft.com/fwlink/?LinkId=238340 for more info. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITesting, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
2017-12-29T13:56:34.4812849Z .
2017-12-29T13:56:34.4968849Z MSTestAdapter failed to discover tests in class 'CodedUITestProject1.CodedUITest1' of assembly 'E:\Temp\VsTests\Acme\CodedUITestProject1.dll'. Reason Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
2017-12-29T13:56:34.6216849Z Unable to load types from the test source 'E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll'. Some or all of the tests in this source may not be discovered. If you are running unit tests for Windows Store apps and referencing custom WinMD type then please visit http://go.microsoft.com/fwlink/?LinkId=238340 for more info. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITesting, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
2017-12-29T13:56:34.6216849Z .
2017-12-29T13:56:34.6216849Z MSTestAdapter failed to discover tests in class 'XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.CodedUITest1' of assembly 'E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll'. Reason Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
2017-12-29T13:56:34.6216849Z MSTestAdapter failed to discover tests in class 'XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.Categories' of assembly 'E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll'. Reason Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
2017-12-29T13:56:34.6216849Z MSTestAdapter failed to discover tests in class 'XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.Account' of assembly 'E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll'. Reason Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
2017-12-29T13:56:34.6216849Z MSTestAdapter failed to discover tests in class 'XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.Login' of assembly 'E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll'. Reason Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified..
2017-12-29T13:56:34.6996849Z No test is available in E:\Temp\VsTests\Acme\CodedUITestProject1.dll E:\Temp\VsTests\Acme\XyzPlc.Acme.Tests.AcmePhoneChatWebApp.UITests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
2017-12-29T13:56:34.7776849Z
2017-12-29T13:56:34.7776849Z Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:.
2017-12-29T13:56:34.8088849Z ##[warning]No results found to publish.
2017-12-29T13:56:35.9164849Z ##[section]Finishing: VsTest - testAssemblies
Environment
Windows Server 2008 R2 running VSTS agent version 2.126.0
Nuget VsTest 15.6.0-preview-20171211-02 x64
The text was updated successfully, but these errors were encountered: