[release/8.0] Stop testing on windows.10.amd64.android.open

[dougbu]

• see Close windows.10.amd64.android.open dnceng#1994
• use windows.11.amd64.android.open instead
• was previously done as part of [main] Update dependencies from 6 repositories #14453 for release/9.0

To double check:

• The right tests are in and the right validation has happened. Guidance: https://github.com/dotnet/arcade/blob/main/Documentation/Validation.md

[dougbu]

/backport to release/6.0

[github-actions]

Started backporting to release/6.0: https://github.com/dotnet/arcade/actions/runs/10776745637

[premun]

@simonrozsival we're seeing strange errors when trying to use the windows.11 (new) queue with the Android phones.

I tried googling for the error it gave: INSTALL_FAILED_VERIFICATION_FAILURE

And found this:
⁠https://stackoverflow.com/questions/15014519/apk-installation-failed-install-failed-verification-failure

Do you know if this is something we should do?
Are you using Windows 11 in runtime already?

[premun]

Ohhh, I also see this:

@ilyas1974 these Android phones were attached to new PCs when they moved, right? So maybe we need to enable USB debugging or trust the machines from the phones?

[dougbu]

the odd thing here (unless we've been merging arcade release/8.0 changes on red) is this PR shouldn't actually be changing anything. we've all been testing on windows.11.amd64.android.open for quite a while. windows.10.amd64.android.open has been forwarded to that queue since mid-February

[premun]

the odd thing here (unless we've been merging arcade release/8.0 changes on red) is this PR shouldn't actually be changing anything. we've all been testing on windows.11.amd64.android.open for quite a while. windows.10.amd64.android.open has been forwarded to that queue since mid-February

I see. Then maybe it's something about the test app we're sending there in the PR build (it's probably years old and somewhere in the netcorenativeassets storage account)

[dougbu]

I see. Then maybe it's something about the test app we're sending there in the PR build (it's probably years old and somewhere in the netcorenativeassets storage account)

I looked a bit deeper. Seems the XHarnessChangeDetection build step isn't working consistently across branches. Or the refactoring of the PR YAML when creating the Test_XHarness stage fixed something. I don't see the XHarness tests running in https://dev.azure.com/dnceng-public/public/_build/results?buildId=802216&view=results (the validation build for #15064) for example.

[dougbu]

I looked a bit deeper. Seems the XHarnessChangeDetection build step isn't working consistently across branches. Or the refactoring of the PR YAML when creating the Test_XHarness stage fixed something. I don't see the XHarness tests running in https://dev.azure.com/dnceng-public/public/_build/results?buildId=802216&view=results (the validation build for #15064) for example.

Put another way, this could be a consistent problem with the queue that skipped tests hid

[premun]

It's possible the 6.0 branch differs a lot from 8.0 to be honest

[dougbu]

trying w/ a newer XHarness CLI

I noticed release/8.0 doesn't have a subscription from dotnet/xharness. that said, the sub for release/6.0 has Update Frequency: None and there's also no sub for release/9.0. right now there's only an active (weekly) sub for 'main' from dotnet/xharness. should there be active (say, weekly) subscriptions for all arcade branches❓

[premun]

I noticed some time ago that 6.0/8.0 Arcade was not actually even going to arcade-validation so I guess we're careful about retriggering all 6.0 branches of all repos? (I don't know myself)

Only we found out during servicing that required updates from Arcade never made it anywhere (like some token removal).

So maybe we try to keep 6.0/8.0 branches low-traffic?

XHarness has a problem of depending on Arcade while Arcade depends on it so keeping it alive will be an endless loop of non-updates going there and back. I think it's probably fine to keep it off.

[dougbu]

I think it's probably fine to keep it off.

alright. /fyi I missed a sub from xharness into release/8.0 but it's set up the same as for release/6.0 — Update Frequency: None. if we trigger those subs every so often and (perhaps) create a similar one for release/9.0, we should be up-to-date enough. the XHarness CLI version in this branch was particularly old…

[dougbu]

no change w/ newer XHarness CLI version 😦 any other suggestions @simonrozsival or @premun❓

[premun]

@dougbu I tried the commands from the stackoverflow thread and they seem to work. But let's re-run the tests at least once or twice to see if we just didn't get lucky with some machines.

[premun]

Oops, there was an auto-complete