"coreclr Pri0 Runtime Tests Run windows arm64 check" timeouts

[EgorBo]

This JOB timeouts in various PRs, e.g.: #72141, #72168, #72084, #72167

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This JOB timeouts in various PRs, e.g.: #72141, #72168, #72084, #72167

Author:	EgorBo
Assignees:	-
Labels:	area-Infrastructure-coreclr, blocking-clean-ci, untriaged
Milestone:	-

[tlakollo]

All the timeouts seem to come from a machine in particular DDARM64-104 in the windows.10.arm64v8.open pool. Contacted first responders to get more information about it.

[MattGal]

The problems with DDARM64-104 are already resolved as of Thursday, but the issue it's hitting is dotnet/arcade#10013. We'll introduce a reboot when this is seen in the future to make it more resilient but the actual root cause (other than "ARM64 machine has slightly too much uptime") is not really known.

That said, the problem DDARM64-104 experienced, while irritating, can't cause timeouts. YOu can even see in Kusto there hasn't been a single work item timeout in the last 3000 work items it ran:

WorkItems
| where MachineName == "DDARM64-104"
| top 3000 by Finished desc 
| summarize count() by Status

as well as the longest any work item with this exit code running , in history, being about 8 minutes:

WorkItems
| where MachineName == "DDARM64-104"
| where ExitCode == -1073741502
| extend rantime = Finished-Started
| summarize avg(rantime), min(rantime), max(rantime)

So, if your jobs are having timeouts, pick a specific one and share with me its correlation id and we can investigate together.

[agocke]

Sounds like this has been resolved.