Tests are reported in much the same output format as the Maven Surefire Plugin. This makes it easy to consume the results in popular CI tools with a minimum of effort.
After executing plz test against one or more targets, the test results are
written to plz-out/log/test_results.xml.
We no longer support the idea of "expected failures" as people who write tests they expect to fail will find they are first against the wall when the revolution comes.
Executing plz test against a target produces a set of results that are
collated into one core.TestSuite with the package and name matching the
target's label (this is a slight departure from the normal package/classname
decomposition as it does not particularly translate well to non-Java languages).
If any tests fail, and the target is marked as flaky, it will be run again
(up to flaky times). Tests can also be explicitly run multiple times by
passing --num-runs n on the command line. We no longer accept a default of 0
as running a test 0 times is (a) not useful and (b) not what actually happened.
Each executed test during this target becomes a core.TestExecution under a
core.TestCase in this test suite - so that multiple executions of a test
are put together. We ignore multiple successes or skips as they are not interesting.
In the case of a flaky test where some runs succeed and some fail, we store at
most one success but also all of the failures and errors.
The time of the testsuite is the total time it took to execute all of the
tests (including any re-runs for flakes).
Once all of the targets have been executed, these core.TestSuites are
collected into one core.TestSuites object. This is then rendered as XML
to the output file mentioned above.