Repeated execution of tests for different language versions

[bjornhellander]

A question about the test class inheritance that goes on between the version specific test projects:
I might be missing something, but it looks like most earlier tests are executed for later language versions as well, but without any change in the test setup. (I see that some tests actually change language version in test sub classes, but it does not seem to be very common.)

Two strategies seem reasonable two me:

1. Executing tests only for the minimum required language version, i.e. the version where the the tested feature was introduced.
2. Executing all tests for all possible language versions.

What I think is happening now is that tests are re-executed in test projects for higher language versions, but still configured with the same language version as in the test project it was inherited from. So they are executed multiple times with exactly the same setup.

Is this intentional?
Or am I just dead wrong? 😁

Asking since its starting to take quite a while to run the tests.

[sharwell]

Is this intentional?

Yes, it helps make sure the behavior is consistent across different versions of Roslyn (sometimes behavior changes in Roslyn require workarounds to handle both situations).

[sharwell]

What I think is happening now is that tests are re-executed in test projects for higher language versions, but still configured with the same language version as in the test project it was inherited from.

Not entirely sure on this one. Would need to check, but I think the default comes from the test project being run, and not the project a test is originally defined in. There are exceptions in cases where an individual test explicitly specifies a language version.

[bjornhellander]

After looking more at this, you are of course correct. Since the test projects for later c# versions reference later Roslyn versions,
they will override whatever Roslyn version was referenced from earlier test projects when inheriting test cases. Each Roslyn version has a default language version, which is the one that will be used in all test cases which do not explicitly say otherwise,
so the expected version will typically be used. Good.

But there are quite a few tests that do set the language version explicitly, for example most tests in the c# 8, 9 and 10 test projects. Should these be changed, so we get the behaviour that you described? That test projects for later language versions inherit earlier test cases, but run them with there own language version instead. I have started removing explicit language versions and have not seen any problems so far.