Build only: TestProvisioner has become very flaky #429

nedtwigg · 2019-08-09T07:26:12Z

PR feat(FeatureClassLoader): support class loading of Java 9+ #426 passed CI, but master failed after it was merged

:testlib:test
com.diffplug.spotless.scala.ScalaFmtStepTest > equality FAILED
    java.lang.AssertionError at ScalaFmtStepTest.java:83
        Caused by: org.gradle.api.artifacts.ResolveException at ScalaFmtStepTest.java:83
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException at ScalaFmtStepTest.java:83
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at ScalaFmtStepTest.java:83
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at ScalaFmtStepTest.java:83
                        Caused by: org.gradle.api.resources.ResourceException at ScalaFmtStepTest.java:83
                            Caused by: org.gradle.internal.resource.transport.http.HttpRequestException at ScalaFmtStepTest.java:83
                                Caused by: org.gradle.internal.impldep.org.apache.http.conn.ConnectTimeoutException at ScalaFmtStepTest.java:83
                                    Caused by: java.net.SocketTimeoutException at ScalaFmtStepTest.java:83
com.diffplug.spotless.markdown.FreshMarkStepTest > equality FAILED
    java.lang.AssertionError at FreshMarkStepTest.java:62
        Caused by: org.gradle.api.artifacts.ResolveException at FreshMarkStepTest.java:62
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException at FreshMarkStepTest.java:62
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at FreshMarkStepTest.java:62
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at FreshMarkStepTest.java:62
com.diffplug.spotless.markdown.FreshMarkStepTest > behavior FAILED
    org.gradle.api.artifacts.ResolveException at FreshMarkStepTest.java:35
        Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException at FreshMarkStepTest.java:35
            Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at FreshMarkStepTest.java:35
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException at FreshMarkStepTest.java:35
78 tests completed, 3 failed
:testlib:test FAILED
FAILURE: Build failed with an exception.

PR Modernize build #428 also passed CI, but master continued to fail after it was merged too

> Task :lib-extra:test
com.diffplug.spotless.extra.java.EclipseJdtFormatterStepTest > testSupportedVersions FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException
com.diffplug.spotless.extra.cpp.EclipseCdtFormatterStepTest > testSupportedVersions FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException
com.diffplug.spotless.extra.wtp.EclipseWtpFormatterStepTest > testSupportedVersions[CSS] FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException
com.diffplug.spotless.extra.wtp.EclipseWtpFormatterStepTest > testSupportedVersions[HTML] FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException
com.diffplug.spotless.extra.wtp.EclipseWtpFormatterStepTest > testSupportedVersions[JS] FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException
com.diffplug.spotless.extra.groovy.GrEclipseFormatterStepTest > testSupportedVersions FAILED
    java.lang.AssertionError
        Caused by: org.gradle.api.artifacts.ResolveException
            Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                        Caused by: org.gradle.api.resources.ResourceException
                            Caused by: org.gradle.internal.resource.transport.http.HttpErrorStatusCodeException

Since the PR's passed before they failed, it seems to be a flaky sort of failure - the very best kind!!

On my box, I get a different failure:

> Task :testlib:test
com.diffplug.spotless.kotlin.KtLintStepTest > worksShyiko FAILED
    org.gradle.api.artifacts.ResolveException at KtLintStepTest.java:47
        Caused by: org.gradle.api.internal.artifacts.ivyservice.DefaultLenientConfiguration$ArtifactResolveException at KtLintStepTest.java:47
            Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                    Caused by: org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.MetaDataParseException
                        Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                            Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                                Caused by: org.gradle.api.internal.artifacts.ivyservice.ivyresolve.parser.MetaDataParseException
                                    Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                                        Caused by: org.gradle.internal.resolve.ModuleVersionResolveException
                                            Caused by: org.gradle.internal.resource.local.FileStoreException
                                                Caused by: org.gradle.api.UncheckedIOException
                                                    Caused by: java.nio.file.NoSuchFileException

78 tests completed, 1 failed

> Task :testlib:test FAILED

FAILURE: Build failed with an exception.

After I removed the travis cache, the build passed (or at least revealed an easy-to-fix bug in #426). I think it must be that TestProvisioner is somehow gunking up gradle's artifact resolution. On my machine, a git clean -d -X -f did not fix the problem. Neither did nuking ~/.gradle/caches...

We've got a man on the inside now ;-) . Hey @JLLeitschuh, we want to turn a list of maven coordinates into files on our harddrive inside of test code. Do you know of a better gradle API to accomplish this?

spotless/testlib/src/main/java/com/diffplug/spotless/TestProvisioner.java

Lines 53 to 68 in babc574

    
           Project project = ProjectBuilder.builder().build(); 
        
           repoConfig.accept(project.getRepositories()); 
        
           return (withTransitives, mavenCoords) -> { 
        
           	Dependency[] deps = mavenCoords.stream() 
        
           			.map(project.getDependencies()::create) 
        
           			.toArray(Dependency[]::new); 
        
           	Configuration config = project.getConfigurations().detachedConfiguration(deps); 
        
           	config.setTransitive(withTransitives); 
        
           	config.setDescription(mavenCoords.toString()); 
        
           	try { 
        
           		return config.resolve(); 
        
           	} catch (ResolveException e) { 
        
           		/* Provide Maven coordinates in exception message instead of static string 'detachedConfiguration' */ 
        
           		throw new ResolveException(config.getDescription(), e); 
        
           	} 
        
           };

The text was updated successfully, but these errors were encountered:

nedtwigg · 2019-08-19T02:39:08Z

Since updating the gradle version, this has become a frequent problem. We should disable travis caching until we can resolve this.

nedtwigg · 2019-09-23T23:34:38Z

Build caching is turned back on. I'm not super confident that it is really fixed, but my box had problems, and those problems are now fixed, so I'm closing this for now.

nedtwigg · 2019-12-30T20:12:14Z

It's now failing close to 100% of the time, but only for KtLintStepTest. Interestingly, that is the only test that uses jcenter(), all of the rest use mavenCentral(). I removed this for now in #499. There is a gradle bug for resolving this here: gradle/gradle#11752

… be flaky and we can close-out #429.

nedtwigg · 2021-12-05T00:32:15Z

Finally fixed! Huzzah!

…ed (#429).

nedtwigg added the bug label Aug 9, 2019

nedtwigg mentioned this issue Aug 9, 2019

Feature/build cleanup #430

Merged

nedtwigg changed the title ~~TestProvisioner has become very flaky~~ Build only: TestProvisioner has become very flaky Aug 9, 2019

This was referenced Aug 19, 2019

Try fixing #427 #434

Merged

Update to Gradle 5.6 #433

Merged

nedtwigg added a commit to ZacSweers/spotless that referenced this issue Aug 19, 2019

Disable some caching as a workaround for diffplug#429

229b38c

nedtwigg mentioned this issue Aug 20, 2019

KtLintStepTest fails on windows (10) #299

Closed

nedtwigg mentioned this issue Sep 23, 2019

Feature/improve test speed #456

Merged

nedtwigg closed this as completed Sep 23, 2019

nedtwigg reopened this Dec 30, 2019

nedtwigg mentioned this issue Mar 11, 2021

Non-reproducible behavior around dependency verification with Spotless plugin gradle/gradle#16500

Closed

nedtwigg added a commit that referenced this issue Dec 4, 2021

Now that we're all mavenCentral and no jcenter, maybe tests won't…

383b827

… be flaky and we can close-out #429.

nedtwigg closed this as completed Dec 5, 2021

nedtwigg added a commit that referenced this issue Dec 5, 2021

shyiko is flaky (#429)

ac073a3

jbduncan mentioned this issue Dec 6, 2021

Fix conflicts between Spotless and Android re: clean task #1014

Merged

nedtwigg added a commit that referenced this issue Dec 6, 2021

Crank up the test retrying so that all of KtLintStepTest can be enabl…

142bf66

…ed (#429).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build only: TestProvisioner has become very flaky #429

Build only: TestProvisioner has become very flaky #429

nedtwigg commented Aug 9, 2019

nedtwigg commented Aug 19, 2019

nedtwigg commented Sep 23, 2019

nedtwigg commented Dec 30, 2019

nedtwigg commented Dec 5, 2021

Build only: TestProvisioner has become very flaky #429

Build only: TestProvisioner has become very flaky #429

Comments

nedtwigg commented Aug 9, 2019

nedtwigg commented Aug 19, 2019

nedtwigg commented Sep 23, 2019

nedtwigg commented Dec 30, 2019

nedtwigg commented Dec 5, 2021