Add configuration option for 'versionsProvidingConfiguration' (#142)

This way, a build setup can be created with a Configuration available
in ALL projects that resolves to the same version of every module
used. Then the metadata input to the transform, to process
'requireAllDefinedDependencies', is the same for a certain Jar
independent of the classpath it is used on.

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Extra Java Module Info Gradle Plugin - Changelog
 
+## Version 1.9
+* [New] [#137](https://github.com/gradlex-org/extra-java-module-info/pull/137) - Configuration option for 'versionsProvidingConfiguration'
+* [New] [#130](https://github.com/gradlex-org/extra-java-module-info/pull/130) - Support classifier in coordinates notation
+* [New] [#138](https://github.com/gradlex-org/extra-java-module-info/pull/138) - 'javaModulesMergeJars' extends 'internal' if available
+* [Fixed] [#129](https://github.com/gradlex-org/extra-java-module-info/pull/129) - Find Jar for coordinates when version in Jar nam differs
+* [Fixed] [#100](https://github.com/gradlex-org/extra-java-module-info/pull/100) - Fix error message about automatic module name mismatch
+
 ## Version 1.8
 * [New] [#99](https://github.com/gradlex-org/extra-java-module-info/issues/99) - Default behavior for 'module(id, name)' notation without configuration block
 * [New] - Use custom mappings from 'java-module-dependencies' for 'known modules' (if available)

README.md

@@ -199,6 +199,64 @@ extraJavaModuleInfo {
 }
 ```
 
+## How can I avoid that the same Jar is transformed multiple times when using requireAllDefinedDependencies?
+
+When using the `requireAllDefinedDependencies` option, all metadata of the dependencies on your classpath is input to the Jar transformation.
+In a multi-project however, each subproject typically has different classpaths and not all metadata is available everywhere.
+This leads to a situation, where Gradle's transformation system does not know if transforming the same Jar will lead to the same result.
+Then, the same Jar is transformed many times. This is not necessary a problem, as the results of the transforms are cached
+and do not run on every build invocation. However, the effect of this is still visible:
+for example when you import the project in IntelliJ IDEA.
+You see the same dependency many times in the _External Libraries_ list and IntelliJ is doing additional indexing work.
+
+To circumvent this, you need to construct a common classpath – as a _resolvable configuration_ – that the transform can use.
+This needs to be done in all subprojects. You use the `versionsProvidingConfiguration` to tell the plugin about the commons classpath.
+
+```
+extraJavaModuleInfo {
+    versionsProvidingConfiguration = "mainRuntimeClasspath"
+}
+```
+
+To create such a common classpath, some setup work is needed.
+And it depends on your overall project structure if and how to do that.
+Here is an example setup you may use:
+
+```
+val consistentResolutionAttribute = Attribute.of("consistent-resolution", String::class.java)
+
+// Define an Outgoing Variant (aka Consumable Configuration) that knows about all dependencies
+configurations.create("allDependencies") {
+    isCanBeConsumed = true
+    isCanBeResolved = false
+    sourceSets.all {
+        extendsFrom(
+            configurations[this.implementationConfigurationName],
+            configurations[this.compileOnlyConfigurationName],
+            configurations[this.runtimeOnlyConfigurationName],
+            configurations[this.annotationProcessorConfigurationName]
+        )
+    }
+    attributes { attribute(consistentResolutionAttribute, "global") }
+}
+
+// Define a "global claspath" (as Resolvable Configuration)
+val mainRuntimeClasspath = configurations.create("mainRuntimeClasspath") {
+    isCanBeConsumed = false
+    isCanBeResolved = true
+    attributes.attribute(consistentResolutionAttribute, "global")
+}
+
+// Add a dependency to the 'main' project(s) (:app ins this example) that transitively 
+// depend on all subprojects to create a depenedency graph wih "everything"
+dependencies { mainRuntimeClasspath(project(":app")) }
+
+// Use the global classpath for consisten resolution (optional)
+configurations.runtimeClasspath {
+    shouldResolveConsistentlyWith(mainRuntimeClasspath)
+}
+```
+
 ## I have many automatic modules in my project. How can I convert them into proper modules and control what they export or require?
 
 The plugin provides a set of `<sourceSet>moduleDescriptorRecommendations` tasks that generate the real module declarations utilizing [jdeps](https://docs.oracle.com/en/java/javase/11/tools/jdeps.html) and dependency metadata.

src/main/java/org/gradlex/javamodule/moduleinfo/ExtraJavaModuleInfoPlugin.java

@@ -53,7 +53,6 @@
 import java.util.Properties;
 import java.util.Set;
 import java.util.stream.Collectors;
-import java.util.stream.Stream;
 
 import static org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE;
 import static org.gradle.api.attributes.Category.LIBRARY;
@@ -202,16 +201,9 @@ private void registerTransform(String fileExtension, Project project, ExtraJavaM
                 p.getMergeJarIds().set(artifacts.map(new IdExtractor()));
                 p.getMergeJars().set(artifacts.map(new FileExtractor(project.getLayout())));
 
-                p.getRequiresFromMetadata().set(project.provider(() -> sourceSets.stream().flatMap(s -> Stream.of(
-                                        s.getRuntimeClasspathConfigurationName(),
-                                        s.getCompileClasspathConfigurationName(),
-                                        s.getAnnotationProcessorConfigurationName()
-                                ))
-                                .flatMap(resolvable -> existingComponentsOfInterest(configurations.getByName(resolvable), extension))
-                                .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, (k1, k2) -> k1)).entrySet().stream()
-                                .collect(Collectors.toMap(Map.Entry::getKey, c -> new PublishedMetadata(c.getKey(), c.getValue(), project)))
-                ));
-
+                Provider<Set<String>> componentsOfInterest = componentsOfInterest(extension);
+                p.getRequiresFromMetadata().set(componentsOfInterest.map(gaSet -> gaSet.stream()
+                        .collect(Collectors.toMap(ga -> ga, ga -> new PublishedMetadata(ga, project, extension)))));
                 p.getAdditionalKnownModules().set(extractFromModuleDependenciesPlugin(project));
             });
             t.getFrom().attribute(artifactType, fileExtension).attribute(javaModule, false);
@@ -245,26 +237,17 @@ private Provider<Map<String, String>> extractFromModuleDependenciesPlugin(Projec
         });
     }
 
-    private Stream<Map.Entry<String, Configuration>> existingComponentsOfInterest(Configuration resolvable, ExtraJavaModuleInfoPluginExtension extension) {
-        Set<String> componentsOfInterest = componentsOfInterest(extension);
-        if (componentsOfInterest.isEmpty()) {
-            return Stream.empty();
-        }
-
-        return resolvable.getIncoming().getResolutionResult().getAllComponents().stream()
-                .filter(c -> componentsOfInterest.contains(ga(c.getId())))
-                .collect(Collectors.toMap(c -> c.getId().toString(), c -> resolvable)).entrySet().stream();
-    }
-
-    private static Set<String> componentsOfInterest(ExtraJavaModuleInfoPluginExtension extension) {
-        return extension.getModuleSpecs().get().values().stream()
+    private static Provider<Set<String>> componentsOfInterest(ExtraJavaModuleInfoPluginExtension extension) {
+        return extension.getModuleSpecs().map(specs -> specs.values().stream()
                 .filter(ExtraJavaModuleInfoPlugin::needsDependencies)
                 .map(ModuleSpec::getIdentifier)
-                .collect(Collectors.toSet());
+                .collect(Collectors.toSet()));
     }
 
     private static boolean needsDependencies(ModuleSpec moduleSpec) {
-        return moduleSpec instanceof ModuleInfo && ((ModuleInfo) moduleSpec).requireAllDefinedDependencies;
+        return moduleSpec instanceof ModuleInfo
+                && ((ModuleInfo) moduleSpec).requireAllDefinedDependencies
+                && IdValidator.isCoordinates(moduleSpec.getIdentifier());
     }
 
     static String ga(ComponentIdentifier id) {

src/main/java/org/gradlex/javamodule/moduleinfo/ExtraJavaModuleInfoPluginExtension.java

@@ -41,6 +41,7 @@ public abstract class ExtraJavaModuleInfoPluginExtension {
     public abstract Property<Boolean> getFailOnMissingModuleInfo();
     public abstract Property<Boolean> getFailOnAutomaticModules();
     public abstract Property<Boolean> getDeriveAutomaticModuleNamesFromFileNames();
+    public abstract Property<String> getVersionsProvidingConfiguration();
 
     /**
      * Add full module information for a given Jar file.

src/main/java/org/gradlex/javamodule/moduleinfo/ExtraJavaModuleInfoTransform.java

@@ -380,14 +380,19 @@ private byte[] addModuleInfo(ModuleInfo moduleInfo, Map<String, List<String>> pr
         moduleVisitor.visitRequire("java.base", 0, null);
 
         if (moduleInfo.requireAllDefinedDependencies) {
-            String fullIdentifier = moduleInfo.getIdentifier() + ":" + version;
-            PublishedMetadata requires = getParameters().getRequiresFromMetadata().get().get(fullIdentifier);
+            String identifier = moduleInfo.getIdentifier();
+            PublishedMetadata requires = getParameters().getRequiresFromMetadata().get().get(identifier);
 
             if (requires == null) {
                 throw new RuntimeException("[requires directives from metadata] " +
                         "Cannot find dependencies for '" + moduleInfo.getModuleName() + "'. " +
                         "Are '" + moduleInfo.getIdentifier() + "' the correct component coordinates?");
             }
+            if (requires.getErrorMessage() != null) {
+                throw new RuntimeException("[requires directives from metadata] " +
+                        "Cannot read metadata for '" + moduleInfo.getModuleName() + "': " +
+                        requires.getErrorMessage());
+            }
 
             for (String ga : requires.getRequires()) {
                 String depModuleName = gaToModuleName(ga);

src/main/java/org/gradlex/javamodule/moduleinfo/IdValidator.java

@@ -25,4 +25,8 @@ static void validateIdentifier(String identifier) {
             throw new RuntimeException("'" + identifier + "' are not valid coordinates (group:name) / is not a valid file name (name-1.0.jar)");
         }
     }
+
+    static boolean isCoordinates(String identifier) {
+        return identifier.matches(COORDINATES_PATTERN);
+    }
 }

src/main/java/org/gradlex/javamodule/moduleinfo/PublishedMetadata.java

@@ -18,36 +18,49 @@
 
 import org.gradle.api.Project;
 import org.gradle.api.artifacts.Configuration;
+import org.gradle.api.artifacts.ConfigurationContainer;
 import org.gradle.api.artifacts.result.DependencyResult;
-import org.gradle.api.artifacts.result.ResolvedComponentResult;
 import org.gradle.api.artifacts.result.ResolvedDependencyResult;
+import org.gradle.api.artifacts.result.UnresolvedDependencyResult;
 import org.gradle.api.attributes.Attribute;
+import org.gradle.api.attributes.Bundling;
 import org.gradle.api.attributes.Category;
+import org.gradle.api.attributes.LibraryElements;
 import org.gradle.api.attributes.Usage;
+import org.gradle.api.attributes.java.TargetJvmEnvironment;
+import org.gradle.api.model.ObjectFactory;
+import org.gradle.api.provider.Provider;
+import org.gradle.api.tasks.SourceSet;
+import org.gradle.api.tasks.SourceSetContainer;
+import org.gradle.util.GradleVersion;
 
 import java.io.Serializable;
 import java.util.ArrayList;
 import java.util.List;
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
+import static java.util.Collections.emptyList;
 import static java.util.Objects.requireNonNull;
 import static org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE;
 import static org.gradle.api.attributes.Category.LIBRARY;
 import static org.gradle.api.attributes.Usage.USAGE_ATTRIBUTE;
 
 public class PublishedMetadata implements Serializable {
     private static final Attribute<String> CATEGORY_ATTRIBUTE_UNTYPED = Attribute.of(CATEGORY_ATTRIBUTE.getName(), String.class);
+    private static final String DEFAULT_VERSION_SOURCE_CONFIGURATION = "definedDependenciesVersions";
 
     private final String gav;
     private final List<String> requires = new ArrayList<>();
     private final List<String> requiresTransitive = new ArrayList<>();
     private final List<String> requiresStaticTransitive = new ArrayList<>();
+    private String errorMessage = null;
 
-    PublishedMetadata(String gav, Configuration origin, Project project) {
+    PublishedMetadata(String gav, Project project, ExtraJavaModuleInfoPluginExtension extension) {
         this.gav = gav;
-        List<String> compileDependencies = componentVariant(origin, project, Usage.JAVA_API);
-        List<String> runtimeDependencies = componentVariant(origin, project, Usage.JAVA_RUNTIME);
+
+        List<String> compileDependencies = componentVariant(extension.getVersionsProvidingConfiguration(), project, Usage.JAVA_API);
+        List<String> runtimeDependencies = componentVariant(extension.getVersionsProvidingConfiguration(), project, Usage.JAVA_RUNTIME);
 
         Stream.concat(compileDependencies.stream(), runtimeDependencies.stream()).distinct().forEach(ga -> {
             if (compileDependencies.contains(ga) && runtimeDependencies.contains(ga)) {
@@ -60,26 +73,73 @@ public class PublishedMetadata implements Serializable {
         });
     }
 
-    private List<String> componentVariant(Configuration origin, Project project, String usage) {
+    private List<String> componentVariant(Provider<String> versionsProvidingConfiguration, Project project, String usage) {
+        Configuration versionsSource;
+        if (versionsProvidingConfiguration.isPresent()) {
+            versionsSource = project.getConfigurations().getByName(versionsProvidingConfiguration.get());
+        } else {
+            // version provider is not configured, create on adhoc based on ALL classpaths of the project
+            versionsSource = maybeCreateDefaultVersionSourcConfiguration(project.getConfigurations(), project.getObjects(),
+                    project.getExtensions().findByType(SourceSetContainer.class));
+        }
+
         Configuration singleComponentVariantResolver = project.getConfigurations().detachedConfiguration(project.getDependencies().create(gav));
         singleComponentVariantResolver.setCanBeConsumed(false);
-        singleComponentVariantResolver.shouldResolveConsistentlyWith(origin);
-        origin.getAttributes().keySet().forEach(a -> {
+        singleComponentVariantResolver.shouldResolveConsistentlyWith(versionsSource);
+        versionsSource.getAttributes().keySet().forEach(a -> {
             @SuppressWarnings("rawtypes") Attribute untypedAttributeKey = a;
             //noinspection unchecked
-            singleComponentVariantResolver.getAttributes().attribute(untypedAttributeKey, requireNonNull(origin.getAttributes().getAttribute(a)));
+            singleComponentVariantResolver.getAttributes().attribute(untypedAttributeKey, requireNonNull(versionsSource.getAttributes().getAttribute(a)));
         });
         singleComponentVariantResolver.getAttributes().attribute(USAGE_ATTRIBUTE, project.getObjects().named(Usage.class, usage));
-        return firstAndOnlyComponent(singleComponentVariantResolver).getDependencies().stream()
-                .filter(PublishedMetadata::filterComponentDependencies)
-                .map(PublishedMetadata::ga)
-                .collect(Collectors.toList());
+        return firstAndOnlyComponentDependencies(singleComponentVariantResolver);
+    }
+
+    private Configuration maybeCreateDefaultVersionSourcConfiguration(ConfigurationContainer configurations, ObjectFactory objects, SourceSetContainer sourceSets) {
+        String name = DEFAULT_VERSION_SOURCE_CONFIGURATION;
+        Configuration existing = configurations.findByName(name);
+        if (existing != null) {
+            return existing;
+        }
+
+        return configurations.create(name, c -> {
+            c.setCanBeResolved(true);
+            c.setCanBeConsumed(false);
+            c.getAttributes().attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage.class, Usage.JAVA_RUNTIME));
+            c.getAttributes().attribute(Category.CATEGORY_ATTRIBUTE, objects.named(Category.class, Category.LIBRARY));
+            c.getAttributes().attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, objects.named(LibraryElements.class, LibraryElements.JAR));
+            c.getAttributes().attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling.class, Bundling.EXTERNAL));
+            if (GradleVersion.current().compareTo(GradleVersion.version("7.0")) >= 0) {
+                c.getAttributes().attribute(TargetJvmEnvironment.TARGET_JVM_ENVIRONMENT_ATTRIBUTE,
+                        objects.named(TargetJvmEnvironment.class, TargetJvmEnvironment.STANDARD_JVM));
+            }
+
+            if (sourceSets != null) {
+                for (SourceSet sourceSet : sourceSets) {
+                    Configuration implementation = configurations.getByName(sourceSet.getImplementationConfigurationName());
+                    Configuration compileOnly = configurations.getByName(sourceSet.getCompileOnlyConfigurationName());
+                    Configuration runtimeOnly = configurations.getByName(sourceSet.getRuntimeOnlyConfigurationName());
+                    Configuration annotationProcessor = configurations.getByName(sourceSet.getAnnotationProcessorConfigurationName());
+                    c.extendsFrom(implementation, compileOnly, runtimeOnly, annotationProcessor);
+                }
+            }
+        });
     }
 
-    private ResolvedComponentResult firstAndOnlyComponent(Configuration singleComponentVariantResolver) {
-        ResolvedDependencyResult onlyResult = (ResolvedDependencyResult) singleComponentVariantResolver.getIncoming().getResolutionResult()
-                .getRoot().getDependencies().iterator().next();
-        return onlyResult.getSelected();
+    private List<String> firstAndOnlyComponentDependencies(Configuration singleComponentVariantResolver) {
+        DependencyResult result = singleComponentVariantResolver
+                .getIncoming().getResolutionResult().getRoot()
+                .getDependencies().iterator().next();
+
+        if (result instanceof UnresolvedDependencyResult) {
+            errorMessage = ((UnresolvedDependencyResult) result).getFailure().getMessage();
+            return emptyList();
+        } else {
+            return ((ResolvedDependencyResult) result).getSelected().getDependencies().stream()
+                    .filter(PublishedMetadata::filterComponentDependencies)
+                    .map(PublishedMetadata::ga)
+                    .collect(Collectors.toList());
+        }
     }
 
     private static boolean filterComponentDependencies(DependencyResult d) {
@@ -113,4 +173,8 @@ public List<String> getRequiresTransitive() {
     public List<String> getRequiresStaticTransitive() {
         return requiresStaticTransitive;
     }
+
+    public String getErrorMessage() {
+        return errorMessage;
+    }
 }