Reevaluate the impact of requiring metadata entries for each class member query

[vjovanov]

TL;DR

The current reflection in Native Image requires metadata entries for each query on java.lang.Class instances. For example, calling clazz.getDeclaredMethods() will require a JSON entry "allDeclaredMethods": true" for that class. This fine-grained approach can be cumbersome as it yields many entries in reflection metadata.

We will investigate the impact of including metadata of all java.lang.Class members for classes registered for reflection. This would include names and signatures of all methods, properties of all fields, and all annotations.

The impact will be evaluated on, Spring Petclinic, Micronaut Shopcart, and a few prominent metadata-repository projects. The following metrics will be taken into account: Max RSS, Startup time, Image Size, and the number of required metadata entries.

Goals

1. Evaluate if we can reduce the amount of required metadata without noticeably increasing the image size.
2. Improve statistics collection for reflection metadata.

Non-goals

1. Change reflection metadata to bloat the produced native images.
2. Change JNI, Serialization, or dynamic proxies.
3. Include bodies of all methods into the reflection metadata.

[loicottet]

Impact of adding all classes, methods and constructors (no fields) from registered classes as queried (not invoked):

Micronaut shopcart:

• Image size: +1.5-2% (avg 1.7%)
• Startup: +0-5% (avg 2.4%, all within the margin of error)
• Max RSS ±2% (avg. 0.3%)

Spring petclinic:

• Image size: +1-1.5% (average 1.1%)
• App startup: +5-18% (average 9.6%)
• Max RSS: ±1.5% (average +0.5%)

The startup time may have been affected by some methods added as reachable to enable running the benchmark with the additional configuration.

Evaluation of the impact on configuration file entries will follow soon. Evaluating the impact of including all elements as invoked/accessed will take more time, if it is at all possible, since the analysis and program behaviour will be affected in more drastic ways.

[loicottet]

Impact on configuration file size (1 entry = 1 class, field, method, or "queryAllMethods"-style field):

Micronaut shopcart:

• 46% reduction in configuration file entries
• 59% potential reduction in configuration file entries when including all default constructors in the image (image size impact not evaluated yet)

Spring petclinic:

• 27% reduction in configuration file entries
• 45% potential reduction in configuration file entries when including all default constructors in the image (image size impact not evaluated yet)

[zakkak]

Hi @loicottet, what's needed to get some Quarkus project(s) included in the evaluation?

I am willing to perform the runs and report the numbers if needed.

[vjovanov]

We can do that as well, @loicottet can you share the code when ready on a PR and notify us here?

The reason we did not evaluate Quarkus is simply because Quarkus uses almost no reflection. We have the tika benchmark in our set of benchmarks, so we should report that here as well.

[maxandersen]

I would expect micronaut and quarkus would be similar in amount of reflections ?

But okay now I see the comments about what it impacts then I better understand it and Afaics confirms my thinking. That the measurement here will be greatly affected by what libraries/extensions are involved in the testing.

If I grok it right - this new change will make it so where before you had to explicitly list defsult constructors and other parts graalvm would now default include it even though it might not be strictly required and thus cost extra in image size.

That means you will get quite different results dependent on what kind library's are involved.

The more default constructors present or just mere number of classes the bigger impact it will have.

Right?

Thus if looking at quarkus apps if you take something like pet clinic which dependencies include java driver, hibernate and a few other things you would find that in quarkus superheroes villain or quarkus todo app too (is my assumption)

But if you take a quarkus app that uses the Kubernetes client at runtime (not just buildtime) which has a huge model class api I would expect quite the different level of impact/measurements.

[vjovanov]

I would expect micronaut and quarkus would be similar in amount of reflections ?

We measured a long time ago and it was almost 0. We will confirm when the results come in.

Right?
Correct. We are first evaluating the impact on image size for: i) including only metadata of reflective classes (i.e., names and signatures), and ii) also including the default constructors (as it seems they make a large portion of the metadata).

We will likely expand the evaluation to the whole metadata repo to cover the different use cases you mention. Of course, if you have some interesting projects, we will gladly include them in the set of benchmarks.

[vjovanov]

The latest results follow: all-queried means all metadata (method signatures, field names, inner classes, etc.) are included when the type is in reflect-config.json. default-constructor means all-queried plus the body of the default constructor. all-constructors means all-queried plus bodies of all constructors, and all-methods means that the bodies of all methods are included.

All values are normalized to the current configuration. The Metadata Repository represents the geomean of all projects in the metadata repository.

The Quarkus numbers are not included as the benchmark was not executable in our CI at the time this experiment was executed. The Spring Boot benchmark for all-methods does not build as it pulls in unsupported native libraries from the JDK.