Support polymorphic subclass serializers with inferred type arguments by refactoring Serializers to bind to KTypes instead of / in addition to KClasss

[ShreckYe]

What is your use-case and why do you need this feature?

The Kotlin compiler can infer some type arguments of a subtype from those of the parent abstract/sealed types. Currently, polymorphic serializers can not make use of such inferred type arguments, either automatically for a sealed type or by registering a polymorphic serializer manually for a concrete parent type and passing the serializer for the concrete subtype with the inferred type arguments. Type parameter information is erased at runtime and one has to register a polymorphic serializer for the inferrable type's upper bound as instructed in the guide. Having such types is a common feature called GADT (generic algebraic data type) in functional programming and is becoming more popular with type parameter support in various languages, so it makes sense to support this with serialization. After searching in issues I see that there are many issues related to polymorphic serialization can be resolved by supporting this feature, such as #944, #1252, and #1784.

Describe the solution you'd like

By reading and trying to refactor part of the implementation code I realized that a major obstacle is that in the current implementation Serializers bind to KClasss and the generated code by the compiler plugin also passes KClass arguments. To support such a feature, the Serializers, or at least the polymorphic Serializers need to also bind to concrete KTypes, and a serializer for a concrete type with specific type arguments should take precedence over a serializer for a class / a type with star-projections.

I tried implementing this myself and I realized that there is a huge amount of code to refactor, not only in this repo but also in the Kotlin compiler repo. Such a refactor to the core APIs may change the behavior of existing functionality and break existing tests if not implemented carefully and maintaining API compatibility might be a challenge. If this proposal is OK and my direction is right I can also try to complete the remaining necessary changes and submit a PR.

[sandwwraith]

This is a valid request, but it is connected with several complications. Let's take a look at the standard example:

@Serializable
abstract class Response<out T, out ERR>
            
@Serializable
class OkResponse<out T>(val data: T) : Response<T, Nothing>()

1. Registration of serializers

Currently, you have to write subclass(OkResponse::class, OkResponse.serializer(???)) during registration where ??? is likely PolymorphicSerializer(Any::class /* or other upper bound */). To correctly handle generics, we need to substitute ??? with actual T serializer, which is known only on runtime. Therefore, subclass should likely accept lambda to construct OkResponse serializer, just like this overload of contextual(): https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#contextual-serialization-and-generic-classes

2. Getting type arguments in runtime.

Unlike contextual, where we know type arguments in compile time, for open polymorphism we know only a base class. (e.g. Response. Surely, we can figure out its type arguments (in e.g. class Data(val r: Response<String, String>) and pass KType instead of KClass to PolymorphicSerializer. However, real problem is that we need to determine type arguments of a subclass here:

kotlinx.serialization/core/commonMain/src/kotlinx/serialization/modules/SerializersModule.kt

Line 153 in cd9f8b0

override fun <T : Any> getPolymorphic(baseClass: KClass<in T>, value: T): SerializationStrategy<T>? {

Here, we have only value of Response type (which is actually our OkResponse). We can't extract from value::class alone that it is OkResponse<String>, as this information is already erased. And there's no way to get it during compilation time either, as it is a runtime polymorphism.

3. Deserialization

It is roughly the same as 2, but in opposite direction. Usually, in json we'll get something like {"type": "com.app.OkResponse", "data": "abc123"}. We can get lambda constructor by com.app.OkResponse string (see point 1), but what type argument serializers should we pass? To solve this problem, we need to embed whole type information into json (e.g. by writing "type": "com.app.OkResponse<kotlin.String>"), which is not widely spread format, as far as I know. We can't even produce such a string because of 2).

To sum up, we do not have enough information to support this in compile time or runtime. I believe an easier way to solve this is to provide some kind of more sensible replacement to ??? than PolymorphicSerializer(Any::class), which would be able to handle most reasonable cases.

[pdvrieze]

To sum up, we do not have enough information to support this in compile time or runtime. I believe an easier way to solve this is to provide some kind of more sensible replacement to ??? than PolymorphicSerializer(Any::class), which would be able to handle most reasonable cases.

When you use a custom serializer for OkResponse it is generally not too hard to do this. There are different cases, but for serialization you could even store the actual member serializer in the response type. For deserialization you can use polymorphic or contextual lookup (or some other way you map responses to concrete types).

In many cases you also want to limit the algebraic data types to be present at the Kotlin side, not in the data representation side. This translation requires a custom serializer anyway (which can also handle polymorphic values where appropriate).

[ShreckYe]

@sandwwraith Thank you for your detailed explanations. I went through the code again and rethought some of the designs. For the complications you mentioned, I still have some questions and I can think of some possible solutions.

And there's no way to get it during compilation time either, as it is a runtime polymorphism.

This is where I don't quite understand. I don't know a general case where such a kind of polymorphism is used where it's only possible to get runtime class information. Usually, if I call encodeToString or decodeFromString I pass a reified T which contains full KType information.

As how I understand kotlinx.serialization works, the compiler plugin first generates companion serializer(...) functions which return KSerializer instances by invoking the corresponding implementation constructors for classes marked with @Serializable (for example, Response.serializer(???) returns a result of PolymorphicSerializer(...)). Such instances contain properties to (de)serialize in its descriptor. In the case that the class has type parameters, such a companion serializer(???) function takes serializer parameters for these type parameters (such as Response.serializer(???) and OkResponse.serializer(???)); when the SerializersModule.serializer<T>() extension function is called in encodeToString/decodeFromString, a KSerializer is constructed with these companion serializer(...) functions per the given KType from reified T; and finally in encodeToString/decodeFromString, an Encoder/Decoder such as StreamingJsonEncoder/StreamingJsonDecoder is constructed and its encodeSerializableValue/decodeSerializableValue function is called to serialize/deserialize each property recursively provided by KSerializer.descriptor.

So I think it is possible to refactor the constructors of PolymorphicSerializer and SealedClassSerializer to take KType arguments which contain full type information. Since SerializersModule.serializer<T>() calls the plugin-generated companion serializer(...)s, and they call the constructors of PolymorphicSerializer and SealedClassSerializer, it's possible to pass such KType information all along and pass the inferred types in the plugin-generated companion serializer(...)s. Such KType serializers can be cached and reused when SerializersModule.serializer<T>() is called again.

However, real problem is that we need to determine type arguments of a subclass here:

kotlinx.serialization/core/commonMain/src/kotlinx/serialization/modules/SerializersModule.kt

Line 153 in cd9f8b0

override fun <T : Any> getPolymorphic(baseClass: KClass<in T>, value: T): SerializationStrategy<T>? {

For example, this new signature can be added or replace the one you mentioned:

https://github.com/huanshankeji/kotlinx.serialization/blob/74ef87728e6bfac1502e63f389d55ef9d93f4d11/core/commonMain/src/kotlinx/serialization/modules/SerializersModule.kt#L173

Possible approaches to determine subtype type arguments

The key point lies in how to refactor a parameterized polymorphic root class's companion serializer(...) to support creating the subclass serializer(...)s with the correct type arguments. There are 3 possible approaches I originally thought of, one with inference to be implemented at runtime and invoked in the root classes' companion serializer(...)s, one with subtyping relation provided by the coder and checked by the compiler, and another with inference to be implemented in the compiler plugin and generating subclass serializer(...) invocations that pass serializers constructed with inferred type arguments.

Approach 1: invoke type inference in the root class's companion serializer(...)

In this way, the function's signature needs not to be changed. A set of type inference algorithms can be adapted here from the Kotlin repo and invoked in the generated functions. However, type inference might be expensive, so these serializers for the concrete classes might need to be cached/memoized in a hashmap for reuse.

For open polymorphism

To quote what you said:

Therefore, subclass should likely accept lambda to construct OkResponse serializer, just like this overload of contextual(): https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serializers.md#contextual-serialization-and-generic-classes

This is The prototypical API I implemented if we adopt this approach:

https://github.com/huanshankeji/kotlinx.serialization/blob/0f9f27420913831ea8a2ba92910ac49ff59506ea/core/commonMain/src/kotlinx/serialization/modules/PolymorphicModuleBuilder.kt#L39-L41

And it can be called like this (ignore the different function names):

polymorphic(Response::class) {
    subclass { args -> OkResponse.serializer(args[0]) }
}

For closed polymorphism

Closed polymorphism works similarly. IMO for sealed classes, it becomes more important to cache concrete type serializers so that performance is not tampered with.

About caching serializers

It shouldn't be too difficult since there is already some caching mechanism implemented in this library.

Approach 2 (problematic): let the user handle subtyping relations in the SerializersModule DSL and let the compiler check them

This is an approach I first thought of but later found to be problematic.

For open polymorphism

In this way, registering a serializer for a concrete root type (given all the type arguments) instead of just the class should be supported. In the SerializersModule DSL, the code should look like this:

https://github.com/huanshankeji/kotlinx.serialization/blob/74ef87728e6bfac1502e63f389d55ef9d93f4d11/formats/json-tests/commonTest/src/kotlinx/serialization/features/PolymorphicWithInferredTypeParameterTest.kt#L21-L23

Or it can be simplified more if possible:

polymorphic<Response<String, Nothing>> {
    subclass<OkResponse<String>>()
}

In this way, the user handles the subtyping relationship explicitly and the compiler checks that OkResponse<String> is a subtype of Response<String, Nothing>.

However, there is another problem that the user can register a subtype with a covariant strict subtype type argument. Like this:

polymorphic<Response<Number, Nothing>> {
    subclass<OkResponse<Int>>()
}

So this design brings some extra complications and seems unpractical currently.

For closed polymorphism

The user can do the same for closed polymorphism but it becomes quite cumbersome. So if the goal is to not specify explicitly in the SerializersModule DSL but to infer types, this approach won't work for sealed classes.

Approach 3 (problematic): add subclasses' serializer parameters in serializer(...) and implement type inference in the compiler plugin

This is another approach I first thought of but later found to be problematic.

To do this, the compiler plugin needs to know the concrete root types and their subtypes to generate serializers for.

For open polymorphism

This does not work for open polymorphism because the subtypes are added in the SerializersModule DSL at runtime.

For closed polymorphism

Assume Response is sealed. Because the compiler plugin can't generate serializers for all Response<T1, T2>s given possible type arguments are infinite, a new annotation similar to @Serializer can be added for specific type arguments:

@ConcreteTypeSerializer(forType = typeOf<Response<String, Nothing>>)
object StringResponseSerializer

Then the compiler plugin can infer the type argument of OkResponse and generate such a serializer:

Response.serializer(String.serializer(), Nothing::class.serializer(), OkResponse.serializer(String.serializer()))

In this case, the serializer for OkResponse<String> under polymorphism becomes OkResponse.serializer(String.serializer()) instead of OkResponse.serializer(PolymorphicSerializer(Any::class)).

The advantage of this approach is that there is no need to copy and adapt the type inference algorithm into this repository, and there is no need for caching. However, the user needs to generate serializers manually for all possible Ts. There is also another complication that StringResponseSerializer may have to be specified explicitly or registered for contextual serialization, given that registering a serializer for a concrete type given all the type arguments becomes supported in contextual serialization.

Summary

To sum up, Approach 1 appears the most feasible among the 3.

Possible implementation details

Refactoring serializers and implementing caching in Approach 1

If I understand it correctly, the semantics of the type parameter T in KSerializer<T> is a concrete type. So in the case of PolymorphicSerializer and SealedClassSerializer the semantics is a bit misused since an instance of either works for values of all possible concrete root types formed by the polymorphic root class given type arguments.

For Approach 1, to ensure performance, type inference can not be run every time so it's necessary to cache subtype serializers (or inferred subtype type arguments) for different root types (or root type type arguments). Inside the generated companion serializer(...) function (Response.serializer(...) in this case), the cache can be checked for a concrete type's serializer and if it's missing, the type inference algorithm can run. In this case, we need to make each instance of PolymorphicSerializer and SealedClassSerializer work for a concrete type. Like this:

https://github.com/huanshankeji/kotlinx.serialization/blob/74ef87728e6bfac1502e63f389d55ef9d93f4d11/core/commonMain/src/kotlinx/serialization/SealedSerializer.kt#L75-L76

In this way, the serializers for Response<String, Nothing> and Response<Int> become 2 separate instances, and we may reuse ParametrizedSerializerCache to cache parameterized polymorphic serializers.

Polymorphic serializers for concrete types

In all these Approaches 1, 2, and 3, PolymorphicSerializer and SealedClassSerializer (subclasses of AbstractPolymorphicSerializer) have to be refactored to bind to concrete types (KTypes), as stated in my issue title.

[pdvrieze]

This is a bit trickier than you seem to see. For example for

And it can be called like this (ignore the different function names):

polymorphic(Response::class) {
    subclass { args -> OkResponse.serializer(args[0]) }
}```

The problem is that for deserialization it isn't enough. And for some formats not enough at all. For example the XML format requires the ability to enumerate all subtypes (as it has annotations to map tag names to type names) - this might work if passing Nothing::class is permitted.
For deserialization you in any case need a mapping from type to deserializer. Currently types are not expected to have parameters (it may be possible to add this in a way that doesn't change the serialized format for existing types, but only for those that are supported with the new features).

As a starting point you can probably get away with a custom special polymorphic serializer (that uses your logic instead of the existing logic).

As to approach 2. It can not work as it is not possible (generically) at runtime to distinguish between OkResponse<String> and OkResponse<Int> (as the type parameter is erased and the serialization library has no generic way to introspect the response for its member type - note that it can't use the serializer to "extract" the member as this code is needed to determine the serializer - also note that there is no requirement for serializers to work any specific way as long as they meet the serializer contract, they can make up properties out of whole cloth).

As to the sealed bit, with annotations:

@ConcreteTypeSerializer(forType = typeOf<Response<String, Nothing>>)
object StringResponseSerializer```

This doesn't work. What Serializer does is to trigger an autogenerated serializer. You can also write it by hand without the annotation. Restrictions on annotations mean that non-source annotations can not have KType parameters (only JVM constants, classes and strings are allowed, at least for the JVM). In any case the type disappears entirely at runtime. What would be possible is to generate a special serializer that retains the type information, but this makes the serializer specific to that type (you can already do that for custom serializers without KType, just specify the full type as type parameter to KSerializer).

As to trying things out, you can do that if you realise that the main thing the compiler plugin does is to generate serializers. You can write your own custom serializers to behave whatever way you want. Even nested type parameters work (for each type parameter a nested serializer is passed (which can have type parameters of its own)). The challenge is when the type parameter is polymorphic as polymorphic serialization erases the subtypes (there is one polymorphic serializer independent of type parameters), if your serializer has special logic you can make the child @Contextual (as PolymorphicSerializer itself does).

[ShreckYe]

The problem is that for deserialization it isn't enough.

The challenge is when the type parameter is polymorphic as polymorphic serialization erases the subtypes (there is one polymorphic serializer independent of type parameters)

This is where I think I can provide a solution. When the user calls a deserialization function such as decodeFromString, we can get a polymorphic root type's KType and create a subclass map of type Map<KClass<out Base>, KType> (or Map<KClass<out Base>, KSerializer<out Base>>). Though at runtime there is only a subclass's KClass<*>, its KType and KSerializer can be recovered with this map. Or to speed up the process, since the type is serialized as a serial name string, we can create a map of type Map<String, KSerializer> that directly maps the serial names to the serializers.

As to approach 2. It can not work as it is not possible (generically) at runtime to distinguish between OkResponse<String> and OkResponse<Int>

Yes, you are right. This is another reason Approach 2 doesn't work which I missed.

This doesn't work. What Serializer does is to trigger an autogenerated serializer. You can also write it by hand without the annotation.

In this approach the subtype serializers have to be specified in the constructor, so the compiler plugin can infer and pass OkResponse.serializer(String.serializer()) in
Response.serializer(String.serializer(), Nothing::class.serializer(), OkResponse.serializer(String.serializer())), saving the coder the trouble of finding the right subtype.

Restrictions on annotations mean that non-source annotations can not have KType parameters (only JVM constants, classes and strings are allowed, at least for the JVM).

Thanks for pointing this out. I actually doubted this but I didn't check since Approach 3 is problematic anyway.

[mgroth0]

This issue is important for something I am trying to do and I'm glad it is being discussed.

To be honest, I do not fully understand everything that has been said. It's a lot of different ideas and issues, but I am pretty sure I get the main point. To sum it up: polymorphic serializers are not able transmit their type argument serializers to their subclasses.

This issue really matters for me so I wanted to contribute in some way, so I hope this is helpful.

First, here is some tests that demonstrate exactly what problem I had, and also contains a custom workaround I developed. I will discuss the custom workaround, its caveats, and its implications below.

Click here to see tests and usage of custom serializer

import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Polymorphic
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.serializer
import matt.json.asolution.solution2.workingGenericPolymorphicSerializer
import kotlin.test.Test
import kotlin.test.assertEquals


@OptIn(ExperimentalSerializationApi::class)
class PolymorphicGenericSerializers {
    private val instance = SubClass(1)

    @Test
    fun test1() {
        assertFullCircleEquality(serializer<SubClass<Int>>(), instance)
    }

    @Test
    fun test2() {
        assertFullCircleEquality(serializer<SuperInterface<Int>>(), instance)
    }

    @Test
    fun test3() {
        assertFullCircleEquality(SubClass.serializer(serializer<Int>()), instance)
    }

    @Test
    fun test4() {
        assertFullCircleEquality(SuperInterface.serializer(serializer<Int>()), instance)
    }


    @Test
    fun test5() {
        assertFullCircleEquality(serializer(SubClass::class, listOf(serializer<Int>()), false), instance)
    }

    @Test
    fun test6() {
        testSolution(SubClass(1))
    }

    private inline fun <reified T> testSolution(instance: SubClass<T>) {
        assertFullCircleEquality(workingGenericPolymorphicSerializer(SuperInterface::class, serializer<T>()), instance)
    }

    private inline fun <reified A, reified B> testSolution2(instance: SuperInterface2<A, B>) {
        assertFullCircleEquality(
            workingGenericPolymorphicSerializer(
                SuperInterface2::class, serializer<A>(),
                serializer<B>()
            ),
            instance
        )
    }

    @Test
    fun test7() {
        testSolution(SubClass(1f))
        testSolution(SubClass("abc"))
        testSolution(SubClass('a'))
        testSolution(SubClass(1.0))
        testSolution(SubClass(1L))
        testSolution(SubClass(listOf(1, 2, 3)))
        testSolution(SubClass(SubClass(1)))
        testSolution(SubClass(mapOf("a" to 1)))
        testSolution2(SubClass2(1, 'a'))
        testSolution2(SubClass2(SubClass(SubClass(1)), SubClass(SubClass2(1.0, 1f))))
    }


    private fun <T> assertFullCircleEquality(
        serializer: KSerializer<T>,
        instance: T,
        json: Json = Json
    ) {
        val serialized = json.encodeToString(serializer, instance)
        val deserialized = json.decodeFromString(serializer, serialized)
        assertEquals(instance, deserialized)
    }
}

@Polymorphic
@Serializable
data class SubClass<T>(override val value: T) : SuperInterface<T>

@Serializable
sealed interface SuperInterface<T> {
    val value: T
}

@Serializable
sealed interface SuperInterface2<A, B>

@Serializable
data class SubClass2<A, B>(
    val a: A,
    val b: B
) : SuperInterface2<A, B>

@Serializable
data class SubClass3<A, B>(
    val a: A,
    val b: B
) : SuperInterface2<A, B>

In my tests above, which run on Kotlin/JVM, "test2" and "test4" fail. These failures demonstrate the high level issue and how it practically affects me. The context in which I am trying to use sealed interfaces with type arguments is in a custom web socket protocol in which the server and client are exchanging serialized messages.

You can also see in the tests above that I used my custom solution "test6" and "test7". To sum up what this does, basically it gives you a polymorphic serializer that transmits the type arguments to subtypes as you would expect in certain contexts.

Click here to see the implementation of the custom serializer

import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.InternalSerializationApi
import kotlinx.serialization.KSerializer
import kotlinx.serialization.SealedClassSerializer
import kotlinx.serialization.serializer
import kotlin.reflect.KClass
import kotlin.reflect.typeOf


@OptIn(InternalSerializationApi::class, ExperimentalSerializationApi::class)
inline fun <reified T : Any> workingGenericPolymorphicSerializer(
    baseClass: KClass<T>,
    vararg typeArgSerializers: KSerializer<*>
): KSerializer<T> {
    val typeArgCount = typeOf<T>().arguments.size
    check(typeArgCount == typeArgSerializers.size) {
        "base class has $typeArgCount type params, so exactly $typeArgCount typeArgSerializers should be provided (not ${typeArgSerializers.size})"
    }
    return when (typeArgCount) {
        0    -> serializer<T>()
        else -> {
            val defaultSerializer = serializer(baseClass, typeArgSerializers.toList(), isNullable = false)
            defaultSerializer as SealedClassSerializer
            val subclasses = baseClass.sealedSubclasses.toTypedArray()
            val subclassSerializers = Array<KSerializer<out T>?>(subclasses.size) { null }
            subclasses.forEachIndexed { i, it ->
                val subclassTypeArgCount = it.typeParameters.size
                when {
                    subclassTypeArgCount > typeArgCount ->
                        error(
                            "this will never work, some type args are undefined here"
                        )

                    subclassTypeArgCount < typeArgCount ->
                        TODO(
                            "gotta sort out which type args are serialized"
                        )

                    else                                -> {
                        val r = serializer(it, typeArgSerializers.toList(), isNullable = false)
                        @Suppress("UNCHECKED_CAST")
                        subclassSerializers[i] = r as KSerializer<out T>
                    }
                }
            }
            return SealedClassSerializer(
                serialName = defaultSerializer.descriptor.serialName,
                baseClass = baseClass,
                subclasses = subclasses,
                subclassSerializers = subclassSerializers.filterNotNull().toTypedArray()
            )
        }
    }
}

Basically, I just create a SealedClassSerializer but substitute the subclass serializers with ones that have a type parameter.

The nice things about this solution are:

• It does not rely on anything format-specific, so it can be used with any format.
• It fixes the main issue for many simple cases
• It is small and simple
• It doesn't require changes to the compiler plugin

The caveats are:

• It requires kotlin.reflect (but it doesn't have to, if this library exposed SealedClassSerializer.class2Serializer)
• It does not work on JS (but it could, if this library exposed SealedClassSerializer.class2Serializer)
  • (It is only tested on JVM, but if native supports getting sealed subclasses than I would assume it works there too)
• This could break if type arg serializers need to be transmitted recursively. The current implementation only transmits to direct subclasses. I haven't put any thought into if this could be recursive, though it seems possible
• You may have noticed the TODO in there, which is because things get a bit more complex if you have to figure out which type arg serializers to transmit
• There also could be issues if subclasses define type args in a different order

Moving forward, I see the potential for some potentially small-ish changes to this library that could make a significant dent in this overall issue:

1. Kotlinx.serialization seems to collect sealed subclass information and store them inside of SealedClassSerializer instances. Because Kotlin/JS has absolutely no way at all of getting sealed subclasses at runtime (as far as I know), it seems like there could be significant value in kotlinx.serialization exposing this information. So, essentially making SealedClassSerializer.class2Serializer public or perhaps through some other new API.
2. This library could adopt or makes its own implementation of a "smarter" polymorphic serializer like the one I have above. It just needs work in a few areas (which I already mentioned):

• recursion through subclasses and transmitting type arg serializers to each
• gaining access to sealed subclass info in kotlin/JS so this can be multiplatform
• sorting out situations like when the subclass has type params in a different order or different ones

[pdvrieze]

Having had a look at #2729 it should be possible for this to be implemented (by the plugin) for the sealed base class case where the child type parameter matches the parent type parameter. In such case you would effectively want a serializer(typeParamSerializer) function instead of serializer() and the member type serializer resolved at compile time. The no-arg overload could even be implemented using PolymorphicSerializer as default.