Generic polymorphism example doesn't work with primitive and collection types

[fluidsonic]

Describe the bug
I'm trying to use example poly 16 with Response<List<String>>. However the library doesn't seem to be able to encode List, String, etc. in a polymorphic context.

To Reproduce

import kotlinx.serialization.*
import kotlinx.serialization.json.*
import kotlinx.serialization.modules.*

@Serializable
abstract class Response<out T>

@Serializable
data class OkResponse<out T>(val data: T) : Response<T>()

fun main() {
    val json = Json {
        serializersModule = SerializersModule {
            polymorphic(Response::class) {
                subclass(OkResponse.serializer(PolymorphicSerializer(Any::class)))
            }
        }
    }

    val response1: Response<List<String>> = OkResponse(listOf("good"))
    println(json.encodeToJsonElement(response1)) // Class 'SingletonList' is not registered for polymorphic serialization in the scope of 'Any'.

    val response2: Response<String> = OkResponse("good")
    println(json.encodeToJsonElement(response2)) // Class 'String' is not registered for polymorphic serialization in the scope of 'Any'.
}

However, using OkResponse instead of Response works (I guess because it's not polymorphic):

val response: OkResponse<List<String>> = OkResponse(listOf("good"))
println(json.encodeToJsonElement(response)) // {"data":["good"]}

The error messages above aren't helpful either.
Using the guide and the error messages the next step would be straightforward:

polymorphic(Any::class) {
    subclass(SingletonList::class) // not possible because class is private
    subclass(String::class) // not possible: Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator.
}

Expected behavior
In the serialization guide there's no mention about these issues.
Primitives and collection types can be encoded in all other cases automatically by kotlinx-serialization. I expect the same here too.

The initial example could either output the following or add a class discriminator like kotlin.String.

{"type":"OkResponse","data":["good"]}
{"type":"OkResponse","data":"good"}

If that's not possible, then:

• the guide should at least explain the limitation
• the guide should mention possible alternatives
• the related error messages should also be improved for standard types

Environment

• Kotlin version: 1.4.21
• Library version: 1.0.1
• Kotlin platforms: JVM

[christofvanhove]

My reply is about the "not possible: Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator" part of the problem. I'm not pretending to give an answer, I'm also just trying to figure things out. The following code below seems to fix your problem. Please comment if you see any issues with it ;-)

import kotlinx.serialization.*
import kotlinx.serialization.descriptors.SerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass

@Serializable
abstract class Response<out T>

@Serializable
data class OkResponse<out T>(val data: T) : Response<T>()

object StringAsObjectSerializer : KSerializer<String> {

    @Serializable
    @SerialName("String")
    data class StringSurrogate(val value: String)

    override val descriptor: SerialDescriptor = StringSurrogate.serializer().descriptor

    override fun serialize(encoder: Encoder, value: String) {
        StringSurrogate.serializer().serialize(encoder, StringSurrogate(value));
    }

    override fun deserialize(decoder: Decoder): String {
        return decoder.decodeSerializableValue(StringSurrogate.serializer()).value
    }
}

fun main() {
    val json = Json {
        serializersModule = SerializersModule {
            polymorphic(Any::class) {
                subclass(StringAsObjectSerializer)
            }
            polymorphic(Response::class) {
                subclass(OkResponse.serializer(PolymorphicSerializer(Any::class)))
            }
        }
    }

    val response2: Response<String> = OkResponse("good")

    //serialization looks OK
    val serializedJson = json.encodeToString(response2)
    println("serializedJson = ${serializedJson}")

    //this works
    val deSerializedJson1 = json.decodeFromString<Response<String>>(serializedJson)
    println("deSerializedJson1 = ${deSerializedJson1}")

    //this doesn't work
    try {
        json.decodeFromString<Response<Any>>(serializedJson)
    } catch (e: Exception) {
        println("why?  ${e}")
    }

    //this works
    val serializer = PolymorphicSerializer(Response::class)
    val deSerializedJson2 = json.decodeFromString(serializer, serializedJson)
    println("deSerializedJson2 = ${deSerializedJson2}")
    
}

[christofvanhove]

Another option for the "not possible: Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator" is useArrayPolymorphism = true. But this results in ugly unpractical json IMHO. But it just works, it's a pitty that there is no option to serialize primitives as json object with type attributes. That would make my solution above obsolete ;-). In Kotlin primitives are just objects, so why not just serialize them like that as well?

import kotlinx.serialization.PolymorphicSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass

@Serializable
abstract class Response<out T>

@Serializable
data class OkResponse<out T>(val data: T) : Response<T>()

fun main() {
    val json = Json {
        useArrayPolymorphism  = true
        serializersModule = SerializersModule {
            polymorphic(Any::class) {
                subclass(String::class)
            }
            polymorphic(Response::class) {
                subclass(OkResponse.serializer(PolymorphicSerializer(Any::class)))
            }
        }
    }

    val response2: Response<String> = OkResponse("good")

    //serialization looks OK
    val serializedJson = json.encodeToString(response2)
    println("serializedJson = ${serializedJson}")

    //this works
    val deSerializedJson1 = json.decodeFromString<Response<String>>(serializedJson)
    println("deSerializedJson1 = ${deSerializedJson1}")

    //this doesn't work
    try {
        json.decodeFromString<Response<Any>>(serializedJson)
    } catch (e: Exception) {
        println("why?  ${e}")
    }

    //this works
    val serializer = PolymorphicSerializer(Response::class)
    val deSerializedJson2 = json.decodeFromString(serializer, serializedJson)
    println("deSerializedJson2 = ${deSerializedJson2}")

}

[fluidsonic]

Thanks, that weird surrogate hack indeed works for String :)
Just not really nice nor efficient.

It also would be even better if

{"type":"OkResponse","data":{"type":"String","value":"good"}}

would become

{"type":"OkResponse","data":"good"}

From JSON and JSON-API point of view {"type":"String","value":"good"} is really odd as you can simply use a plain string. Same for other types natively supported by JSON.

Regarding why json.decodeFromString<Response<Any>>(…) fails see https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/polymorphism.md#static-parent-type-lookup-for-polymorphism
Any (or any other type) is not considered polymorphic until we explicitly say so.

[christofvanhove]

I think the problem with {"type":"OkResponse","data":"good"} is that it cannot be deserialized again. For a String it might theoretically work (json string to Kotlin String), but with a number it is impossible to guess the correct type. How should {"type":"OkResponse","data":3} be deserialized? Is 3 a Byte,Int,Double,... ?

[lnhrdt]

Has anyone been able to find a way to encode Lists in a polymorphic context? Or determine conclusively that it's not supported (or a current limitation)?

I am encountering the same/similar issues when trying something as in @fluidsonic's example:

val response1: Response<List<String>> = OkResponse(listOf("good"))

[christofvanhove]

I think there are 2 solutions to encode Lists in a polymorphic context. See my comments above. Both work without issues as far as I know. You can choose either one depending on which output you like best.

1. use arraypolymorphism
2. add custom serializers for the primitive types (I use this in production because I like the output better)

[StarGuardian]

I found very beautiful solution for case with sealed classes:

@Serializable(with = OperationResultSerializer::class)
sealed interface OperationResult<T>

@Serializable
class OperationSuccess<T>(val value:T): OperationResult<T>

@Serializable
class OperationFailed(val error: String): OperationResult<Nothing>

@OptIn(InternalSerializationApi::class)
class OperationResultSerializer<T>(valueSerializer: KSerializer<T>): KSerializer<OperationResult<T>> {
    private val serializer = SealedClassSerializer(
        OperationResult::class.simpleName!!,
        OperationResult::class,
        arrayOf(OperationSuccess::class, OperationFailed::class),
        arrayOf(OperationSuccess.serializer(valueSerializer), OperationFailed.serializer())
    )

    override val descriptor: SerialDescriptor = serializer.descriptor
    @Suppress("UNCHECKED_CAST")
    override fun deserialize(decoder: Decoder): OperationResult<T> { return serializer.deserialize(decoder) as OperationResult<T> }
    override fun serialize(encoder: Encoder, value: OperationResult<T>) { serializer.serialize(encoder, value) }
}

Nothing more! encodeToString and decodeFromString works perfectly event in case OperationResult<List<Int>>!

Kotlin version: 1.9.10
Kotlin serialization version: 1.6.0

[SettingDust]

My workaround based on @christofvanhove 's method

@Serializable data class PolymorphicSurrogate<T>(val value: T)

inline fun <reified T> PolymorphicPrimitiveSerializer() =
    PolymorphicPrimitiveSerializer(serializer<T>())

inline fun <reified T> SerializersModule.PolymorphicPrimitiveSerializer() =
    PolymorphicPrimitiveSerializer(serializer<T>())

class PolymorphicPrimitiveSerializer<T>(private val serializer: KSerializer<T>) : KSerializer<T> {
    override val descriptor = SerialDescriptor(serializer.descriptor.serialName, PolymorphicSurrogate.serializer(serializer).descriptor)

    override fun serialize(encoder: Encoder, value: T) =
        PolymorphicSurrogate.serializer(serializer).serialize(encoder, PolymorphicSurrogate(value))

    override fun deserialize(decoder: Decoder) =
        PolymorphicSurrogate.serializer(serializer).deserialize(decoder).value
}