MismatchedInputException: Cannot deserialize instance of com.fasterxml.jackson.databind.node.ObjectNode out of VALUE_NULL token

[Stexxen]

When using readerForUpdating and updating an existing objectNode to a null Node this exception is triggered

com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of `com.fasterxml.jackson.databind.node.ObjectNode` out of VALUE_NULL token
 at [Source: UNKNOWN; line: -1, column: -1]

	at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59)
	at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1468)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1242)
	at com.fasterxml.jackson.databind.DeserializationContext.handleUnexpectedToken(DeserializationContext.java:1148)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer$ObjectDeserializer.deserialize(JsonNodeDeserializer.java:107)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer$ObjectDeserializer.deserialize(JsonNodeDeserializer.java:82)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.updateObject(JsonNodeDeserializer.java:361)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.updateObject(JsonNodeDeserializer.java:373)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer$ObjectDeserializer.deserialize(JsonNodeDeserializer.java:120)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer$ObjectDeserializer.deserialize(JsonNodeDeserializer.java:82)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:2081)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1696)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1711)

Version information
Jackson : 2.11.4
Java : openjdk-15.0.2

To Reproduce

  @Test
  void test() throws IOException {

    ObjectMapper objectMapper = new ObjectMapper();

    ObjectNode node = objectMapper.createObjectNode();
    node.set("test", objectMapper.createObjectNode());

    ObjectNode update = objectMapper.createObjectNode();
    update.set("test", NullNode.getInstance());

    ObjectReader objectReader = objectMapper.readerForUpdating(node);
    ObjectNode afterUpdateNode = objectReader.readValue(update, ObjectNode.class);

  }

Expected behavior
In the code above node starts as

{
  "test": {}
}

and updated with the object update containing

{
  "test": null
}

after the update is applied afterUpdateNode should contain the following

{
  "test": null
}

But instead the above exception is raised. This looks to be same as #2325 but that was closed by the originator

[cowtowncoder]

Thank you for reporting this; I agree that it sounds like a flaw in implementation.

[Stexxen]

Thankyou for looking at this issue.

Unfortunately I know very little about Jackson internals, but my first thought is this code here https://github.com/FasterXML/jackson-databind/blob/2.12/src/main/java/com/fasterxml/jackson/databind/deser/std/JsonNodeDeserializer.java#L391

                if (old instanceof ObjectNode) {
                    JsonNode newValue = updateObject(p, ctxt, (ObjectNode) old);
                    if (newValue != old) {
                        node.set(key, newValue);
                    }
                    continue;
                }

If we could change it to something like...
if the old node and the new node are instances of ObjectNode only then try to merge, otherwise it can only be a replace as the types are different.
This would then fall through and get set to a NULL_VALUE at line https://github.com/FasterXML/jackson-databind/blob/2.12/src/main/java/com/fasterxml/jackson/databind/deser/std/JsonNodeDeserializer.java#L434

I've tried this in a debugger by skipping the test on line 391 and it outputs expected results, but because of the TokenParser I'm not sure how to check that the new 'thing' would be an instance of ObjectNode

[cowtowncoder]

Thanks; I should be able to figure out. It may be just question of parent (enclosing) ObjectNode dealing with null -> NullNode change, I hope. But I hope to also see if ArrayNode might have similar issues or not.

[cowtowncoder]

Same problem also happened to ArrayNode. Also noticed that duplicate detection would misfire for update case. Changed these as well; passes original test, although I would be interested in more testing as JsonNode merging test coverage is bit shallow.

Fixed in 2.12 for 2.12.1; slightly concerned about XML use case (since 2.12 had big improvements to JsonNode-from-XML use case). No new XML format test failures, but then again testing on that side is bit limited as well.
But we'll solve XML-related problems if and when they get uncovered.

[Stexxen]

@cowtowncoder Many thanks for resolving, i was looking at adding test cases to NodeMergeTest but the in the copy I've cloned the ObjectWriter here https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/ObjectWriter.java#L177

requires this file https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/cfg/PackageVersion.java.in but cannot due to the .in suffix on the filename.

What purpose does the .in suffix serve? Should I just rename it to make it compile?

[Stexxen]

Sorry please ignore that I've just seen the comments in the file I added.

/**
 * Automatically generated from PackageVersion.java.in during
 * packageVersion-generate execution of maven-replacer-plugin in
 * pom.xml.
 */

Doh!.

[cowtowncoder]

Unfortunately I have been unable to make it so that IDE auto-import would auto-generate that file: running mvn compile will create it.