root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 3 Packages Projects Releases Wiki Activity

KAFKA-10477: Fix JsonConverter regression to treat MISSING nodes as NULL nodes (#9306) 

Fixes a regression introduced in `JsonConverter` with previous upgrades from Jackson Databind 2.9.x to 2.10.x. Jackson Databind version 2.10.0 included a backward-incompatible behavioral change to use `JsonNodeType.MISSING` (and `MissingNode`, the subclass of `JsonNode` that has a type of `MISSING`) instead of `JsonNodeType.NULL` / `NullNode`. See https://github.com/FasterXML/jackson-databind/issues/2211 for details of this change.

This change makes recovers the older `JsonConverter` behavior of returning null on empty input.

Added two unit tests for this change. Both unit tests were independently tested with earlier released versions and passed on all versions that used Jackson 2.9.x and earlier, and failed on all versions that used 2.10.x and that did not have the fixed included in the PR. Both of the new unit tests pass with this fix to `JsonConverter`.

Author: Shaik Zakir Hussain <zhussain@confluent.io>
Reviewer: Randall Hauch <rhauch@gmail.com>
This commit is contained in:
Shaik Zakir Hussain 2020-10-02 19:51:11 +05:30 committed by Randall Hauch
parent f716986ac4
commit 94fd8ff345
2 changed files with 31 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java
View File

@ -729,6 +729,7 @@ public class JsonConverter implements Converter, HeaderConverter {
} else {
switch (jsonValue.getNodeType()) {
case NULL:
case MISSING:
// Special case. With no schema
return null;
case BOOLEAN:
@ -751,7 +752,6 @@ public class JsonConverter implements Converter, HeaderConverter {
break;
case BINARY:
case MISSING:
case POJO:
default:
schemaType = null;

30
connect/json/src/test/java/org/apache/kafka/connect/json/JsonConverterTest.java
View File

@ -195,6 +195,36 @@ public class JsonConverterTest {
assertEquals(SchemaAndValue.NULL, converted);
}
/**
* When schemas are disabled, empty data should be decoded to an empty envelope.
* This test verifies the case where `schemas.enable` configuration is set to false, and
* {@link JsonConverter} converts empty bytes to {@link SchemaAndValue#NULL}.
*/
@Test
public void emptyBytesToConnect() {
// This characterizes the messages with empty data when Json schemas is disabled
Map<String, Boolean> props = Collections.singletonMap("schemas.enable", false);
converter.configure(props, true);
SchemaAndValue converted = converter.toConnectData(TOPIC, "".getBytes());
assertEquals(SchemaAndValue.NULL, converted);
}
/**
* When schemas are disabled, fields are mapped to Connect maps.
*/
@Test
public void schemalessWithEmptyFieldValueToConnect() {
// This characterizes the messages with empty data when Json schemas is disabled
Map<String, Boolean> props = Collections.singletonMap("schemas.enable", false);
converter.configure(props, true);
String input = "{ \"a\": \"\", \"b\": null}";
SchemaAndValue converted = converter.toConnectData(TOPIC, input.getBytes());
Map<String, String> expected = new HashMap<>();
expected.put("a", "");
expected.put("b", null);
assertEquals(new SchemaAndValue(null, expected), converted);
}
@Test
public void nullSchemaPrimitiveToConnect() {
SchemaAndValue converted = converter.toConnectData(TOPIC, "{ \"schema\": null, \"payload\": null }".getBytes());