mirror of https://github.com/apache/kafka.git
b3c06b21d9
Kafka Streams checkpoints the wrong offset when a task is closed during
restoration. If under exactly-once processing guarantees a
TaskCorruptedException happens, the affected task is closed dirty, its
state content is wiped out and the task is re-initialized. If during
the following restoration the task is closed cleanly, the task writes
the offsets that it stores in its record collector to the checkpoint
file. Those offsets are the offsets that the task wrote to the changelog
topics. In other words, the task writes the end offsets of its changelog
topics to the checkpoint file. Consequently, when the task is
initialized again on the same Streams client, the checkpoint file is
read and the task assumes it is fully restored although the records
between the last offsets the task restored before closing clean and
the end offset of the changelog topics are missing locally.
The fix is to clear the offsets in the record collector on close.
Cherry-pick of
|
||
|---|---|---|
| .. | ||
| .scalafmt.conf | ||
| checkstyle.xml | ||
| import-control-core.xml | ||
| import-control-jmh-benchmarks.xml | ||
| import-control-metadata.xml | ||
| import-control-server-common.xml | ||
| import-control-storage.xml | ||
| import-control.xml | ||
| java.header | ||
| suppressions.xml | ||