kafka/examples
Federico Valeri 4cb20379c7
MINOR: Add retry mechanism to EOS example (#15561)
In the initial EOS example, a retry logic was implemented within the resetToLastCommittedPositions method. During refactoring, this logic was removed becasue a poison pill prevented the example from reaching the final phase of consuming from the output topic.

In this change, I suggest to add it back, but with a retry limit defined as MAX_RETRIES. Once this limit is reached, the problematic batch will be logged and skipped, allowing the processor to move on and process remaining records. If some records are skipped, the example will still hit the hard timeout (2 minutes), but after consuming all processed records.

Reviewers: Luke Chen <showuon@gmail.com>
2024-03-27 16:31:27 +08:00
..
bin KAFKA-9922: Update demo instructions in examples README (#8559) 2020-04-29 19:31:26 -07:00
src/main/java/kafka/examples MINOR: Add retry mechanism to EOS example (#15561) 2024-03-27 16:31:27 +08:00
README.md KAFKA-14752: Kafka examples improvements - demo changes (#13517) 2023-05-12 10:39:12 +08:00

README.md

Kafka client examples

This module contains some Kafka client examples.

  1. Start a Kafka 2.5+ local cluster with a plain listener configured on port 9092.
  2. Run examples/bin/java-producer-consumer-demo.sh 10000 to asynchronously send 10k records to topic1 and consume them.
  3. Run examples/bin/java-producer-consumer-demo.sh 10000 sync to synchronous send 10k records to topic1 and consume them.
  4. Run examples/bin/exactly-once-demo.sh 6 3 10000 to create input-topic and output-topic with 6 partitions each, start 3 transactional application instances and process 10k records.