2021-03-30 06:39:10 +08:00
|
|
|
KRaft (Kafka Raft)
|
|
|
|
==================
|
|
|
|
KRaft (Kafka Raft) is a protocol based on the [Raft Consensus Protocol](https://www.usenix.org/system/files/conference/atc14/atc14-paper-ongaro.pdf)
|
|
|
|
tailored for Apache Kafka.
|
2020-09-23 02:32:44 +08:00
|
|
|
|
2023-05-10 16:40:20 +08:00
|
|
|
This is used by Apache Kafka in the KRaft (Kafka Raft Metadata) mode. We
|
2021-03-30 06:39:10 +08:00
|
|
|
also have a standalone test server which can be used for performance testing. We describe the details to set this up below.
|
2020-09-23 02:32:44 +08:00
|
|
|
|
|
|
|
### Run Single Quorum ###
|
2024-10-17 11:32:27 +08:00
|
|
|
bin/test-kraft-server-start.sh --config config/kraft.properties --replica-directory-id b8tRS7h4TJ2Vt43Dp85v2A
|
2020-09-23 02:32:44 +08:00
|
|
|
|
|
|
|
### Run Multi Node Quorum ###
|
2021-03-30 06:39:10 +08:00
|
|
|
Create 3 separate KRaft quorum properties as the following:
|
2020-09-23 02:32:44 +08:00
|
|
|
|
2021-03-30 06:39:10 +08:00
|
|
|
`cat << EOF >> config/kraft-quorum-1.properties`
|
2020-09-23 02:32:44 +08:00
|
|
|
|
2021-01-31 09:05:31 +08:00
|
|
|
node.id=1
|
2020-09-23 02:32:44 +08:00
|
|
|
listeners=PLAINTEXT://localhost:9092
|
2021-04-16 11:48:53 +08:00
|
|
|
controller.listener.names=PLAINTEXT
|
2021-01-22 02:26:23 +08:00
|
|
|
controller.quorum.voters=1@localhost:9092,2@localhost:9093,3@localhost:9094
|
2021-03-30 06:39:10 +08:00
|
|
|
log.dirs=/tmp/kraft-logs-1
|
2020-09-23 02:32:44 +08:00
|
|
|
EOF
|
|
|
|
|
2021-03-30 06:39:10 +08:00
|
|
|
`cat << EOF >> config/kraft-quorum-2.properties`
|
2020-09-23 02:32:44 +08:00
|
|
|
|
2021-01-31 09:05:31 +08:00
|
|
|
node.id=2
|
2020-09-23 02:32:44 +08:00
|
|
|
listeners=PLAINTEXT://localhost:9093
|
2021-04-16 11:48:53 +08:00
|
|
|
controller.listener.names=PLAINTEXT
|
2021-01-22 02:26:23 +08:00
|
|
|
controller.quorum.voters=1@localhost:9092,2@localhost:9093,3@localhost:9094
|
2021-03-30 06:39:10 +08:00
|
|
|
log.dirs=/tmp/kraft-logs-2
|
2020-09-23 02:32:44 +08:00
|
|
|
EOF
|
|
|
|
|
2021-03-30 06:39:10 +08:00
|
|
|
`cat << EOF >> config/kraft-quorum-3.properties`
|
2020-09-23 02:32:44 +08:00
|
|
|
|
2021-01-31 09:05:31 +08:00
|
|
|
node.id=3
|
2020-09-23 02:32:44 +08:00
|
|
|
listeners=PLAINTEXT://localhost:9094
|
2021-04-16 11:48:53 +08:00
|
|
|
controller.listener.names=PLAINTEXT
|
2021-01-22 02:26:23 +08:00
|
|
|
controller.quorum.voters=1@localhost:9092,2@localhost:9093,3@localhost:9094
|
2021-03-30 06:39:10 +08:00
|
|
|
log.dirs=/tmp/kraft-logs-3
|
2020-09-23 02:32:44 +08:00
|
|
|
EOF
|
|
|
|
|
|
|
|
Open up 3 separate terminals, and run individual commands:
|
|
|
|
|
2024-10-17 11:32:27 +08:00
|
|
|
bin/test-kraft-server-start.sh --config config/kraft-quorum-1.properties --replica-directory-id b8tRS7h4TJ2Vt43Dp85v2A
|
|
|
|
bin/test-kraft-server-start.sh --config config/kraft-quorum-2.properties --replica-directory-id Nkij_D9XRiYKNb41SiJo7Q
|
|
|
|
bin/test-kraft-server-start.sh --config config/kraft-quorum-3.properties --replica-directory-id 4-e97nI7eHPYKfEDtW8rtQ
|
2020-10-23 08:51:05 +08:00
|
|
|
|
2020-10-28 03:10:13 +08:00
|
|
|
Once a leader is elected, it will begin writing to an internal
|
2021-04-09 01:24:23 +08:00
|
|
|
`__raft_performance_test` topic with a steady workload of random data.
|
2020-10-28 03:10:13 +08:00
|
|
|
You can control the workload using the `--throughput` and `--record-size`
|
2021-03-30 06:39:10 +08:00
|
|
|
arguments passed to `test-kraft-server-start.sh`.
|