From 18226ff0be6f98795b98dd505d8ac0b4f3cf8c07 Mon Sep 17 00:00:00 2001 From: Vahid Hashemian Date: Mon, 9 May 2016 17:29:15 -0700 Subject: [PATCH] KAFKA-3608; Fix ZooKeeper structures and output format in documentation Author: Vahid Hashemian Reviewers: Gwen Shapira Closes #1257 from vahidhashemian/KAFKA-3608 --- docs/implementation.html | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/implementation.html b/docs/implementation.html index be81227c906..0a36c22d9f8 100644 --- a/docs/implementation.html +++ b/docs/implementation.html @@ -282,7 +282,7 @@ When an element in a path is denoted [xyz], that means that the value of xyz is

Broker Node Registry

-/brokers/ids/[0...N] --> host:port (ephemeral node)
+/brokers/ids/[0...N] --> {"jmx_port":...,"timestamp":...,"endpoints":[...],"host":...,"version":...,"port":...} (ephemeral node)
 

This is a list of all present broker nodes, each of which provides a unique logical broker id which identifies it to consumers (which must be given as part of its configuration). On startup, a broker node registers itself by creating a znode with the logical broker id under /brokers/ids. The purpose of the logical broker id is to allow a broker to be moved to a different physical machine without affecting consumers. An attempt to register a broker id that is already in use (say because two servers are configured with the same broker id) results in an error. @@ -292,7 +292,7 @@ Since the broker registers itself in ZooKeeper using ephemeral znodes, this regi

Broker Topic Registry

-/brokers/topics/[topic]/[0...N] --> nPartitions (ephemeral node)
+/brokers/topics/[topic]/partitions/[0...N]/state --> {"controller_epoch":...,"leader":...,"version":...,"leader_epoch":...,"isr":[...]} (ephemeral node)
 

@@ -317,7 +317,7 @@ The consumers in a group divide up the partitions as fairly as possible, each pa

In addition to the group_id which is shared by all consumers in a group, each consumer is given a transient, unique consumer_id (of the form hostname:uuid) for identification purposes. Consumer ids are registered in the following directory.

-/consumers/[group_id]/ids/[consumer_id] --> {"topic1": #streams, ..., "topicN": #streams} (ephemeral node)
+/consumers/[group_id]/ids/[consumer_id] --> {"version":...,"subscription":{...:...},"pattern":...,"timestamp":...} (ephemeral node)
 
Each of the consumers in the group registers under its group and creates a znode with its consumer_id. The value of the znode contains a map of <topic, #streams>. This id is simply used to identify each of the consumers which is currently active within a group. This is an ephemeral node so it will disappear if the consumer process dies.

@@ -327,7 +327,7 @@ Each of the consumers in the group registers under its group and creates a znode Consumers track the maximum offset they have consumed in each partition. This value is stored in a ZooKeeper directory if offsets.storage=zookeeper.

-/consumers/[group_id]/offsets/[topic]/[broker_id-partition_id] --> offset_counter_value ((persistent node)
+/consumers/[group_id]/offsets/[topic]/[partition_id] --> offset_counter_value ((persistent node)
 

Partition Owner registry

@@ -337,7 +337,7 @@ Each broker partition is consumed by a single consumer within a given consumer g

-/consumers/[group_id]/owners/[topic]/[broker_id-partition_id] --> consumer_node_id (ephemeral node)
+/consumers/[group_id]/owners/[topic]/[partition_id] --> consumer_node_id (ephemeral node)
 

Broker node registration