From d2a267b111d23d6b98f2784382095b9ae5ddf886 Mon Sep 17 00:00:00 2001 From: Vahid Hashemian Date: Thu, 29 Sep 2016 19:37:20 -0700 Subject: [PATCH] KAFKA-3697; Clean up website documentation of client usage This is to imply that the Java consumer/producer are the recommended consumer/producer now. Author: Vahid Hashemian Reviewers: Jason Gustafson Closes #1921 from vahidhashemian/KAFKA-3697 --- docs/configuration.html | 13 +++++++------ docs/documentation.html | 4 ++-- docs/implementation.html | 4 ++-- docs/ops.html | 8 ++++---- docs/quickstart.html | 10 +++++----- docs/security.html | 2 +- docs/upgrade.html | 2 +- 7 files changed, 22 insertions(+), 21 deletions(-) diff --git a/docs/configuration.html b/docs/configuration.html index 54286919884..35f1475a234 100644 --- a/docs/configuration.html +++ b/docs/configuration.html @@ -70,9 +70,14 @@ Below is the configuration of the Java producer:

3.3 Consumer Configs

-We introduce both the old 0.8 consumer configs and the new consumer configs respectively below. +In 0.9.0.0 we introduced the new Java consumer as a replacement for the older Scala-based simple and high-level consumers. +The configs for both new and old consumers are described below. -

3.3.1 Old Consumer Configs

+

3.3.1 New Consumer Configs

+Below is the configuration for the new consumer: + + +

3.3.2 Old Consumer Configs

The essential old consumer configurations are the following:
    @@ -239,10 +244,6 @@ The essential old consumer configurations are the following:

    More details about consumer configuration can be found in the scala class kafka.consumer.ConsumerConfig.

    -

    3.3.2 New Consumer Configs

    -Since 0.9.0.0 we have been working on a replacement for our existing simple and high-level consumers. The code is considered beta quality. Below is the configuration for the new consumer: - -

    3.4 Kafka Connect Configs

    Below is the configuration of the Kafka Connect framework. diff --git a/docs/documentation.html b/docs/documentation.html index f4f1ddc0e7f..07ffe849e12 100644 --- a/docs/documentation.html +++ b/docs/documentation.html @@ -46,8 +46,8 @@ Prior releases: 0.7.x, 3.2 Producer Configs
  • 3.3 Consumer Configs
  • 3.4 Kafka Connect Configs
  • 3.5 Kafka Streams Configs diff --git a/docs/implementation.html b/docs/implementation.html index 91e17a68a71..12846fb1e8e 100644 --- a/docs/implementation.html +++ b/docs/implementation.html @@ -40,9 +40,9 @@ class Producer { The goal is to expose all the producer functionality through a single API to the client. -The new producer - +The Kafka producer
      -
    • can handle queueing/buffering of multiple producer requests and asynchronous dispatch of the batched data - +
    • can handle queueing/buffering of multiple producer requests and asynchronous dispatch of the batched data:

      kafka.producer.Producer provides the ability to batch multiple produce requests (producer.type=async), before serializing and dispatching them to the appropriate kafka broker partition. The size of the batch can be controlled by a few config parameters. As events enter a queue, they are buffered in a queue, until either queue.time or batch.size is reached. A background thread (kafka.producer.async.ProducerSendThread) dequeues the batch of data and lets the kafka.producer.EventHandler serialize and send the data to the appropriate kafka broker partition. A custom event handler can be plugged in through the event.handler config parameter. At various stages of this producer queue pipeline, it is helpful to be able to inject callbacks, either for plugging in custom logging/tracing code or custom monitoring logic. This is possible by implementing the kafka.producer.async.CallbackHandler interface and setting callback.handler config parameter to that class.

      • diff --git a/docs/ops.html b/docs/ops.html index a59e134c749..75657386f8d 100644 --- a/docs/ops.html +++ b/docs/ops.html @@ -143,7 +143,7 @@ my-group my-topic 1 0 0 -Note, however, after 0.9.0, the kafka.tools.ConsumerOffsetChecker tool is deprecated and you should use the kafka.admin.ConsumerGroupCommand (or the bin/kafka-consumer-groups.sh script) to manage consumer groups, including consumers created with the new consumer API. +NOTE: Since 0.9.0.0, the kafka.tools.ConsumerOffsetChecker tool has been deprecated. You should use the kafka.admin.ConsumerGroupCommand (or the bin/kafka-consumer-groups.sh script) to manage consumer groups, including consumers created with the new consumer API.

      Managing Consumer Groups

      @@ -183,7 +183,7 @@ The process of migrating data is manually initiated but fully automated. Under t

      The partition reassignment tool can be used to move partitions across brokers. An ideal partition distribution would ensure even data load and partition sizes across all brokers. The partition reassignment tool does not have the capability to automatically study the data distribution in a Kafka cluster and move partitions around to attain an even load distribution. As such, the admin has to figure out which topics or partitions should be moved around.

      -The partition reassignment tool can run in 3 mutually exclusive modes - +The partition reassignment tool can run in 3 mutually exclusive modes:

      • --generate: In this mode, given a list of topics and a list of brokers, the tool generates a candidate reassignment to move all partitions of the specified topics to the new brokers. This option merely provides a convenient way to generate a partition reassignment plan given a list of topics and target brokers.
      • --execute: In this mode, the tool kicks off the reassignment of partitions based on the user provided reassignment plan. (using the --reassignment-json-file option). This can either be a custom reassignment plan hand crafted by the admin or provided by using the --generate option
        • @@ -900,9 +900,9 @@ The following metrics are available on producer/consumer/connector instances. F -

        New producer monitoring

        +

        Producer monitoring

        -The following metrics are available on new producer instances. +The following metrics are available on producer instances. diff --git a/docs/quickstart.html b/docs/quickstart.html index 7654d5cf11b..32d61256543 100644 --- a/docs/quickstart.html +++ b/docs/quickstart.html @@ -78,7 +78,7 @@ Run the producer and then type a few messages into the console to send to the se Kafka also has a command line consumer that will dump out messages to standard output. 
        -> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning
+> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
 This is a message
 This is another message
        @@ -159,7 +159,7 @@ Let's publish a few messages to our new topic: Now let's consume these messages: 
        -> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic
+> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic my-replicated-topic
 ...
 my test message 1
 my test message 2
@@ -181,7 +181,7 @@ Topic:my-replicated-topic	PartitionCount:1	ReplicationFactor:3	Configs:
        But the messages are still be available for consumption even though the leader that took the writes originally is down: 
        -> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic my-replicated-topic
+> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic my-replicated-topic
 ...
 my test message 1
 my test message 2
@@ -236,7 +236,7 @@ Note that the data is being stored in the Kafka topic 
        connect-test
        , s
 data in the topic (or use custom consumer code to process it):
 
 
        -> bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic connect-test --from-beginning
+> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic connect-test --from-beginning
 {"schema":{"type":"string","optional":false},"payload":"foo"}
 {"schema":{"type":"string","optional":false},"payload":"bar"}
 ...
@@ -333,7 +333,7 @@ We can now inspect the output of the WordCount demo application by reading from
 
        
 
 
        -> bin/kafka-console-consumer.sh --zookeeper localhost:2181 \
+> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 \
             --topic streams-wordcount-output \
             --from-beginning \
             --formatter kafka.tools.DefaultMessageFormatter \
diff --git a/docs/security.html b/docs/security.html
index a00bbf654da..2e77c936e62 100644
--- a/docs/security.html
+++ b/docs/security.html
@@ -715,7 +715,7 @@ To enable ZooKeeper authentication on brokers, there are two necessary steps:
 	
      •  Set the configuration property zookeeper.set.acl in each broker to true
        • 
 
 
-The metadata stored in ZooKeeper is such that only brokers will be able to modify the corresponding znodes, but znodes are world readable. The rationale behind this decision is that the data stored in ZooKeeper is not sensitive, but inappropriate manipulation of znodes can cause cluster disruption. We also recommend limiting the access to ZooKeeper via network segmentation (only brokers and some admin tools need access to ZooKeeper if the new consumer and new producer are used).
+The metadata stored in ZooKeeper for the Kafka cluster is world-readable, but can only be modified by the brokers. The rationale behind this decision is that the data stored in ZooKeeper is not sensitive, but inappropriate manipulation of that data can cause cluster disruption. We also recommend limiting the access to ZooKeeper via network segmentation (only brokers and some admin tools need access to ZooKeeper if the new Java consumer and producer clients are used).
 
 
        7.6.2 Migrating clusters
        
 If you are running a version of Kafka that does not support security or simply with security disabled, and you want to make the cluster secure, then you need to execute the following steps to enable ZooKeeper authentication with minimal disruption to your operations:
diff --git a/docs/upgrade.html b/docs/upgrade.html
index 1eaa3559c75..7b16ab0b6ec 100644
--- a/docs/upgrade.html
+++ b/docs/upgrade.html
@@ -207,7 +207,7 @@ work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded to 0.9
     
      •  The default Kafka JVM performance options (KAFKA_JVM_PERFORMANCE_OPTS) have been changed in kafka-run-class.sh. 
        • 
     
      •  The kafka-topics.sh script (kafka.admin.TopicCommand) now exits with non-zero exit code on failure. 
        • 
     
      •  The kafka-topics.sh script (kafka.admin.TopicCommand) will now print a warning when topic names risk metric collisions due to the use of a '.' or '_' in the topic name, and error in the case of an actual collision. 
        • 
-    
      •  The kafka-console-producer.sh script (kafka.tools.ConsoleProducer) will use the new producer instead of the old producer be default, and users have to specify 'old-producer' to use the old producer. 
        • 
+    
      •  The kafka-console-producer.sh script (kafka.tools.ConsoleProducer) will use the Java producer instead of the old Scala producer be default, and users have to specify 'old-producer' to use the old producer. 
        • 
     
      •  By default all command line tools will print all logging messages to stderr instead of stdout. 
        •