KAFKA-2766: use new producer by default in tooling

Also update the API docs for new consumer.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Gwen Shapira

Closes #448 from guozhangwang/K2766

core/src/main/scala/kafka/tools/ConsoleProducer.scala

@@ -39,10 +39,10 @@ object ConsoleProducer {
         reader.init(System.in, getReaderProps(config))
 
         val producer =
-          if(config.useNewProducer) {
+          if(config.useOldProducer) {
-            new NewShinyProducer(getNewProducerProps(config))
-          } else {
             new OldProducer(getOldProducerProps(config))
+          } else {
+            new NewShinyProducer(getNewProducerProps(config))
           }
 
         Runtime.getRuntime.addShutdownHook(new Thread() {
@@ -239,7 +239,7 @@ object ConsoleProducer {
             .withRequiredArg
             .describedAs("producer_prop")
             .ofType(classOf[String])
-    val useNewProducerOpt = parser.accepts("new-producer", "Use the new producer implementation.")
+    val useOldProducerOpt = parser.accepts("old-producer", "Use the old producer implementation.")
 
     val options = parser.parse(args : _*)
     if(args.length == 0)
@@ -247,7 +247,7 @@ object ConsoleProducer {
     CommandLineUtils.checkRequiredArgs(parser, options, topicOpt, brokerListOpt)
 
     import scala.collection.JavaConversions._
-    val useNewProducer = options.has(useNewProducerOpt)
+    val useOldProducer = options.has(useOldProducerOpt)
     val topic = options.valueOf(topicOpt)
     val brokerList = options.valueOf(brokerListOpt)
     ToolsUtils.validatePortOrDie(parser,brokerList)

core/src/main/scala/kafka/tools/ProducerPerformance.scala

@@ -34,7 +34,10 @@ import org.apache.log4j.Logger
 
 /**
  * Load test for the producer
+ *
+ * @note This class will be replaced by org.apache.kafka.tools.ProducerPerformance after the old producer client is removed
  */
+@deprecated
 object ProducerPerformance extends Logging {
 
   def main(args: Array[String]) {

core/src/test/scala/kafka/tools/ConsoleProducerTest.scala

@@ -18,9 +18,8 @@
 package kafka.tools
 
 import kafka.producer
-import kafka.tools.ConsoleProducer.{LineMessageReader, MessageReader,ProducerConfig}
+import kafka.tools.ConsoleProducer.LineMessageReader
-import org.apache.kafka.clients.producer.{KafkaProducer, ProducerConfig}
+import org.apache.kafka.clients.producer.KafkaProducer
-import joptsimple.UnrecognizedOptionException
 import org.junit.Assert
 import org.junit.Test
 

docs/api.html

@@ -19,7 +19,7 @@ We are in the process of rewritting the JVM clients for Kafka. As of 0.8.2 Kafka
 
 <h3><a id="producerapi">2.1 Producer API</a></h3>
 
-As of the 0.8.2 release we encourage all new development to use the new Java producer. This client is production tested and generally both faster and more fully featured than the previous Scala client. You can use this client by adding a dependency on the client jar using the following maven co-ordinates:
+As of the 0.8.2 release we encourage all new development to use the new Java producer. This client is production tested and generally both faster and more fully featured than the previous Scala client. You can use this client by adding a dependency on the client jar using the following example maven co-ordinates (you can change the version numbers with new releases):
 <pre>
 	&lt;dependency&gt;
 	    &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
@@ -152,3 +152,17 @@ Providing a horizontally scalable solution for aggregating and loading data into
 <p>
 Usage information on the hadoop consumer can be found <a href="https://github.com/linkedin/camus/">here</a>.
 </p>
+
+
+<h3><a id="newconsumerapi">2.5 New Consumer API</a></h3>
+As of the 0.9.0 release we have added a replacement for our existing simple and high-level consumers. This client is considered beta quality. You can use this client by adding a dependency on the client jar using the following example maven co-ordinates (you can change the version numbers with new releases):
+<pre>
+	&lt;dependency&gt;
+	    &lt;groupId&gt;org.apache.kafka&lt;/groupId&gt;
+	    &lt;artifactId&gt;kafka-clients&lt;/artifactId&gt;
+	    &lt;version&gt;0.9.0.0&lt;/version&gt;
+	&lt;/dependency&gt;
+</pre>
+
+Examples showing how to use the producer are given in the
+<a href="http://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/producer/KafkaConsumer.html" title="Kafka 0.9.0 Javadoc">javadocs</a>.

docs/upgrade.html

@@ -39,6 +39,7 @@ Note: Bumping the protocol version and restarting can be done any time after the
     <li> The default Kafka JVM performance options (KAFKA_JVM_PERFORMANCE_OPTS) have been changed in kafka-run-class.sh. </li>
     <li> The kafka-topics.sh script (kafka.admin.TopicCommand) now exits with non-zero exit code on failure. </li>
     <li> The kafka-topics.sh script (kafka.admin.TopicCommand) will now print a warning when topic names risk metric collisions due to the use of a '.' or '_' in the topic name, and error in the case of an actual collision. </li>
+    <li> The kafka-console-producer.sh script (kafka.tools.ConsoleProducer) will use the new producer instead of the old producer be default, and users have to specify 'old-producer' to use the old producer. </li>
     <li> By default all command line tools will print all logging messages to stderr instead of stout. </li>
 </ul>
 
@@ -47,6 +48,7 @@ Note: Bumping the protocol version and restarting can be done any time after the
 <ul>
     <li> Altering topic configuration from the kafka-topics.sh script (kafka.admin.TopicCommand) has been deprecated. Going forward, please use the kafka-configs.sh script (kafka.admin.ConfigCommand) for this functionality. </li>
     <li> The kafka-consumer-offset-checker.sh (kafka.tools.ConsumerOffsetChecker) has been deprecated. Going forward, please use kafka-consumer-groups.sh (kafka.admin.ConsumerGroupCommand) for this functionality. </li>
+    <li> The kafka.tools.ProducerPerformance class has been deprecated. Going forward, please use org.apache.kafka.tools.ProducerPerformance for this functionality (kafka-producer-perf-test.sh will also be changed to use the new class). </li>
 </ul>
 
 <h4>Upgrading from 0.8.1 to 0.8.2.0</h4>