root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity
15,214 Commits 84 Branches 277 Tags 600 MiB
Commit Graph

5638 Commits

Author SHA1 Message Date
Joel Jacob Koshy 190bd3ff71 KAFKA-385 Fix race condition between checkSatisfied and expire in RequestPurgatory; fixed constant expiration of follower fetch requests as checkSatisfied was not getting called; add metrics to the RequestPurgatory; add a KafkaTimer convenience class; patched by Joel Koshy; reviewed by Jun Rao and Jay Kreps. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1374069 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-16 21:26:40 +00:00
Jun Rao 0fe89f7a49 KafkaController.RequestSendThread can throw exception on broker socket; patched by Yang Ye; reviewed by Jun Rao; KAFKA-459, KAFKA-460 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373448 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-15 15:07:41 +00:00
Edward Jay Kreps 9eb99fde9a KAFKA-323 Add more parameters to the log4j appender. Patch by Jose Quinteiro. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373067 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 20:35:48 +00:00
Edward Jay Kreps a8790784df KAFKA-439 Fix bad javadoc. Patch from Jim Plush. Reviewed and modified by Jay Kreps. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373052 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 19:47:54 +00:00
Jun Rao ee855fe1aa remove ZK dependency on producer; patched by Yang Ye; reviewed by Jun Rao; KAFKA-369 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1372724 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-14 04:17:27 +00:00
Neha Narkhede b995133387 Minor checkin to fix replication system test and jmx tool 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1370941 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-08 20:17:31 +00:00
Jun Rao 0223fd8270 recommit: revisit the become leader and become follower state change operations using V3 design; patched by Yang Ye; reviewed by Neha Narkhede and Jun Rao; kafka-343 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1368092 13f79535-47bb-0310-9956-ffa450edef68
 2012-08-01 16:13:59 +00:00
Neha Narkhede 96ed9af620 KAFKA-384 Fix intermittent unit test failures and remove Thread.sleep statements; patched by Neha Narkhede; reviewed by Joel Koshy, Jun Rao and Jay Kreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367821 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 23:27:11 +00:00
Jun Rao c21a7979c1 revert commit to KAFKA-343 due to unit test failures 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367811 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 22:50:59 +00:00
Jun Rao 2a9d42c243 revisit the become leader and become follower state change operations using V3 design; patched by Yang Ye; reviewed by Neha Narkhede and Jun Rao; kafka-343 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367619 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-31 15:33:29 +00:00
Jun Rao 19fbdb3099 deal with empty TopicData list in producer and fetch request; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-412 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1366244 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-27 00:16:08 +00:00
Jun Rao 86de21a71c NullPointerException in ConsumerFetcherManager; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-418 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1366238 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-26 23:56:59 +00:00
Neha Narkhede b49de724f1 KAFKA-405 Improve high watermark maintenance to store high watermarks for all partitions in a single .highwatermark file; patched by Neha Narkhede; reviewed by Jay Kreps and Jun Rao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1365841 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-25 23:42:07 +00:00
Neha Narkhede 842da7bfcd KAFKA-350 Enable message replication in the presence of failures; patched by Neha Narkhede; reviewed by Jun Rao and Jay Kreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1365199 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-24 18:13:01 +00:00
Neha Narkhede c0e6ca95bf Minor change: Deleting the doc for kafka.producer.Producer's constructor. It is causing NoSuchElementException causing jenkins build to fail 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1364001 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-20 22:36:19 +00:00
Edward Jay Kreps 16b0f44559 KAFKA-371 Refactoring of LogManager. Reviewed by Neha. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363542 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-19 21:02:15 +00:00
Neha Narkhede 9e44db6af0 KAFKA-352 Unknown topic error code handling for all requests; patched by Neha Narkhede; reviewed by Jay Kreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363507 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-19 19:51:12 +00:00
Edward Jay Kreps 0d0f1fcfb3 KAFKA-371 Patch from Jonathan Creasy reviewed by me. Correctly handle the case of an empty string as topic name or invalid partition number. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363022 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-18 17:16:28 +00:00
Jun Rao 89661d3b67 ZookeeperConsumerConnector needs to connect to new leader after leadership change; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-362 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1360261 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-11 16:16:26 +00:00
Jun Rao 3bade0b5c8 broker failure system test broken on replication branch; patched by John Fung; reviewed by Joel Koshy and Jun Rao; KAFKA-306 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1359813 13f79535-47bb-0310-9956-ffa450edef68
 2012-07-10 18:08:07 +00:00
Joel Jacob Koshy e7efee02c1 KAFKA-353 Tie producer-side ack with high watermark and progress of replicas; patched by Joel Koshy; reviewed by Jun Rao, Jay Kreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1355578 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-29 22:05:57 +00:00
Jun Rao cb5be0aff1 Consumer doesn't receive all data; patched by Jun Rao; reviewed by John Fung; KAFKA-372 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1354094 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-26 16:23:14 +00:00
Jun Rao 505cf9d618 using MultiFetch in the follower; patched by Jun Rao; reviewed by Joel Koshy and Neha Narkhede; KAFKA-339 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1353086 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-23 07:14:24 +00:00
Jun Rao 0efd609fe7 Exception java.util.NoSuchElementException: None.get appears inconsistently; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-370 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1352980 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-22 17:31:38 +00:00
Jun Rao 1a8511684c upgrade ZKClient to allow conditional updates in ZK; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-337 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351546 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-19 00:32:18 +00:00
Prashanth Menon eff68ce1d0 Create topic support (revisit based on v3 design); patched by Prashanth Menon; reviewed by Jun Rao; KAFKA-329 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351188 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-18 01:17:44 +00:00
Joe Stein efdc57bc58 KAFKA-348 rebase branch from trunk patch by Jun Rao reviewed by Joe Stein 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351112 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-17 15:32:29 +00:00
Jun Rao 83c82a3ecc embedded controller; patched by Yang Ye; reviewed by Jun Rao; KAFKA-335 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1350291 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-14 15:17:35 +00:00
Jun Rao 7d3a782e50 trivial change to add some logging info in DefaultEventHandler 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1350046 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-13 22:36:20 +00:00
Jun Rao f99177766a Add admin RPC requests; clean up Response objects; patched by Yang Ye; reviewed by Jun Rao; KAFKA-349; KAFKA-336 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1346697 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-06 01:28:46 +00:00
Neha Narkhede 045f4fdca4 KAFKA-46: Message replication feature without failures; patched by Neha Narkhede; reviewed by Jun Rao, Jay Kreps, Prashanth Menon 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1344964 13f79535-47bb-0310-9956-ffa450edef68
 2012-06-01 01:53:19 +00:00
Joe Stein 822963bb9d KAFKA-348 merge trunk to branch 1239902:1310937 patch by Joe Stein reviewed by Jun Rao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1344526 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-31 01:51:23 +00:00
Prashanth Menon 904708fae8 Fix retry logic for producers; patched by Prashanth Menon; reviewed by Jun Rao, Neha Narkhede; KAFKA-49 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1343255 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-28 13:45:51 +00:00
Jun Rao 23b422bc44 upgrade ZKClient to allow conditional updates in ZK; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-337 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1342487 13f79535-47bb-0310-9956-ffa450edef68
 2012-05-25 01:49:22 +00:00
Edward Jay Kreps 1bedde76c2 KAFKA-48 Patch to add "long poll" support to fetch requests. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1332413 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-30 21:34:49 +00:00
Neha Narkhede d73355017a KAFKA-301 Implement broker startup procedure; patched by Neha Narkhede; reviewed by Jun Rao and Jay Kreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1329509 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-23 23:32:59 +00:00
Neha Narkhede 79a3b31f26 KAFKA-320 testZKSendWithDeadBroker fails intermittently due to ZKNodeExistsException; patched by nehanarkhede; reviewed by junrao and prashanth menon 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1310661 13f79535-47bb-0310-9956-ffa450edef68
 2012-04-07 02:12:26 +00:00
Jun Rao a260bfacae SyncProducer does not correctly timeout; patched by Prashanth Menon; reviewed by Jun Rao, Neha Narkhede; KAFKA-305 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1305454 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-26 17:12:30 +00:00
Neha Narkhede fedb9d6c19 KAFKA-307 Refactor server code to reduce interdependencies; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1304473 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-23 16:46:03 +00:00
Neha Narkhede 854fb29c26 KAFKA-300 Leader election; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1303473 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-21 17:29:32 +00:00
Jun Rao 4dd7f95569 Add acknowledgement to the produce request; patched by Prashanth Menon; reviewed by Jun Rao; KAFKA-49 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1300435 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-14 01:35:46 +00:00
Joe Stein 88660999a8 KAFKA-240 missing producer response 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1297593 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-06 17:45:17 +00:00
Joe Stein f8042d5b39 KAFKA-240 ProducerRequest wire format protocol update and related changes 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1296577 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-03 05:46:43 +00:00
Neha Narkhede 9fd2d4ffdb KAFKA-239 Refactoring code to wire new ZK data structures and making partitions logical; patched by Neha Narkhede; reviewed by Jun Rao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1295861 13f79535-47bb-0310-9956-ffa450edef68
 2012-03-01 21:15:26 +00:00
Jun Rao 6e36047fae disable dup topic in consumer request; patched by Prashanth Menon; reviewed by Jun Rao; KAFKA-240 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1244650 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-15 19:08:08 +00:00
Jun Rao a5fb217293 new consumer request format; patched by Prashanth Menon; reviewed by Jun Rao and Jay Kreps; KAFKA-240 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1243407 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-13 03:58:37 +00:00
Jun Rao 21da30456f refactor the async producer; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-253 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1241754 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-08 02:59:54 +00:00
Neha Narkhede e3b529df80 Merging commits 1230840:1239902 from trunk 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1239937 13f79535-47bb-0310-9956-ffa450edef68
 2012-02-02 23:40:39 +00:00
Neha Narkhede af0ceff79b KAFKA-238 getTopicMetadata API; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1235492 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-24 20:54:11 +00:00
Jun Rao ff668f4fc9 create/delete ZK path for a topic in an admin tool; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-237 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1231400 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 23:48:27 +00:00
Neha Narkhede cd91e6770b Checking in some files left from the last checkin for KAFKA-202 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1231298 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 21:15:29 +00:00
Neha Narkhede 60d88a3ac9 KAFKA-202 Make request processing in kafka asynchronous; patched by jaykreps; reviewed by junrao and nehanarkhede 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1230845 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-13 00:03:02 +00:00
Jun Rao 7bd3e169f1 trivial fix to make output of ConsumerOffsetChecker sorted 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1229876 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-11 02:53:04 +00:00
Neha Narkhede a3c742ef23 KAFKA-245 Pull zkclient jar from Maven; patched by pyritschard; reviewed by nehanarkhede 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1229794 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-10 23:17:45 +00:00
Neha Narkhede b0484eccf3 KAFKA-228 Reduce duplicate messages served by the kafka consumer for uncompressed topics; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1228504 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-07 00:01:25 +00:00
Jun Rao 66be5eedf3 Support configurable send / receive socket buffer size in server; patched by John Fung; reviewed by Jun Rao; KAFKA-200 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1228480 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-06 22:50:29 +00:00
Neha Narkhede 336410a777 KAFKA-227 Broker failure system test; patched by johnfung; reviewed by nehanarkhede 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1227916 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-06 00:43:46 +00:00
Jun Rao aac7668195 Make backoff time during consumer rebalance configurable; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-234 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1227417 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-05 01:12:43 +00:00
Jun Rao a7d4c2cf23 avoid logging stacktrace directly; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-231 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1226599 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-03 01:55:23 +00:00
Edward Jay Kreps 5eb4badfc9 KAFKA-229 Patch from Joe Stein. Log full exception stack trace instead of just message. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1226549 13f79535-47bb-0310-9956-ffa450edef68
 2012-01-02 20:51:36 +00:00
Jun Rao e5a4ec1806 SyncProducer connect may return failed connection on reconnect; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-226 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1215027 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-16 02:21:02 +00:00
Jun Rao 31f6f02aa2 Bug in mirroring code causes mirroring to halt; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-225 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1213990 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 23:55:52 +00:00
Jun Rao bbba103d76 Embedded consumer doesn't shut down if the server can't start; patched by Jun Rao; reviewed by Neha Narkhede and Jay Kreps; KAFKA-197 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1213546 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-13 03:03:38 +00:00
Jun Rao 9a3e2104b5 trivial change to catch all throwables in ProducerSendThread 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1211747 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-08 02:38:46 +00:00
Neha Narkhede f53eb16f87 KAFKA 222 Mavenize contrib;patched by nehanarkhede; reviewed jakobhoman and junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1211616 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-07 20:14:15 +00:00
Neha Narkhede 07611f3174 KAFKA:212 IllegalThreadStateException in kafka mirroring;patched by nehanarkhede; reviewed by junrao and jaykreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1208919 13f79535-47bb-0310-9956-ffa450edef68
 2011-12-01 00:46:25 +00:00
Jun Rao 078bd591e1 make # of consumer rebalance retries configurable; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-213 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1207645 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 21:56:14 +00:00
Jun Rao cb2564c45e SyncProducer should log host and port if can't connect; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-214 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1207642 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-28 21:53:13 +00:00
Jun Rao c65e771d47 BoundedByteBufferReceive hides OutOfMemoryError; patched by Chris Burroughs; reviewed by Neha Narkhede; KAFKA-204 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205663 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 23:41:23 +00:00
Jun Rao eefd133656 revert previous commit r1205659 because of wrong commit message 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205662 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 23:40:23 +00:00
Jun Rao 285acbec41 AsyncProducerStats is not a singleton; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-207 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205659 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 23:25:14 +00:00
Jun Rao e98bf881a5 javaapi ZookeeperConsumerConnectorTest duplicates many tests in the scala version; patched by Jun Rao; reviewed by Kay Kreps; KAFKA-210 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205535 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 18:43:48 +00:00
Neha Narkhede 56b48d3fb6 KAFKA-176 Fix existing perf tools; patched by nehanarkhede; reviewed by junrao and jaykreps 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205525 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 18:19:06 +00:00
Edward Jay Kreps cb4486656e KAFKA-171 Do a single write for request sends. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205313 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 07:29:37 +00:00
Edward Jay Kreps 1280b71a35 KAFKA-193 Add a logging helper trait and use it where logging is used. Patch from Joe Stein. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1205311 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-23 07:21:16 +00:00
Jun Rao 09f5ce21b3 AsyncProducerStats is not a singleton; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-207 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1204768 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 00:58:39 +00:00
Jun Rao af923c3018 Avoid creating a new topic by the consumer; patched by Taylor Gautier; reviewed by Jun Rao; KAFKA-101 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1204764 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-22 00:35:09 +00:00
Jun Rao c51b940179 Add Snappy Compression as a Codec; patched by Joe Stein; reviewed by Neha Narkhede and Jun Rao; KAFKA-187 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1202045 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-15 05:02:08 +00:00
Jun Rao 0153d4bb59 change ProducerShell to use high level producer; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-195 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1198849 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-07 18:11:10 +00:00
Neha Narkhede 4f9000c412 KAFKA-192 CompressionUtilTest does not run and fails when it does;patched by joestein;reviewed by nehanarkhede 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1198536 13f79535-47bb-0310-9956-ffa450edef68
 2011-11-06 19:07:20 +00:00
Neha Narkhede d9a8af7782 KAFKA-161 Producer using broker list does not load balance requests across multiple partitions on a broker; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1188333 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 19:41:35 +00:00
Edward Jay Kreps 41f14a3e96 KAFKA-166 Add JMX perf tool. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1188288 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 18:52:12 +00:00
Edward Jay Kreps 76957e5e3a KAFKA-150 Make the error message for duplicate node ids in zk more self-explanatory. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1188255 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-24 17:47:20 +00:00
Jun Rao 81f8d0dd83 ZK consumer gets into infinite loop if a message is larger than fetch size; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-160 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1186570 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-19 23:52:58 +00:00
Jun Rao c0fc483d9f ZK consumer may lose a chunk worth of message during rebalance; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-154 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1182028 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 19:11:38 +00:00
Jun Rao 84c06d4285 Include payload size in DumpLogSegments; patched by Chris Burroughs; reviewed by Jun Rao; KAFKA-152 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1181329 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-11 01:10:37 +00:00
Neha Narkhede fa09aacdbd KAFKA-141 Check and make sure that the files that have been donated have been updated to reflect the new ASF copyright;patched by nehanarkhede; reviewd by jjkoshy 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1180185 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 19:51:28 +00:00
Neha Narkhede f34dfc002d Reverting accidental commit 1180110, which is pending review for KAFKA-141 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1180117 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 17:33:43 +00:00
Neha Narkhede a99becbe49 Minor patch, removing the .gitigore file 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1180110 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-07 17:18:56 +00:00
Neha Narkhede 4de899c24d KAFKA-89 consumer should initialize to a valid offset;patched by junrao; reviewed by jjkoshy 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179796 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 19:57:38 +00:00
Neha Narkhede 101af58f6c KAFKA-141; Added the ASF header to the kafka source and test files where it was missing, deleted the CONTRIBUTORS file to match the convention of other Apache projects; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179788 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 19:45:38 +00:00
Jun Rao 0a9aea7160 add optional mx4j support; patched by Chris Burroughs; reviewed by Jun Rao; KAFKA-78 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179501 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 05:10:09 +00:00
Jun Rao 881eaf14e5 trivial change to print out last offset in DumpLogSegments 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179500 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-06 05:04:40 +00:00
Neha Narkhede 4f28442864 KAFKA-145 Kafka server mirror shutdown bug; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179466 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 22:48:10 +00:00
Neha Narkhede bac6d9c3fa KAFKA-146 testUnreachableServer sporadically fails; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1179043 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-05 02:49:56 +00:00
Neha Narkhede dd62e77f90 KAFKA-129 ZK-based producer can throw an unexpceted exception when sending a message; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1178997 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 22:43:00 +00:00
Chris Burroughs a7b4731947 obsessive compulsive tag team: Replace tabs with spaces 
patch by jkreps; reviewed by cburroughs for KAFKA-114

git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1178670 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 01:41:25 +00:00
Edward Jay Kreps 59a5c3215a KAFKA-130 Add a "console producer" that sends messages from standard input 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1178659 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-04 00:54:14 +00:00
Edward Jay Kreps 031a559e89 KAFKA-125 On Linux, the test can throw SocketException instead of EOFException, that is cool too. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1178553 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 20:11:36 +00:00
Jun Rao 596c8d1af7 JMX bean that reports #message/sec in consumer; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-136 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1178298 13f79535-47bb-0310-9956-ffa450edef68
 2011-10-03 04:57:06 +00:00
Jun Rao efc7a8a0cd Expose total metrics through MBeansp; patched by Pierre-Yves Ritschard; reviewed by Jun Rao; KAFKA-140 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1177798 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 19:36:07 +00:00
Jun Rao da35513245 trival fix to make hash code positive; patched by Joel Koshy; reviewed by Jun Rao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1177516 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-30 05:44:13 +00:00
Jun Rao 1b0feb6345 Bug in the queue timeout logic of the async producer; patched by Neha Narkhede; reviewed by Jun Rao; KAFKA-138 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1176672 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 00:51:51 +00:00
Jun Rao 4092ceeb5a Consumer needs a pluggable decoder; patched by Joel Koshy; reviewed by Jun Rao; KAFKA-3 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1176671 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-28 00:42:30 +00:00
Jun Rao 5fbac30f86 Log flush should complete upon broker shutdown; patched by Joel Koshy; reviewed by Jun Rao; KAFKA-126 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1175966 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-26 17:34:48 +00:00
Edward Jay Kreps 8efb536aff KAFKA-132 Patch from Scott Andreas to avoid flush when there is nothing to flush. Looks like due to atime attribute in the filesystem this can lead to flushing file metadata. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1171889 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-17 03:31:15 +00:00
Edward Jay Kreps 54f49314af KAFKA-70 Patch from Prashanth Menon to add space-based retention setting. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1171886 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-17 03:21:06 +00:00
Jun Rao 6af334d69a Tool to watch consumer offsets and lag; patched by Joel Joshy; reviewed by Jun Rao; KAFKA-127 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1167405 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-09 23:49:45 +00:00
Jun Rao c8e507aba4 fix string formatting; write data to stdout in ConsumerShell 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1164353 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-02 05:19:15 +00:00
Jun Rao ec21a5e841 DumpLogSegments outputs wrong offsets; kafka-128 patched by junrao; reviewed by Joel 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1163929 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-01 07:49:31 +00:00
Edward Jay Kreps c247dfa884 KAFKA-124 Console consumer does not stop consuming if the program reading from standard out dies. Check for errors on the output stream and exit if no one is listening. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1163911 13f79535-47bb-0310-9956-ffa450edef68
 2011-09-01 05:47:21 +00:00
Chris Burroughs 92b1dd9252 Enable logging for tests, at a reasonable verbosity 
patch by cburroughs; reviewed by junrao for KAFKA-113

git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1163693 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-31 16:53:38 +00:00
Jun Rao d72b388236 more info in DumpLogSegments; KAFKA-122 ; patched by Jun; reviewed by Neha 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1161664 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-25 17:47:03 +00:00
Neha Narkhede b5ea7c9055 A new file part of KAFKA-116 was accidentally not checked in. This commit fixes it; patched by nehanarkhede; reviewed by chrisburroughs 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1161276 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-24 21:10:03 +00:00
Neha Narkhede 72544aa51e Code clean up in FetcherRunnable and ZookeeperConsumerConnector; KAFKA-120; patched by junrao; reviewed by nehanarkhede 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160953 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-24 01:33:59 +00:00
Neha Narkhede bf30ae996f The FetcherRunnable busy waits on empty fetch requests; KAFKA-117; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160952 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-24 01:27:42 +00:00
Neha Narkhede 4cdc982e22 Producer performance tool should use the new blocking async producer instead of the sleep timeout hack; KAFKA-118; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160947 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-24 00:50:28 +00:00
Neha Narkhede c79c4f6f94 AsyncProducer shutdown logic causes data loss; KAFKA-116; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160946 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-24 00:44:22 +00:00
Chris Burroughs e311cbb6bb Explicitly list ZooKeeper as a dependency instead of including jar in-tree 
patch by cburroughs; reviewed by nehanarkhede and hsaputra for KAFKA-90


git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160905 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-23 22:22:13 +00:00
Neha Narkhede 2e05fbfa22 Improve the command line tools in the bin directory to use the compression feature correctly; KAFKA-112; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160529 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-23 02:36:24 +00:00
Neha Narkhede a484c1bcd1 Kafka server access log does not log request details coming from a MultiProduceRequest; KAFKA-115; patched by nehanarkhede; reviewed by junrao 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1160527 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-23 02:31:56 +00:00
Edward Jay Kreps 4a688c5d6c KAFKA-99 Enforce a max request size in socket server to avoid running out of memory with very large requests. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159837 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-20 04:08:06 +00:00
Neha Narkhede 4f56e44100 ByteBufferMessageSet iterator bug returning incorrect offsets after reading a compressed empty message set KAFKA-111; patched by Jun; reviewed by Neha 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159466 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-19 01:09:26 +00:00
Neha Narkhede 76ccf2211d Bug in the collate logic of the DefaultEventHandler dispatches empty list of messages using the producer KAFKA-110; patched by Neha; reviewed by Jun 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159465 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-19 01:00:07 +00:00
Neha Narkhede 28984e6c18 Some new unit tests for ByteBufferMessageSet iterator KAFKA-108; patched by Jun; reviewed by Neha 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159461 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-19 00:51:20 +00:00
Neha Narkhede 900496eeda CompressionUtils introduces a GZIP header while compressing empty message sets KAFKA-109; patched by Neha; reviewed by Jun 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159459 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-19 00:47:34 +00:00
Neha Narkhede 8a07bdb71e Bug in serialize and collate logic in the DefaultEventHandler KAFKA-107; patched by Neha; reviewed by Jun 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1159452 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-19 00:29:09 +00:00
Jun Rao 82b42163a5 change whitelist config for mirroring; patched by Joel; KAFKA-103 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1158969 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-17 23:34:08 +00:00
Edward Jay Kreps 6d1b6f2827 KAFKA-98 Fix hardcoded ports in unit tests so you can run them while running kafka. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1157922 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-15 17:34:24 +00:00
Edward Jay Kreps 7976993f9e KAFKA-102 Patch from Blake Matheny. Clean up logging. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1157920 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-15 17:31:35 +00:00
Jun Rao 944da579ba KafkaServer can throw a NullPointerException during startup if zookeeper is down; KAFKA-94 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1156940 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-12 02:09:56 +00:00
Jun Rao 49f3b40179 auto-discovery of topics for mirroring; patched by Joel; reviewed by Jun; KAFKA-74 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1156393 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-10 22:32:23 +00:00
Henry Saputra d0c980dab2 KAFKA-93 | add license to missed files and remove LinkedIn copyright line per ASF guideline. Thanks to Joel Koshy for pointing it out 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1156299 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-10 18:32:09 +00:00
Henry Saputra 462dfda513 KAFKA-93 | Change and add ASF source header to follow standard ASF source header (http://www.apache.org/legal/src-headers.html). 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1156232 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-10 15:33:21 +00:00
Jun Rao f7046c247d Options in SyncProducerConfig and AsyncProducerConfig can leak, KAFKA-83 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1154804 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-07 23:56:05 +00:00
Jun Rao 96b8e03dd1 commit offset before consumer shutdown KAFKA-84; rename util.StringSerializer to ZKStringSerializer to avoid confusion with producer.StringSerializer 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1154719 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-07 16:29:55 +00:00
Jun Rao 665f1d0527 upgrade zkclient jar; #KAFKA-82 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1154064 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-05 01:36:43 +00:00
Edward Jay Kreps 642da2f28c Initial checkin of Kafka to Apache SVN. This corresponds to 709afe4ec7 except that git specific files have been removed and code has been put into trunk/branches/site/etc. This is just a copy of master, branches and history are not being converted since we can't find a good tool for it. 
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1152970 13f79535-47bb-0310-9956-ffa450edef68
 2011-08-01 23:41:24 +00:00