Commit Graph

12596 Commits

Author SHA1 Message Date
Jun Rao 93e9dbd026 update leaderAndISR in ZK conditionally; patched by Yang Ye; reviewed by Jun Rao; KAFKA-428
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1377157 13f79535-47bb-0310-9956-ffa450edef68
2012-08-25 00:06:33 +00:00
Jun Rao 21326cb9f2 Time based log segment rollout; patched by Swapnil Ghike; reviewed by Jun Rao, Neha Narkhede; KAFKA-475
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1377093 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 20:38:27 +00:00
Jun Rao 9a84e408c1 message size not checked at the server; patched by Swapnil Ghike; reviewed by Jun Rao, Neha Narkhede; KAFKA-469
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1376943 13f79535-47bb-0310-9956-ffa450edef68
2012-08-24 14:32:10 +00:00
Jun Rao 0d6a4dff09 TopicCount.constructTopicCount isn't thread-safe; patched by Jun Rao; reviewed by Joel Koshy and Neha Narkhede; kafka-379
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1376598 13f79535-47bb-0310-9956-ffa450edef68
2012-08-23 17:15:17 +00:00
Neha Narkhede 5861046d09 KAFKA-440 Regression/system test framework; patched by John Fung; reviewed by Neha Narkhede
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1376147 13f79535-47bb-0310-9956-ffa450edef68
2012-08-22 17:16:26 +00:00
Jun Rao 663d254433 refactor ReplicaManager; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-351
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1375670 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 17:17:29 +00:00
Jun Rao 7baccc6c33 remove support for format for magic byte 0 in 0.8; patched by Swapnil Ghike; reviewed by Jun Rao; KAFKA-461
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1375367 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 01:52:50 +00:00
Jun Rao 27aaebc874 Create a generic Kafka thread; patched by Yang Ye; reviewed by Jun Rao, Jay Kreps; KAFKA-56
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1375360 13f79535-47bb-0310-9956-ffa450edef68
2012-08-21 01:10:00 +00:00
Jun Rao be521175dc failed ERROR messages in LazyInitProducerTest; patched by Yang Ye; reviewed by Jun Rao; kafka-467
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1374514 13f79535-47bb-0310-9956-ffa450edef68
2012-08-18 05:26:59 +00:00
Jun Rao 12a3ccafa1 KafkaController NPE in SessionExpireListener; patched by Yang Ye; reviewed by Jun Rao, Neha Narkhede; KAFKA-464
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1374467 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 23:09:05 +00:00
Jun Rao 60735b7ad6 enforce broker.id to be a non-negative integer; patched by Swapnil Ghike; reviewed by Jun Rao, Neha Narkhede; KAFKA-424
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1374440 13f79535-47bb-0310-9956-ffa450edef68
2012-08-17 21:09:15 +00:00
Joel Jacob Koshy 190bd3ff71 KAFKA-385 Fix race condition between checkSatisfied and expire in RequestPurgatory; fixed constant expiration of follower fetch requests as checkSatisfied was not getting called; add metrics to the RequestPurgatory; add a KafkaTimer convenience class; patched by Joel Koshy; reviewed by Jun Rao and Jay Kreps.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1374069 13f79535-47bb-0310-9956-ffa450edef68
2012-08-16 21:26:40 +00:00
Jun Rao 0fe89f7a49 KafkaController.RequestSendThread can throw exception on broker socket; patched by Yang Ye; reviewed by Jun Rao; KAFKA-459, KAFKA-460
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373448 13f79535-47bb-0310-9956-ffa450edef68
2012-08-15 15:07:41 +00:00
Edward Jay Kreps 9eb99fde9a KAFKA-323 Add more parameters to the log4j appender. Patch by Jose Quinteiro.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373067 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 20:35:48 +00:00
Edward Jay Kreps 623e2b7e53 KAFKA-424. Fix command line usage description. Patch from Tommie.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373056 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 19:59:11 +00:00
Edward Jay Kreps d482531810 KAFKA-425 Wrong class name in scripts. Patch from Akira Kitada reviewed by Jay Kreps.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373054 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 19:56:36 +00:00
Edward Jay Kreps a8790784df KAFKA-439 Fix bad javadoc. Patch from Jim Plush. Reviewed and modified by Jay Kreps.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1373052 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 19:47:54 +00:00
Jun Rao ee855fe1aa remove ZK dependency on producer; patched by Yang Ye; reviewed by Jun Rao; KAFKA-369
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1372724 13f79535-47bb-0310-9956-ffa450edef68
2012-08-14 04:17:27 +00:00
Neha Narkhede b995133387 Minor checkin to fix replication system test and jmx tool
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1370941 13f79535-47bb-0310-9956-ffa450edef68
2012-08-08 20:17:31 +00:00
Neha Narkhede 8ceec6eff0 KAFKA-380 Enhance single_host_multi_brokers test with failure to trigger leader re-election in replication; patched by John Fung; reviewed by Neha Narkhede, Jun Rao and Joel Koshy
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1369888 13f79535-47bb-0310-9956-ffa450edef68
2012-08-06 17:27:05 +00:00
Jun Rao 0223fd8270 recommit: revisit the become leader and become follower state change operations using V3 design; patched by Yang Ye; reviewed by Neha Narkhede and Jun Rao; kafka-343
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1368092 13f79535-47bb-0310-9956-ffa450edef68
2012-08-01 16:13:59 +00:00
Neha Narkhede 96ed9af620 KAFKA-384 Fix intermittent unit test failures and remove Thread.sleep statements; patched by Neha Narkhede; reviewed by Joel Koshy, Jun Rao and Jay Kreps
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367821 13f79535-47bb-0310-9956-ffa450edef68
2012-07-31 23:27:11 +00:00
Jun Rao c21a7979c1 revert commit to KAFKA-343 due to unit test failures
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367811 13f79535-47bb-0310-9956-ffa450edef68
2012-07-31 22:50:59 +00:00
Jun Rao 2a9d42c243 revisit the become leader and become follower state change operations using V3 design; patched by Yang Ye; reviewed by Neha Narkhede and Jun Rao; kafka-343
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1367619 13f79535-47bb-0310-9956-ffa450edef68
2012-07-31 15:33:29 +00:00
Jun Rao 19fbdb3099 deal with empty TopicData list in producer and fetch request; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-412
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1366244 13f79535-47bb-0310-9956-ffa450edef68
2012-07-27 00:16:08 +00:00
Jun Rao 86de21a71c NullPointerException in ConsumerFetcherManager; patched by Jun Rao; reviewed by Neha Narkhede; KAFKA-418
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1366238 13f79535-47bb-0310-9956-ffa450edef68
2012-07-26 23:56:59 +00:00
Jun Rao 2e073db8dc single_host_multi_brokers system test fails on laptop; patched by John Fung; reviewed by Jun Rao; kafka-413
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1366085 13f79535-47bb-0310-9956-ffa450edef68
2012-07-26 16:59:06 +00:00
Neha Narkhede b49de724f1 KAFKA-405 Improve high watermark maintenance to store high watermarks for all partitions in a single .highwatermark file; patched by Neha Narkhede; reviewed by Jay Kreps and Jun Rao
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1365841 13f79535-47bb-0310-9956-ffa450edef68
2012-07-25 23:42:07 +00:00
Neha Narkhede 842da7bfcd KAFKA-350 Enable message replication in the presence of failures; patched by Neha Narkhede; reviewed by Jun Rao and Jay Kreps
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1365199 13f79535-47bb-0310-9956-ffa450edef68
2012-07-24 18:13:01 +00:00
Neha Narkhede c0e6ca95bf Minor change: Deleting the doc for kafka.producer.Producer's constructor. It is causing NoSuchElementException causing jenkins build to fail
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1364001 13f79535-47bb-0310-9956-ffa450edef68
2012-07-20 22:36:19 +00:00
Edward Jay Kreps 16b0f44559 KAFKA-371 Refactoring of LogManager. Reviewed by Neha.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363542 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 21:02:15 +00:00
Neha Narkhede 9e44db6af0 KAFKA-352 Unknown topic error code handling for all requests; patched by Neha Narkhede; reviewed by Jay Kreps
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363507 13f79535-47bb-0310-9956-ffa450edef68
2012-07-19 19:51:12 +00:00
Edward Jay Kreps 0d0f1fcfb3 KAFKA-371 Patch from Jonathan Creasy reviewed by me. Correctly handle the case of an empty string as topic name or invalid partition number.
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1363022 13f79535-47bb-0310-9956-ffa450edef68
2012-07-18 17:16:28 +00:00
Jun Rao 89661d3b67 ZookeeperConsumerConnector needs to connect to new leader after leadership change; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-362
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1360261 13f79535-47bb-0310-9956-ffa450edef68
2012-07-11 16:16:26 +00:00
Jun Rao 3bade0b5c8 broker failure system test broken on replication branch; patched by John Fung; reviewed by Joel Koshy and Jun Rao; KAFKA-306
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1359813 13f79535-47bb-0310-9956-ffa450edef68
2012-07-10 18:08:07 +00:00
Jun Rao d79919d827 broker failure system test broken on replication branch; patched by John Fung; reviewed by Joel Koshy and Jun Rao; KAFKA-306
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1359812 13f79535-47bb-0310-9956-ffa450edef68
2012-07-10 18:05:54 +00:00
Joel Jacob Koshy 489b09c120 KAFKA-373 Fix trunk broker failure test to work with mirror maker; patched by John Fung; reviewed by Joel Koshy
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1358407 13f79535-47bb-0310-9956-ffa450edef68
2012-07-06 20:44:51 +00:00
Joel Jacob Koshy e7efee02c1 KAFKA-353 Tie producer-side ack with high watermark and progress of replicas; patched by Joel Koshy; reviewed by Jun Rao, Jay Kreps
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1355578 13f79535-47bb-0310-9956-ffa450edef68
2012-06-29 22:05:57 +00:00
Jun Rao cb5be0aff1 Consumer doesn't receive all data; patched by Jun Rao; reviewed by John Fung; KAFKA-372
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1354094 13f79535-47bb-0310-9956-ffa450edef68
2012-06-26 16:23:14 +00:00
Jun Rao 505cf9d618 using MultiFetch in the follower; patched by Jun Rao; reviewed by Joel Koshy and Neha Narkhede; KAFKA-339
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1353086 13f79535-47bb-0310-9956-ffa450edef68
2012-06-23 07:14:24 +00:00
Jun Rao 0efd609fe7 Exception java.util.NoSuchElementException: None.get appears inconsistently; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-370
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1352980 13f79535-47bb-0310-9956-ffa450edef68
2012-06-22 17:31:38 +00:00
Joe Stein a16448b71e KAFKA-368 use the pig core jar from maven instead of distributing it patch by Joe Stein reviewed by Jun Rao and Neha Narkhede
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1352146 13f79535-47bb-0310-9956-ffa450edef68
2012-06-20 15:04:53 +00:00
Joe Stein 1af23bb263 KAFKA-368 use the pig core jar from maven instead of distributing it patch by Joe Stein reviewed by Jun Rao and Neha Narkhede
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1352145 13f79535-47bb-0310-9956-ffa450edef68
2012-06-20 15:04:04 +00:00
Jun Rao 1a8511684c upgrade ZKClient to allow conditional updates in ZK; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-337
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351546 13f79535-47bb-0310-9956-ffa450edef68
2012-06-19 00:32:18 +00:00
Jun Rao 16b600d4fd add jmx beans in broker to track # bytes in consumer; patched by Jun Rao; reviewed by Joel Koshy; KAFKA-336
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1351544 13f79535-47bb-0310-9956-ffa450edef68
2012-06-19 00:27:00 +00:00
Prashanth Menon eff68ce1d0 Create topic support (revisit based on v3 design); patched by Prashanth Menon; reviewed by Jun Rao; KAFKA-329
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351188 13f79535-47bb-0310-9956-ffa450edef68
2012-06-18 01:17:44 +00:00
Joe Stein efdc57bc58 KAFKA-348 rebase branch from trunk patch by Jun Rao reviewed by Joe Stein
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1351112 13f79535-47bb-0310-9956-ffa450edef68
2012-06-17 15:32:29 +00:00
Joe Stein f8ce071f3f KAFKA-365 change copyright in NOTICE to current year, reviewed by Jun Rao
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/trunk@1350349 13f79535-47bb-0310-9956-ffa450edef68
2012-06-14 17:47:20 +00:00
Joe Stein ebcf7fc10d KAFKA-365 change copyright in NOTICE to current year, reviewed by Jun Rao
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1350349 13f79535-47bb-0310-9956-ffa450edef68
2012-06-14 17:47:20 +00:00
Jun Rao 965edbbcd8 system test to validate consistency of replicas; patched by John Fung; reviewed by Jun Rao; KAFKA-341
git-svn-id: https://svn.apache.org/repos/asf/incubator/kafka/branches/0.8@1350316 13f79535-47bb-0310-9956-ffa450edef68
2012-06-14 16:23:30 +00:00