Commit Graph

29 Commits

Author SHA1 Message Date
Jason Gustafson 6dc9743125 KAFKA-2886: Handle sink task rebalance failures by stopping worker task
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Liquan Pei <liquanpei@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #767 from hachikuji/KAFKA-2886
2016-01-15 09:28:43 -08:00
Jason Gustafson 58def1cba0 MINOR: speed up connect startup when full connector class name is provided
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Liquan Pei <liquanpei@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #746 from hachikuji/worker-startup-improvement
2016-01-08 15:04:27 -08:00
Gwen Shapira b93f48f749 KAFKA-2422: Allow copycat connector plugins to be aliased to simpler names
…names

Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #687 from gwenshap/KAFKA-2422
2016-01-04 15:01:58 -05:00
Kishore Senji 2679524604 KAFKA-3055; Fix JsonConverter mangling the Schema in Connect
Author: ksenji <ksenji@ebay.com>

Reviewers: Dong Lin <lindong28@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #722 from ksenji/trunk
2016-01-04 11:47:31 -05:00
glikson bde49e3711 MINOR: Use "new line" delimeters in a portable way in a unit test
This fixes failure of the FileStreamSinkTaskTest unit test on Windows

Signed-off-by: glikson <gliksonil.ibm.com>

Author: glikson <glikson@il.ibm.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #708 from glikson/file-ut-newline
2015-12-22 17:07:15 -08:00
manasvigupta a0d21407cb KAFKA-3009; Disallow star imports
Summary of code changes
------------------------------------
1) Added a new Checkstyle rule to flag any code using star imports
2) Fixed ALL existing code violations using star imports

Testing
-----------
Local build was successful
ALL JUnits ran successfully on local.

ewencp - Request you to please review changes. Thank you !

I state that the contribution is my original work and I license the work to the project under the project's open source license.

Author: manasvigupta <manasvigupta@yahoo.co.in>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #700 from manasvigupta/KAFKA-3009
2015-12-21 13:30:59 -08:00
Ismael Juma 8c754c45af MINOR: Change return type of `Schema.read` to be `Struct` instead of `Object`
We always return a `Struct` from `Schema.read` and this means that
we can remove a large number of casts.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #684 from ijuma/schema-read-should-return-struct
2015-12-18 10:27:03 -08:00
Gwen Shapira fdac34e6ab MINOR: Updating comment that fell out of sync with code
Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #681 from gwenshap/minor-herder-comment
2015-12-15 16:18:39 -08:00
Vahid Hashemian 3f3358b6d4 MINOR: Fix typos in code comments
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Gwen Shapira

Closes #673 from vahidhashemian/typo02/fix_typos_in_code_comments
2015-12-15 13:46:57 -08:00
Edward Ribeiro 6e5bd2497a KAFKA-2974; `==` is used incorrectly in a few places in Java code
A few issues found via static analysis.

Author: Edward Ribeiro <edward.ribeiro@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Gwen Shapira, Sriharsha Chintalapani, Guozhang Wang

Closes #652 from ijuma/use-equals-instead-of-==
2015-12-09 20:34:09 -08:00
Dong Lin ef92a8ae74 KAFKA-2668; Add a metric that records the total number of metrics
onurkaraman becketqin Do you have time to review this patch? It addresses the ticket that jjkoshy filed in KAFKA-2668.

Author: Dong Lin <lindong28@gmail.com>

Reviewers: Onur Karaman <okaraman@linkedin.com>, Joel Koshy <jjkoshy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>

Closes #328 from lindong28/KAFKA-2668
2015-12-08 19:43:05 -08:00
Jason Gustafson ee6b5e044c KAFKA-2667; Fix assertion depending on hash map order in KafkaBasedLogTest.testSendAndReadToEnd
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #642 from hachikuji/KAFKA-2667
2015-12-08 17:48:52 -08:00
Jason Gustafson cd54fc8816 KAFKA-2931: add system test for consumer rolling upgrades
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma, Guozhang Wang

Closes #619 from hachikuji/KAFKA-2931
2015-12-03 21:17:51 -08:00
Ewen Cheslack-Postava 75c7abd826 KAFKA-2906: Fix Connect javadocs, restrict only to api subproject, and clean up javadoc warnings.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #599 from ewencp/kafka-2906-connect-javadocs
2015-11-30 05:26:32 +08:00
Ewen Cheslack-Postava 7b3d1bf6a3 KAFKA-2867: Fix missing WorkerSourceTask synchronization and handling of InterruptException.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #566 from ewencp/kafka-2867-fix-source-sync-and-interrupt
2015-11-20 10:04:40 -08:00
Ewen Cheslack-Postava 0f00ec97ae KAFKA-2859: Fix deadlock in WorkerSourceTask.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #554 from ewencp/kafka-2859-deadlock-worker-source-task
2015-11-18 14:19:37 -08:00
Ewen Cheslack-Postava f1169f1da8 KAFKA-2848; Use client SSL/SASL config utilities in Kafka Connect to avoid duplication of configs.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #539 from ewencp/kafka-2848-reuse-ssl-sasl-client-configs
2015-11-16 22:00:34 -08:00
Ewen Cheslack-Postava 1408c670ea KAFKA-2807: Fix Kafka Connect packaging and move VerifiableSource/Sink into runtime jar.
Gradle does not handle subprojects with the same name (top-level tools vs
connect/tools) properly, making the dependency impossible to express correctly
since we need to move the ThroughputThrottler class into the top level tools
project. Moving the current set of tools into the runtime jar works fine since
they are only used for system tests at the moment.

Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #512 from ewencp/kafka-2807-redux
2015-11-12 11:11:56 -08:00
Ewen Cheslack-Postava c6b8de4e68 KAFKA-2807: Move ThroughputThrottler back to tools jar to fix upgrade tests.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #499 from ewencp/kafka-2807-relocate-throughput-throttler
2015-11-11 15:55:12 -08:00
Grant Henke 04827e6e99 KAFKA-2752: Follow up to fix checkstlye
Author: Grant Henke <granthenke@gmail.com>

Reviewers: Ewen Cheslack-Postava

Closes #492 from granthenke/fix
2015-11-10 15:57:02 -08:00
Ewen Cheslack-Postava 8db55618d5 KAFKA-2752: Add VerifiableSource/Sink connectors and rolling bounce Copycat system tests.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Ben Stopford, Geoff Anderson, Guozhang Wang

Closes #432 from ewencp/kafka-2752-copycat-clean-bounce-test
2015-11-10 14:54:15 -08:00
Ewen Cheslack-Postava 403d89edea KAFKA-2798: Use prefixedd configurations for Kafka Connect producer and consumer settings so they do not conflict with the distributed herder's settings.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #486 from ewencp/kafka-2798-conflicting-herder-producer-consumer-configs
2015-11-10 11:07:26 -08:00
Ewen Cheslack-Postava 590a4616a1 KAFKA-2786: Only respond to SinkTask onPartitionsRevoked after the WorkerSinkTask has finished starting up.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #476 from ewencp/kafka-2786-on-partitions-assigned-only-after-start
2015-11-09 20:43:07 -08:00
Ewen Cheslack-Postava 83eaf3284f KAFKA-2379: Add basic documentation for Kafka Connect.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #475 from ewencp/kafka-2379-connect-docs
2015-11-09 16:29:30 -08:00
Ewen Cheslack-Postava 75f2b8c8b4 KAFKA-2782: Fix KafkaBasedLogTest assertion and move it to the main test thread.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #463 from ewencp/kafka-2782-fix-kafka-based-log-test-assertion
2015-11-09 10:39:34 -08:00
Ewen Cheslack-Postava 13ba57dcfc KAFKA-2778: Use zero loss settings by default for Connect source producers.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #459 from ewencp/kafka-2778-connect-source-zero-loss-settings
2015-11-09 10:36:57 -08:00
Ewen Cheslack-Postava bc76e6704e KAFKA-2775: Move exceptions into API package for Kafka Connect.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #457 from ewencp/kafka-2775-exceptions-in-api-package
2015-11-09 10:27:18 -08:00
Ewen Cheslack-Postava e9fc7b8c84 KAFKA-2776: Fix lookup of schema conversion cache size in JsonConverter.
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #458 from ewencp/kafka-2776-json-converter-cache-config-fix
2015-11-09 10:19:27 -08:00
Ewen Cheslack-Postava f2031d4063 KAFKA-2774: Rename Copycat to Kafka Connect
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Gwen Shapira

Closes #456 from ewencp/kafka-2774-rename-copycat
2015-11-08 22:11:03 -08:00