Commit Graph

589 Commits

Author SHA1 Message Date
cmolter 5d81639907 KAFKA-5890; records.lag should use tags for topic and partition rather than using metric name.
This is the implementation of KIP-225.
It marks the previous metrics as deprecated in the documentation and adds new metrics using tags.

Testing verifies that both the new and the old metric report the same value.

Author: cmolter <cmolter@apple.com>

Reviewers: Jiangjie (Becket) Qin <becket.qin@gmail.com>

Closes #4362 from lahabana/kafka-5890
2018-01-14 16:18:39 -08:00
RichardYuSTUG 1d1c857596 4385
Author: RichardYuSTUG <yohan.richard.yu2@gmail.com>
Author: Prasanna Gautam <prasannagautam@gmail.com>
Author: Dong Lin <lindong28@gmail.com>
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #4385 from ConcurrencyPractitioner/doc-kafka-4499
2018-01-12 17:27:52 -08:00
Richard Yu b8aa1761c3 KAFKA-6265: Add #queryableStoreName() to GlobalKTable
A spinoff of original pull request #4340 for resolving conflicts.

Author: RichardYuSTUG <yohan.richard.yu2@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #4413 from ConcurrencyPractitioner/kafka-6265-2
2018-01-11 09:54:02 -08:00
Jun Rao 92529ac6d8
Merge pull request #3567 from bobrik/unclean-docs-clarification
KAFKA-4711: fix docs onunclean.leader.election.enable default
2018-01-10 13:32:16 -08:00
Joel Hamill 06597ba26b MINOR: Menu updates and navigation (#4405)
* Menu updates and navigation

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2018-01-09 16:28:49 -08:00
Jorge Quilcate Otoya 2900af6b33 Updating docs for streams app reset tool (#4401)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2018-01-08 12:02:29 -08:00
Joel Hamill 26270983cd MINOR: Fix menu ordering in streams docs
https://issues.apache.org/jira/browse/KAFKA-6419

Related: https://github.com/apache/kafka-site/pull/115

Author: Joel Hamill <joel-hamill@users.noreply.github.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4390 from joel-hamill/fix-streams-menu-order
2018-01-08 11:43:38 -08:00
Joel Hamill d63f94797c MINOR: fix image 404s in streams doc
Fixes image 404s in streams docs (e.g. https://kafka.apache.org/documentation/streams/developer-guide/interactive-queries.html).

Related https://github.com/apache/kafka-site/pull/114

Author: Joel Hamill <joel-hamill@users.noreply.github.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4389 from joel-hamill/fix-404-streams
2018-01-08 09:38:58 -08:00
Matt Farmer e1c5d0c119 MINOR: Add documentation for KAFKA-6086 (ProductionExceptionHandler) (#4395)
* Update streams documentation to describe production exception handler

* Add a mention of the ProductionExceptionHandler in the upgrade guide
2018-01-08 11:33:23 +00:00
Guozhang Wang f8593b0958 Fix wrong property mentioned in doc; Author: Praveen K Palaniswamy <yourspraveen@gmail.com> 2018-01-02 10:07:54 -08:00
Guozhang Wang 5995453f4c MINOR: Web docs for KIP-220
1. added functions for KafkaStreams and KafkaClientSupplier.
2. added prefix for admin client in StreamsConfig.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>, Matthias J. Sax <matthias@confluent.io>

Closes #4338 from guozhangwang/K6150-doc-changes
2017-12-22 11:53:04 -08:00
Joel Hamill 3e2fe17c08 MINOR: Improve Streams Dev Guide content on web docs
This PR migrates content from CP Streams Dev Guide.

Here is the top-level page:
![image](https://user-images.githubusercontent.com/11722533/33904945-df9cf804-df31-11e7-93aa-52385961522c.png)

Here is a child page:
![image](https://user-images.githubusercontent.com/11722533/33904976-f2eafabe-df31-11e7-918c-fbf95db0f76b.png)

See related: https://github.com/apache/kafka-site/pull/112

Author: Joel Hamill <joel-hamill@users.noreply.github.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4252 from joel-hamill/20171122-migrate-cp-dev-guide
2017-12-21 11:15:54 -08:00
Rajini Sivaram 35c1be7469 KAFKA-5746; Document new broker metrics added for health checks
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4026 from rajinisivaram/MINOR-KIP-188-metrics-docs
2017-12-20 13:37:42 +00:00
Ismael Juma 88c2b6849a KAFKA-6317; Maven artifact for kafka should not depend on log4j
It should only depend on slf4j-api (like kafka-clients). The
release tarball still includes log4j and slf4j-log4j12.

Manually verified that there are no duplicate dependencies
in the release tarball and `./gradlew core:dependencies`
looks good.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>

Closes #4297 from ijuma/kafka-6317-kafka-slf4j-api-only
2017-12-19 15:15:27 +00:00
Manikumar Reddy b088822134 KAFKA-5764; Add toLowerCase support to sasl.kerberos.principal.to.local rule (KIP-203)
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #3800 from omkreddy/KAFKA-5764-REGEX
2017-12-12 15:47:57 -08:00
Guozhang Wang 72eec0a04c KAFKA-6334: minor typo fix in web docs 2017-12-11 18:55:13 -08:00
Dong Lin 078fd21365 KAFKA-6174; Add methods in AdminClient Options classes to restore binary compatibility with 0.11
From 0.11 to 1.0, we moved `DescribeClusterOptions timeoutMs(Integer timeoutMs)` from
DescribeClusterOptions to AbstractOptions (similarly for other Options classes). This can
cause code compiled against 0.11.0.x to fail when it is executed with 1.0 kafka-clients jar.
This patch adds back these methods to restore binary compatibility with 0.11.

Author: Dong Lin <lindong28@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4257 from lindong28/KAFKA-6174
2017-12-06 15:56:56 +02:00
Jason Gustafson 062c5568e1 KAFKA-6238; Fix 1.0.0 upgrade instructions relating to the message format version
The upgrade instructions concerning the message format versions did not account
for upgrades from versions prior to 0.11.0.x.

Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4256 from hachikuji/KAFKA-6328
2017-11-23 14:46:21 +00:00
Jeff Klukas 049342e440 KAFKA-3073: Add topic regex support for Connect sinks
There are more methods that had to be touched than I anticipated when writing [the KIP](https://cwiki.apache.org/confluence/display/KAFKA/KIP-215%3A+Add+topic+regex+support+for+Connect+sinks).

The implementation here is now complete and includes a test that verifies that there's a call to `consumer.subscribe(Pattern, RebalanceHandler)` when `topics.regex` is provided.

Author: Jeff Klukas <jeff@klukas.net>

Reviewers: Randall Hauch <rhauch@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #4151 from jklukas/connect-topics.regex
2017-11-21 16:01:16 -08:00
Manjula K ca7f3dab04 MINOR: Adding secondary nav to Streams webpage
Added secondary navigation to Streams sub-pages:
 - quickstart.html
 - core-concepts.html
 - developer-guide.html
 - tutorial.html

Author: Manjula K <manjula@kafka-summit.org>

Reviewers: Joel Hamill <joel-hamill@users.noreply.github.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #4222 from manjuapu/streams-nav
2017-11-15 14:43:11 -08:00
Joel Hamill 48f5f048bc MINOR: kafka-site introduction section improvements
*Clarify multi-tenant support, geo-replication, and some grammar fixes.*

Author: Joel Hamill <joel-hamill@users.noreply.github.com>

Reviewers: GUozhang Wang

Closes #4212 from joel-hamill/intro-cleanup
2017-11-15 14:32:00 -08:00
Adem Efe Gencer 86062e9a78 KAFKA-6157; Fix repeated words words in JavaDoc and comments.
Author: Adem Efe Gencer <agencer@linkedin.com>

Reviewers: Jiangjie Qin <becket.qin@gmail.com>

Closes #4170 from efeg/bug/typoFix
2017-11-05 18:00:43 -08:00
Guozhang Wang 520b313628 HOTFIX: Add missing template ref in upgrade section 2017-11-03 10:10:35 -07:00
Guozhang Wang 487436b1a4 MINOR: Update docs for new version
1. Update the Streams hello world examples with the new API.
2. Update the version references in various places.
3. Update version templates to 1.1.x.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Damian Guy <damian.guy@gmail.com>, Derrick Or <derrickor@gmail.com>

Closes #4169 from guozhangwang/KMINOR-streams-docs
2017-11-03 08:51:44 -07:00
Manjula K 5178702715 MINOR: Adding Trivago logo to Streams landing page
guozhangwang Please review

Author: Manjula K <manjula@kafka-summit.org>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4164 from manjuapu/ny-trivago-logos
2017-10-31 13:22:56 -07:00
Manikumar Reddy 9b53c31d08 MINOR: Update docs wrt topic deletion being enabled by default
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3835 from omkreddy/update-delete-topic-doc
2017-10-27 14:41:46 +01:00
Vahid Hashemian ebe749b4ee MINOR: Improve a Windows quickstart instruction
The output of `wmic` can be very long and could truncate the search keywords in the existing command. If those keywords are truncated no process is returned in the output. An update is suggested to the command by which the query is performed inside the `wmic` command itself instead of using pipes and `find`.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4083 from vahidhashemian/minor/improve_quickstart_for_windows_wmic
2017-10-26 13:46:11 -07:00
Manikumar Reddy adb9d5ae76 MINOR: Add missing semicolon to example jaas configuration
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #4101 from omkreddy/SCRAM-DOCS
2017-10-26 14:24:29 +01:00
Joel Hamill 5f779ca3f3 MINOR: Fix typo dev guide title
related to https://github.com/apache/kafka-site/pull/103

Author: Joel Hamill <git config --global user.email>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4133 from joel-hamill/dev-guide-title
2017-10-25 10:11:09 -07:00
Guozhang Wang ef49145200 MINOR: a few web doc and javadoc fixes
1. Added missing Javadocs in public interfaces.
2. Added missing upgrade web docs.
3. Minor improvements on exception messages.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Damian Guy <damian.guy@gmail.com>, Matthias J. Sax <matthias@confluent.io>, Antony Stubbs <antony.stubbs@gmail.com>

Closes #4071 from guozhangwang/KMinor-javadoc-gaps
2017-10-16 16:50:59 -07:00
Manikumar Reddy 76a45e0dce MINOR: Update `config/consumer.properties` to have new consumer properties
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4055 from omkreddy/update-consumer-props
2017-10-12 14:28:55 +01:00
Manjula K f6e724b127 MINOR: Redesign of Streams page to include video & logos
guozhangwang Please review.

Author: Manjula K <manjula@kafka-summit.org>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4059 from manjuapu/redesign-streams-page
2017-10-11 15:16:12 -07:00
Bill Bejeck dac990aab3 KAFKA-6025: small fix for streams tutorial
Author: Bill Bejeck <bill@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #4053 from bbejeck/KAFKA_6025_fix_streams_tutorial
2017-10-10 15:07:53 -07:00
Guozhang Wang e2e8d4a57a MINOR: KIP-161 upgrade docs change
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #4036 from guozhangwang/KMinor-kip-161-docs
2017-10-06 17:57:32 -07:00
Manikumar Reddy 10cd98cc89 KAFKA-5547; Return TOPIC_AUTHORIZATION_FAILED error if no describe access for topics
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #3924 from omkreddy/KAFKA-5547-TOPIC-AUTHRO
2017-10-06 12:51:30 -07:00
Derrick Or f29203d022 Update docs to reflect kafka trademark status
Updated a couple places in docs with the 'registered' trademark symbol.

Author: Derrick Or <derrickor@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #4028 from derrickdoo/kafka-trademark-status
2017-10-05 13:31:18 -07:00
Joel Hamill ff5fc9dd1e Rename streams tutorial and quickstart
Changed these topic titles:
- Write your own Streams Applications -> Tutorial: Write a Streams Application
- Play with a Streams Application -> Run the Streams Demo Application

Author: Joel Hamill <joel@Joel-Hamill-Confluent.local>
Author: Joel Hamill <11722533+joel-hamill@users.noreply.github.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #4017 from joel-hamill/joel-hamill/streams-titles
2017-10-05 09:30:38 -07:00
Damian Guy 93b71e7dee MINOR: update streams quickstart for KIP-182
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3984 from dguy/quickstart-update
2017-10-04 12:19:40 -07:00
Joel Hamill d985513b22 MINOR: streams dev guide fixup
Author: Joel Hamill <joel@Joel-Hamill-Confluent.local>
Author: Joel Hamill <11722533+joel-hamill@users.noreply.github.com>

Reviewers: Derrick Or <derrickor@gmail.com>, Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3862 from joel-hamill/joel-hamill/streams-dev-guide
2017-10-04 11:28:59 -07:00
Randall Hauch 11afff0990 KAFKA-5990: Enable generation of metrics docs for Connect (KIP-196)
A new mechanism was added recently to the Metrics framework to make it easier to generate the documentation. It uses a registry with a MetricsNameTemplate for each metric, and then those templates are used when creating the actual metrics. The metrics framework provides utilities that can generate the HTML documentation from the registry of templates.

This change moves the recently-added Connect metrics over to use these templates and to then generate the metric documentation for Connect.

This PR is based upon #3975 and can be rebased once that has been merged.

Author: Randall Hauch <rhauch@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3987 from rhauch/kafka-5990
2017-10-04 11:05:50 -07:00
Rajini Sivaram 9949e1ed1b KAFKA-6004; Allow authentication providers to override error message
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4015 from rajinisivaram/KAFKA-6004-auth-exception
2017-10-04 13:44:46 -04:00
Jason Gustafson 5383f9bed0 MINOR: Use SecurityProtocol in AuthenticationContext
Since we removed the unused `TRACE` option from `SecurityProtocol`, it now seems safer to expose it from `AuthenticationContext`. Additionally this patch exposes javadocs under security.auth and relocates the `Login` and `AuthCallbackHandler` to a non-public package.

Author: Jason Gustafson <jason@confluent.io>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #3863 from hachikuji/use-security-protocol-in-auth-context
2017-10-04 09:20:21 -07:00
Rajini Sivaram c4b8a7eab4 KAFKA-5738; Upgrade note for cumulative count metric (KIP-187)
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4014 from rajinisivaram/MINOR-upgrade-KIP-187
2017-10-04 10:53:18 -04:00
Rajini Sivaram a202da742d KAFKA-4764; Upgrade notes for authentication failure handling (KIP-152)
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #4013 from rajinisivaram/MINOR-upgrade-auth-failure
2017-10-04 10:25:08 -04:00
Tom Bentley 9e35a2aa78 MINOR: Add <p>/</p> to connect docs, because HTML render doesn't respect blank lines
This just adds `<p>`/`</p>` to paragraphs in Kafak Connect docs.

ewencp are you a good person to review this?

Author: Tom Bentley <tbentley@redhat.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #4008 from tombentley/MINOR-connect-docs-paragraph
2017-10-03 10:25:09 -07:00
Dong Lin 716330a5b2 KAFKA-5843; Mx4jLoader.maybeLoad should only be executed if kafka_mx4jenable is set to true
Author: Dong Lin <lindong28@gmail.com>
Author: Ralph Weires

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3797 from lindong28/KAFKA-5843
2017-10-03 11:27:22 +01:00
Damian Guy cc84686a4a MINOR: additional kip-182 doc updates
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Ismael Juma <ismael@juma.me.uk>

Closes #3971 from dguy/kip-182-docs
2017-10-02 13:20:49 -07:00
Vahid Hashemian cdbf806e2d KAFKA-5495; Update docs to use `kafka-consumer-groups.sh` for checking consumer offsets
And remove the deprecated `ConsumerOffsetChecker` example.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #3405 from vahidhashemian/KAFKA-5495
2017-10-02 12:46:05 -07:00
Bill Bejeck 39d5cdcccf KAFKA-5985; update javadoc regarding closing iterators
Author: Bill Bejeck <bill@confluent.io>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Michael G. Noll <michael@confluent.io>, Damian Guy <damian.guy@gmail.com>

Closes #3994 from bbejeck/KAFKA-5985_document_need_to_close_iterators
2017-10-02 11:49:22 -07:00
Ismael Juma 4f4f995325 KAFKA-5746; Return 0.0 from Metric.value() instead of throwing exception
This is less likely to break custom metric reporters and since the method
is deprecated, people will be warned about this potential issue.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>, Xavier Léauté <xavier@confluent.io>

Closes #3996 from ijuma/avoid-exception-in-measurable-value
2017-10-01 01:29:33 +01:00
Rajini Sivaram 021d8a8e96 KAFKA-5746; Add new metrics to support health checks (KIP-188)
Adds new metrics to support health checks:
1. Error rates for each request type, per-error code
2. Request size and temporary memory size
3. Message conversion rate and time
4. Successful and failed authentication rates
5. ZooKeeper latency and status
6. Client version

Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3705 from rajinisivaram/KAFKA-5746-new-metrics
2017-09-28 21:58:59 +01:00
Manikumar Reddy 8dfdcfd02c KAFKA-5750; Elevate log messages for denials to INFO in SimpleAclAuthorizer
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3698 from omkreddy/KAFKA-5750
2017-09-27 16:39:12 +01:00
Manjula K 5b943ca8a9 MINOR:Updated Rabobank description
dguy Please review

Author: Manjula K <manjula@kafka-summit.org>
Author: manjuapu <manjula@confluent.io>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3963 from manjuapu/customer-logo-stream
2017-09-27 09:26:57 +01:00
Richard Yu b8be86b805 KAFKA-5765; Move merge() from StreamsBuilder to KStream
This is the polished version.
1. The old merge() method in StreamsBuilder has been removed,
2. The merge() method in KStreamBuilder was changed so that it would use the single variable argument
rather than several variable arguments in the KStreamImpl implementation
3. The merge() method in KStream has been declared as final and tests have been added to test correctness.

Author: Richard Yu <richardyu@Richards-Air.attlocal.net>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3916 from ConcurrencyPractitioner/trunk
2017-09-26 09:42:53 +01:00
Dong Lin 4e43a7231d MINOR: Improve documentation of AdminClient and fix exception thrown
Make documentation consistent across methods and throw
IllegalStateException instead of IllegalArgumentException in
some cases.

Also include a couple of minor fixes in upgrade.html.

Author: Dong Lin <lindong28@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3781 from lindong28/minor-admin-client-comment
2017-09-26 08:20:26 +01:00
Matthias J. Sax a96e28eac1 MINOR: Update Streams quickstart to create output topic with compaction enabled
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3949 from mjsax/minor-update-streams-quickstart
2017-09-25 16:17:01 +08:00
Mickael Maison a0f533266a KAFKA-3356: Remove ConsumerOffsetChecker
Author: Mickael Maison <mickael.maison@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #3036 from mimaison/KAFKA-3356
2017-09-24 10:08:52 +08:00
Manjula K 1fd70c7c94 MINOR - Adding New York Times logo to streams page
Author: Manjula K <manjula@kafka-summit.org>
Author: manjuapu <manjula@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3950 from manjuapu/customer-logo-stream
2017-09-23 10:53:10 +08:00
Manjula K 36afd1095c MINOR: Adding LINE corp logo to streams page
Author: Manjula K <manjula@kafka-summit.org>
Author: manjuapu <manjula@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3940 from manjuapu/customer-logo-stream
2017-09-22 07:11:06 +08:00
Mickael Maison c8c6ab6324 KAFKA-5735; KIP-190: Handle client-ids consistently
Developed with edoardocomar

Author: Mickael Maison <mickael.maison@gmail.com>

Reviewers: Edoardo Comar <ecomar@uk.ibm.com>, Rajini Sivaram <rajinisivaram@googlemail.com>

Closes #3906 from mimaison/KAFKA-5735
2017-09-21 21:16:57 +01:00
Damian Guy f29391c490 MINOR: add upgrade note for KIP-173 topic configs
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3921 from dguy/minor-kip-173-docs
2017-09-21 18:19:24 +08:00
manjuapu c82be0f303 MINOR: Adding See how Kafka Streams is being used section to Streams page
Author: manjuapu <manjula@confluent.io>
Author: Manjula K <manjula@kafka-summit.org>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3914 from manjuapu/customer-logo-stream
2017-09-20 08:59:05 -07:00
Florian Hussonnois 398714b758 KAFKA-5839: Upgrade Guide doc changes for KIP-130
Author: Florian Hussonnois <florian.hussonnois@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3811 from fhussonnois/KAFKA-5839

minor fixes
2017-09-20 16:50:08 +08:00
Manikumar Reddy 83bdcdbaef MINOR: Refer consistently to server.properties in security docs
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3788 from omkreddy/RULE-DOC
2017-09-19 05:33:02 +01:00
Jakub Scholz a64fe2ed88 KAFKA-5918: Fix minor typos and errors in the Kafka Streams turotial
I found several minor issues with the Kafka Streams tutorial:
* Some typos
  * "As shown above, it illustrate that the constructed ..." instead of "As shown above, it illustrate_s_ that the constructed ..."
  * "same as Pipe.java below" instead of "same as Pipe.java _above_"
  * Wrong class name in the `LineSplit` example
* Incorrect imports for the code examples
  * Missing `import org.apache.kafka.streams.kstream.KStream;` in `LineSplit` and `WordCount` example
* Unnecessary (and potentially confusing) split by whitespaces in the `WorkCount` class (the split into words happened already in `LineSplit`)

Author: Jakub Scholz <www@scholzj.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3883 from scholzj/stream-tutorial-typos
2017-09-19 07:58:06 +08:00
Damian Guy f7b1add686 MINOR: update tutorial doc to match ak-site
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3847 from dguy/minor-doc-update
2017-09-13 13:22:06 -07:00
Guozhang Wang 9d2437a464 MINOR: Un-hide the tutorial buttons on web docs
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3850 from guozhangwang/KMinor-unhide-button
2017-09-13 13:12:09 -07:00
Damian Guy 2db1e4423f KAFKA-5852: Add filter, filterNot, mapValues and Materialized to KTable
Add overloads of `filter`, `filterNot`, `mapValues` that take `Materialized` as a param to `KTable`. Deprecate overloads using `storeName` and `storeSupplier`

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3807 from dguy/ktable-filter-map
2017-09-08 14:01:58 -07:00
Damian Guy 4769e3d92a KAFKA-5815; add Printed class and KStream#print(printed)
Part of KIP-182
- Add `Printed` class and `KStream#print(Printed)`
- deprecate all other `print` and `writeAsText` methods

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3768 from dguy/kafka-5652-printed
2017-09-08 18:22:04 +01:00
Damian Guy e16b9143df KAFKA-5853; implement WindowedKStream
Add the `WindowedKStream` interface and implementation of methods that don't require `Materialized`

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3809 from dguy/kgrouped-stream-windowed-by
2017-09-08 16:49:18 +01:00
Damian Guy d0ee6ed36b KAFKA-5832; add Consumed and change StreamBuilder to use it
Added `Consumed` class.
Updated `StreamBuilder#stream`, `StreamBuilder#table`, `StreamBuilder#globalTable`

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3784 from dguy/kip-182-stream-builder
2017-09-08 08:21:48 +01:00
Damian Guy 9cbb9f0939 KAFKA-5650; add StateStoreBuilder interface and implementations
Part of KIP-182

- Add `StateStoreBuilder` interface and `WindowStateStoreBuilder`, `KeyValueStateStoreBuilder`, and `SessionStateStoreBuilder` implementations
- Add `StoreSupplier`, `WindowBytesStoreSupplier`, `KeyValueBytesStoreSupplier`, `SessionBytesStoreSupplier` interfaces and implementations
- Add new methods to `Stores` to create the newly added `StoreSupplier` and `StateStoreBuilder` implementations
- Update `Topology` and `InternalTopology` to use the interfaces

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3767 from dguy/kafka-5650
2017-09-07 09:39:46 +01:00
Damian Guy 667cd60dc6 KAFKA-5816; add Produced class, KStream#to(topic, Produced), and KStream#through(topic, Produced)
Add the `Produced` class and `KStream` overloads that use it:
`KStream#to(String, Produced)`
`KStream#through(String, Produced)`
Deprecate all other to and through methods accept the single param methods that take a topic param

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3770 from dguy/kafka-5652-produced
2017-09-07 08:54:10 +01:00
Boyang Chen b041c8d87d MINOR: close iterator on doc example
The iterator interface usage has some examples missing explicit close operation after usage. We should remind the user to do so because un-closed iterator will leave the underlying file descriptor open, thus eating up memory.
guozhangwang Ishiihara

Author: Boyang Chen <bychen@pinterest.com>
Author: Boyang Chen <bchen11@outlook.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3714 from abbccdda/add_iterator_close_on_doc

minor fixes
2017-09-06 22:12:10 -07:00
Guozhang Wang 9b85cf9ed0 MINOR: KIP-138 renaming of string names
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com>

Closes #3796 from guozhangwang/kip-138-minor-renames
2017-09-06 18:39:40 -07:00
Matthias J. Sax 212bce6e3f MINOR: fix typos on web doc's upgrade guide
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3804 from mjsax/hotfix
2017-09-06 15:42:52 -07:00
Damian Guy b687c06800 KAFKA-5817; Add Serialized class and overloads to KStream#groupBy and KStream#groupByKey
Part of KIP-182
- Add the `Serialized` class
- implement overloads of `KStream#groupByKey` and KStream#groupBy`
- deprecate existing methods that have more than default arguments

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3772 from dguy/kafka-5817
2017-09-06 10:43:14 +01:00
James Cheng 2fb5664bf4 KAFKA-5597: Autogenerate producer sender metrics
Subtask of https://issues.apache.org/jira/browse/KAFKA-3480

The changes are very similar to what was done for the consumer in https://issues.apache.org/jira/browse/KAFKA-5191 (pull request https://github.com/apache/kafka/pull/2993)

Author: James Cheng <jylcheng@yahoo.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Rajini Sivaram <rajinisivaram@googlemail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #3535 from wushujames/producer_sender_metrics_docs

Fix one minor naming bug
2017-09-05 17:38:58 -07:00
Matthias J. Sax cd59976eea KAFKA-5823: Extend upgrade section for KIP-120
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Sharon Liu <sharonliu.cup@gmail.com>, Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #3787 from mjsax/kafka-5823-kip120-docs
2017-09-05 16:54:00 -07:00
Damian Guy caddae19ea MINOR: fix scalaVersion variable in templateData.js
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3790 from dguy/fix-js-tempate
2017-09-05 11:45:21 +01:00
Eno Thereska 493c2aad59 MINOR: add table of contents
Added a simple table of contents for the developer section.

Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3760 from enothereska/minor-docs-toc
2017-09-04 14:28:50 +01:00
Guozhang Wang ce04b14358 MINOR: KIP-160 docs
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3733 from guozhangwang/KMinor-kip160-docs
2017-08-30 11:17:55 -07:00
Guozhang Wang f5310d645c MINOR: doc changes for KIP-138
1. Core concepts (added the stream time definition), upgrade guide and developer guide.
2. Related Java docs changes.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Damian Guy <damian.guy@gmail.com>

Closes #3732 from guozhangwang/KMinor-kip138-docs
2017-08-30 09:45:16 -07:00
Eno Thereska bd54d2e3e0 MINOR: stateful docs for aggregates
Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3730 from enothereska/minor-docs-aggregates
2017-08-30 10:09:08 +01:00
Michal Borowiecki 5648dcc3e5 MINOR: Fix doc typos and grammar
This is contributed by mihbor on various doc fixes including:

https://github.com/apache/kafka/pull/3224
https://github.com/apache/kafka/pull/3226
https://github.com/apache/kafka/pull/3229

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3746 from guozhangwang/KMinor-doc-typos
2017-08-26 16:33:33 -07:00
Guozhang Wang 21fea170d8 MINOR: add upgrade section for 1.0.0
1. Add upgrade section for 1.0.0, including Streams API changes section.
2. Add metrics name changes section.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3687 from guozhangwang/KMinor-metrics-upgrade-guide
2017-08-24 09:12:12 -07:00
Eno Thereska 5e65c0da45 MINOR: Stateless transformation documentation
Needs to come after https://github.com/apache/kafka/pull/3701
Originally reviewed as part of #3490.

Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3711 from enothereska/minor-docs-stateless
2017-08-24 14:53:39 +01:00
Eno Thereska ecea150bde MINOR: Document how to create source streams and tables
Originally reviewed as part of https://github.com/apache/kafka/pull/3490.

Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3701 from enothereska/minor-docs-create-source-streams
2017-08-22 14:14:35 +01:00
Guozhang Wang b6102cf4b2 HOTFIX: missing imports and version on web docs 2017-08-11 13:53:47 -07:00
Guozhang Wang d2774e302f KAFKA-5727: Add Streams quickstart tutorial as an archetype project
0. Minor fixes on the existing examples to merge all on a single input topic; also do not use `common.utils.Exit` as it is for internal usage only.

1. Add the archetype project for the quickstart. Steps to try it out:

  a. `mvn install` on the quickstart directory.
  b. `mvn archetype:generate \
-DarchetypeGroupId=org.apache.kafka \
-DarchetypeArtifactId=streams-quickstart-java \
-DarchetypeVersion=1.0.0-SNAPSHOT \
-DgroupId=streams-quickstart \
-DartifactId=streams-quickstart \
-Dversion=0.1 \
-Dpackage=StreamsQuickstart \
-DinteractiveMode=false` at any directory to create the project.
  c. build the streams jar with version `1.0.0-SNAPSHOT` to local maven repository with `./gradlew installAll`; `cd streams-quickstart; mvn clean package`
  d. create the input / output topics, start the console producer and consumer.
  e. start the program: `mvn exec:java -Dexec.mainClass=StreamsQuickstart.Pipe/LineSplit/WordCount`.
  f. type data on console producer and observe data on console consumer.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Eno Thereska <eno.thereska@gmail.com>

Closes #3630 from guozhangwang/KMinor-streams-quickstart-tutorial
2017-08-11 12:19:28 -07:00
Bill Bejeck b2b529522d MINOR: First cut at porting State Store docs to AK
Author: Bill Bejeck <bill@confluent.io>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3629 from bbejeck/docs-updates-for-kip-167
2017-08-11 09:35:26 +01:00
Jason Gustafson 017a21c604 MINOR: Remove unneeded error handlers in deprecated request objects
These handlers were previously used on the broker to handle uncaught exceptions, but now the broker users the new Java request objects exclusively.

Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3646 from hachikuji/remove-old-request-error-handlers
2017-08-10 11:07:53 -07:00
Ismael Juma 69d3d4aad6 KAFKA-5470; Replace -XX:+DisableExplicitGC with -XX:+ExplicitGCInvokesConcurrent in kafka-run-class
This is important because Bits.reserveMemory calls System.gc() hoping to free native
memory in order to avoid throwing an OutOfMemoryException. This call is currently
a no-op due to -XX:+DisableExplicitGC.

It's worth mentioning that -XX:MaxDirectMemorySize can be used to increase the
amount of native memory available for allocation of direct byte buffers.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Apurva Mehta <apurva@confluent.io>, Manikumar Reddy <manikumar.reddy@gmail.com>, Jason Gustafson <jason@confluent.io>

Closes #3371 from ijuma/kafka-5470-explicit-gc-invokes-concurrent
2017-08-09 23:38:57 +01:00
Damian Guy 2010aa067f MINOR: add memory management section to streams docs
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Eno Thereska <eno.thereska@gmail.com>

Closes #3604 from dguy/memory-management-docs
2017-08-07 10:17:17 +01:00
Guozhang Wang 5704d8f05f HOTFIX: add the hyper link for storage
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3617 from guozhangwang/KHotfix-introduction-hyperlink
2017-08-04 09:16:57 -07:00
Eno Thereska 6bee1e9e57 HOTFIX: Fixes to metric names of Streams
A couple of fixes to metric names to match the KIP
- Removed extra strings in the metric names that are already in the tags
- add a separate metric for "all"

Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3491 from enothereska/hotfix-metric-names
2017-08-03 14:16:47 -07:00
Guozhang Wang 125d69caee KAFKA-5671 Followup: Remove reflections in unit test classes
1. Remove rest deprecation warnings in streams:jar.

2. Consolidate all unit test classes' reflections to access internal topology builder from packages other than `o.a.k.streams`. We need to refactor the hierarchies of StreamTask, StreamThread and KafkaStreams to remove these hacky reflections.

3. Minor fixes such as reference path, etc.

4. Minor edits on web docs for the describe function under developer-guide.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Damian Guy <damian.guy@gmail.com>

Closes #3603 from guozhangwang/K5671-followup-comments
2017-08-02 15:13:02 -07:00
Matthias J. Sax da2205578b KAFKA-5671: Add StreamsBuilder and Deprecate KStreamBuilder
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3602 from mjsax/kafka-5671-add-streamsbuilder
2017-07-31 15:28:59 -07:00
Matthias J. Sax 1844bf2b2f KAFKA-5670: (KIP-120) Add Topology and deprecate TopologyBuilder
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3590 from mjsax/kafka-3856-replace-topology-builder-by-topology
2017-07-28 16:46:34 -07:00
Dong Lin 4003c9384b KAFKA-5341; Add UnderMinIsrPartitionCount and per-partition UnderMinIsr metrics (KIP-164)
Author: Dong Lin <lindong28@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3583 from lindong28/KAFKA-5341
2017-07-28 11:28:33 -07:00
Ismael Juma 5effe72390 MINOR: Next release will be 1.0.0
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3580 from ijuma/bump-to-1.0.0-SNAPSHOT
2017-07-26 13:01:41 -07:00
Robin Moffatt f4d1305bfe MINOR: Fix typo in SMT doc: s/RegexpRouter/RegexRouter
Author: Robin Moffatt <robin@rmoff.net>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3576 from rmoff/patch-1
2017-07-26 09:17:44 -07:00
Guozhang Wang 91c207c2c6 MINOR: Make streams quick start more interactive
1. Make the WordCountDemo application to not stop automatically but via "ctrl-C".
2. Update the quickstart html file to let users type input messages one-by-one, and observe added output in an interactive manner.
3. Some minor fixes on the parent documentation page pointing to streams sub-pages, added a new recommended Scala version number.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Damian Guy <damian.guy@gmail.com>

Closes #3515 from guozhangwang/KMinor-interactive-quickstart
2017-07-25 11:34:16 -07:00
Ivan Babrou b887eeb721
KAFKA-4711: fix docs on unclean.leader.election.enable default 2017-07-24 12:06:12 -07:00
Dong Lin fc93fb4b61 KAFKA-4763; Handle disk failure for JBOD (KIP-112)
Author: Dong Lin <lindong28@gmail.com>

Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Onur Karaman <okaraman@linkedin.com>

Closes #2929 from lindong28/KAFKA-4763
2017-07-22 12:35:32 -07:00
Ismael Juma 91b5fc737b MINOR: Give correct instructions for retaining previous unclear leader election behaviour
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>

Closes #3561 from ijuma/fix-upgrade-note-for-unclean-leader-election
2017-07-22 10:36:12 +01:00
Derrick Or 300f48018c MINOR: Add Streams landing page
Content and assets for the updated Streams API landing page

Author: Derrick Or <derrickor@gmail.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3540 from derrickdoo/streams-landing-page
2017-07-21 19:16:14 -07:00
Damian Guy 1110f66fa7 KAFKA-3741; allow users to specify default topic configs for internal topics
Allow users to specify default topic configs for streams internal topics by supplying properties from `TopicConfig` with a prefix.
Supplied defaults are used when creating the internal topics. They are overridden by the configs supplied along with the `InternalTopicConfig`

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3459 from dguy/kafka-3741
2017-07-21 12:15:40 +01:00
Vahid Hashemian beb0ad9bbb MINOR: Make Kafka Connect step in quickstart more Windows friendly
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2498 from vahidhashemian/doc/connect_quickstart_update
2017-07-20 22:06:46 -07:00
ppatierno 1d2d0bac9e MINOR: Fixed misleading reference to HTTPS instead of SSL support in the doc
Author: ppatierno <ppatierno@live.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3487 from ppatierno/ssl-doc-https
2017-07-20 20:16:15 -07:00
ppatierno 8a81566214 KAFKA-5410; Fix taskClass() method name in Connector and flush() signature in SinkTask
Author: ppatierno <ppatierno@live.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #3269 from ppatierno/connect-doc
2017-07-20 19:51:09 -07:00
Vahid Hashemian 5bb53e034e KAFKA-5534; KafkaConsumer `offsetForTimes` result should include partitions with no offset
For topics that support timestamp search, if no offset is found for a partition, the partition should still be included in the result with a `null` offset value. This `KafkaConsumer` method currently excludes such partitions from the result.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #3460 from vahidhashemian/KAFKA-5534
2017-07-20 17:38:30 -07:00
Jeff Klukas 0816e47bfe MINOR: Docs update, Java clients use Kafka Metrics
This contribution is my original work and I license the work to the project under the project's open source license.

Author: Jeff Klukas <jeff@klukas.net>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3249 from jklukas/docs-no-more-yammer-metrics-in-client
2017-07-20 12:31:17 +01:00
Vahid Hashemian 3bfc073f03 MINOR: Make 'Topic-Level Configs' a doc section for easier access
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3415 from vahidhashemian/doc/make_topic_config_a_section
2017-07-19 14:25:02 +01:00
Gwen Shapira ea8845bbd6 KAFKA-5384: Enable topic deletion by default
https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default

Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Ismael Juma

Closes #3241 from gwenshap/KAFKA-5384
2017-07-18 08:57:06 -07:00
Guozhang Wang 6a5cd67b9b HOTFIX: fix a few typos on streams quickstart
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3510 from guozhangwang/KHotfix-streams-quickstart
2017-07-10 10:16:02 -07:00
Eno Thereska 7bfe008ae1 MINOR: Move quickstart under streams
Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Michael G. Noll <michael@confluent.io>, Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #3494 from enothereska/minor-quickstart-docs

rename section name and bold font for section names
2017-07-09 17:35:14 -07:00
Damian Guy 75f1b757e1 HOTFIX: fix paths in streams index
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3502 from dguy/doc-fixes
2017-07-07 15:10:16 +01:00
Ismael Juma c574c58a16 MINOR: Improve versioning in docs when a full version is required
Also fix quickstart.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3501 from ijuma/introduce-full-version-variable-in-docs
2017-07-07 14:41:54 +01:00
huxihx 6d203361a0 KAFKA-5508; Documentation for altering topics
Operations documentation should guide user to employ `kafka-configs.sh` to add/remove configs for a topic.

Author: huxihx <huxi_2b@hotmail.com>

Reviewers: Damian Guy <damian.guy@gmail.com>

Closes #3429 from huxihx/KAFKA-5508_Doc_for_altering_topics
2017-07-06 10:10:03 +01:00
Damian Guy 6cae5ec668 MINOR: add IQ docs to streams documentation
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #3484 from dguy/iq-doc
2017-07-05 14:15:43 -07:00
Guozhang Wang 2c6e9a9653 MINOR: Follow-up Streams doc changes to break into sub-pages
Also fixed a bunch of broken links (details can be found in 34f8ecea0d)

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3473 from guozhangwang/KMinor-streams-doc-breakdown
2017-07-03 15:35:47 +01:00
Apurva Mehta 6471822079 KAFKA-5020; Update message format in implementation docs
Author: Apurva Mehta <apurva@confluent.io>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #3360 from apurvam/KAFKA-5020-message-format-docs-update
2017-06-30 09:22:25 -07:00
Michal Borowiecki efb060c57f KAFKA-5233; KIP-138: Change punctuate semantics
Implementation for KIP-138: Change punctuate semantics

Author: Michal Borowiecki <michal.borowiecki@openbet.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bbejeck@gmail.com>, Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3055 from mihbor/KIP-138
2017-06-28 11:26:02 +01:00
Kevin Sweeney 785d8e20ca MINOR: Provide link to ZooKeeper within Quickstart
Author: Kevin Sweeney <restlessdesign@users.noreply.github.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3372 from restlessdesign/patch-1
2017-06-22 14:03:39 +01:00
Tom Bentley a6799f4e14 KAFKA-4059; API Design section under Implementation is out of date
It describes the old deprecated clients and it's better to just
remove it.

The contribution is my original work and I license the work to the
project under the project's open source license.

Author: Tom Bentley <tbentley@redhat.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3385 from tombentley/KAFKA-4059
2017-06-22 13:42:40 +01:00
Jason Gustafson e6e2631743 MINOR: Detail message/batch size implications for conversion between old and new formats
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3373 from hachikuji/fetch-size-upgrade-notes
2017-06-21 14:04:19 -07:00
Matthias J. Sax cae5977ed0 MINOR: explain producer naming within Streams
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Bill Bejeck <bbejeck@gmail.com>, Damian Guy <damian.guy@gmail.com>

Closes #3378 from mjsax/minor-producer-naming
2017-06-21 09:32:46 +01:00
Jason Gustafson cb3952a4fd KAFKA-5021; Update delivery semantics documentation for EoS (KIP-98)
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>

Closes #3388 from hachikuji/KAFKA-5021
2017-06-20 16:56:38 -07:00
Guozhang Wang f28fc1100b MINOR: Add Processing Guarantees to Streams docs
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Apurva Mehta <apurva@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Jason Gustafson <jason@confluent.io>

Closes #3345 from guozhangwang/KMinor-streams-eos-docs
2017-06-20 16:29:08 -07:00
Edoardo Comar 491774bd52 KAFKA-5290; Docs need clarification on meaning of 'committed' to the log
based on conversations with vahidhashemian rajinisivaram apurvam

The docs didn't make clear that what gets committed and what gets not may depend on the producer acks.

Author: Edoardo Comar <ecomar@uk.ibm.com>

Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Apurva Mehta <apurva@confluent.io>, Jason Gustafson <jason@confluent.io>

Closes #3035 from edoardocomar/DOC-clarification-on-committed
2017-06-20 14:21:53 -07:00
Hao Chen 9790d5aba0 MINOR: Remove version in the uses page
This is contributed by Hao Chen.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #3366 from guozhangwang/KMinor-fix-streams-link-in-uses
2017-06-20 09:34:59 -07:00
Michal Borowiecki 5b063252e2 MINOR: fix quotes for consistent rendering
“ as opposed to " don't render consistently across browsers. On current Kafka website they render correctly in Firefox but not Chrome (“runs”) - charset issue?

Author: Michal Borowiecki <michal.borowiecki@openbet.com>
Author: mihbor <mbor81@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3227 from mihbor/patch-7
2017-06-16 13:09:31 -07:00
Michal Borowiecki 518f141ed0 MINOR: (docs) in section, not at section
See https://english.stackexchange.com/questions/158981/at-this-section-vs-in-this-section and https://forum.wordreference.com/threads/at-in-within-the-section-we-can-find.374188/

Author: mihbor <mbor81@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3228 from mihbor/patch-8
2017-06-16 13:07:18 -07:00
Manikumar Reddy O d06a0db8b8 MINOR: Specify keyalg RSA for SSL key generation commands
Author: Manikumar Reddy O <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3305 from omkreddy/ssl-doc
2017-06-12 22:43:58 +01:00
Ismael Juma c57cfdf4fa KAFKA-5411: AdminClient javadoc and documentation improvements
- Show AdminClient configs in the docs.
- Update Javadoc config so that public classes exposed by
the AdminClient are included.
- Version and table of contents fixes.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Colin Mccabe, Gwen Shapira

Closes #3271 from ijuma/kafka-5411-admin-client-javadoc-configs
2017-06-08 15:27:30 -07:00
Eno Thereska d1a279dcc7 MINOR: Cache metrics were missing
Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3262 from enothereska/minor-missing-metric
2017-06-07 20:52:54 -07:00
Guozhang Wang 57b0d0fe57 MINOR: syntax brush for java / bash / json / text
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Derrick Or <derrickor@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #3214 from guozhangwang/KMinor-doc-java-brush
2017-06-07 15:17:25 +01:00
Damian Guy ad8e7cf5e8 MINOR: update stream docs for kip-134
Add a section in the streams docs about the broker config introduced in KIP-134

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll, Bill Bejeck, Guozhang Wang

Closes #3179 from dguy/kip134-doc
2017-06-06 09:37:13 -07:00
Bill Bejeck 1c70e7ac12 MINOR: Adding deprecated KTable methods to docs from KIP-114
Author: Bill Bejeck <bill@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3215 from bbejeck/kip114-doc
2017-06-05 11:24:29 -07:00
Matthias J. Sax d4a070fcfa MINOR: update docs with regard to KIP-123
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #3232 from mjsax/minor-update-docs-for-kip-123
2017-06-04 22:42:44 -07:00
Jason Gustafson 8711e5937f KAFKA-5019; Upgrades notes for idempotent/transactional features and new message format
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3212 from hachikuji/KAFKA-5019
2017-06-03 06:36:29 +01:00
Ismael Juma 4959444afc KAFKA-5236; Increase the block/buffer size when compressing with Snappy or Gzip
We had originally increased Snappy’s block size as part of KAFKA-3704. However,
we had some issues with excessive memory usage in the producer and we reverted
it in 7c6ee8d5e.

After more investigation, we fixed the underlying reason why memory usage seemed
to grow much more than expected via KAFKA-3747 (included in 0.10.0.1).

In 0.10.2, we changed the broker to use the same classes as the producer and the
broker’s block size for Snappy was changed from 32 KB to 1KB. As reported in
KAFKA-5236, the on disk size is, in some cases, 50% larger when the data is compressed
with 1 KB instead of 32 KB as the block size.

As discussed in KAFKA-3704, it may be worth making this configurable and/or allocate
the compression buffers from the producer pool. However, for 0.11.0.0, I think the
simplest thing to do is to default to 32 KB for Snappy (the default if no block size
is provided).

I also increased the Gzip buffer size. 1 KB is too small and the default is smaller
still (512 bytes). 8 KB (which is the default buffer size for BufferedOutputStream)
seemed like a reasonable default.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #3205 from ijuma/kafka-5236-snappy-block-size
2017-06-02 21:20:02 +01:00
Guozhang Wang 4c8ec795a0 MINOR: Add upgrade notes for KAFKA-2358
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3200 from guozhangwang/KMinor-KAFKA-2358-upgrade-notes
2017-06-01 18:24:54 -07:00
Matthias J. Sax 1e350b860a MINOR: update Streams docs for KIP-123
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Ismael Juma, Ewen Chesklack-Postava, Bill Bejeck, Guozhang Wang

Closes #3194 from mjsax/minor-update-docs-for-kip-123
2017-06-01 15:27:41 -07:00
Damian Guy adb4ea74ab MINOR: add upgrade not for group.initial.rebalance.delay.ms
Add a new entry in upgrade.html for `group.initial.rebalance.delay.ms`

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>

Closes #3189 from dguy/relabance-delay
2017-06-01 13:42:12 -07:00
Matthias J. Sax d9479275fd MINOR: Improve streams config parameters
Adjust "importance level" and add explanation to the docs.

Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Eno Thereska <eno.thereska@gmail.com>, Guozhang Wang <wangguoz@gmail.com>

Closes #2855 from mjsax/minor-improve-streams-config-parameters
2017-05-31 14:53:28 -07:00
Rajini Sivaram b38b74bb77 MINOR: Fix doc for producer throttle time metrics
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>

Closes #3169 from rajinisivaram/MINOR-producer-metrics
2017-05-30 16:35:25 -07:00
xinlihua f0745cd514 KAFKA-4603: Disallow abbreviations in OptionParser constructor
KAFKA-4603 the command parsed error
Using "new OptionParser" might result in parse error

Change all the OptionParser constructor in Kafka into "new OptionParser(false)"

Author: xinlihua <xin.lihua1@zte.com.cn>
Author: unknown <00067310@A23338408.zte.intra>
Author: auroraxlh <xin.lihua1@zte.com.cn>
Author: xin <xin.lihua1@zte.com.cn>

Reviewers: Damian Guy, Guozhang Wang

Closes #2349 from auroraxlh/fix_OptionParser_bug
2017-05-30 13:53:32 -07:00
Jiangjie Qin 6b03497915 KAFKA-5344; set message.timestamp.difference.max.ms back to Long.MaxValue
Author: Jiangjie Qin <becket.qin@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #3163 from becketqin/KAFKA-5344
2017-05-30 15:44:34 +01:00
James Cheng 0bc4f75eed KAFKA-5191: Autogenerate Consumer Fetcher metrics
Autogenerate docs for the Consumer Fetcher's metrics. This is a smaller subset of the original PR https://github.com/apache/kafka/pull/1202.

CC ijuma benstopford hachikuji

Author: James Cheng <jylcheng@yahoo.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>

Closes #2993 from wushujames/fetcher_metrics_docs
2017-05-26 15:34:20 -07:00
Jason Gustafson cea319a4ad KAFKA-4935; Deprecate client checksum API and compute lazy partial checksum for magic v2
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk>

Closes #3123 from hachikuji/KAFKA-4935
2017-05-25 08:21:01 +01:00
Michael André Pearce 1c7fdd2843 KAFKA-4208; Add record headers upgrade doc
Update upgrade.html

Raising this now, as KIP-118 is pulled from release as such submitting this without java 8 changes.

As per remaining review comment from https://github.com/apache/kafka/pull/2772, updating the upgrade notes.

Author: Michael André Pearce <michael.andre.pearce@me.com>
Author: Michael Andre Pearce <Michael.Andre.Pearce@me.com>

Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2991 from michaelandrepearce/KIP-82
2017-05-21 14:34:19 -07:00
Rajini Sivaram 670193f22d KAFKA-4957; Request rate quotas documentation
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #3069 from rajinisivaram/KAFKA-4957
2017-05-16 16:36:25 -07:00
Ismael Juma bcf447e93e KAFKA-4422; Drop support for Scala 2.10 (KIP-119)
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>

Closes #2956 from ijuma/kafka-4422-drop-support-scala-2.10
2017-05-11 08:08:11 +01:00
Ismael Juma 59b918ec2b KAFKA-3763; Remove deprecated APIs for 0.11.0.0
This only removes deprecated methods,
fields and constructors in a small number of classes.

Deprecated producer configs is tracked via KAFKA-3353
and the old clients and related (tools, etc.) won't
be removed in 0.11.0.0.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2995 from ijuma/kafka-3763-remove-deprecated-0.11
2017-05-08 10:29:31 -07:00
Ismael Juma 3bcadbfb47 KAFKA-3353; Remove deprecated producer configs
These configs have been deprecated since 0.9.0.0:
block.on.buffer.full, metadata.fetch.timeout.ms and timeout.ms

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2987 from ijuma/kafka-3353-remove-deprecated-producer-configs
2017-05-08 10:00:04 -07:00
Colin P. Mccabe 124ddf5779 KAFKA-5162; Add a reference to AdminClient to docs/api.html
Author: Colin P. Mccabe <cmccabe@confluent.io>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Ismael Juma <ismael@juma.me.uk>

Closes #2958 from cmccabe/KAFKA-5162
2017-05-05 05:08:17 +01:00
Ryan P 8d74920163 KAFKA-3754; Add GC log retention policy to limit size of log
Add a default log retention policy to keep GC logs from growing too large

Author: Ryan P <ryan.n.pridgeon@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1431 from rnpridgeon/KAFKA-3754
2017-05-03 10:51:32 +01:00
sharad-develop 46da01a4a7 KAFKA-4623; Default unclean.leader.election.enabled to false (KIP-106)
Author: sharad.develop <sharad.develop@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2625 from sharad-develop/KAFKA-4623
2017-05-02 16:27:28 +01:00
Gwen Shapira 3c93fa321b HOTFIX: HTML formatting error in upgrade docs
Introduced in PR #2824. Already fixed in the
website github.

Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2844 from gwenshap/docs-hotfix

(cherry picked from commit 5f728532ac)
Signed-off-by: Ismael Juma <ismael@juma.me.uk>
2017-04-12 11:09:21 +01:00
Eno Thereska a74f035844 MINOR: Added changes in 0.10.2.1
Author: Eno Thereska <eno@confluent.io>

Reviewers: Gwen Shapira

Closes #2824 from enothereska/minor-docs-0.10.2.1

(cherry picked from commit 5a68fa7e4e)
Signed-off-by: Gwen Shapira <cshapi@gmail.com>
2017-04-11 17:08:04 -07:00
sunnykrgupta d2f5589afc MINOR: Fix typo in consumer ACL example
Author: sunnykrgupta <sunnygupta.kr@gmail.com>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2839 from sunnykrGupta/trunk
2017-04-11 11:28:58 +01:00
shuguo zheng 1ce6aa5503 KAFKA-4964; Use correct keystore/trustore name in documentation
Author: shuguo zheng <zheng.shuguo@zte.com.cn>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2749 from zhengsg/local
2017-03-31 11:08:12 +01:00
Manikumar Reddy O 81721f8c53 MINOR: Doc change related to ZK sasl configs
Author: Manikumar Reddy O <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2620 from omkreddy/MINOR-ZK-CHANGE
2017-03-30 13:50:46 +01:00
Dong Lin 8b05ad406d KAFKA-4586; Add purgeDataBefore() API (KIP-107)
Author: Dong Lin <lindong28@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jiangjie Qin <becket.qin@gmail.com>

Closes #2476 from lindong28/KAFKA-4586
2017-03-28 09:59:44 -07:00
Jason Gustafson 5bd06f1d54 KAFKA-4816; Message format changes for idempotent/transactional producer (KIP-98)
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Jun Rao <junrao@gmail.com>, Apurva Mehta <apurva@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2614 from hachikuji/exactly-once-message-format
2017-03-24 19:38:43 +00:00
Damian Guy 2269eed242 KAFKA-4919: Document that stores must not be closed when Processors are closed
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll, Eno Thereska, Matthias J. Sax, Elias Levy, Guozhang Wang

Closes #2725 from dguy/minor-processor-java-doc
2017-03-23 09:47:45 -07:00
Gwen Shapira 95ef40dd31 MINOR: Adding example to SMT documentation
Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2721 from gwenshap/improve_smt_docs
2017-03-22 23:06:26 -07:00
Michael G. Noll 666beecec8 MINOR: MyProcessor doc example should implement, not extend `Processor`
Author: Michael G. Noll <michael@confluent.io>

Reviewers: Eno Thereska, Guozhang Wang

Closes #2723 from miguno/trunk-streams-docs-typo-fix
2017-03-22 12:23:07 -07:00
Ryan P 05690f0c85 MINOR: Fix zookeeper-security-migration documentation example
Incorrect option in example

https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/admin/ZkSecurityMigrator.scala#L71

Author: Ryan P <ryan.n.pridgeon@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2714 from rnpridgeon/patch-1
2017-03-21 09:52:16 +00:00
Vahid Hashemian be11272818 MINOR: Fix a documentation typo
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>

Closes #2674 from vahidhashemian/minor/fix_typos_1703
2017-03-15 11:50:57 -07:00
Matthias J. Sax 2064a2a55d HOTFIX: fix broken link for wordcount demo example
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2644 from mjsax/hotfixBrokerLink
2017-03-06 09:52:17 -08:00
Sönke Liebau b1272500bd KAFKA-4567 - Connect Producer and Consumer ignore ssl parameters…
KAFKA-4567 - Connect Producer and Consumer ignore ssl parameters configured for worker

Added brief explanation to the docs about parameter inheritance of Kafka consumers and producers from the worker config.

Author: Sönke Liebau <soenke.liebau@opencore.com>

Reviewers: Gwen Shapira

Closes #2511 from soenkeliebau/KAFKA-4567
2017-03-05 10:28:36 -08:00
Guozhang Wang 6a55d41fe3 HOTFIX: add js template to upgrade.html
Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2591 from guozhangwang/KHotfix-explicit-version
2017-02-23 13:21:49 -08:00
Jiangjie Qin 1f2ee5f0a9 KAFKA-4340; Follow-up fixing system test failures and handling non default log.retention.ms
Author: Jiangjie Qin <becket.qin@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2544 from becketqin/KAFKA-4340_follow_up
2017-02-17 02:40:34 -08:00
Matthias J. Sax af182480d2 MINOR: fix indention in <pre> tags
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2553 from mjsax/hotfixDocs2
2017-02-15 17:02:15 -08:00
Damian Guy abe5e0e489 MINOR: add session windows doc to streams.html
Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll, Matthias J. Sax, Guozhang Wang

Closes #2547 from dguy/session-window-doc
2017-02-15 16:39:52 -08:00
Eno Thereska 2e1b00f262 MINOR: Stream metrics documentation
Author: Eno Thereska <eno@confluent.io>

Reviewers: Matthias J. Sax, Guozhang Wang

Closes #2542 from enothereska/minor-streams-metrics
2017-02-15 16:32:02 -08:00
Matthias J. Sax 2c91b324d4 HOTFIX: fixed section incompatible Steams API changes
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Damian Guy, Guozhang Wang

Closes #2492 from mjsax/hotfixDocs
2017-02-15 13:17:29 -08:00
Vahid Hashemian 601d4040d4 MINOR: Fix quickstart in docs
Reverting some of the recent changes to quickstart doc. Further explanation is provided inline.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2539 from vahidhashemian/doc/fix_quickstart_issues
2017-02-10 15:06:54 -08:00
Damian Guy 1c45f79c91 MINOR: add GlobalKTable doc to streams.html
Update streams.html with GlobalKTable docs

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Michael G. Noll, Matthias J. Sax, Guozhang Wang

Closes #2516 from dguy/global-tables-doc
2017-02-10 09:45:13 -08:00
Guozhang Wang a15fcea799 MINOR: add architecture section and configure / execution for streams
1. Added an architecture section.
2. Added a configuration / execution sub-section to developer guide.

Minor tweaks and a bunch of missing fixes from `kafka-site` repo.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Matthias J. Sax <matthias@confluent.io>, Derrick Or <derrickor@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2488 from guozhangwang/KMinor-streams-docs-second-pass
2017-02-09 19:13:23 -08:00
Grant Henke b5dd39dda6 KAFKA-4525; Kafka should not require SSL truststore password
Author: Grant Henke <ghenke@cloudera.com>

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2246 from granthenke/truststore-password
2017-02-10 02:57:35 +00:00
Alex Loddengaard 8bd8751aa7 MINOR: changes to the production broker configuration docs.
Author: Alex Loddengaard <alexloddengaard@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #2519 from alexlod/production-config-docs
2017-02-08 17:18:55 -08:00
huxi be98865dfc KAFKA-4727; A Production server configuration needs to be updated
1. Update value for queued.max.requests to 500
2. Removed invalid config 'controller.message.queue.size'
3. Removed flush configs including 'log.flush.interval.messages', 'log.flush.interval.ms' and 'log.flush.scheduler.interval.ms'

Author: huxi <huxi@zhenrongbao.com>

Reviewers: Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>

Closes #2490 from amethystic/kafka4727_server_config_doc_update
2017-02-07 08:40:30 -08:00
Matthias J. Sax ab4ebb4bb9 MINOR: added upgrade and API changes to docs
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Ismael Juma, Will Marshall, Damian Guy, Guozhang Wang, Michael G. Noll

Closes #2461 from mjsax/addStreamsUpdateSecton
2017-02-02 21:36:27 -08:00
Guozhang Wang c8baf28545 MINOR: Update Streams docs: quickstart and concepts
Added figures for topology and table-stream duality; added sections about aggregations; misc code fixes.

Author: Guozhang Wang <wangguoz@gmail.com>

Reviewers: Damian Guy, Sriram Subramanian

Closes #2482 from guozhangwang/KMinor-streams-docs-first-pass
2017-02-02 10:35:58 -08:00
Onur Karaman 063d534c51 KAFKA-3959: enforce offsets.topic.replication.factor upon __consumer_offsets auto topic creation (KIP-115)
Kafka brokers have a config called "offsets.topic.replication.factor" that specify the replication factor for the "__consumer_offsets" topic. The problem is that this config isn't being enforced. If an attempt to create the internal topic is made when there are fewer brokers than "offsets.topic.replication.factor", the topic ends up getting created anyway with the current number of live brokers. The current behavior is pretty surprising when you have clients or tooling running as the cluster is getting setup. Even if your cluster ends up being huge, you'll find out much later that __consumer_offsets was setup with no replication.

The cluster not meeting the "offsets.topic.replication.factor" requirement on the internal topic is another way of saying the cluster isn't fully setup yet.

The right behavior should be for "offsets.topic.replication.factor" to be enforced. Topic creation of the internal topic should fail with GROUP_COORDINATOR_NOT_AVAILABLE until the "offsets.topic.replication.factor" requirement is met. This closely resembles the behavior of regular topic creation when the requested replication factor exceeds the current size of the cluster, as the request fails with error INVALID_REPLICATION_FACTOR.

Author: Onur Karaman <okaraman@linkedin.com>

Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2177 from onurkaraman/KAFKA-3959
2017-02-01 19:55:06 -08:00
Ewen Cheslack-Postava cea4613533 MINOR: Update docs for version 0.10.2.0
Author: Ewen Cheslack-Postava <me@ewencp.org>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2480 from ewencp/docs-version-0.10.2.0
2017-02-01 10:25:24 -08:00
Colin P. Mccabe 4a14adbb63 KAFKA-4664; Update docs/protocol.html with KIP-97 information
Author: Colin P. Mccabe <cmccabe@confluent.io>

Reviewers: Gwen Shapira <cshapi@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2387 from cmccabe/KAFKA-4664
2017-01-27 21:39:27 +00:00
Ismael Juma 1956b46252 KAFKA-4578; Upgrade notes for 0.10.2.0
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>, Michael G. Noll

Closes #2445 from ijuma/kafka-4578-upgrade-notes-0.10.2
2017-01-27 11:10:28 +00:00
Vahid Hashemian 0c83eddcda MINOR: Escape '<' and '>' symbols in quickstart streams code snippet
This was missing from [an earlier PR](https://github.com/apache/kafka/pull/2247) that escaped these symbols in another section of the doc.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2453 from vahidhashemian/doc/escape_lt_gt_in_streams_code
2017-01-26 15:04:35 -08:00
Vahid Hashemian 63cf0e5877 MINOR: Update consumer group describe output in the documentation
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2409 from vahidhashemian/doc/update_consumer_group_describe_output
2017-01-24 16:24:19 -08:00
Hojjat Jafarpour 62206de4e5 KAFKA-4060 Follow-up: update docs accordingly
Updated the docs with changes in KAFKA-4060.

Author: Hojjat Jafarpour <hojjat@Hojjat-Jafarpours-MBP.local>
Author: Hojjat Jafarpour <hojjat@HojjatJpoursMBP.attlocal.net>

Reviewers: Ismael Juma, Matthias J. Sax, Guozhang Wang

Closes #2377 from hjafarpour/KAFKA-4060-docs-update
2017-01-23 10:21:06 -08:00
Shikhar Bhushan a8aa756166 KAFKA-3209: KIP-66: more single message transforms
Renames `HoistToStruct` SMT to `HoistField`.

Adds the following SMTs:
`ExtractField`
`MaskField`
`RegexRouter`
`ReplaceField`
`SetSchemaMetadata`
`ValueToKey`

Adds HTML doc generation and updates to `connect.html`.

Author: Shikhar Bhushan <shikhar@confluent.io>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2374 from shikhar/more-smt
2017-01-20 16:15:15 -08:00
Rajini Sivaram 666abafcc5 KAFKA-4589; SASL/SCRAM documentation
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Gwen Shapira <cshapi@gmail.com>, Sriharsha Chintalapani <harsha@hortonworks.com>, Jun Rao <junrao@gmail.com>

Closes #2369 from rajinisivaram/KAFKA-4589
2017-01-19 08:35:39 -08:00
Rajini Sivaram b4d8668d6d KAFKA-4363; Documentation for sasl.jaas.config property
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2316 from rajinisivaram/KAFKA-4363
2017-01-17 11:16:29 +00:00
Rajini Sivaram 80d6c64c1b KAFKA-4626; Add KafkaConsumer#close change to upgrade notes
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #2366 from rajinisivaram/KAFKA-4626
2017-01-13 09:32:46 -08:00
dasl c063172c23 MINOR: Fix error in design docs
Author: dasl <dasl-@users.noreply.github.com>

Reviewers: Gwen Shapira

Closes #2357 from dasl-/trunk
2017-01-12 11:28:46 -08:00
huxi 29d456cd21 KAFKA-4351; MirrorMaker with new consumer should support comma-separated regex
This makes it consistent with MirrorMaker with the old consumer.

Author: huxi <huxi@zhenrongbao.com>
Author: amethystic <huxi_2b@hotmail.com>

Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Ismael Juma <ismael@juma.me.uk>

Closes #2072 from amethystic/kafka-4351_Regex_behavior_change_for_new_consumer
2016-12-31 12:22:23 +00:00
Ismael Juma 76169f9e24 KAFKA-3284: Remove beta label from security documentation
4 release cycles (0.9.0.0, 0.10.0.0, 0.10.1.0, 0.10.2.0) should be enough
to remove the beta label.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2286 from ijuma/kafka-3284-security-beta-label
2016-12-22 08:15:21 -08:00
Gwen Shapira 7599d5d78b MINOR: Fix document header/footer links
Based on:  https://github.com/apache/kafka-site/pull/27

I recommend also merging into 10.1.0.0 branch to avoid mishaps when re-publishing docs.

Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2077 from gwenshap/protocol_doc_fix
2016-12-21 13:33:39 -08:00
Scott Ferguson 05d7664317 MINOR: fixup typo in ops.html
pretty boring docfix, "no" -> "not"

Author: Scott Ferguson <smferguson@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2269 from smferguson/fixup_typo_in_add_remove_topics
2016-12-21 11:37:32 -08:00
Hikiko Murakami e55205b810 MINOR: Fix typo on introduction page
Author: ash <ashishg@qburst.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2259 from ashishg-qburst/intro
2016-12-15 17:26:44 -08:00
Derrick Or 53428694a6 Separate Streams documentation and setup docs with easy to set variables
- Seperate Streams documentation out to a standalone page.
- Setup templates to use handlebars.js
- Create template variables to swap in frequently updated values like version number from a single file templateData.js

Author: Derrick Or <derrickor@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #2245 from derrickdoo/docTemplates
2016-12-13 17:59:49 -08:00
Vahid Hashemian 6626b058c7 MINOR: Fix Streams examples in documentation
Performed minor cleanup and escaped `<` and `>` so code examples are shown correctly in the browser.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Matthias J. Sax, Guozhang Wang

Closes #2247 from vahidhashemian/doc/fix_streams_doc
2016-12-13 09:44:11 -08:00
Matthias J. Sax 9bed8fbcfc KAFKA-4393: Improve invalid/negative TS handling
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Michael G. Noll, Eno Thereska, Damian Guy, Guozhang Wang

Closes #2117 from mjsax/kafka-4393-improveInvalidTsHandling
2016-12-09 16:17:36 -08:00
Stig Rohde Døssing 1503f7603c KAFKA-4387; KafkaConsumer should properly handle interrupts
See https://issues.apache.org/jira/browse/KAFKA-4387

Author: Stig Rohde Døssing <sdo@it-minds.dk>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #2110 from srdo/KAFKA-4387
2016-11-29 19:57:13 -08:00
Alex Loddengaard 6f0cbe7212 MINOR: doc fix related to monitoring consumer lag
Author: Alex Loddengaard <alexloddengaard@gmail.com>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #1814 from alexlod/consumer-lag-doc-fix
2016-11-29 09:57:22 -08:00
Ismael Juma 079ea89b36 KAFKA-4417: Update build dependencies for 0.10.2 cycle
Notes on the updates:
* Gradle to 3.2: better incremental build and faster IDE import times (https://docs.gradle.org/3.2/release-notes)
* zkclient to 0.10: it now uses slf4j-api instead of log4j
* zookeeper to 3.4.9: a few important bug fixes (http://zookeeper.apache.org/doc/r3.4.9/releasenotes.html)
* jackson to 2.8.5: lots of updates (https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.6, https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.7, https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.8)
* jersey to 2.24: bug fixes (https://jersey.java.net/release-notes/2.23.html, https://jersey.java.net/release-notes/2.24.html)
* jopt to 5.0.3: minor improvements, the major version bump is due to requiring Java 7 instead of 6 (https://pholser.github.io/jopt-simple/changes.html)
* argparse4j to 0.7.0: minor tweaks and improvements (https://github.com/tatsuhiro-t/argparse4j/blob/argparse4j-0.6.0/NEWS, https://github.com/tatsuhiro-t/argparse4j/blob/argparse4j-0.7.0/NEWS)
* Gradle plugins
* bcpkix to 1.55: quite a few additions, but nothing that matters to us (http://www.bouncycastle.org/releasenotes.html)

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2144 from ijuma/gradle-3.2-and-deps
2016-11-17 14:43:00 -08:00
Matthias J. Sax 6972d9476f MINOR: add upgrade guide for Kafka Streams API
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Michael G. Noll, Eno Thereska

Closes #2114 from mjsax/updateDocUpgradeSection
2016-11-09 15:31:11 -08:00
Apurva Mehta c62dc28ceb MINOR: Fix documentation of compaction
Removed a duplicate line and also cleaned up some of the language around compaction guarantees.

Author: Apurva Mehta <apurva.1618@gmail.com>

Reviewers: Gwen Shapira

Closes #2089 from apurvam/fix-documentation-of-compaction and squashes the following commits:

03c5bdd [Apurva Mehta] Fix line length to be consistent with the rest of the file
0af1a86 [Apurva Mehta] MINOR: fix duplicate line in docs for compaction.
2016-11-01 18:17:42 -07:00
Eno Thereska c2a8b86117 MINOR: Added more basic concepts to the documentation
Author: Eno Thereska <eno.thereska@gmail.com>

Reviewers: Michael G. Noll, Matthias J. Sax, Guozhang Wang

Closes #2030 from enothereska/minor-kip63-docs
2016-10-19 14:21:53 -07:00
Ben Stopford b8cfa167ed DOC: Documentation for Throttled Replication
Author: Ben Stopford <benstopford@gmail.com>

Reviewers: Jun Rao, Gwen Shapira

Closes #2033 from benstopford/throttling-docs
2016-10-18 14:24:52 -07:00
Manikumar Reddy O 3c02e5a20f MINOR: Remove duplicate doc headers
Author: Manikumar Reddy O <manikumar.reddy@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2021 from omkreddy/LATEST-DOC-CHANGE
2016-10-18 14:09:09 -07:00
Jason Gustafson 9662e466d9 MINOR: Some images should be centered in the documentation
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #2029 from hachikuji/center-some-images
2016-10-14 10:04:13 -07:00
Vahid Hashemian ae9532c6b3 MINOR: Fixed broken links in the documentation
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2010 from vahidhashemian/doc/fix_hyperlinks
2016-10-11 20:25:35 -07:00
Andrea Cosentino 9179bbc067 KAFKA-4130; Link to Varnish architect notes is broken
Author: Andrea Cosentino <ancosen@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1835 from oscerd/KAFKA-4130
2016-10-11 09:32:36 -07:00
Jason Gustafson 2e82bc0a10 MINOR: Add images missing from documentation
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>

Closes #2008 from hachikuji/add-missing-images
2016-10-10 19:58:53 -07:00
Vahid Hashemian 8e9e63138d MINOR: Fix table of contents and section numbers in the documentation
Removed a non-existing reference in table of contents and fixed some section numbers.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2004 from vahidhashemian/doc/fix_toc
2016-10-10 17:09:52 -07:00
Vahid Hashemian e972d2afd3 MINOR: Fix typos in documentation
And improve readability by adding proper punctuations.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2002 from vahidhashemian/doc/fix_typos
2016-10-10 15:58:37 -07:00
Derrick Or 06d6d98bed MINOR: Make documentation follow latest template
Make the latest version of our docs follow the latest site template structure.

Author: Derrick Or <derrickor@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #2003 from derrickdoo/docs-updates
2016-10-10 14:46:36 -07:00
Gwen Shapira bf98c47389 KAFKA-4244; Fix formatting issues in documentation
Author: Gwen Shapira <cshapi@gmail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1966 from gwenshap/KAFKA-4244
2016-10-10 13:39:05 -07:00
Jakub Dziworski 7a5133d554 MINOR: Fixed introduction doc - wrong streams api link
Author: Jakub Dziworski <jakub.dziworski@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jason Gustafson <jason@confluent.io>

Closes #1996 from JakubDziworski/doc_fix_streams_link
2016-10-10 10:15:47 -07:00
Jakub Dziworski fb22685fa9 MINOR: Fixed incomplete sentence in introduction docs
Author: Jakub Dziworski <jakub.dziworski@gmail.com>

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jason Gustafson <jason@confluent.io>

Closes #1994 from JakubDziworski/doc_fix
2016-10-10 09:26:37 -07:00
Vahid Hashemian 179d069857 MINOR: Update Quickstart in documentation to account for Windows platforms
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1990 from vahidhashemian/doc/quickstart_update_windows
2016-10-09 10:32:01 -07:00
Rajini Sivaram 3c8925946d KAFKA-4252; Fix purgatory metric name in documentation
Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1980 from rajinisivaram/KAFKA-4252
2016-10-06 11:19:43 -07:00
Ismael Juma 4efa833f45 MINOR: Clarify 0.10.1.0 upgrade docs
This is a minor change to fix the most glaring issues. We have another JIRA to revamp the upgrade docs.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jeff Klukas <jeff@klukas.net>, Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>

Closes #1971 from ijuma/kafka-4257-upgrade-docs-inconsitencies
2016-10-06 09:39:26 -07:00
Jason Gustafson d8d2f3aac4 MINOR: Tweak upgrade note on KIP-62 to include request.timeout.ms
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1960 from hachikuji/add-note-on-request-timeout
2016-10-04 10:38:49 -07:00
Jason Gustafson 1a396c9cf2 MINOR: Add upgrade notes for KIP-62
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Guozhang Wang <wangguoz@gmail.com>

Closes #1956 from hachikuji/add-kip-62-upgrade-notes
2016-10-03 21:17:11 -07:00
pilo 91d025e063 MINOR: Clarify doc on consumption of topics
In doc it stays:

_"Our topic is divided into a set of totally ordered partitions, each of which is consumed by one consumer at any given time."_

And consumer is described as:

_"We'll call **processes** that subscribe to topics and process the feed of published messages **consumers**."_

Which might lead to a wrong conclusion - that each partition can be read by one process at any given time.

I think this statements misses information about **consumer groups**, so i propose:

_"Our topic is divided into a set of totally ordered partitions, each of which is consumed by exactly one consumer (from each subscribed consumer groups) at any given time"_

This contribution is my original work and I license the work to the project under the project's open source license.

Author: pilo <jakub.pilimon@4finance.com>

Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>

Closes #1900 from pilloPl/minor/doc-fix
2016-10-03 19:35:54 -07:00
Ismael Juma cf0bf7c7a2 MINOR: Tweak implementation of `FetchRequest.shuffle` and upgrade.html improvements
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1955 from ijuma/kip-74-follow-up
2016-10-04 02:18:19 +01:00
Jason Gustafson 7115c66aef MINOR: Follow-up minor improvements/cleanup for KAFKA-3396
Author: Jason Gustafson <jason@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1946 from hachikuji/followup-for-kafka-3396
2016-10-03 16:49:50 -07:00
Edoardo Comar 8124f6e099 KAFKA-3396; Ensure Describe access is required to detect topic existence
Reopening of https://github.com/apache/kafka/pull/1428

Author: Edoardo Comar <ecomar@uk.ibm.com>
Author: Mickael Maison <mickael.maison@gmail.com>

Reviewers: Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #1908 from edoardocomar/KAFKA-3396
2016-09-30 23:07:51 -07:00
Vahid Hashemian d2a267b111 KAFKA-3697; Clean up website documentation of client usage
This is to imply that the Java consumer/producer are the recommended consumer/producer now.

Author: Vahid Hashemian <vahidhashemian@us.ibm.com>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1921 from vahidhashemian/KAFKA-3697
2016-09-29 19:37:20 -07:00
Jiangjie Qin aa506a6919 KAFKA-4194; Follow-up improvements/testing for ListOffsets v1 (KIP-79)
Author: Jiangjie Qin <becket.qin@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #1897 from becketqin/KAFKA-4194
2016-09-28 17:58:08 -07:00
Jay Kreps 16f85e84ce MINOR: Improve introduction section in docs to better cover connect and streams. Make uses and ecosystem pages stand alone. 2016-09-28 16:30:21 -07:00
Ismael Juma 61d3378bc8 MINOR: Remove no longer required --new-consumer switch in docs
Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1905 from ijuma/no-new-consumer-switch-in-examples
2016-09-27 16:49:46 -07:00
Sumit Arrawatia 36242b846a KAFKA-4151; Update public docs for Cluster Id (KIP-78)
- Updated implementation docs with details on Cluster Id generation.
- Mention cluster id in "noteworthy changes for 0.10.1.0" in upgrade docs.

Author: Sumit Arrawatia <sumit.arrawatia@gmail.com>
Author: arrawatia <sumit.arrawatia@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1895 from arrawatia/kip-78-docs
2016-09-24 10:23:18 +01:00
Jiangjie Qin eaaa433fc9 KAFKA-4148; Support ListOffsetRequest v1 and search offsets by timestamp in consumer (KIP-79)
Author: Jiangjie Qin <becket.qin@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>

Closes #1852 from becketqin/KAFKA-4148
2016-09-19 18:38:27 -07:00
Ismael Juma 3281b3c904 KAFKA-3283; Remove beta from new consumer documentation
Include a few clean-ups (also in producer section), mention deprecation plans and reorder so that the new consumer documentation is before the old consumers.

Author: Ismael Juma <ismael@juma.me.uk>

Reviewers: Jason Gustafson <jason@confluent.io>

Closes #1880 from ijuma/remove-beta-from-new-consumer-documentation
2016-09-19 14:26:26 -07:00
Rajini Sivaram f27a6f319a KAFKA-4079; Documentation for secure quotas
Details in KIP-55.

Author: Rajini Sivaram <rajinisivaram@googlemail.com>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #1847 from rajinisivaram/KAFKA-4079
2016-09-19 12:16:45 -07:00
Matthias J. Sax b8683026ea HOTFIX: changed quickstart donwload from 0.10.0.0 to 0.10.0.1
Author: Matthias J. Sax <matthias@confluent.io>

Reviewers: Ismael Juma <ismael@juma.me.uk>

Closes #1869 from mjsax/hotfix-doc
2016-09-17 22:45:29 +01:00
Eric Wasserman 5f040cd77f KAFKA-1981; Make log compaction point configurable
Now uses LogSegment.largestTimestamp to determine age of segment's messages.

Author: Eric Wasserman <eric.wasserman@gmail.com>

Reviewers: Jun Rao <junrao@gmail.com>

Closes #1794 from ewasserman/feat-1981
2016-09-11 20:45:05 -05:00
Mathieu Fenniak ca3f3a492c MINOR: Document that Connect topics should use compaction
Update documentation for Kafka Connect distributed’s config.storage.topic, offset.storage.topic, and status.storage.topic configuration values to indicate that all three should refer to compacted topics.

Author: Mathieu Fenniak <mathieu.fenniak@replicon.com>

Reviewers: Jason Gustafson

Closes #1832 from mfenniak/kafka-connect-topic-docs
2016-09-07 17:34:24 -07:00
Jiangjie Qin af9fc503de KAFKA-4099; Fix the potential frequent log rolling
Author: Jiangjie Qin <becket.qin@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>

Closes #1809 from becketqin/KAFKA-4099
2016-09-02 12:49:34 -07:00
Damian Guy 268cff7049 KAFKA-4112: Remove alpha quality label from Kafka Streams in docs
Rephrase 'alpha quality' wording in Streams section of api.html.
Couple of other minor fixes in streams.html

Author: Damian Guy <damian.guy@gmail.com>

Reviewers: Guozhang Wang, Ismael Juma, Michael G. Noll

Closes #1811 from dguy/kstreams-312
2016-09-01 12:21:55 -07:00
Andrew Otto b4ddd021b4 MINOR: Update MirrorMaker docs to remove multiple --consumer.config options
See:
- https://issues.apache.org/jira/browse/KAFKA-1650
- https://mail-archives.apache.org/mod_mbox/kafka-users/201512.mbox/%3CCAHwHRrUeTq_-EHXiUXdrbgHcRt-0E_t0+5kOYaF9Qy4aNVqYkAmail.gmail.com%3E

Author: Andrew Otto <acotto@gmail.com>

Reviewers: Gwen Shapira

Closes #1654 from ottomata/mirror-maker-doc-fix
2016-08-25 20:32:08 -07:00
Alex Loddengaard 20155ef87e MINOR: doc changes for QueueTimeMs JMX metrics.
Author: Alex Loddengaard <alexloddengaard@gmail.com>

Reviewers: Gwen Shapira

Closes #1706 from alexlod/doc-queuetimems-jmx
2016-08-25 20:29:20 -07:00