Commit Graph

709 Commits

Author SHA1 Message Date
Ismael Juma dbe618c63c
KAFKA-9907: Switch default build to Scala 2.13 (#8537)
Scala 2.13.2 introduces support for suppressing warnings,
which makes it possible to enable fatal warnings. This is
useful enough from a development perspective to justify
this change.

In addition, Scala 2.13.2 also has a Vector implementation
with significant performance improvements and encoding
of String matches to switches.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-04-23 07:02:54 -07:00
Sönke Liebau 402f2c1ece
MINOR: Some html fixes in Streams DSL documentation (#8503)
* Minor: Some html fixes Streams DSL documentation:
 - duplicate id
 - duplicate opening <span> element
 - surplus closing </div> tag

* Replaced opening/closing quotation mark codes with &quot; (they caused w3c validation to complain).

* Replaced right arrow that wasn't rendered correctly with &rarr;

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-04-23 10:44:36 +01:00
showuon a91b06708b
KAFKA-9863: replace the deprecated --zookeeper options in the documentation (#8482)
Reviewers: Ron Dagostino <rdagostino@confluent.io>, Colin P. McCabe <cmccabe@apache.org>
2020-04-22 13:42:35 -07:00
Liam Clarke-Hutchinson 7004fc22db
Add explicit grace period to tumbling window example (#8520) 2020-04-21 14:42:08 -05:00
Sönke Liebau cc4e3aa302
MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (#8505)
Currently, tumbling windows are defined as "a special case of hopping time windows" in the streams docs, but hopping windows are only explained in a subsequent section.
I think it would make sense to switch the order of these paragraphs around. To me this also makes more sense semantically.

Testing
Built the site and checked that everything looks ok and html is valid (or at least didn't contain any new warnings that were caused by this change).

Reviewers: Bill Bejeck <bbejeck@apache.org>
2020-04-17 11:25:51 -04:00
Konstantine Karantasis 8595267260
KAFKA-9810: Document Connect Root REST API on / (#8408)
Document the supported endpoint at the top-level (root) REST API resource and the information that it returns when a request is made to a Connect worker.

Fixes an omission in documentation after KAFKA-2369 and KAFKA-6311 (KIP-238)

Reviewers: Toby Drake <tobydrake7@gmail.com>, Soenke Liebau <soenke.liebau@opencore.com>
2020-04-03 13:27:27 -07:00
Boyang Chen 0520bc6c7c
MINOR: Update docs for KIP-530 and KIP-562 (#8388)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-30 13:06:54 -07:00
Scott 43f4372531
MINOR: Fix code example reference to SchemaBuilder call in Connect's documentation (#3029)
Simple doc fix in a code snippet in connect.html

Co-authored-by: Scott Ferguson <smferguson@gmail.com>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Konstantine Karantasis <konstantine@confluent.io>
2020-03-28 18:34:07 -07:00
Boyang Chen 4018f5e8b3
KAFKA-9760: Add KIP-447 protocol change to upgrade notes (#8350)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-03-26 09:40:49 -07:00
Jim Galasyn 0f48446690
DOCS-3625: Add section to config topic: parameters controlled by Kafka Streams (#8268)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 16:21:27 -07:00
Boyang Chen 4c9e56b358
KAFKA-9758: Doc changes for KIP-523 and KIP-527 (#8343)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 09:31:00 -07:00
Sönke Liebau eb7b8295ac
MINOR: Add clarifying "additionally" to streams aggregator docs that is missing in one place. (#8237)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-23 13:17:00 -07:00
Hossein Torabi afe26e2992
KAFKA-9563: Fix Kafka Connect documentation around consumer and producer overrides (#8124)
Kafka Connect main doc required a fix to distinguish between worker level producer and consumer overrides and per-connector level producer and consumer overrides, after the latter were introduced with KIP-458.  

* [KAFKA-9563] Fix Kafka connect consumer and producer override documentation

Co-authored-by: Konstantine Karantasis <konstantine@confluent.io>

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-03-22 20:07:21 -07:00
Alaa Zbair 5ccd3cd46d
KAFKA-8842: : Reading/Writing confused in Connect QuickStart Guide
In step 7 of the QuickStart guide, "Writing data from the console and writing it back to the console" should be "Reading data from the console and writing it back to the console".

Co-authored-by: Alaa Zbair <alaa.zbair@grenoble-inp.org>

Reviewer: Konstantine Karantasis <konstantine@confluent.io>
2020-03-21 22:00:33 -07:00
Agam Brahma 24d05aa601
KAFKA-9553; Improve measurement for loading groups and transactions (#8155)
This patch modifies the loading time metric to account for time spent waiting for the loading time task to be scheduled.

Reviewers: Jason Gustafson <jason@confluent.io>
2020-03-19 23:35:16 -07:00
Lee Dongjin 75268cb8e9 KAFKA-9327: Document GroupMetadata metrics
This commit adds documentation on following `GroupMetadata` gauges:

- `NumOffsets`
- `NumGroups`
- `NumGroupsPreparingRebalance`
- `NumGroupsCompletingRebalance`
- `NumGroupsEmpty`
- `NumGroupsStable`
- `NumGroupsDead`

cc/ gwenshap

Author: Lee Dongjin <dongjin@apache.org>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #7890 from dongjinleekr/feature/KAFKA-9327
2020-03-02 12:14:51 +05:30
Bruno Cadonna d2c42125aa
Adapt docs about metrics of Streams according to KIP-444 (#8171)
Adapts the docs about metrics of Streams according to https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%253A+Augment+metrics+for+Kafka+Streams

Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2020-02-27 09:48:53 -08:00
Sönke Liebau 30ab2297f1
KAFKA-9308: Reworded the ssl part of the security documentation (#8009)
Reworded the ssl part of the security documentation to fix various issues (mainly as noted by this jira, the problem that SAN extension values are not copied to certificates) and add some recommendations.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-25 16:23:56 +00:00
Ron Dagostino 9d53ad794d KAFKA-9567: Docs, system tests for ZooKeeper 3.5.7
These changes depend on [KIP-515: Enable ZK client to use the new TLS supported authentication](https://cwiki.apache.org/confluence/display/KAFKA/KIP-515%3A+Enable+ZK+client+to+use+the+new+TLS+supported+authentication), which was only added to 2.5.0. The upgrade to ZooKeeper 3.5.7 was merged to both 2.5.0 and 2.4.1 via https://issues.apache.org/jira/browse/KAFKA-9515, but this change must only be merged to 2.5.0 (it will break the system tests if merged to 2.4.1).

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Andrew Choi <li_andchoi@microsoft.com>

Closes #8132 from rondagostino/KAFKA-9567
2020-02-25 19:59:55 +05:30
Konstantine Karantasis bbfecaef72 MINOR: Document endpoints for connector topic tracking (KIP-558)
Update the site documentation to include the endpoints introduced with KIP-558 and a short paragraph on how this feature is used in Connect.

Author: Konstantine Karantasis <konstantine@confluent.io>

Reviewers: Toby Drake <tobydrake7@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #8148 from kkonstantine/kip-558-docs
2020-02-21 12:25:35 -08:00
Lee Dongjin b28aa4ece6 KAFKA-9586: Fix errored json filename in ops documentation
This PR is the counterpart of apache/kafka-site#253.

cc/ omkreddy

Author: Lee Dongjin <dongjin@apache.org>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #8149 from dongjinleekr/feature/KAFKA-9586
2020-02-21 18:49:11 +05:30
Ron Dagostino d9b8b86bdd KAFKA-9575: Mention ZooKeeper 3.5.7 upgrade
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*

*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #8139 from rondagostino/KAFKA-9575
2020-02-21 18:45:14 +05:30
Gunnar Morling 04b6641c14
MINOR: Wording fix in Streams DSL docs (#5692)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-20 11:35:27 +00:00
Lee Dongjin 9942e20a58
MINOR: fix omitted 'next.' in interactive queries documentation (#7883)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-02-19 10:23:59 -08:00
Rajini Sivaram b1449f683c
MINOR: Add upgrade note about TLSv1 and TLSv1.1 being disabled in 2.5.0 (#8128)
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-02-17 11:37:15 +00:00
Mitch 96c69da8c1
KAFKA-8507; Unify connection name flag for command line tool [KIP-499] (#8023)
This change updates ConsoleProducer, ConsumerPerformance, VerifiableProducer, and VerifiableConsumer classes to add and prefer the --bootstrap-server flag for defining the connection point of the Kafka cluster. This change is part of KIP-499: https://cwiki.apache.org/confluence/display/KAFKA/KIP-499+-+Unify+connection+name+flag+for+command+line+tool.

Reviewers: Ron Dagostino <rdagostino@confluent.io>, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>,  Chia-Ping Tsai <chia7712@gmail.com>, Jason Gustafson <jason@confluent.io>
2020-02-13 13:44:51 -08:00
Ron Dagostino 342f13a838 KAFKA-8843: KIP-515: Zookeeper TLS support
Signed-off-by: Ron Dagostino <rdagostinoconfluent.io>

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com>

Closes #8003 from rondagostino/KAFKA-8843
2020-02-08 21:16:48 +05:30
David Arthur 7e776b0462
Bump trunk to 2.6.0-SNAPSHOT (#8026) 2020-02-03 13:04:56 -05:00
commandini 9d96964792 MINOR: Update introduction page in Kafka documentation
Although APIs section in Kafka documentation lists 5 core APIs (https://kafka.apache.org/documentation/#api), introduction page in Kafka documentation lists 4 of them. I've added the missing list element to fix this incoherence.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-20 22:19:41 +05:30
Rajini Sivaram dbeaba5d9e
KAFKA-8847; Deprecate and remove usage of supporting classes in kafka.security.auth (#7966)
Removes references to the old scala Acl classes from kafka.security.auth (Acl, Operation, ResourceType, Resource etc.) and replaces these with the Java API. Only the old SimpleAclAuthorizer, AuthorizerWrapper used to wrap legacy authorizer instances and tests using SimpleAclAuthorizer continue to use the old API. Deprecates the old scala API.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-17 13:20:34 +00:00
Guozhang Wang 8c21fa837d
MINOR: Add upgrade guide for KAFKA-8421 (#7925)
Co-Authored-By: A. Sophie Blee-Goldman <ableegoldman@gmail.com>

Reviewers: A. Sophie Blee-Goldman <ableegoldman@gmail.com>
2020-01-13 12:40:06 -08:00
Bill Bejeck 03f763df8a
MINOR: Fixes for adding new DSL naming page (#7899)
Added re-direct for new page and added link to Developer Guide page
Reviewers:  Matthias J. Sax <mjsax@apache.org>, Sophie Blee-Goldman <sophie@confluent.io>,
2020-01-06 18:42:06 -05:00
Ismael Juma 6dc6f6a60d
KAFKA-9324: Drop support for Scala 2.11 (KIP-531) (#7859)
* Adjust build and documentation.
* Use lambda syntax for SAM types in `core`, `streams-scala` and
`connect-runtime` modules.
* Remove `runnable` and `newThread` from `CoreUtils` as lambda
syntax for SAM types make them unnecessary.
* Remove stale comment in `FunctionsCompatConversions`,
`KGroupedStream`, `KGroupedTable' and `KStream` about Scala 2.11,
the conversions are needed for Scala 2.12 too.
* Deprecate `org.apache.kafka.streams.scala.kstream.Suppressed`
and use `org.apache.kafka.streams.kstream.Suppressed` instead.
* Use `Admin.create` instead of `AdminClient.create`. Static methods
in Java interfaces can be invoked since Scala 2.12. I noticed that
MirrorMaker 2 uses `AdminClient.create`, but I did not change them
as Connectors have restrictions on newer client APIs.
* Improve efficiency in a few `Gauge` implementations by avoiding
unnecessary intermediate collections.
* Remove pointless `Option.apply` in `ZookeeperClient`
`SessionState` metric.
* Fix unused import/variable and other compiler warnings.
* Reduce visibility of some vals/defs.

Reviewers: Manikumar Reddy <manikumar@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Gwen Shapira <gwen@confluent.io>
2020-01-06 19:51:01 +01:00
Mickael Maison 46980d632c
MINOR: Update authorization primitives in security.html (#7509)
Since 2.3, there are 5 new APIs: ElectPreferredLeaders, IncrementalAlterConfigs, AlterPartitionReassignments, DescribePartitionReassignments and OffsetDelete

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-06 17:52:50 +00:00
A. Sophie Blee-Goldman e812784e74 fix rows (#7837)
Minor followup to #7825, should be cherry-picked to 2.4

Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-12-23 12:05:57 -05:00
Manikumar Reddy 96edabb6b8 MINOR: Update ZooKeeper upgrade notes
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Magnus Edenhill <magnus@edenhill.se>, Colin Patrick McCabe <cmccabe@apache.org>

Closes #7818 from omkreddy/zk-note
2019-12-20 00:47:12 +05:30
Harsha Laxman 6e56ba4b7d Update introduction.html (#7853)
Fixing small grammar issue with plural versions

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-12-19 13:04:38 +00:00
Mickael Maison c6e25bb362 MINOR: Fix static membership link in Streams upgrade notes (#7439)
Fix broken link

Reviewers: John Roesler <vvcephei@apache.org>
2019-12-18 11:21:57 -06:00
Guozhang Wang 763cb9683f HOTFIX: remove non-os streams example repo reference from web docs 2019-12-16 14:08:30 -08:00
wcarlson5 8b57f6cb3a KAFKA-6049: Add auto-repartitioning for cogroup (#7792)
Follow up to PR #7538 (KIP-150)

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2019-12-13 14:07:21 -08:00
A. Sophie Blee-Goldman 9567921fd4 MINOR: add UPGRADE_FROM to config docs (#7825)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2019-12-12 18:43:37 -08:00
A. Sophie Blee-Goldman 48da0c6c95 port paragrpah from CP docs (#7808)
The AK Streams architecture docs should explain how the maximum parallelism is determined
Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-12-09 16:35:17 -05:00
ravowlga123 deafc56fed KAFKA-8953: Rename UsePreviousTimeOnInvalidTimestamp to UsePartitionTimeOnInvalidTimestamp (#7633)
Implements KIP-530

Reviewer: Matthias J. Sax <matthias@confluent.io>
2019-12-08 19:23:19 -08:00
A. Sophie Blee-Goldman cd5618f866 MINOR: clarify node grouping of input topics using pattern subscription (#7793)
Updates the HTML docs and the javadoc.

Reviewers: John Roesler <vvcephei@apache.org>
2019-12-06 16:03:42 -06:00
Fábio Silva a2aa0a458b KAFKA-9255: Add 'timestamp' field to 'message' entry (#7764)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-12-02 14:39:28 +00:00
Sönke Liebau d7513693a3 KAFKA-9226: Updated documentation section on log deletion policies. (#7738)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-11-27 12:08:06 +00:00
Simon Clark af17468fa6 MINOR: Standardise "variable-length" vs "variable length" (#7720)
Both were used in the same sentence, which isn't necessarily clear.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-11-21 13:36:56 +00:00
Xin Wang 0ee9bf326d MINOR: Add missing parentheses in docs/streams/tutorial.html (#7696)
Reviewers: Jason Gustafson <jason@confluent.io>
2019-11-18 08:41:40 -08:00
Michael Gyarmathy f44272e392 fix typo in processor-api developer guide docs (#7689)
Fixed a small typo on the Processor API page of the Kafka Streams developer guide docs. ("buildeer" changed to "builder")

Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-11-14 10:32:35 -05:00
Bill Bejeck ed8810127f
KAFKA-9072: Add Topology naming to the dev guide (#7629)
Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <mjsax@apache.org>, Sophie Blee-Goldman <sophie@confluent.io>
2019-11-13 10:54:47 -05:00