Commit Graph

1010 Commits

Author SHA1 Message Date
Josep Prat 054d5f9e91
MINOR: fix code listings security.html (#10770)
Fix examples under security.html so they use the right bash icon (`>`
instead of `$`) and also uses the right tool for showing code listings.

Reviewers: Luke Chen <showuon@gmail.com>, Bruno Cadonna <cadonna@apache.org>
2021-06-01 13:20:35 +02:00
prince-mahajan db288e4a64
Update implementation.html (#10771)
Fixing the link to a cited blog. The existing link now points to a steroid website so need to pull the blog from internet archives.
2021-05-26 17:30:34 -07:00
A. Sophie Blee-Goldman bbe170af70
MINOR: deprecate TaskMetadata constructor and add KIP-740 notes to upgrade guide (#10755)
Quick followup to KIP-740 to actually deprecate this constructor, and update the upgrade guide with what we changed in KIP-740. I also noticed the TaskId#parse method had been modified previously, and should be re-added to the public TaskId class. It had no tests, so now it does

Reviewers: Matthias J. Sax <mjsax@confluent.io>, Luke Chen <showuon@gmail.com>
2021-05-26 10:35:12 -07:00
Josep Prat aa25176e77
MINOR: Kafka Streams code samples formating unification (#10651)
Code samples are now unified and correctly formatted.
Samples under Streams use consistently the prism library.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2021-05-21 17:27:42 +02:00
Daniyar Yeralin 6d1ae8bc00
KAFKA-8326: Introduce List Serde (#6592)
Introduce List serde for primitive types or custom serdes with a serializer and a deserializer according to KIP-466

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>, Matthias J. Sax <mjsax@conflunet.io>, John Roesler <roesler@confluent.io>, Michael Noll <michael@confluent.io>
2021-05-13 15:54:00 -07:00
Vito Jeng fae0784ce3
KAFKA-5876: KIP-216 Part 4, Apply InvalidStateStorePartitionException for Interactive Queries (#10657)
KIP-216, part 4 - apply InvalidStateStorePartitionException

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-05-10 17:29:58 -07:00
Matthias J. Sax 6a5992a814
KAFKA-8531: Change default replication factor config (#10532)
Implements KIP-733

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>
2021-05-05 16:16:06 -07:00
Vito Jeng 816f5c3b86
KAFKA-5876: KIP-216 Part 3, Apply StreamsNotStartedException for Interactive Queries (#10597)
KIP-216 Part 3: Throw StreamsNotStartedException if KafkaStreams state is CREATED

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-05-03 13:53:35 -07:00
Guozhang Wang 3ec6317ee6
KAFKA-12683: Remove deprecated UsePreviousTimeOnInvalidTimestamp (#10557)
Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-05-01 08:31:41 -07:00
Lee Dongjin 0a1ef2e982
MINOR: Group KafkaController, ReplicaManager metrics in documentation (#7891)
Some minor edits to the docs

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-04-30 12:50:37 -07:00
A. Sophie Blee-Goldman 3805f3706f
KAFKA-12574: KIP-732, Deprecate eos-alpha and replace eos-beta with eos-v2 (#10573)
Deprecates the following 

1. StreamsConfig.EXACTLY_ONCE
2. StreamsConfig.EXACTLY_ONCE_BETA
3. Producer#sendOffsetsToTransaction(Map offsets, String consumerGroupId)

And introduces a new StreamsConfig.EXACTLY_ONCE_V2 config. Additionally, this PR replaces usages of the term "eos-beta" throughout the code with the term "eos-v2"

Reviewers: Matthias J. Sax <mjsax@confluent.io>
2021-04-28 13:22:15 -07:00
JoelWee 0ea440b2af
KAFKA-6435: KIP-623 Add internal topics option to streamResetter (#8923)
Allow user to specify subset of internal topics to clean up with application reset tool

Reviewers: Boyang Chen <boyang@confluent.io>, Anna Sophie Blee-Goldman <ableegoldman@apache.org>, Walker Carlson <wcarlson@confluent.io>
2021-04-27 15:44:53 -07:00
Vito Jeng c972ac929e
KAFKA-5876: Apply UnknownStateStoreException for Interactive Queries (#9821)
KIP-216: IQ should throw different exceptions for different errors, Part 2

Reviewers: Matthias J. Sax <mjsax@confluent.io>, Anna Sophie Blee-Goldman <ableegoldman@apache.org>, Bruno Cadonna <cadonna@confluent.io>
2021-04-23 12:09:32 -07:00
Guozhang Wang 87b24025ce HOTFIX: typo in Streams DSL docs 2021-04-21 10:25:54 -07:00
Guozhang Wang 035449eb55
KAFKA-12633: Remove deprecated APIs in TopologyTestDriver (#10508)
As well as related test classes.

Reviewers: John Roesler <vvcephei@apache.org>
2021-04-18 10:46:01 -07:00
Tom Bentley f8f1769256
KIP-145: Add SMTs, HeaderFrom, DropHeaders and InsertHeader (#9549)
These SMTs were originally specified in KIP-145 but never implemented
at the time.

HeaderTo is not included since its original specification doesn't deal with
the fact that there can be >1 header with the same name, but a field can only
have a single value (which could be an array, but not if the headers for
the given name had different schemas).

Reviewers: Chris Egerton <chrise@confluent.io>, Mickael Maison <mickael.maison@gmail.com>
2021-04-16 15:11:25 +01:00
Ismael Juma 89933f21f2
KAFKA-12612: Remove `checksum` from ConsumerRecord/RecordMetadata for 3.0 (#10470)
The methods have been deprecated since 0.11 without replacement since
message format 2 moved the checksum to the record batch (instead of the
record).

Unfortunately, we did not deprecate the constructors that take a checksum
(even though we intended to) so we cannot remove them. I have deprecated
them for removal in 4.0 and added a single non deprecated constructor to
`ConsumerRecord` and `RecordMetadata` that take all remaining parameters.
`ConsumerRecord` could do with one additional convenience constructor, but
that requires a KIP and hence should be done separately.

Also:
* Removed `ChecksumMessageFormatter`, which is technically not public
API, but may have been used with the console consumer.
* Updated all usages of `ConsumerRecord`/`RecordMetadata` constructors
to use the non deprecated ones.
* Added tests for deprecated `ConsumerRecord/`RecordMetadata`
constructors.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Jacot <djacot@confluent.io>
2021-04-14 14:38:37 -07:00
David Jacot 23633cd1ec
KAFKA-12584; Remove deprecated `Sum` and `Total` classes (#10511)
`Sum` and `Total` classes were deprecated and replaced by `WindowedSum` and `CumulativeSum` in 2.4. This patch removes them for 3.0.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>
2021-04-13 11:48:17 +02:00
dengziming 88eb24db40
KAFKA-12637: Remove deprecated PartitionAssignor interface (#10512)
Remove PartitionAssignor and related classes, update docs and move unit test

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-04-12 18:37:01 -07:00
A. Sophie Blee-Goldman c608d8480e
KAFKA-7606: Remove deprecated options from StreamsResetter (#10411)
Remove deprecated --zookeeper and --execute flags

Reviewers: Matthias J. Sax <mjsax@confluent.io>
2021-04-12 18:24:18 -07:00
Lee Dongjin 0b0eaefc60
KAFKA-12408: Document omitted ReplicaManager metrics (#10258)
Reviewers: Tom Bentley <tbentley@redhat.com>
2021-04-12 15:35:15 +01:00
Andrew Lee c9edb8ad84
MINOR: Fix typo in docs (#10423)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2021-04-09 12:12:52 -07:00
Jorge Esteban Quilcate Otoya db0323e9ba
KAFKA-12449: Remove deprecated WindowStore#put (#10293)
Removes `WindowStore#put(K,V)` that was deprecated via KIP-474.

Reviewers: Matthias J. Sax <matthias@confluent.io>
2021-04-09 11:49:37 -07:00
Jim Galasyn e4b6ffae5d
MINOR: Fix left/outer join descriptions in Streams DSL reference topic (#10453)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2021-04-09 11:41:52 -07:00
David Jacot ff3b2d67a1
KAFKA-12591; Remove deprecated `quota.producer.default` and `quota.consumer.default` configurations (#10427)
`quota.producer.default` and `quota.consumer.default` were deprecated in AK 0.11.0.0. Dynamic default quotas must be used instead. This patch removes them for AK 3.0. 

Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Ismael Juma <ismael@juma.me.uk>
2021-04-09 18:11:34 +02:00
cyc 8d356b034b
KAFKA-12492: Fix the formatting of example RocksDBConfigSetter (#10486)
Fix the formatting of example RocksDBConfigSetter due to the un-arranged spaces within <pre> tag.

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-04-08 20:55:52 -07:00
Guozhang Wang d44e01be48 HOTFIX: add missing upgrade docs 2021-04-08 08:49:33 -07:00
Guozhang Wang 3ca5a3bb78
KAFKA-12568: Remove deprecated APIs in KStream, KTable and Joined (#10421)
This is related to KIP-307 / KIP-372 / KIP-479.

Reviewers: John Roesler <vvcephei@apache.org>
2021-04-07 17:38:43 -07:00
Marco Aurelio Lotz 33d0445b84
KAFKA-5146: remove Connect dependency from Streams module (#10131)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Ismael Juma <ismael@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2021-04-06 17:47:52 -07:00
Ismael Juma 2f36001987
KAFKA-12579: Remove various deprecated clients classes/methods for 3.0 (#10438)
* Remove `ExtendedSerializer` and `ExtendedDeserializer`, deprecated since 2.1.
The extra functionality was also made available in `Serializer` and `Deserializer`.
* Remove `close(long, TimeUnit)` from the producer, consumer and admin client,
deprecated since 2.0 for the consumer and 2.2 for the rest. The replacement is `close(Duration)`.
* Remove `ConsumerConfig.addDeserializerToConfig` and `ProducerConfig.addSerializerToConfig`,
deprecated since 2.7 with no replacement. These methods were not intended to be public API
and are likely not used much (if at all).
* Remove `NoOffsetForPartitionException.partition()`, deprecated since 0.11. `partitions()`
should be used instead.
* Remove `MessageFormatter.init(Properties)`, deprecated since 2.7. The `configure(Map)`
method should be used instead.
* Remove `kafka.common.MessageFormatter`, deprecated since 2.7.
`org.apache.kafka.common.MessageFormatter` should be used instead.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Jacot <djacot@confluent.io>
2021-04-06 08:54:53 -07:00
Ismael Juma 976e78e405
KAFKA-12590: Remove deprecated kafka.security.auth.Authorizer, SimpleAclAuthorizer and related classes in 3.0 (#10450)
These were deprecated in Apache Kafka 2.4 (released in December 2019) to be replaced
by `org.apache.kafka.server.authorizer.Authorizer` and `AclAuthorizer`.

As part of KIP-500, we will implement a new `Authorizer` implementation that relies
on a topic (potentially a KRaft topic) instead of `ZooKeeper`, so we should take the chance
to remove related tech debt in 3.0.

Details on the issues affecting the old Authorizer interface can be found in the KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-504+-+Add+new+Java+Authorizer+Interface

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Ron Dagostino <rdagostino@confluent.io>
2021-04-03 08:23:26 -07:00
Matthias J. Sax 57723f0c97
MINOR: remove KTable.to from the docs (#10464)
Reviewers: Bill Bejeck <bill@confluent.io>
2021-04-02 09:36:35 -07:00
Adil Houmadi cc00dffade
MINOR: Fix typo in MirrorMaker v2 documentation (#10433)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2021-04-01 16:08:28 +01:00
Ismael Juma 2342ec1d1c
KAFKA-12600: Remove deprecated config value `default` for client config `client.dns.lookup` (#10458)
The config has been deprecated since Kafka 2.6 (released ~1 year before
3.0), but it was the default before it got deprecated. As such, it's
reasonably unlikely that people would have set it explicitly.

Given the confusing `default` name even though it's _not_ the default, I
think we should remove it in 3.0.

Also remove `ClientDnsLookup.DEFAULT` (not public API), which unlocks
a number of code simplications.

Reviewers: David Jacot <djacot@confluent.io>
2021-04-01 07:59:59 -07:00
David Jacot 6d7a9012dc
KAFKA-8405; Remove deprecated `kafka-preferred-replica-election` command (#10443)
The `kafka-preferred-replica-election` command was deprecated in 2.4. This path removes it for 3.0. `kafka-leader-election` can be used instead.

Reviewers: Ismael Juma <ismael@juma.me.uk>, Chia-Ping Tsai <chia7712@gmail.com>, Jason Gustafson <jason@confluent.io>
2021-03-31 20:45:18 +02:00
Bruno Cadonna 58d4e527d0
MINOR: Fix docs for end-to-end record latency metrics (#10449)
Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-03-31 11:02:16 -07:00
David Jacot 667578860a
KAFKA-12577; Remove deprecated `ConfigEntry` constructor for 3.0 (#10436)
ConfigEntry's public constructor was deprecated in 1.1.0. This patch removes it in AK 3.0.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>
2021-03-31 11:28:29 +02:00
Ismael Juma a7251ce174
KAFKA-12581: Remove deprecated `Admin.electPreferredLeaders` (#10440)
`Admin.electLeaders` is the replacement since the deprecation in Apache Kafka 2.4.0.
The methods were originally introduced in Apache Kafka 2.2.0, so they were only
non deprecated for two releases.

Reviewers: David Jacot <djacot@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
2021-03-30 22:04:13 -07:00
Ismael Juma e8754ba7a0
KAFKA-12578: Remove deprecated security classes/methods for 3.0 (#10435)
More specifically, remove deprecated:
- Constants in SslConfigs
- Constants in SaslConfigs
- AclBinding constructor
- AclBindingFilter constructor
- PrincipalBuilder and DefaultPrincipalBuilder classes
- ResourceFilter

Also simplify tests and code that no longer have to handle the removed `PrincipalBuilder`.

These removals seem non controversial. There is a straightforward alternative. The
deprecations happened in 1.0.0 and 2.0.0.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2021-03-30 22:02:16 -07:00
David Jacot 0018c3c089
KAFKA-12573; Remove deprecated `Metric#value` (#10425)
Reviewers: Ismael Juma <ismael@juma.me.uk>
2021-03-30 09:28:48 +02:00
Walker Carlson e971d94eb3
KAFKA-8784: remove default close for RocksDBConfigSetter (#10416)
Remove the default close implementation for RocksDBConfigSetter to avoid accidental memory leaks via C++ backed objects which are constructed but not closed by the user

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-03-29 14:26:39 -07:00
Guozhang Wang d5fd491bf7
KAFKA-7106: remove deprecated Windows APIs (#10378)
1. Remove all deprecated APIs in KIP-328.
2. Remove deprecated APIs in Windows in KIP-358.

Reviewers: John Roesler <vvcephei@apache.org>
2021-03-28 12:33:40 -07:00
Guozhang Wang b8058829bb
KAFKA-12562: Remove deprecated APIs in KafkaStreams and returned state classes (#10412)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>
2021-03-28 12:20:31 -07:00
Luke Chen 69baa03404
MINOR: fix the wrong/missing anchor in docs caused link error (#10413)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2021-03-28 11:39:53 -07:00
Matthias J. Sax bd9a5da9d1
MINOR: update Streams upgrade notes with removed APIs (#10420)
Reviewer: Guozhang Wang <guozhang@confluent.io>
2021-03-28 11:36:42 -07:00
Guozhang Wang 7071ded2a6
KAFKA-12524: Remove deprecated segments() (#10379)
Reviewers: Boyang Chen <boyang@confluent.io>
2021-03-23 21:05:42 -07:00
John Roesler 7c45976c8d MINOR: bump documentation.html to 3.0 2021-03-22 10:59:17 -05:00
A. Sophie Blee-Goldman 5c22e03e9e
MINOR: add missing docs for record-e2e-latency metrics (#10251)
Need to add missing docs for the record-e2e-latency metrics, and new TRACE recording level

Reviewers: Walker Carlson <wcarlson@confluent.io>
2021-03-04 14:42:56 -08:00
Ismael Juma 7a3ebbebbc
KAFKA-12415 Prepare for Gradle 7.0 and restrict transitive scope for non api dependencies (#10203)
Gradle 7.0 is required for Java 16 compatibility and it removes a number of
deprecated APIs. Fix most issues preventing the upgrade to Gradle 7.0.
The remaining ones are more complicated and should be handled
in a separate PR. Details of the changes:

* Release tarball no longer includes includes test, sources, javadoc and test sources jars (these
are still published to the Maven Central repository).
* Replace `compile` with `api` or `implementation` - note that `implementation`
dependencies appear with `runtime` scope in the pom file so this is a (positive)
change in behavior
* Add missing dependencies that were uncovered by the usage of `implementation`
* Replace `testCompile` with `testImplementation`
* Replace `runtime` with `runtimeOnly` and `testRuntime` with `testRuntimeOnly`
* Replace `configurations.runtime` with `configurations.runtimeClasspath`
* Replace `configurations.testRuntime` with `configurations.testRuntimeClasspath` (except for
the usage in the `streams` project as that causes a cyclic dependency error)
* Use `java-library` plugin instead of `java`
* Use `maven-publish` plugin instead of deprecated `maven` plugin - this changes the
commands used to publish and to install locally, but task aliases for `install` and
`uploadArchives` were added for backwards compatibility
* Removed `-x signArchives` line from the readme since it was wrong (it was a
no-op before and it fails now, however)
* Replaces `artifacts` block with an approach that works with the `maven-publish` plugin
* Don't publish `jmh-benchmark` module - the shadow jar is pretty large and not
particularly useful (before this PR, we would publish the non shadow jars)
* Replace `version` with `archiveVersion`, `baseName` with `archiveBaseName` and
`classifier` with `archiveClassifier`
* Update Gradle and plugins to the latest stable version (7.0 is not stable yet)
* Use `plugin` DSL to configure plugins
* Updated notable changes for 3.0

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Randall Hauch <rhauch@gmail.com>
2021-03-04 11:22:22 -08:00
Michael G. Noll 1fd75bf1ed
KAFKA-12393: Document multi-tenancy considerations (#334) (#10263)
KAFKA-12393: Document multi-tenancy considerations
Addressed review feedback by @dajac and @rajinisivaram
Ported from apache/kafka-site#334

Reviewers: Bill Bejeck <bbejeck@apache.org>
2021-03-04 10:47:48 -05:00
Lee Dongjin ea005cc700
KAFKA-12407: Document Controller Health Metrics (#10257)
Reviewers: Luke Chen <showuon@gmail.com>, Dong Lin <lindong28@gmail.com>
2021-03-04 22:24:50 +08:00
Matthias J. Sax e2a0d0c90e
MINOR: bump release version to 3.0.0-SNAPSHOT (#10186)
Reviewers: Konstantine Karantasis <konstantine@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
2021-02-24 17:49:18 -08:00
vamossagar12 8bd0e218d8
MINOR: Add release documentation for KIP-614 (#10051)
Reviewers: Bruno Cadonna <bruno@confluent.io>, John Roesler <vvcephei@apache.org>
2021-02-24 13:34:38 -06:00
Luke Chen 5a9435afd1
KAFKA-12350: Correct the default value in doc (#10165)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2021-02-25 01:52:34 +08:00
A. Sophie Blee-Goldman 0605b690e9
MINOR: document restriction against running multiple Streams apps on same state.dir (#10187)
Reviewers: Leah Thomas <lthomas@confluent.io>, Almog Gavra <almog@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2021-02-23 08:39:19 -08:00
Jim Galasyn 9728b4ff92
KAFKA-12160: Remove max.poll.interval.ms from config docs (#10182)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2021-02-22 17:06:25 -08:00
Matthias J. Sax b35ca4349d
KAFKA-9274: Throw TaskCorruptedException instead of TimeoutException when TX commit times out (#10072)
Part of KIP-572: follow up work to PR #9800. It's not save to retry a TX commit after a timeout, because it's unclear if the commit was successful or not, and thus on retry we might get an IllegalStateException. Instead, we will throw a TaskCorruptedException to retry the TX if the commit failed.

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>
2021-02-19 13:36:07 -08:00
Luke Chen b17f70ed66
MINOR: Fix broken link in quickstart.html (#10161)
Update the old anchor #intro_topic to #intro_concepts_and_terms

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2021-02-19 11:36:12 +00:00
Justine Olshan 16a8f2c3c4 MINOR: Add note about topic IDs to upgrade doc (#10125)
Reviewers: Jun Rao <junrao@gmail.com>
2021-02-17 09:48:08 -08:00
Jim Galasyn da58ce4065
MINOR: Clarify config names for EOS versions 1 and 2 (#9670)
Reviewers: Boyang Chen <boyang@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2021-02-16 17:11:24 -08:00
Cheng Tan 19506b6c01
Fill in the 2.8 release note for Authorizer (#9865)
Reviewers: Ismael Juma <ismael@juma.me.uk>, Rajini Sivaram <rajinisivaram@googlemail.com>
2021-02-10 11:24:13 +00:00
Luke Chen 8bd5ceb3d2
MINOR: Update security doc
Author: Luke Chen <showuon@gmail.com>

    Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2021-02-08 18:16:47 +05:30
John Roesler 1f240ce179 bump to 2.9 development version 2021-02-07 09:25:36 -06:00
Luke Chen 3a766bd30d MINOR: remove the indent in security doc
Reading the security doc recently, and one thing annoys me: the long indent in front of each command, ex:

![image](https://user-images.githubusercontent.com/43372967/106253151-155a1e80-6252-11eb-97f7-e8f4f60c6047.png)

![image](https://user-images.githubusercontent.com/43372967/106253176-1be89600-6252-11eb-845b-b8e478534fd7.png)

![image](https://user-images.githubusercontent.com/43372967/106253249-3589dd80-6252-11eb-82e1-45fe188b26d6.png)

https://kafka.apache.org/documentation/#security_ssl_key

Removing all the prefix indent in the commands. I'll create another PR to kafka-site if this change accepts. Thanks.

Author: Luke Chen <showuon@gmail.com>

Reviewers: Tom Bentley <tbentley@redhat.com>, Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #10002 from showuon/doc_code_indent
2021-02-06 19:50:54 +05:30
Matthias J. Sax d01d97ed66
MINOR: add docs for KIP-680 (#10048)
Reviewers: Bill Bejeck <bill@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>
2021-02-04 20:18:20 -08:00
Matthias J. Sax 016128c920
MINOR: Extends RocksDB docs (#10046)
We recommend users to switch to jemalloc for RocksDB.

Co-authored-by: Jim Galasyn <jim.galasyn@confluent.io>

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Rohan Desai <rohan@confluent.io>, A. Sophie Blee-Goldman <sophie@confluent.io>
2021-02-04 18:50:19 -08:00
Ivan Ponomarev 5552da3a20
KAFKA-5488: Add type-safe split() operator (#9107)
Implements KIP-418, that deprecated the `branch()` operator in favor of the newly added and type-safe `split()` operator.

Reviewers: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2021-02-04 16:23:35 -08:00
leah 8bdfc5770d
MINOR: adding docs for failed stream thread metric (#9974)
Add docs for KIP-663.

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Walker Carlson <wcarlson@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2021-02-02 10:12:22 -08:00
leah f5a2fbac6d
KAFKA-10366 & KAFKA-9649: Implement KIP-659 to allow TimeWindowedDeserializer and TimeWindowedSerde to handle window size (#9253)
See KIP details and discussions here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-659%3A+Improve+TimeWindowedDeserializer+and+TimeWindowedSerde+to+handle+window+size

Deprecates methods that allow users to skip setting a window size when one is needed. Adds a window size streams config to allow the timeWindowedDeserializer to calculate window end time.

Reviewers: Walker Carlson <wcarlson@confluent.io>, John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>
2021-02-01 16:20:35 -08:00
Walker Carlson eff51a82f2
MINOR: update docs for KIP-663, KIP-696, and KIP-671 (#9969)
Co-authored-by: Bruno Cadonna <bruno@confluent.io>

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bruno Cadonna <bruno@confluent.io>
2021-01-29 09:00:56 -08:00
Michael G. Noll f4983f4172
KAFKA-8930: MirrorMaker v2 documentation (#324) (#9983)
This adds a new user-facing documentation "Geo-replication (Cross-Cluster Data Mirroring)" section to the Kafka Operations documentation that covers MirrorMaker v2.

Was already merged to kafka-site via apache/kafka-site#324.
Reviewers: Bill Bejeck <bbejeck@apache.org>
2021-01-27 11:51:34 -05:00
leah 6c8033abf3
MINOR: Add docs for StreamJoined changes (#9951)
Add docs for KIP-689.

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2021-01-26 11:38:13 -08:00
Bruno Cadonna add160d522
KAFKA-9924: Add docs for RocksDB properties-based metrics (#9895)
Document the new properties-based metrics for RocksDB

Reviewers: Leah Thomas <lthomas@confluent.io>, Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2021-01-19 13:30:23 -08:00
Matthias J. Sax 2e0c686ec1
MINOR: improve KafkaStreams replication factor documentation (#9829)
Reviewers: Jason Gustafson <jason@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>
2021-01-06 11:31:05 -08:00
Matthias J. Sax e3b0ea81ca
MINOR: add missing quickstart.html file (#9722)
Reviewers: Guozhang Wang <guozhang@confluent.io>, Bill Bejeck <bill@confluent.io>
2020-12-22 16:45:50 -08:00
Matthias J. Sax bb813310c7
MINOR: remove dangling quickstart-*.html (#9721)
Reviewers: Guozhang Wang <guozhang@confluent.io>
2020-12-22 16:19:43 -08:00
Luke Chen 3717ab0ff2
KAFKA-10776: Add version attribute in RequestsPerSec metrics documentation (#9661)
See https://cwiki.apache.org/confluence/display/KAFKA/KIP-272%3A+Add+API+version+tag+to+broker%27s+RequestsPerSec+metric

Reviewers: Ismael Juma <ismael@juma.me.uk>
2020-12-15 06:39:07 -08:00
high.lee 88c8180957
KAFKA-8147: Update upgrade notes for KIP-446 (#8965)
Reviewer: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2020-12-04 12:56:59 -07:00
Jim Galasyn 03cff6cb59
MINOR: Fix KTable-KTable foreign-key join example (#9683)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-12-03 13:17:01 -07:00
James Cheng 16eb1f5cd1
KAFKA-10473: Add docs on partition size-on-disk, and other log-related metrics (#9276)
kafka.log,type=Log,name=Size
kafka.log,type=Log,name=NumLogSegments
kafka.log,type=Log,name=LogStartOffset
kafka.log,type=Log,name=LogEndOffset

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-12-03 11:18:12 -08:00
fml2 90bc7e9777
KAFKA-10722: Described the types of the used state stores (#9607)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-11-30 17:25:47 -07:00
Tom Bentley 6add0594a7
MINOR: Remove erroneous extra <code> in design doc (#9657)
Reviewers: Bill Bejeck <bbejeck@apache.org>
2020-11-30 15:41:14 -05:00
Tom Bentley 91679f247a
KAFKA-10692: Add delegation.token.secret.key, deprecate ...master.key (#9623)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-11-19 15:26:25 +00:00
zhangyue19921010 156584915a
MINOR: Remove the extra brackets in the demo code (#9586)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-11-11 19:57:40 +08:00
Bill Bejeck 27153e2268
MINOR: Add back section taken out by mistake (#9544)
Reviewers: Matthias J. Sax <mjsax@apache.org>
2020-11-04 14:44:29 -05:00
bill b5a1c2649b update version in quickstart to current 2020-11-04 09:16:30 -05:00
Bill Bejeck e847f057e3
KAFKA-10679: [Streams] migrate kafka-site updated docs to kafka/docs (#9554)
During the AK website upgrade, changes made to kafka-site weren't migrated back to kafka-docs.

This PR is an attempt at porting the streams changes to kafka/docs

For the most part, the bulk of the changes in the PR are cosmetic.

For testing:

I reviewed the PR diffs
Rendered the changes locally

Reviewers: John Roesler <john@confluent.io>
2020-11-04 08:30:10 -05:00
Igor Soarez b3e77dfad9
KAFKA-10036: Improve handling and documentation of Suppliers (#9000)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-11-03 17:30:15 -08:00
Bill Bejeck b4f00d7ef6
KAFKA-10679: Migrate upgrade changes from site to kafka/docs (#9551)
During the AK website upgrade, changes made to kafka-site weren't migrated back to kafka-docs.

This PR is an initial attempt at porting the changes to kafka/docs, but it does not include the streams changes. Those will come in a separate PR.

For the most part, the bulk of the changes in the PR are cosmetic. Only the introduction.html has substantial changes, but it's a direct port from the live documentation.

For testing:

I reviewed the PR diffs
Rendered the changes locally

Reviewers: Matthias J. Sax <mjsax@apache.org>
2020-11-03 13:40:44 -05:00
Kengo Seki fb3d26d5b7
MINOR: Fix an example in the Kafka Streams tutorial to be compilable (#6647)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-11-03 10:49:49 +08:00
Bill Bejeck d39382d3e2
MINOR: Move upgraded docs from site to kafak docs (#9532)
Reviewers: Matthias J. Sax <mjsax@apache.org>
2020-10-31 07:52:14 -04:00
Alexander Iskuskov 51c8c69e3a
MINOR: Fix documentation for KIP-585 (#9524)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-10-29 10:08:00 +08:00
Bill Bejeck d5c77695a2
MINOR: Add KIP-431 to upgrade.html file (#9514)
Reviewers: Matthias J. Sax <mjsax@apache.org>
2020-10-27 19:35:33 -04:00
Bill Bejeck 81465a1951
MINOR: Add KIP-584 to upgrade.html file (#9511)
Reviewers: Kowshik Prakasam <kprakasam@confluent.io>, Manikumar Reddy <manikumar.reddy@gmail.com>, Boyang Chen <boyang@confluent.io>
2020-10-27 14:35:02 -04:00
Bill Bejeck 714d125772
MINOR: Update docs to point to next release add notable features for 2.7 (#9483)
Reviewers: Matthias J. Sax <mjsax@apache.org>
2020-10-22 17:57:41 -04:00
Okada Haruki 94ccd4d25e
MINOR: Fix timestampDelta type in doc (#8870)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-10-22 16:42:53 +08:00
Emre Hasegeli 850e9922c1
MINOR: Fix upgrade.mode references (#5645)
Reviewers: John Roesler <john@confluent.io>, Andrew Choi <andrew.choi@uwaterloo.ca>, Chia-Ping Tsai <chia7712@gmail.com>
2020-10-20 22:17:56 +08:00
Matthias Merdes d841b912d1
MINOR: inconsistent naming for the output topic in the stream documentation (#9265)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-10-18 23:19:49 +08:00
bill 4d3036bb4e Updating trunk versions after cutting branch for 2.7 2020-10-08 07:47:36 -04:00
Rajini Sivaram 7be8bd8cbf
KAFKA-10338; Support PEM format for SSL key and trust stores (KIP-651) (#9345)
Adds support for SSL key and trust stores to be specified in PEM format either as files or directly as configuration values.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-10-06 19:13:43 +01:00
Matthias J. Sax a15387f34d
KAFKA-9274: Revert deprecation of `retries` for producer and admin clients (#9333)
Reviewer: John Roesler <john@confluent.io>
2020-09-30 12:13:34 -07:00
Chia-Ping Tsai 821c1ac664
KAFKA-10479; Throw exception if users try to update non-reconfigurable configs of existing listeners (#9284)
The previous dynamic configuration validation did not actually compare new configs to original configs as intended, so the expected exception was not thrown.

Reviewers: Ismael Juma <ismael@juma.me.uk>, Rajini Sivaram <rajinisivaram@googlemail.com>, Jason Gustafson <jason@confluent.io>
2020-09-29 12:15:08 -07:00
Ankit Kumar e98b38cb1e
KAFKA-8360: Docs do not mention RequestQueueSize JMX metric (#9325)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Viktor Somogyi-Vass <viktorsomogyi@gmail.com>

Co-authored-by: akumar <akumar@cloudera.com>
2020-09-29 10:32:17 +02:00
A. Sophie Blee-Goldman c2789934e4
MINOR: add docs for 2.7 TRACE-level e2e latency metrics (#9339)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-09-26 20:37:48 -07:00
A. Sophie Blee-Goldman 3ef6c045e2
MINOR: Streams docs fixes (#9308)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-09-19 17:25:47 -07:00
Luke Chen 77f6175d2e
update source link in interactive query page (#9261)
Currently, the source reference are all pointing to the 1.0 version codes,
which is obviously wrong. Update to the current dotVersion.

Reviewers: John Roesler <vvcephei@apache.org>
2020-09-14 10:11:03 -05:00
leah 8260d7cdfb
KAFKA-5636: Add Sliding Windows documentation (#9264)
Add necessary documentation for KIP-450, adding sliding window aggregations to KStreams

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2020-09-10 21:55:11 -07:00
Yuriy Badalyantc d0111d3bcd
KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616) (#8955)
Wildcard import of the old org.apache.kafka.streams.scala.Serdes leads
to a name clash because some of implicits has the same names as types
from the scala's std lib. The new oak.streams.scala.serialization.Serdes is
the same as the old Serdes, but without name clashes.
The old one is marked as deprecated.

Also, add missing serdes for UUID, ByteBuffer and Short types in
the new Serdes.

Implements: KIP-616

Reviewers: John Roesler <vvcephei@apache.org>
2020-09-01 16:48:40 -05:00
John Thomas c8453e9a68
KAFKA-10316: Consider renaming getter method for Interactive Queries (#9146)
- updates docs for KIP-648

Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-08-21 12:39:24 -07:00
Matthias J. Sax a5aadeb31a
MINOR: fix HTML (#9127)
Reviewers: Guozhang Wang <guozhang@confluent.io>, Boyang Chen <boyang@confluent.io>
2020-08-06 11:22:14 -07:00
Matthias J. Sax b351493543
KAFKA-9274: Remove `retries` for global task (#9047)
- part of KIP-572
 - removed the usage of `retries` in `GlobalStateManger`
 - instead of retries the new `task.timeout.ms` config is used

Reviewers: John Roesler <john@confluent.io>, Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-08-05 14:14:18 -07:00
Jason Gustafson 4e2546ab83
MINOR: Add notes for 2.6 on reassignment tool changes (#9109)
Add some notable changes to the reassignment tool for the 2.6 release.

Reviewers: Randall Hauch <rhauch@gmail.com>
2020-07-31 10:20:34 -07:00
Ismael Juma 6004ee9f22
MINOR: Recommend Java 11 (#9080)
Java 11 has been recommended for a while, but the ops section had not been updated.
Also added `-XX:+ExplicitGCInvokesConcurrent` which has been in `kafka-run-class`
for a while. Finally, tweaked the text slightly to read better.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-07-26 12:25:30 -07:00
Matthias J. Sax 194c56fce2
KAFKA-9274: Mark `retries` config as deprecated and add new `task.timeout.ms` config (#8864)
- part of KIP-572
 - deprecates producer config `retries` (still in use)
 - deprecates admin config `retries` (still in use)
 - deprecates Kafka Streams config `retries` (will be ignored)
 - adds new Kafka Streams config `task.timeout.ms` (follow up PRs will leverage this new config)

Reviewers: John Roesler <john@confluent.io>, Jason Gustafson <jason@confluent.io>, Randall Hauch <randall@confluent.io>
2020-07-21 12:19:13 -07:00
A. Sophie Blee-Goldman 9b30276e58
KAFKA-9161: add docs for KIP-441 and KIP-613 and other configs that need fixing (#9027)
Add docs for KIP-441 and KIP-613.
Fixed some miscellaneous unrelated issues in the docs:
* Adds some missing configs to the Streams config docs: max.task.idle.ms,topology.optimization, default.windowed.key.serde.inner.class, and default.windowed.value.serde.inner.class
* Defines the previously-undefined default windowed serde class configs, including choosing a default (null) and giving them a doc string, so the yshould nwo show up in the auto-generated general Kafka config docs
* Adds a note to warn users about the rocksDB bug that prevents setting a strict capacity limit and counting write buffer memory against the block cache

Reviewers: Bruno Cadonna <bruno@confluent.io>, John Roesler <vvcephei@apache.org>
2020-07-20 16:27:28 -05:00
Rajini Sivaram 9c8f75c4b6
KAFKA-10223; Use NOT_LEADER_OR_FOLLOWER instead of non-retriable REPLICA_NOT_AVAILABLE for consumers (#8979)
Brokers currently return NOT_LEADER_FOR_PARTITION to producers and REPLICA_NOT_AVAILABLE to consumers if a replica is not available on the broker during reassignments. Non-Java clients treat REPLICA_NOT_AVAILABLE as a non-retriable exception, Java consumers handle this error by explicitly matching the error code even though it is not an InvalidMetadataException. This PR renames NOT_LEADER_FOR_PARTITION to NOT_LEADER_OR_FOLLOWER and uses the same error for producers and consumers. This is compatible with both Java and non-Java clients since all clients handle this error code (6) as retriable exception. The PR also makes ReplicaNotAvailableException a subclass of InvalidMetadataException.
    - ALTER_REPLICA_LOG_DIRS continues to return REPLICA_NOT_AVAILABLE. Retained this for compatibility since this request never returned NOT_LEADER_FOR_PARTITION earlier. 
   -  MetadataRequest version 0 also returns REPLICA_NOT_AVAILABLE as topic-level error code for compatibility. Newer versions filter these out and return Errors.NONE, so didn't change this.
   - Partition responses in MetadataRequest return REPLICA_NOT_AVAILABLE to indicate that one of the replicas is not available. Did not change this since NOT_LEADER_FOR_PARTITION is not suitable in this case.

Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Bob Barrett <bob.barrett@confluent.io>
2020-07-17 20:05:11 +01:00
Matthias J. Sax 934033bd17
MINOR: update required MacOS version (#9025)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
2020-07-15 10:09:27 -07:00
Jim Galasyn d945b287ff
KAFKA-6453: Document how timestamps are computed for aggregations and joins (#9009)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-07-13 15:01:14 -07:00
Jim Galasyn 7e66848200
MINOR: Restore stream-table duality description (#8995)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-07-09 13:50:29 -07:00
Tom Bentley cfadb8b8d9
MINOR: Define the term tombstone, since it's used elsewhere in the docs (#3480)
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-07-09 16:27:52 +05:30
Jim Galasyn 11a8549479
MINOR: document timestamped state stores (#8920)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-07-06 18:09:24 -07:00
Gal Margalit 5dade71a8f
MINOR: Update introduction.html (#8969) 2020-07-02 05:21:29 +05:30
Aakash Shah 16eb9c0bfe
KAFKA-10153: Error Reporting in Connect Documentation (#8858)
Added a section about error reporting in Connect documentation, and another about how to safely use the new errant record reporter in SinkTask implementations.

Author: Aakash Shah <ashah@confluent.io>
Reviewer: Randall Hauch <rhauch@gmail.com>
2020-07-01 10:18:16 -05:00
Matthias J. Sax 712cc5d073
KAFKA-10168: fix StreamsConfig parameter name variable (#8865)
Implements KIP-626.

Reviewers: Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
2020-06-19 17:41:06 -07:00
Tom Bentley a1f429d4f6
MINOR: Documentation for KIP-585 (#8839)
* Add documentation for using transformation predicates.
* Add `PredicateDoc` for generating predicate config docs, following the style of `TransformationDoc`.
* Fix the header depth mismatch.
* Avoid generating HTML ids based purely on the config name since there
are very likely to conflict (e.g. #name). Instead allow passing a function
which can be used to generate an id from a config key.

The docs have been generated and tested locally. 

Reviewer: Konstantine Karantasis <konstantine@confluent.io>
2020-06-15 22:07:01 -07:00
Matthias J. Sax 922ed9d2af
MINOR: fix HTML markup (#8823)
Reviewers: Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
2020-06-08 12:18:01 -04:00
Badai Aqrandista 50c3012890
KAFKA-9313: Set `use_all_dns_ips` as the new default for `client.dns.lookup` (KIP-602) (#8644)
This applies to the producer, consumer, admin client, connect worker
and inter broker communication.

`ClientDnsLookup.DEFAULT` has been deprecated and a warning
will be logged if it's explicitly set in a client config.

Reviewers: Mickael Maison <mickael.maison@gmail.com>, Ismael Juma <ismael@juma.me.uk>
2020-06-04 06:21:52 -07:00
Nikolay 8b22b81596
KAFKA-9320: Enable TLSv1.3 by default (KIP-573) (#8695)
1. Enables `TLSv1.3` by default with Java 11 or newer.
2. Add unit tests that cover the various TLSv1.2 and TLSv1.3 combinations.
3. Extend `benchmark_test.py` and `replication_test.py` to run with 'TLSv1.2'
or 'TLSv1.3'.

Reviewers: Ismael Juma <ismael@juma.me.uk>
2020-06-02 15:34:43 -07:00
Randall Hauch 19e40788e7
Bump trunk to 2.7.0-SNAPSHOT (#8746) 2020-06-01 21:23:09 -05:00
feyman2016 77f123391c
Kafka-10064 Add documentation for KIP-571 (#8760)
* Update documentation for KIP-571

* update doc for KIP-571

* fix comments
2020-06-01 17:19:26 -07:00
Randall Hauch ed3c42bd9b
MINOR: Update documentation.html to refer to 2.6 (#8745) 2020-05-28 12:56:30 -05:00
Randall Hauch 9e2d4b508d
MINOR: Update documentation.html to refer to 2.5 (#8744) 2020-05-28 12:52:04 -05:00
Jim Galasyn bfc9c9909d
MINOR: Remove retries entry from Streams config table (#8733)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-05-27 12:24:15 -07:00
Paul 075bbcfec4
KAFKA-7523: Add ConnectedStoreProvider to Processor API (#6824)
Implements KIP-401:
 - Add ConnectedStoreProvider interface
 - let Processor/[*]Transformer[*]Suppliers extend ConnectedStoreProvider
 - allows to add and connect state stores to processors/transformers implicitly

Reviewers: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2020-05-27 10:57:14 -07:00
Navinder Pal Singh Brar 4b97e50eea
MINOR: Added doc for KIP-535 and updated it for KIP-562 (#8395)
Reviewers: Boyang Chen <boyang@confluent.io>, Vinoth Chandar <vchandar@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2020-05-26 16:24:00 -07:00
Valeria 9aeb78b990
KAFKA-6755: Allow literal value for MaskField SMT (#6284)
Implemented KIP-437 by adding a new optional configuration property for the `MaskField` transformation that allows users to define a replacement literal for specific fields in matching records.

Author: Valeria Vasylieva <valeria.vasylieva@gmail.com>
Reviewer: Randall Hauch <rhauch@gmail.com>
2020-05-24 09:22:40 -05:00
Jim Galasyn 9111374974
MINOR: Improve security documentation for Kafka Streams #8710
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-05-22 11:40:25 -07:00
Matthias J. Sax 27824baa21
KAFKA-10003: Mark KStream.through() as deprecated and update Scala API (#8679)
- part of KIP-221

Co-authored-by: John Roesler <john@confluent.io>
2020-05-22 08:41:28 -07:00
showuon 82fb6e9473
MINOR: Update stream documentation (#8622)
fix broken links
rephrase a sentence
update the version number
Reviewers: Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@apache.org>
2020-05-19 11:05:55 -04:00
showuon 3ec5e8e652
MINOR: Small fixes in the documentation (#8623)
These minor documentation fixes included: 

1. fix broken links
2. remove redundant sentences
3. fix content format issue

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-05-19 07:31:06 -07:00
Matthias J. Sax f04fd7e9fd
MINOR: updated MacOS compatibility statement for RocksDB (#8687)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-05-18 12:43:35 -07:00
showuon b72ab51e4d
Remove redundant TOC and introduction in Running Streams Applications (#8686)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-05-18 09:30:19 -07:00
Levani Kokhreidze 8a0fcd1695
KAFKA-8611: update documentation for KIP-221 (#8558)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-05-15 15:28:23 -07:00
Matthias J. Sax 318063a16a
KAFKA-9466: Update Kafka Streams docs for KIP-447 (#8621)
Reviewers: Boyang Chen <boyang@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-05-11 19:11:58 -07:00
showuon 91efb30dfa
MINOR: Improve formatting in docs (#8611)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-05-08 15:32:50 -07:00
Jorge Esteban Quilcate Otoya 4803c779de
MINOR: Use `firstTimestamp` instead of `baseTimestamp` in docs (#8584)
In the docs, the `baseTimestamp` term is used when `firstTimestamp` is defined in the on-disk format and implementation.

Reviewers: Jason Gustafson <jason@confluent.io>
2020-05-01 14:43:47 -07:00
Viktor Somogyi 74fdbc42dd
KAFKA-8835; KIP-352 docs update (#7434)
Doc updates for KIP-352.

Reviewers: Stanislav Kozlovski <stanislav_kozlovski@outlook.com>, Jason Gustafson <jason@confluent.io>
2020-04-30 08:36:01 -07:00
showuon e4e28883d4
MINOR: Fix typo and rephrase content in ops docs (#8581)
1. fix typo: `atleast` -> `at least`
2. add missing `--` from `--bootstrap-servers` argument to be consistent
3. rephrase a sentence, to make it more clear:

before: `LinkedIn is currently running JDK 1.8 u5 (looking to upgrade to a newer version) with the G1 collector`

It will misguide the users to use JDK 1.8 u5, while the JDK 1.8 u251 is already released, which will include many important bug fixes. I did some rephrase as below:

after: `At the time this is written, LinkedIn is running JDK 1.8 u5 (looking to upgrade to a newer version) with the G1 collector`

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-04-29 23:06:21 -07:00
showuon 417d9b1801
MINOR: Fix broken JMX link in docs by adding missing starting double quote (#8587)
The href attribute missed the starting double quote, so the hyperlink is interpreted as https://docs.oracle.com/.../agent.html", with a redundant tailing double quote. 

Add the missing starting double quote back to fix this issue.

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-04-29 20:25:33 -07:00
Jorge Esteban Quilcate Otoya 57961c33de
MINOR: Fix partition numbering from 0 to P-1 instead of P in docs (#8572)
Small typo fix

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-04-28 12:10:03 -07:00
showuon 2bc87de08e
MINOR: Update the link to the Raft paper in docs (#8560)
Update the link to the Raft paper pointing to the Usenix conference link.

old broken link: https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf
new link: https://www.usenix.org/system/files/conference/atc14/atc14-paper-ongaro.pdf

Reviewers: Boyang Chen <boyang@confluent.io>, Konstantine Karantasis <konstantine@confluent.io>
2020-04-27 23:25:46 -07:00
Ismael Juma dbe618c63c
KAFKA-9907: Switch default build to Scala 2.13 (#8537)
Scala 2.13.2 introduces support for suppressing warnings,
which makes it possible to enable fatal warnings. This is
useful enough from a development perspective to justify
this change.

In addition, Scala 2.13.2 also has a Vector implementation
with significant performance improvements and encoding
of String matches to switches.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-04-23 07:02:54 -07:00
Sönke Liebau 402f2c1ece
MINOR: Some html fixes in Streams DSL documentation (#8503)
* Minor: Some html fixes Streams DSL documentation:
 - duplicate id
 - duplicate opening <span> element
 - surplus closing </div> tag

* Replaced opening/closing quotation mark codes with &quot; (they caused w3c validation to complain).

* Replaced right arrow that wasn't rendered correctly with &rarr;

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-04-23 10:44:36 +01:00
showuon a91b06708b
KAFKA-9863: replace the deprecated --zookeeper options in the documentation (#8482)
Reviewers: Ron Dagostino <rdagostino@confluent.io>, Colin P. McCabe <cmccabe@apache.org>
2020-04-22 13:42:35 -07:00
Liam Clarke-Hutchinson 7004fc22db
Add explicit grace period to tumbling window example (#8520) 2020-04-21 14:42:08 -05:00
Sönke Liebau cc4e3aa302
MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (#8505)
Currently, tumbling windows are defined as "a special case of hopping time windows" in the streams docs, but hopping windows are only explained in a subsequent section.
I think it would make sense to switch the order of these paragraphs around. To me this also makes more sense semantically.

Testing
Built the site and checked that everything looks ok and html is valid (or at least didn't contain any new warnings that were caused by this change).

Reviewers: Bill Bejeck <bbejeck@apache.org>
2020-04-17 11:25:51 -04:00
Konstantine Karantasis 8595267260
KAFKA-9810: Document Connect Root REST API on / (#8408)
Document the supported endpoint at the top-level (root) REST API resource and the information that it returns when a request is made to a Connect worker.

Fixes an omission in documentation after KAFKA-2369 and KAFKA-6311 (KIP-238)

Reviewers: Toby Drake <tobydrake7@gmail.com>, Soenke Liebau <soenke.liebau@opencore.com>
2020-04-03 13:27:27 -07:00
Boyang Chen 0520bc6c7c
MINOR: Update docs for KIP-530 and KIP-562 (#8388)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-30 13:06:54 -07:00
Scott 43f4372531
MINOR: Fix code example reference to SchemaBuilder call in Connect's documentation (#3029)
Simple doc fix in a code snippet in connect.html

Co-authored-by: Scott Ferguson <smferguson@gmail.com>

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Konstantine Karantasis <konstantine@confluent.io>
2020-03-28 18:34:07 -07:00
Boyang Chen 4018f5e8b3
KAFKA-9760: Add KIP-447 protocol change to upgrade notes (#8350)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-03-26 09:40:49 -07:00
Jim Galasyn 0f48446690
DOCS-3625: Add section to config topic: parameters controlled by Kafka Streams (#8268)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 16:21:27 -07:00
Boyang Chen 4c9e56b358
KAFKA-9758: Doc changes for KIP-523 and KIP-527 (#8343)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 09:31:00 -07:00
Sönke Liebau eb7b8295ac
MINOR: Add clarifying "additionally" to streams aggregator docs that is missing in one place. (#8237)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-23 13:17:00 -07:00
Hossein Torabi afe26e2992
KAFKA-9563: Fix Kafka Connect documentation around consumer and producer overrides (#8124)
Kafka Connect main doc required a fix to distinguish between worker level producer and consumer overrides and per-connector level producer and consumer overrides, after the latter were introduced with KIP-458.  

* [KAFKA-9563] Fix Kafka connect consumer and producer override documentation

Co-authored-by: Konstantine Karantasis <konstantine@confluent.io>

Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2020-03-22 20:07:21 -07:00
Alaa Zbair 5ccd3cd46d
KAFKA-8842: : Reading/Writing confused in Connect QuickStart Guide
In step 7 of the QuickStart guide, "Writing data from the console and writing it back to the console" should be "Reading data from the console and writing it back to the console".

Co-authored-by: Alaa Zbair <alaa.zbair@grenoble-inp.org>

Reviewer: Konstantine Karantasis <konstantine@confluent.io>
2020-03-21 22:00:33 -07:00
Agam Brahma 24d05aa601
KAFKA-9553; Improve measurement for loading groups and transactions (#8155)
This patch modifies the loading time metric to account for time spent waiting for the loading time task to be scheduled.

Reviewers: Jason Gustafson <jason@confluent.io>
2020-03-19 23:35:16 -07:00
Lee Dongjin 75268cb8e9 KAFKA-9327: Document GroupMetadata metrics
This commit adds documentation on following `GroupMetadata` gauges:

- `NumOffsets`
- `NumGroups`
- `NumGroupsPreparingRebalance`
- `NumGroupsCompletingRebalance`
- `NumGroupsEmpty`
- `NumGroupsStable`
- `NumGroupsDead`

cc/ gwenshap

Author: Lee Dongjin <dongjin@apache.org>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #7890 from dongjinleekr/feature/KAFKA-9327
2020-03-02 12:14:51 +05:30
Bruno Cadonna d2c42125aa
Adapt docs about metrics of Streams according to KIP-444 (#8171)
Adapts the docs about metrics of Streams according to https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%253A+Augment+metrics+for+Kafka+Streams

Reviewers: Boyang Chen <boyang@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2020-02-27 09:48:53 -08:00
Sönke Liebau 30ab2297f1
KAFKA-9308: Reworded the ssl part of the security documentation (#8009)
Reworded the ssl part of the security documentation to fix various issues (mainly as noted by this jira, the problem that SAN extension values are not copied to certificates) and add some recommendations.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-25 16:23:56 +00:00
Ron Dagostino 9d53ad794d KAFKA-9567: Docs, system tests for ZooKeeper 3.5.7
These changes depend on [KIP-515: Enable ZK client to use the new TLS supported authentication](https://cwiki.apache.org/confluence/display/KAFKA/KIP-515%3A+Enable+ZK+client+to+use+the+new+TLS+supported+authentication), which was only added to 2.5.0. The upgrade to ZooKeeper 3.5.7 was merged to both 2.5.0 and 2.4.1 via https://issues.apache.org/jira/browse/KAFKA-9515, but this change must only be merged to 2.5.0 (it will break the system tests if merged to 2.4.1).

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Andrew Choi <li_andchoi@microsoft.com>

Closes #8132 from rondagostino/KAFKA-9567
2020-02-25 19:59:55 +05:30
Konstantine Karantasis bbfecaef72 MINOR: Document endpoints for connector topic tracking (KIP-558)
Update the site documentation to include the endpoints introduced with KIP-558 and a short paragraph on how this feature is used in Connect.

Author: Konstantine Karantasis <konstantine@confluent.io>

Reviewers: Toby Drake <tobydrake7@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>

Closes #8148 from kkonstantine/kip-558-docs
2020-02-21 12:25:35 -08:00
Lee Dongjin b28aa4ece6 KAFKA-9586: Fix errored json filename in ops documentation
This PR is the counterpart of apache/kafka-site#253.

cc/ omkreddy

Author: Lee Dongjin <dongjin@apache.org>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #8149 from dongjinleekr/feature/KAFKA-9586
2020-02-21 18:49:11 +05:30
Ron Dagostino d9b8b86bdd KAFKA-9575: Mention ZooKeeper 3.5.7 upgrade
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*

*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>

Closes #8139 from rondagostino/KAFKA-9575
2020-02-21 18:45:14 +05:30
Gunnar Morling 04b6641c14
MINOR: Wording fix in Streams DSL docs (#5692)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-20 11:35:27 +00:00
Lee Dongjin 9942e20a58
MINOR: fix omitted 'next.' in interactive queries documentation (#7883)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-02-19 10:23:59 -08:00
Rajini Sivaram b1449f683c
MINOR: Add upgrade note about TLSv1 and TLSv1.1 being disabled in 2.5.0 (#8128)
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-02-17 11:37:15 +00:00
Mitch 96c69da8c1
KAFKA-8507; Unify connection name flag for command line tool [KIP-499] (#8023)
This change updates ConsoleProducer, ConsumerPerformance, VerifiableProducer, and VerifiableConsumer classes to add and prefer the --bootstrap-server flag for defining the connection point of the Kafka cluster. This change is part of KIP-499: https://cwiki.apache.org/confluence/display/KAFKA/KIP-499+-+Unify+connection+name+flag+for+command+line+tool.

Reviewers: Ron Dagostino <rdagostino@confluent.io>, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>,  Chia-Ping Tsai <chia7712@gmail.com>, Jason Gustafson <jason@confluent.io>
2020-02-13 13:44:51 -08:00
Ron Dagostino 342f13a838 KAFKA-8843: KIP-515: Zookeeper TLS support
Signed-off-by: Ron Dagostino <rdagostinoconfluent.io>

Author: Ron Dagostino <rdagostino@confluent.io>

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com>

Closes #8003 from rondagostino/KAFKA-8843
2020-02-08 21:16:48 +05:30
David Arthur 7e776b0462
Bump trunk to 2.6.0-SNAPSHOT (#8026) 2020-02-03 13:04:56 -05:00
commandini 9d96964792 MINOR: Update introduction page in Kafka documentation
Although APIs section in Kafka documentation lists 5 core APIs (https://kafka.apache.org/documentation/#api), introduction page in Kafka documentation lists 4 of them. I've added the missing list element to fix this incoherence.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-20 22:19:41 +05:30
Rajini Sivaram dbeaba5d9e
KAFKA-8847; Deprecate and remove usage of supporting classes in kafka.security.auth (#7966)
Removes references to the old scala Acl classes from kafka.security.auth (Acl, Operation, ResourceType, Resource etc.) and replaces these with the Java API. Only the old SimpleAclAuthorizer, AuthorizerWrapper used to wrap legacy authorizer instances and tests using SimpleAclAuthorizer continue to use the old API. Deprecates the old scala API.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-17 13:20:34 +00:00
Guozhang Wang 8c21fa837d
MINOR: Add upgrade guide for KAFKA-8421 (#7925)
Co-Authored-By: A. Sophie Blee-Goldman <ableegoldman@gmail.com>

Reviewers: A. Sophie Blee-Goldman <ableegoldman@gmail.com>
2020-01-13 12:40:06 -08:00
Bill Bejeck 03f763df8a
MINOR: Fixes for adding new DSL naming page (#7899)
Added re-direct for new page and added link to Developer Guide page
Reviewers:  Matthias J. Sax <mjsax@apache.org>, Sophie Blee-Goldman <sophie@confluent.io>,
2020-01-06 18:42:06 -05:00
Ismael Juma 6dc6f6a60d
KAFKA-9324: Drop support for Scala 2.11 (KIP-531) (#7859)
* Adjust build and documentation.
* Use lambda syntax for SAM types in `core`, `streams-scala` and
`connect-runtime` modules.
* Remove `runnable` and `newThread` from `CoreUtils` as lambda
syntax for SAM types make them unnecessary.
* Remove stale comment in `FunctionsCompatConversions`,
`KGroupedStream`, `KGroupedTable' and `KStream` about Scala 2.11,
the conversions are needed for Scala 2.12 too.
* Deprecate `org.apache.kafka.streams.scala.kstream.Suppressed`
and use `org.apache.kafka.streams.kstream.Suppressed` instead.
* Use `Admin.create` instead of `AdminClient.create`. Static methods
in Java interfaces can be invoked since Scala 2.12. I noticed that
MirrorMaker 2 uses `AdminClient.create`, but I did not change them
as Connectors have restrictions on newer client APIs.
* Improve efficiency in a few `Gauge` implementations by avoiding
unnecessary intermediate collections.
* Remove pointless `Option.apply` in `ZookeeperClient`
`SessionState` metric.
* Fix unused import/variable and other compiler warnings.
* Reduce visibility of some vals/defs.

Reviewers: Manikumar Reddy <manikumar@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Gwen Shapira <gwen@confluent.io>
2020-01-06 19:51:01 +01:00
Mickael Maison 46980d632c
MINOR: Update authorization primitives in security.html (#7509)
Since 2.3, there are 5 new APIs: ElectPreferredLeaders, IncrementalAlterConfigs, AlterPartitionReassignments, DescribePartitionReassignments and OffsetDelete

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2020-01-06 17:52:50 +00:00
A. Sophie Blee-Goldman e812784e74 fix rows (#7837)
Minor followup to #7825, should be cherry-picked to 2.4

Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-12-23 12:05:57 -05:00
Manikumar Reddy 96edabb6b8 MINOR: Update ZooKeeper upgrade notes
Author: Manikumar Reddy <manikumar.reddy@gmail.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Magnus Edenhill <magnus@edenhill.se>, Colin Patrick McCabe <cmccabe@apache.org>

Closes #7818 from omkreddy/zk-note
2019-12-20 00:47:12 +05:30
Harsha Laxman 6e56ba4b7d Update introduction.html (#7853)
Fixing small grammar issue with plural versions

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-12-19 13:04:38 +00:00
Mickael Maison c6e25bb362 MINOR: Fix static membership link in Streams upgrade notes (#7439)
Fix broken link

Reviewers: John Roesler <vvcephei@apache.org>
2019-12-18 11:21:57 -06:00
Guozhang Wang 763cb9683f HOTFIX: remove non-os streams example repo reference from web docs 2019-12-16 14:08:30 -08:00
wcarlson5 8b57f6cb3a KAFKA-6049: Add auto-repartitioning for cogroup (#7792)
Follow up to PR #7538 (KIP-150)

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2019-12-13 14:07:21 -08:00
A. Sophie Blee-Goldman 9567921fd4 MINOR: add UPGRADE_FROM to config docs (#7825)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2019-12-12 18:43:37 -08:00
A. Sophie Blee-Goldman 48da0c6c95 port paragrpah from CP docs (#7808)
The AK Streams architecture docs should explain how the maximum parallelism is determined
Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-12-09 16:35:17 -05:00
ravowlga123 deafc56fed KAFKA-8953: Rename UsePreviousTimeOnInvalidTimestamp to UsePartitionTimeOnInvalidTimestamp (#7633)
Implements KIP-530

Reviewer: Matthias J. Sax <matthias@confluent.io>
2019-12-08 19:23:19 -08:00
A. Sophie Blee-Goldman cd5618f866 MINOR: clarify node grouping of input topics using pattern subscription (#7793)
Updates the HTML docs and the javadoc.

Reviewers: John Roesler <vvcephei@apache.org>
2019-12-06 16:03:42 -06:00
Fábio Silva a2aa0a458b KAFKA-9255: Add 'timestamp' field to 'message' entry (#7764)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-12-02 14:39:28 +00:00
Sönke Liebau d7513693a3 KAFKA-9226: Updated documentation section on log deletion policies. (#7738)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-11-27 12:08:06 +00:00
Simon Clark af17468fa6 MINOR: Standardise "variable-length" vs "variable length" (#7720)
Both were used in the same sentence, which isn't necessarily clear.

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2019-11-21 13:36:56 +00:00
Xin Wang 0ee9bf326d MINOR: Add missing parentheses in docs/streams/tutorial.html (#7696)
Reviewers: Jason Gustafson <jason@confluent.io>
2019-11-18 08:41:40 -08:00
Michael Gyarmathy f44272e392 fix typo in processor-api developer guide docs (#7689)
Fixed a small typo on the Processor API page of the Kafka Streams developer guide docs. ("buildeer" changed to "builder")

Reviewers: Bill Bejeck <bbejeck@gmail.com>
2019-11-14 10:32:35 -05:00