Commit Graph

265 Commits

Author SHA1 Message Date
leah f5a2fbac6d
KAFKA-10366 & KAFKA-9649: Implement KIP-659 to allow TimeWindowedDeserializer and TimeWindowedSerde to handle window size (#9253)
See KIP details and discussions here: https://cwiki.apache.org/confluence/display/KAFKA/KIP-659%3A+Improve+TimeWindowedDeserializer+and+TimeWindowedSerde+to+handle+window+size

Deprecates methods that allow users to skip setting a window size when one is needed. Adds a window size streams config to allow the timeWindowedDeserializer to calculate window end time.

Reviewers: Walker Carlson <wcarlson@confluent.io>, John Roesler <vvcephei@apache.org>, Guozhang Wang <wangguoz@gmail.com>
2021-02-01 16:20:35 -08:00
Walker Carlson eff51a82f2
MINOR: update docs for KIP-663, KIP-696, and KIP-671 (#9969)
Co-authored-by: Bruno Cadonna <bruno@confluent.io>

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bruno Cadonna <bruno@confluent.io>
2021-01-29 09:00:56 -08:00
leah 6c8033abf3
MINOR: Add docs for StreamJoined changes (#9951)
Add docs for KIP-689.

Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2021-01-26 11:38:13 -08:00
Matthias J. Sax 2e0c686ec1
MINOR: improve KafkaStreams replication factor documentation (#9829)
Reviewers: Jason Gustafson <jason@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>
2021-01-06 11:31:05 -08:00
high.lee 88c8180957
KAFKA-8147: Update upgrade notes for KIP-446 (#8965)
Reviewer: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2020-12-04 12:56:59 -07:00
Jim Galasyn 03cff6cb59
MINOR: Fix KTable-KTable foreign-key join example (#9683)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-12-03 13:17:01 -07:00
fml2 90bc7e9777
KAFKA-10722: Described the types of the used state stores (#9607)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-11-30 17:25:47 -07:00
Bill Bejeck e847f057e3
KAFKA-10679: [Streams] migrate kafka-site updated docs to kafka/docs (#9554)
During the AK website upgrade, changes made to kafka-site weren't migrated back to kafka-docs.

This PR is an attempt at porting the streams changes to kafka/docs

For the most part, the bulk of the changes in the PR are cosmetic.

For testing:

I reviewed the PR diffs
Rendered the changes locally

Reviewers: John Roesler <john@confluent.io>
2020-11-04 08:30:10 -05:00
Igor Soarez b3e77dfad9
KAFKA-10036: Improve handling and documentation of Suppliers (#9000)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-11-03 17:30:15 -08:00
Kengo Seki fb3d26d5b7
MINOR: Fix an example in the Kafka Streams tutorial to be compilable (#6647)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-11-03 10:49:49 +08:00
Matthias Merdes d841b912d1
MINOR: inconsistent naming for the output topic in the stream documentation (#9265)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2020-10-18 23:19:49 +08:00
Matthias J. Sax a15387f34d
KAFKA-9274: Revert deprecation of `retries` for producer and admin clients (#9333)
Reviewer: John Roesler <john@confluent.io>
2020-09-30 12:13:34 -07:00
A. Sophie Blee-Goldman c2789934e4
MINOR: add docs for 2.7 TRACE-level e2e latency metrics (#9339)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-09-26 20:37:48 -07:00
A. Sophie Blee-Goldman 3ef6c045e2
MINOR: Streams docs fixes (#9308)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-09-19 17:25:47 -07:00
Luke Chen 77f6175d2e
update source link in interactive query page (#9261)
Currently, the source reference are all pointing to the 1.0 version codes,
which is obviously wrong. Update to the current dotVersion.

Reviewers: John Roesler <vvcephei@apache.org>
2020-09-14 10:11:03 -05:00
leah 8260d7cdfb
KAFKA-5636: Add Sliding Windows documentation (#9264)
Add necessary documentation for KIP-450, adding sliding window aggregations to KStreams

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2020-09-10 21:55:11 -07:00
Yuriy Badalyantc d0111d3bcd
KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616) (#8955)
Wildcard import of the old org.apache.kafka.streams.scala.Serdes leads
to a name clash because some of implicits has the same names as types
from the scala's std lib. The new oak.streams.scala.serialization.Serdes is
the same as the old Serdes, but without name clashes.
The old one is marked as deprecated.

Also, add missing serdes for UUID, ByteBuffer and Short types in
the new Serdes.

Implements: KIP-616

Reviewers: John Roesler <vvcephei@apache.org>
2020-09-01 16:48:40 -05:00
John Thomas c8453e9a68
KAFKA-10316: Consider renaming getter method for Interactive Queries (#9146)
- updates docs for KIP-648

Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-08-21 12:39:24 -07:00
Matthias J. Sax a5aadeb31a
MINOR: fix HTML (#9127)
Reviewers: Guozhang Wang <guozhang@confluent.io>, Boyang Chen <boyang@confluent.io>
2020-08-06 11:22:14 -07:00
Matthias J. Sax b351493543
KAFKA-9274: Remove `retries` for global task (#9047)
- part of KIP-572
 - removed the usage of `retries` in `GlobalStateManger`
 - instead of retries the new `task.timeout.ms` config is used

Reviewers: John Roesler <john@confluent.io>, Boyang Chen <boyang@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-08-05 14:14:18 -07:00
Matthias J. Sax 194c56fce2
KAFKA-9274: Mark `retries` config as deprecated and add new `task.timeout.ms` config (#8864)
- part of KIP-572
 - deprecates producer config `retries` (still in use)
 - deprecates admin config `retries` (still in use)
 - deprecates Kafka Streams config `retries` (will be ignored)
 - adds new Kafka Streams config `task.timeout.ms` (follow up PRs will leverage this new config)

Reviewers: John Roesler <john@confluent.io>, Jason Gustafson <jason@confluent.io>, Randall Hauch <randall@confluent.io>
2020-07-21 12:19:13 -07:00
A. Sophie Blee-Goldman 9b30276e58
KAFKA-9161: add docs for KIP-441 and KIP-613 and other configs that need fixing (#9027)
Add docs for KIP-441 and KIP-613.
Fixed some miscellaneous unrelated issues in the docs:
* Adds some missing configs to the Streams config docs: max.task.idle.ms,topology.optimization, default.windowed.key.serde.inner.class, and default.windowed.value.serde.inner.class
* Defines the previously-undefined default windowed serde class configs, including choosing a default (null) and giving them a doc string, so the yshould nwo show up in the auto-generated general Kafka config docs
* Adds a note to warn users about the rocksDB bug that prevents setting a strict capacity limit and counting write buffer memory against the block cache

Reviewers: Bruno Cadonna <bruno@confluent.io>, John Roesler <vvcephei@apache.org>
2020-07-20 16:27:28 -05:00
Matthias J. Sax 934033bd17
MINOR: update required MacOS version (#9025)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
2020-07-15 10:09:27 -07:00
Jim Galasyn d945b287ff
KAFKA-6453: Document how timestamps are computed for aggregations and joins (#9009)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-07-13 15:01:14 -07:00
Jim Galasyn 7e66848200
MINOR: Restore stream-table duality description (#8995)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-07-09 13:50:29 -07:00
Jim Galasyn 11a8549479
MINOR: document timestamped state stores (#8920)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-07-06 18:09:24 -07:00
Matthias J. Sax 712cc5d073
KAFKA-10168: fix StreamsConfig parameter name variable (#8865)
Implements KIP-626.

Reviewers: Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
2020-06-19 17:41:06 -07:00
Matthias J. Sax 922ed9d2af
MINOR: fix HTML markup (#8823)
Reviewers: Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
2020-06-08 12:18:01 -04:00
feyman2016 77f123391c
Kafka-10064 Add documentation for KIP-571 (#8760)
* Update documentation for KIP-571

* update doc for KIP-571

* fix comments
2020-06-01 17:19:26 -07:00
Jim Galasyn bfc9c9909d
MINOR: Remove retries entry from Streams config table (#8733)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-05-27 12:24:15 -07:00
Paul 075bbcfec4
KAFKA-7523: Add ConnectedStoreProvider to Processor API (#6824)
Implements KIP-401:
 - Add ConnectedStoreProvider interface
 - let Processor/[*]Transformer[*]Suppliers extend ConnectedStoreProvider
 - allows to add and connect state stores to processors/transformers implicitly

Reviewers: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2020-05-27 10:57:14 -07:00
Navinder Pal Singh Brar 4b97e50eea
MINOR: Added doc for KIP-535 and updated it for KIP-562 (#8395)
Reviewers: Boyang Chen <boyang@confluent.io>, Vinoth Chandar <vchandar@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2020-05-26 16:24:00 -07:00
Jim Galasyn 9111374974
MINOR: Improve security documentation for Kafka Streams #8710
Reviewers: Matthias J. Sax <matthias@confluent.io>
2020-05-22 11:40:25 -07:00
Matthias J. Sax 27824baa21
KAFKA-10003: Mark KStream.through() as deprecated and update Scala API (#8679)
- part of KIP-221

Co-authored-by: John Roesler <john@confluent.io>
2020-05-22 08:41:28 -07:00
showuon 82fb6e9473
MINOR: Update stream documentation (#8622)
fix broken links
rephrase a sentence
update the version number
Reviewers: Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@apache.org>
2020-05-19 11:05:55 -04:00
Matthias J. Sax f04fd7e9fd
MINOR: updated MacOS compatibility statement for RocksDB (#8687)
Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-05-18 12:43:35 -07:00
showuon b72ab51e4d
Remove redundant TOC and introduction in Running Streams Applications (#8686)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-05-18 09:30:19 -07:00
Levani Kokhreidze 8a0fcd1695
KAFKA-8611: update documentation for KIP-221 (#8558)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-05-15 15:28:23 -07:00
Matthias J. Sax 318063a16a
KAFKA-9466: Update Kafka Streams docs for KIP-447 (#8621)
Reviewers: Boyang Chen <boyang@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2020-05-11 19:11:58 -07:00
Sönke Liebau 402f2c1ece
MINOR: Some html fixes in Streams DSL documentation (#8503)
* Minor: Some html fixes Streams DSL documentation:
 - duplicate id
 - duplicate opening <span> element
 - surplus closing </div> tag

* Replaced opening/closing quotation mark codes with &quot; (they caused w3c validation to complain).

* Replaced right arrow that wasn't rendered correctly with &rarr;

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-04-23 10:44:36 +01:00
Liam Clarke-Hutchinson 7004fc22db
Add explicit grace period to tumbling window example (#8520) 2020-04-21 14:42:08 -05:00
Sönke Liebau cc4e3aa302
MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (#8505)
Currently, tumbling windows are defined as "a special case of hopping time windows" in the streams docs, but hopping windows are only explained in a subsequent section.
I think it would make sense to switch the order of these paragraphs around. To me this also makes more sense semantically.

Testing
Built the site and checked that everything looks ok and html is valid (or at least didn't contain any new warnings that were caused by this change).

Reviewers: Bill Bejeck <bbejeck@apache.org>
2020-04-17 11:25:51 -04:00
Boyang Chen 0520bc6c7c
MINOR: Update docs for KIP-530 and KIP-562 (#8388)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-30 13:06:54 -07:00
Jim Galasyn 0f48446690
DOCS-3625: Add section to config topic: parameters controlled by Kafka Streams (#8268)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 16:21:27 -07:00
Boyang Chen 4c9e56b358
KAFKA-9758: Doc changes for KIP-523 and KIP-527 (#8343)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 09:31:00 -07:00
Sönke Liebau eb7b8295ac
MINOR: Add clarifying "additionally" to streams aggregator docs that is missing in one place. (#8237)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-23 13:17:00 -07:00
Gunnar Morling 04b6641c14
MINOR: Wording fix in Streams DSL docs (#5692)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-20 11:35:27 +00:00
Lee Dongjin 9942e20a58
MINOR: fix omitted 'next.' in interactive queries documentation (#7883)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2020-02-19 10:23:59 -08:00
Mitch 96c69da8c1
KAFKA-8507; Unify connection name flag for command line tool [KIP-499] (#8023)
This change updates ConsoleProducer, ConsumerPerformance, VerifiableProducer, and VerifiableConsumer classes to add and prefer the --bootstrap-server flag for defining the connection point of the Kafka cluster. This change is part of KIP-499: https://cwiki.apache.org/confluence/display/KAFKA/KIP-499+-+Unify+connection+name+flag+for+command+line+tool.

Reviewers: Ron Dagostino <rdagostino@confluent.io>, Stanislav Kozlovski <stanislav_kozlovski@outlook.com>,  Chia-Ping Tsai <chia7712@gmail.com>, Jason Gustafson <jason@confluent.io>
2020-02-13 13:44:51 -08:00
Bill Bejeck 03f763df8a
MINOR: Fixes for adding new DSL naming page (#7899)
Added re-direct for new page and added link to Developer Guide page
Reviewers:  Matthias J. Sax <mjsax@apache.org>, Sophie Blee-Goldman <sophie@confluent.io>,
2020-01-06 18:42:06 -05:00