[KIP-962](https://cwiki.apache.org/confluence/display/KAFKA/KIP-962%3A+Relax+non-null+key+requirement+in+Kafka+Streams)
The key requirments got relaxed for the followinger streams dsl operator:
left join Kstream-Kstream: no longer drop left records with null-key and call ValueJoiner with 'null' for right value.
outer join Kstream-Kstream: no longer drop left/right records with null-key and call ValueJoiner with 'null' for right/left value.
left-foreign-key join Ktable-Ktable: no longer drop left records with null-foreign-key returned by the ForeignKeyExtractor and call ValueJoiner with 'null' for right value.
left join KStream-Ktable: no longer drop left records with null-key and call ValueJoiner with 'null' for right value.
left join KStream-GlobalTable: no longer drop records when KeyValueMapper returns 'null' and call ValueJoiner with 'null' for right value.
Reviewers: Walker Carlson <wcarlson@apache.org>
Part of KIP-985.
Updates JavaDocs for `RangeQuery` and `ReadOnlyKeyValueStore` with regard to ordering guarantees.
Updates Kafka Streams upgrade guide with KIP information.
Reviewer: Matthias J. Sax <matthias@confluent.io>
KIP-914 introduced a new boolean isLatest into Change to indicate whether a change update represents the latest for the key. Even though Change is serialized into the table repartition topic, the new boolean does not need to be serialized in, because the table repartition map processor performs an optimization to drop records for which isLatest = false. If not for this optimization, the downstream table aggregate would have to drop such records instead, and isLatest would need to be serialized into the repartition topic.
In light of the possibility that isLatest may need to be serialized into the repartition topic in the future, e.g., if other downstream processors are added which need to distinguish between records for which isLatest = true vs isLatest = false, this PR reserves repartition topic formats which include isLatest. Reserving these formats now comes at no additional cost to users since a rolling bounce is already required for the upcoming release due to KIP-904. If we don't reserve them now and instead have to add them later, then another bounce would be required at that time. Reserving formats is cheap, so we choose to do it now.
Reviewers: Matthias J. Sax <matthias@confluent.io>
Moved the information about the BooleanSerde addition from general upgrade notes to notes in Streams.
Reviewers: Walker Carlson <wcarlson@confluent.io>, Divij Vaidya <diviv@amazon.com>
Documentation only—Minor clarification on how max.warmup.replicas works; specifically, that one "warmup replica" corresponds to a Task that is restoring its state. Also clarifies how max.warmup.replicas interacts with probing.rebalance.interval.ms.
Reviewers: Lucas Brutschy <lbrutschy@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
The table of (stateless) transformations uses the transformation name in the first column and a description in the second column. I adjusted the transformation name for FlatMapValues accordingly.
Reviewers: Matthias J. Sax <mjsax@apache.org>, Bill Bejeck <bbejeck@apache.org>
This PR is part of a series implementing the self-join rewriting. As part of it, we decided to clean up the TOPOLOGY_OPTIMIZATION_CONFIG and make it a list of optimization rules. Acceptable values are: NO_OPTIMIZATION, OPTIMIZE which applies all optimization rules or a comma separated list of specific optimizations.
Reviewers: Guozhang Wang <guozhang@apache.org>, John Roesler <vvcephei@apache.org>
Update the quickstart HTML pages for Kafka and Kafka Stream to include how to quickly start and
experiment with a Kafka cluster using KRaft in addition to ZooKeeper.
Reviews: Colin Patrick McCabe <cmccabe@apache.org>, Chase Thomas <forlack@users.noreply.github.com>, Luke Chen <showuon@gmail.com>
This PR is a mirror of apache/kafka-site#433 which used placeholder images for the Kafka Streams that users need to click in order to load the iframe with the corresponding video.
Reviewers: Mickael Maison <mimaison@apache.org>
Prior to this commit FK response sink routed FK results to
SubscriptionResolverJoinProcessorSupplier using the primary key.
There are cases, where this behavior is incorrect. For example,
if KTable key serde differs from the data source serde which might
happen without a key changing operation.
Instead of determining the resolver partition by serializing the PK
this patch includes target partition in SubscriptionWrapper payloads.
Default FK response-sink partitioner extracts the correct partition
from the value and routes the message accordingly.
Reviewers: Matthias J. Sax <matthias@confluent.io>
Adds documentation for KIP-708: Rack awareness for Kafka Streams
Co-authored-by: Bruno Cadonna <cadonna@apache.org>
Reviewers: Luke Chen <showuon@gmail.com>, Bruno Cadonna <cadonna@apache.org>
In the RocksDb memory management doc, we mentioned in the footnote that there's a rocksdb bug caused the strict_capacity_limit boolean parameter in the LRUCache constructor can't be set to true. However, the bug is already fixed in 6.11.4, and we're using 6.22 now, so the note can be removed.
Reviewer: Bruno Cadonna <cadonna@apache.org>
Right now, we have scattered uses of `SerDes` throughout the docs. These should be updated to be `Serdes`, as that's what we commonly use now.
Reviewers: Boyang Chen <bchen11@outlook.com>
There were a few followup things to address from #10926, most importantly a number of updates to the javadocs. Also includes a few missing verification checks.
Reviewers: Guozhang Wang <guozhang@confluent.io>, Matthias J. Sax <mjsax@apache.org>, Israel Ekpo
Florin Akermann
Hanyu Zheng
Matthias J. Sax
Lucia Cerchie
Hao Li
Iblis Lin
Jim Galasyn
minjian.cai
Victoria Xia
Spacrocket
Farooq Qaiser
Iblis Lin
Bill Bejeck
littlehorse-eng
Matthias Seiler
Luke Chen
liuzc9
Shay Lin
Vicky Papavasileiou
Nikolay
Colin Patrick McCabe
José Armando García Sancio
Divij Vaidya
Mickael Maison
Marco Aurelio Lotz
Okada Haruki
Amir M. Saeid
Milind Mantri
Aleksandr Sorokoumov
Levani Kokhreidze
aSemy
Andy Chambers
Guozhang Wang
A. Sophie Blee-Goldman