Commit Graph

53 Commits

Author SHA1 Message Date
Bill Bejeck e847f057e3
KAFKA-10679: [Streams] migrate kafka-site updated docs to kafka/docs (#9554)
During the AK website upgrade, changes made to kafka-site weren't migrated back to kafka-docs.

This PR is an attempt at porting the streams changes to kafka/docs

For the most part, the bulk of the changes in the PR are cosmetic.

For testing:

I reviewed the PR diffs
Rendered the changes locally

Reviewers: John Roesler <john@confluent.io>
2020-11-04 08:30:10 -05:00
leah 8260d7cdfb
KAFKA-5636: Add Sliding Windows documentation (#9264)
Add necessary documentation for KIP-450, adding sliding window aggregations to KStreams

Reviewers: A. Sophie Blee-Goldman <sophie@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2020-09-10 21:55:11 -07:00
Yuriy Badalyantc d0111d3bcd
KAFKA-10020: Create a new version of a scala Serdes without name clash (KIP-616) (#8955)
Wildcard import of the old org.apache.kafka.streams.scala.Serdes leads
to a name clash because some of implicits has the same names as types
from the scala's std lib. The new oak.streams.scala.serialization.Serdes is
the same as the old Serdes, but without name clashes.
The old one is marked as deprecated.

Also, add missing serdes for UUID, ByteBuffer and Short types in
the new Serdes.

Implements: KIP-616

Reviewers: John Roesler <vvcephei@apache.org>
2020-09-01 16:48:40 -05:00
Matthias J. Sax 922ed9d2af
MINOR: fix HTML markup (#8823)
Reviewers: Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@gmail.com>
2020-06-08 12:18:01 -04:00
Paul 075bbcfec4
KAFKA-7523: Add ConnectedStoreProvider to Processor API (#6824)
Implements KIP-401:
 - Add ConnectedStoreProvider interface
 - let Processor/[*]Transformer[*]Suppliers extend ConnectedStoreProvider
 - allows to add and connect state stores to processors/transformers implicitly

Reviewers: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2020-05-27 10:57:14 -07:00
Matthias J. Sax 27824baa21
KAFKA-10003: Mark KStream.through() as deprecated and update Scala API (#8679)
- part of KIP-221

Co-authored-by: John Roesler <john@confluent.io>
2020-05-22 08:41:28 -07:00
showuon 82fb6e9473
MINOR: Update stream documentation (#8622)
fix broken links
rephrase a sentence
update the version number
Reviewers: Sophie Blee-Goldman <sophie@confluent.io>, Boyang Chen <boyang@confluent.io>, Bill Bejeck <bbejeck@apache.org>
2020-05-19 11:05:55 -04:00
Levani Kokhreidze 8a0fcd1695
KAFKA-8611: update documentation for KIP-221 (#8558)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-05-15 15:28:23 -07:00
Sönke Liebau 402f2c1ece
MINOR: Some html fixes in Streams DSL documentation (#8503)
* Minor: Some html fixes Streams DSL documentation:
 - duplicate id
 - duplicate opening <span> element
 - surplus closing </div> tag

* Replaced opening/closing quotation mark codes with &quot; (they caused w3c validation to complain).

* Replaced right arrow that wasn't rendered correctly with &rarr;

Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-04-23 10:44:36 +01:00
Liam Clarke-Hutchinson 7004fc22db
Add explicit grace period to tumbling window example (#8520) 2020-04-21 14:42:08 -05:00
Sönke Liebau cc4e3aa302
MINOR: Switch order of sections on tumbling and hopping windows in streams doc. Tumbling windows are defined as "special case of hopping time windows" - but hopping windows currently only explained later in the docs. (#8505)
Currently, tumbling windows are defined as "a special case of hopping time windows" in the streams docs, but hopping windows are only explained in a subsequent section.
I think it would make sense to switch the order of these paragraphs around. To me this also makes more sense semantically.

Testing
Built the site and checked that everything looks ok and html is valid (or at least didn't contain any new warnings that were caused by this change).

Reviewers: Bill Bejeck <bbejeck@apache.org>
2020-04-17 11:25:51 -04:00
Boyang Chen 4c9e56b358
KAFKA-9758: Doc changes for KIP-523 and KIP-527 (#8343)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-25 09:31:00 -07:00
Sönke Liebau eb7b8295ac
MINOR: Add clarifying "additionally" to streams aggregator docs that is missing in one place. (#8237)
Reviewer: Matthias J. Sax <matthias@confluent.io>
2020-03-23 13:17:00 -07:00
Gunnar Morling 04b6641c14
MINOR: Wording fix in Streams DSL docs (#5692)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2020-02-20 11:35:27 +00:00
Ismael Juma 6dc6f6a60d
KAFKA-9324: Drop support for Scala 2.11 (KIP-531) (#7859)
* Adjust build and documentation.
* Use lambda syntax for SAM types in `core`, `streams-scala` and
`connect-runtime` modules.
* Remove `runnable` and `newThread` from `CoreUtils` as lambda
syntax for SAM types make them unnecessary.
* Remove stale comment in `FunctionsCompatConversions`,
`KGroupedStream`, `KGroupedTable' and `KStream` about Scala 2.11,
the conversions are needed for Scala 2.12 too.
* Deprecate `org.apache.kafka.streams.scala.kstream.Suppressed`
and use `org.apache.kafka.streams.kstream.Suppressed` instead.
* Use `Admin.create` instead of `AdminClient.create`. Static methods
in Java interfaces can be invoked since Scala 2.12. I noticed that
MirrorMaker 2 uses `AdminClient.create`, but I did not change them
as Connectors have restrictions on newer client APIs.
* Improve efficiency in a few `Gauge` implementations by avoiding
unnecessary intermediate collections.
* Remove pointless `Option.apply` in `ZookeeperClient`
`SessionState` metric.
* Fix unused import/variable and other compiler warnings.
* Reduce visibility of some vals/defs.

Reviewers: Manikumar Reddy <manikumar@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Gwen Shapira <gwen@confluent.io>
2020-01-06 19:51:01 +01:00
wcarlson5 8b57f6cb3a KAFKA-6049: Add auto-repartitioning for cogroup (#7792)
Follow up to PR #7538 (KIP-150)

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2019-12-13 14:07:21 -08:00
A. Sophie Blee-Goldman cd5618f866 MINOR: clarify node grouping of input topics using pattern subscription (#7793)
Updates the HTML docs and the javadoc.

Reviewers: John Roesler <vvcephei@apache.org>
2019-12-06 16:03:42 -06:00
Bill Bejeck ed8810127f
KAFKA-9072: Add Topology naming to the dev guide (#7629)
Reviewers: Jim Galasyn <jim.galasyn@confluent.io>, Matthias J. Sax <mjsax@apache.org>, Sophie Blee-Goldman <sophie@confluent.io>
2019-11-13 10:54:47 -05:00
Adam Bellemare aab14fb843 MINOR: Add documentation for foreign-key joins (KIP-213) (#7535)
Reviewers: John Roesler <john@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2019-10-24 11:35:55 -07:00
Matthias J. Sax e85d671dee
MINOR: replace `late` with `out-of-order` in JavaDocs and docs (#7274)
Reviewers: Bill Bejeck <bill@confluent.io>, John Roesler <john@confluent.io>
2019-09-19 18:28:24 -07:00
Matthias J. Sax 56c64803fa
KAFKA-3729: Revert adding Serde auto-config (#6630)
* Revert "MINOR: Add unit test for SerDe auto-configuration (#6610)"

This reverts commit 172fbb2dd5.

* Revert "[KAFKA-3729] Auto-configure non-default SerDes passed alongside the topology builder (#6461)"

This reverts commit e56ebbffca.

The two merged PRs introduce a breaking change. Reverting to preserve backward compatibility. Jira ticket reopened.

Reviewers: Ted Yu <yuzhihong@gmail.com>, Guozhang Wang <guozhang@confluent.io>
2019-05-01 09:33:53 +02:00
Ted Yu e56ebbffca [KAFKA-3729] Auto-configure non-default SerDes passed alongside the topology builder (#6461)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2019-04-20 11:30:20 -07:00
Bill Bejeck 5e2d062267
KAFKA-8210: Fix link for streams table duality (#6573)
Reviewers: Victoria Bialas <vicky@confluent.io>
2019-04-13 18:54:06 -04:00
John Roesler 8ae985705f KAFKA-7768: Use absolute paths for javadoc (#6100)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2019-01-07 13:30:49 -08:00
slim 6a20e3cd4c KAFKA-7768: Add version to java html urls (#6094)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2019-01-07 09:21:33 -08:00
Matthias J. Sax a1807d4914 MINOR: Improve GlobalKTable docs (#5996)
Reviewers: Jim Galasyn, Michael G. Noll, John Roesler, Bill Bejeck, Guozhang Wang
2018-12-04 16:21:18 -08:00
Ismael Juma b4722cc16f
KAFKA-7524: Recommend Scala 2.12 and use it for development (#5530)
Scala 2.12 has better support for newer Java versions and includes additional
compiler warnings that are helpful during development. In addition, Scala 2.11
hasn't been supported by the Scala community for a long time, the soon to be
released Spark 2.4.0 will finally support Scala 2.12 (this was the main reason
preventing many from upgrading to Scala 2.12) and Scala 2.13 is at the RC stage.
It's time to start recommending the Scala 2.12 build as we prepare support for
Scala 2.13 and start thinking about removing support for Scala 2.11.

In the meantime, Jenkins will continue to build all supported Scala versions (including
Scala 2.11) so the PR and trunk jobs will fail if people accidentally use methods
introduced in Scala 2.12.

Reviewers: Ewen Cheslack-Postava <me@ewencp.org>
2018-10-28 11:31:39 -07:00
John Roesler 4b7148a5b6 KAFKA-7223: Suppression documentation (#5787)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>
2018-10-15 21:04:01 -07:00
Bill Bejeck 8182c4fc7d MINOR: doc changes for KIP-372 (#5790)
Reviewers: Matthias J. Sax <matthias@confluent.io>, John Roesler <john@confluent.io>
2018-10-15 19:01:21 -07:00
Matthias J. Sax 0b417b8331
MINOR: updates docs for KIP-358 (#5796)
Reviewers: Guozhang Wang <guozhang@confluent.io>, Jim Galasyn <jim.galasyn@confluent.io>
2018-10-15 17:22:03 -07:00
Bill Bejeck 862e09f994 HOTFIX: Fix broken links (#5676)
Reviewers: Joel Hamill <11722533+joel-hamill@users.noreply.github.com>, Guozhang Wang <wangguoz@gmail.com>
2018-10-03 18:27:15 -07:00
huxi aa7358e8cc KAFKA-7326: KStream.print() should flush on each line for PrintStream (#5579)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bill Bejeck <bill@confluent.io>, Kamal Chandraprakash <kamal.chandraprakash@gmail.com>
2018-09-01 09:48:50 -07:00
lucapette 2514a42e84 KAFKA-7269: Add docs for KStream.merge (#5512)
Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>
2018-08-29 09:34:54 -07:00
Guozhang Wang d3e264e773
MINOR: update web docs and examples of Streams with Java8 syntax (#5249)
Reviewers: John Roesler <john@confluent.io>, Bill Bejeck <bill@confluent.io>, Damian Guy <damian@confluent.io>
2018-06-21 10:02:58 -07:00
Guozhang Wang 718d6f2475
MINOR: Remove deprecated KafkaStreams constructors in docs (#5118)
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2018-06-04 13:43:20 -07:00
Guozhang Wang f33e9a346e KAFKA-4936: Add dynamic routing in Streams (#5018)
implements KIP-303

Reviewers: Bill Bejeck <bill@confluent.io>, John Roesler <john@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2018-05-30 11:54:53 -07:00
Joan Goyeau 1e0793c4ce Update doc of curried joins and aggregates (#5053)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2018-05-30 11:03:58 -07:00
Andy Coates 4e1c8ffd0d KAFKA-6849: add transformValues methods to KTable. (#4959)
See the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-292%3A+Add+transformValues%28%29+method+to+KTable

This PR adds the transformValues method to the KTable interface. The semantics of the call are the same as the methods of the same name on the KStream interface.

Fixes KAFKA-6849

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2018-05-18 16:06:50 -07:00
Guozhang Wang d4204e8b14
MINOR: fix broken links in streams doc (#5025)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2018-05-16 13:23:20 -07:00
Joel Hamill c14b0ad9ee MINOR - Fix typo in Streams Dev Guide (#4972)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>
2018-05-11 10:27:07 -07:00
Joan Goyeau b88d70b532 MINOR: Make Serdes less confusing in Scala (#4963)
Serdes are confusing in the Scala wrapper:

* We have wrappers around Serializer, Deserializer and Serde which are not very useful.
* We have Serdes in 2 places org.apache.kafka.common.serialization.Serde and in DefaultSerdes, instead we should be having only one place where to find all the Serdes.

I wanted to do this PR before the release as this is a breaking change.
This shouldn't add more so the current tests should be enough.

Reviewers: Debasish Ghosh <dghosh@acm.org>, Guozhang Wang <guozhang@confluent.io>
2018-05-08 09:15:31 -07:00
Sean Glover 893e044515 MINOR: Build and code sample updates for Kafka Streams DSL for Scala (#4949)
Several build and documentation updates were required after the merge of KAFKA-6670: Implement a Scala wrapper library for Kafka Streams.

Encode Scala major version into streams-scala artifacts.
To differentiate versions of the kafka-streams-scala artifact across Scala major versions it's required to encode the version into the artifact name before its published to a maven repository. This is accomplished by following a similar release process as kafka core, which encodes the Scala major version and then runs the build for each major version of Scala supported. This is considered standard practice when releasing Scala libraries, but is not handled for us automatically with the basic Scala for Gradle support.

After this change you can generate and install the kafka-streams-scala artifact into the local maven repository:

$ ./gradlew -PscalaVersion=2.11 install
$ ./gradlew -PscalaVersion=2.12 install

Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>
2018-05-06 20:55:12 -07:00
manjuapu 0143a65091 Fix streams web doc configs tables (#4943)
Reviewers: Guozhang Wang <wangguoz@gmail.com>
2018-04-27 16:15:55 -07:00
manjuapu 3ce14a84a8 MINOR: Streams web doc table fix (#4942) 2018-04-27 14:20:29 -07:00
Debasish Ghosh b2e4db01b6 KAFKA-6670: Implement a Scala wrapper library for Kafka Streams
This PR implements a Scala wrapper library for Kafka Streams. The library is implemented as a project under streams, namely `:streams:streams-scala`. The PR contains the following:

* the library implementation of the wrapper abstractions
* the test suite
* the changes in `build.gradle` to build the library jar

The library has been tested running the tests as follows:

```
$ ./gradlew -Dtest.single=StreamToTableJoinScalaIntegrationTestImplicitSerdes streams:streams-scala:test
$ ./gradlew -Dtest.single=StreamToTableJoinScalaIntegrationTestImplicitSerdesWithAvro streams:streams-scala:test
$ ./gradlew -Dtest.single=WordCountTest streams:streams-scala:test
```

Author: Debasish Ghosh <ghosh.debasish@gmail.com>
Author: Sean Glover <seglo@randonom.com>

Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Ismael Juma <ismael@juma.me.uk>, John Roesler <john@confluent.io>, Damian Guy <damian@confluent.io>, Guozhang Wang <wangguoz@gmail.com>

Closes #4756 from debasishg/scala-streams
2018-04-23 13:33:35 -07:00
John Roesler adbf31ab1d KAFKA-6473: Add MockProcessorContext to public test-utils (#4736)
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Bill Bejeck <bill@confluent.io>
2018-03-27 14:03:24 -07:00
huxi a9c16a8110 KAFKA-6663: Doc for `GlobalKTable` should be corrected. (#4723)
Reviewers: John Roesler <john@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>, Damian Guy <damian@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2018-03-16 13:24:14 -07:00
huxi d2aca95aeb MINOR: Fix incorrect expression for KTable in stream doc. (#4718)
In dsi-api.html, when reading from Kafka to a KTable, the doc says "In the case of a KStream.." where `Kstream` here is not correct. They should be `KTable`.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2018-03-15 11:34:33 -07:00
Guozhang Wang 077fd9ced3
HOTFIX: Fix broken javadoc links on web docs(#4543)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2018-02-08 09:31:58 -08:00
Joel Hamill 6ea53d22e2 MINOR: Fix some streams web doc nits (#4411)
Reviewers: Derrick Or <derrickor@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
2018-01-29 10:09:32 -08:00