Commit Graph

2334 Commits

Author SHA1 Message Date
John Roesler 8eaca9087a MINOR: Add generation to consumer assignor logs 2022-09-04 09:16:19 -05:00
Guozhang Wang 8380d2edf4
KAFKA-10199: Handle exceptions from state updater (#12519)
1. In state updater, when handling task corrupted exception due to invalid restoring offset, first delete the affected partitions from the checkpoint before reporting it back to the stream thread. This is to mimic the same behavior in stream threads's StateManager#handleCorruption#closeDirtyAndRevive. It's cleaner to do so inside the restore thread, plus it enables us to optimize by only deleting those corrupted partitions, and not all.
2. In the state manager, handle the drained exceptions as follows (this is the same as handling all exceptions from handleAssignment): 1) Task-migrated, throw all the way to stream-thread as handleTaskMigrated, 2) any fatal Streams exception, throw all the way to stream-thread to trigger exception handler, 3) Task-corrupted, throw to the stream-thread as handleCorruption. Note that for 3), we would specially distinguish if the corrupted-tasks are already closed (when they are thrown from handleAssignment or not (when they are thrown from the state updater).

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-09-02 17:50:23 -07:00
A. Sophie Blee-Goldman 7ec10ce19a
HOTFIX: fix PriorityQueue iteration to assign warmups in priority order (#12585)
Based on a patch submitted to the confluentinc fork & then abandoned. Needed some updates and minor expansion but more or less just re-applied the changes proposed in confluentinc#697.

Original PR has a very detailed justification for these changes but the tl;dr of it is that apparently the PriorityQueue's iterator does not actually guarantee to return elements in priority order.

Reviewer: Luke Chen <showuon@gmail.com>
2022-09-02 18:14:34 +08:00
John Roesler 8b64a9e235
MINOR: Demystify rebalance schedule log (#12582)
Reviewers: Bruno Cadonna <cadonna@apache.org>, Bill Bejeck <bbejeck@apache.org>
2022-09-01 16:34:03 -05:00
Bruno Cadonna 2bef80e360
KAFKA-10199: Remove changelog unregister from state updater (#12573)
Changelogs are already unregistered when tasks are closed.
There is no need to also unregister them in the state
updater.

In future, when we will only have the state updater without
the old code path, we should consider registering and
unregistering the changelogs within the state updater.

Reviewer: Guozhang Wang <wangguoz@gmail.com>
2022-09-01 14:29:39 +02:00
Bruno Cadonna bc8f7d07d9
KAFKA-10199: Shutdown state updater on task manager shutdown (#12569)
When the task manager is shutdown, the state updater should also
shutdown. After the shutdown of the state updater, the tasks
in its output queues should be closed.

Reviewer: Guozhang Wang <wangguoz@gmail.com>
2022-08-31 20:45:53 +02:00
Divij Vaidya 140faf9f2b
KAFKA-13036: Replace EasyMock and PowerMock with Mockito for RocksDBMetricsRecorderTest (#12459)
Changes:
- Migrate to Mockito
- Add more assertive checks using verify
- Minor indentation fixes

Reviewers: Dalibor Plavcic <dalibor.os@proton.me>, Bruno Cadonna <cadonna@apache.org>
2022-08-30 19:25:26 +02:00
Bruno Cadonna 7b07b2676b
KAFKA-10199: Remove tasks from state updater on shutdown (#12562)
The state updater removes its updating and paused task on shutdown.
The removed tasks are added to the output queue for removed tasks.

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Walker Carlson <wcarlson@confluent.io>
2022-08-29 18:29:21 +02:00
Bruno Cadonna 0e6a3fa978
KAFKA-10199: Handle restored tasks output by state updater (#12554)
Once the state updater restored an active task it puts it
into an output queue. The stream thread reads the restored
active task from the output queue and after it verified
that the task is still owned by the stream thread it transits
it to RUNNING.

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Walker Carlson <wcarlson@confluent.io>
2022-08-29 18:26:02 +02:00
Mickael Maison 0507597597
KAFKA-10360: Allow disabling JMX Reporter (KIP-830) (#12046)
This implements KIP-830: https://cwiki.apache.org/confluence/display/KAFKA/KIP-830%3A+Allow+disabling+JMX+Reporter
It adds a new configuration `auto.include.jmx.reporter` that can be set to false to disable the JMX Reporter. This configuration is deprecated and will be removed in the next major version.

Reviewers: Tom Bentley <tbentley@redhat.com>, Christo Lolov <christo_lolov@yahoo.com>
2022-08-24 18:30:31 +02:00
Bruno Cadonna f191126550
KAFKA-10199: Introduce task registry (#12549)
Currently the task manager stores the tasks it manages in an
internally. We recently extracted the code to store and retrieve
tasks into its own class Tasks. However, the task manager creates
the Tasks object internally and during testing of the task
manager we do not have access to it which makes testing the task
manager quite complex.

This commit externalizes the data structure that the task manager
uses to store and rerieve tasks. It introduces the TasksRegistry
interface and lets the Tasks object implementing TasksRegistry.
The Tasks object is passed into the task manager via its
constructor. Passing the TasksRegistry dependency to the task
manager from outside faciliates simpler testing of the task
manager.

Reviewers: Guozhang Wang <wangguoz@gmail.com>, Walker Carlson <wcarlson@confluent.io>
2022-08-24 08:19:40 +02:00
Bruno Cadonna add4ca6c7f
KAFKA-10199: Remove tasks from state updater on revoked and lost partitions (#12547)
Removes tasks from the state updater when the input partitions of the tasks are revoked or partitions are lost during a rebalance.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-08-22 11:50:50 -07:00
Alex Sorokoumov 5c77c544c6
KAFKA-13769 Fix version check in SubscriptionStoreReceiveProcessorSupplier (#12535)
This patch fixes another incorrect version check in the FK code and adds unit tests that would have caught this bug.

Reviewers: John Roesler <vvcephei@apache.org>
2022-08-18 13:20:04 -05:00
Jason Gustafson 0243bb98a7
HOTFIX: Revert KAFKA-10199 which is causing compilation failures (#12532)
Compilation is failing after these two commits:
```
> Task :streams:compileJava
/Users/jgustafson/Projects/kafka/streams/src/main/java/org/apache/kafka/streams/processor/internals/TaskManager.java:852: error: cannot find symbol
                        tasks.addPendingTaskToClose(restoringTask.id());
                             ^
  symbol:   method addPendingTaskToClose(org.apache.kafka.streams.processor.TaskId)
  location: variable tasks of type org.apache.kafka.streams.processor.internals.Tasks
1 error
```

Also here:
```

[2022-08-17T20:58:20.912Z] > Task :streams:compileTestJava

[2022-08-17T20:58:20.912Z] /home/jenkins/jenkins-agent/workspace/Kafka_kafka-pr_PR-12530/streams/src/test/java/org/apache/kafka/streams/processor/internals/TaskManagerTest.java:822: error: method setupForRevocation(Set<Task>,Set<Task>) is already defined in class TaskManagerTest

[2022-08-17T20:58:20.912Z]     private TaskManager setupForRevocation(final Set<Task> tasksInStateUpdater,
```
 This patch reverts them.

Reviewers: Ismael Juma <ismael@juma.me.uk>
2022-08-17 14:29:49 -07:00
Bruno Cadonna b47c4d8598
KAFKA-10199: Remove tasks from state updater on revocation (#12520)
Removes tasks from the state updater when the input partitions of the tasks are revoked during a rebalance.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-08-17 11:13:34 -07:00
Bruno Cadonna 9f20f89953
KAFKA-10199: Remove tasks from state updater on partition lost (#12521)
Removes tasks from the state updater when the input partitions of the tasks are lost during a rebalance.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-08-17 11:12:30 -07:00
Guozhang Wang dc72f6ec02
KAFKA-10199: Handle task closure and recycling from state updater (#12466)
1. Within the tryCompleteRestore function of the thread, try to drain the removed tasks from state updater and handle accordingly: 1) for recycle, 2) for closure, 3) for update input partitions.
2. Catch up on some unit test coverage from previous PRs.
3. Some minor cleanups around exception handling.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-08-15 19:33:46 -07:00
Bruno Cadonna 75d89931e0
MINOR: Add setting input partitions for task mocks (#12510)
We recently added a builder to create task mocks for unit
tests. This PR adds the functionality to add input partitions
to task mocks when the builder is used.

Reviewers: Walker Carlson <wcarlson@confluent.io>, A. Sophie Blee-Goldman <ableegoldman@apache.org>
2022-08-15 11:16:32 -07:00
Bruno Cadonna 4a3e92b1ab
KAFKA-10199: Expose read only task from state updater (#12497)
The state updater exposes tasks that are in restoration
to the stream thread. To ensure that the stream thread
only accesses the tasks to read from the tasks without
modifying any internal state, this PR introduces a
read-only task that throws an exception if the caller
tries to modify the internal state of a task.

This PR also returns read-only tasks from
DefaultStateUpdater#getTasks().

Reviewer: Guozhang Wang <wangguoz@gmail.com>
2022-08-12 17:03:50 +02:00
Guozhang Wang 5ceaa588ee
HOTFIX / KAFKA-14130: Reduce RackAwarenesssTest to unit Test (#12476)
While working on KAFKA-13877, I feel it's an overkill to introduce the whole test class as an integration test, since all we need is to just test the assignor itself which could be a unit test. Running this suite with 9+ instances takes long time and is still vulnerable to all kinds of timing based flakiness. A better choice is to reduce it as a unit test, similar to HighAvailabilityStreamsPartitionAssignorTest that just test the behavior of the assignor itself, rather than creating many instances hence depend on various timing bombs to not explode.

Since we mock everything, there's no flakiness anymore. Plus we greatly reduced the test runtime (on my local machine, the old integration takes about 35 secs to run the whole suite, while the new one take 20ms on average).

Reviewers: Divij Vaidya <diviv@amazon.com>, Dalibor Plavcic
2022-08-03 15:36:59 -07:00
Guozhang Wang 3202459394
KAFKA-13877: Fix flakiness in RackAwarenessIntegrationTest (#12468)
In the current test, we check for tag distribution immediately after everyone is on the running state, however due to the fact of the follow-up rebalances, "everyone is now in running state" does not mean that the cluster is now stable. In fact, a follow-up rebalance may occur, upon which the local thread metadata would return empty which would cause the distribution verifier to fail.

Reviewers: Divij Vaidya <diviv@amazon.com>, Luke Chen <showuon@gmail.com>
2022-08-03 09:17:38 -07:00
Hao Li f7ac5d3d00
Minor: enable index for emit final sliding window (#12461)
Enable index for sliding window emit final case as it's faster to fetch windows for particular key

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-07-29 14:47:25 -07:00
Christo Lolov 54af64c33a
KAFKA-14108: Ensure both JUnit 4 and JUnit 5 tests run (#12441)
When the migration of the Streams project to JUnit 5 started with PR #12285, we discovered that the migrated tests were not run by the PR builds. This PR ensures that Streams' tests that are written in JUnit 4 and JUnit 5 are run in the PR builds.

Co-authored-by: Divij Vaidya <diviv@amazon.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Bruno Cadonna <cadonna@apache.org>
2022-07-29 17:21:25 +02:00
Bruno Cadonna 5f7c99dd77
MINOR: Remove code of removed metric (#12453)
When we removed metric skipped-records in 3.0 we missed to
remove some code related to that metric.

Reviewer: Guozhang Wang <wangguoz@gmail.com>
2022-07-29 16:53:01 +02:00
Bruno Cadonna a5d71e1550
MINOR: Fix static mock usage in ThreadMetricsTest (#12454)
Before this PR the calls to the static methods on StreamsMetricsImpl were just calls and not a verification on the mock. This miss happened during the switch from EasyMock to Mockito.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-07-28 13:32:46 -07:00
Bruno Cadonna 2724cc9920
KAFKA-10199: Bookkeep tasks during assignment for use with state updater (#12442)
Bookkeeps tasks to be recycled, closed, and updated during handling of the assignment. The bookkeeping is needed for integrating the state updater.

These change is hidden behind internal config STATE_UPDATER_ENABLED. If the config is false Streams should not use the state updater and behave as usual.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-07-28 13:28:47 -07:00
Guozhang Wang 06f47c3b51
KAFKA-10199: Further refactor task lifecycle management (#12439)
1. Consolidate the task recycle procedure into a single function within the task. The current procedure now becomes: a) task.recycleStateAndConvert, at end of it the task is in closed while its stateManager is retained, and the manager type has been converted; 2) create the new task with old task's fields and the stateManager inside the creators.
2. Move the task execution related metadata into the corresponding TaskExecutionMetadata class, including the task idle related metadata (e.g. successfully processed tasks); reduce the number of params needed for TaskExecutor as well as Tasks.
3. Move the task execution related fields (embedded producer and consumer) and task creators out of Tasks and migrated into TaskManager. Now the Tasks is only a bookkeeping place without any task mutation logic.
4. When adding tests, I realized that we should not add task to state updater right after creation, since it was not initialized yet, while state updater would validate that the task's state is already restoring / running. So I updated that logic while adding unit tests.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-07-27 17:29:05 -07:00
Alex Sorokoumov d076b7ad0e
KAFKA-13769: Add tests for ForeignJoinSubscriptionProcessorSupplier (#12437)
Reviewers: Adam Bellemare <adam.bellemare@gmail.com>, John Roesler <vvcephei@apache.org>
2022-07-27 13:58:12 -05:00
Bruno Cadonna f191e4781e
MINOR: Use builder for mock task in DefaultStateUpdaterTest (#12436)
Reviewer: Guozhang Wang <wangguoz@gmail.com>
2022-07-26 10:12:20 +02:00
Bruno Cadonna 5a52601691
KAFKA-10199: Add tasks to state updater when they are created (#12427)
This PR introduces an internal config to enable the state updater. If the state updater is enabled newly created tasks are added to the state updater. Additionally, this PR introduces a builder for mocks for tasks.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-07-21 12:37:17 -07:00
Christo Lolov 569a358a3f
KAFKA-14001: Migrate streams module to JUnit 5 - Part 1 (#12285)
This pull request addresses https://issues.apache.org/jira/browse/KAFKA-14001. It is the first of a series of pull requests which address the move of Kafka Streams tests from JUnit 4 to JUnit 5.

Reviewers: Divij Vaidya <diviv@amazon.com>, Bruno Cadonna <cadonna@apache.org>
2022-07-21 17:27:53 +02:00
James Hughes ff7cbf264c
KAFKA-14076: Fix issues with KafkaStreams.CloseOptions (#12408)
- used static memberId was incorrect
- need to remove all threads/members from the group
- need to use admit client correctly

Add test to verify fixes.

Reviewers: Matthias J. Sax <matthias@confluent.io>
2022-07-21 07:35:29 -07:00
Guozhang Wang c9b6e19b3b
KAFKA-10199: Cleanup TaskManager and Task interfaces (#12397)
In order to integrate with the state updater, we would need to refactor the TaskManager and Task interfaces. This PR achieved the following purposes:

    Separate active and standby tasks in the Tasks placeholder, plus adding pendingActiveTasks and pendingStandbyTasks into Tasks. The exposed active/standby tasks from the Tasks set would only be mutated by a single thread, and the pending tasks hold for those tasks that are assigned but cannot be actively managed yet. For now they include two scenarios: a) tasks from unknown sub-topologies and hence cannot be initialized, b) tasks that are pending for being recycled from active to standby and vice versa. Note case b) would be added in a follow-up PR.

    Extract any logic that mutates a task out of the Tasks / TaskCreators. Tasks should only be a place for maintaining the set of tasks, but not for manipulations of a task; and TaskCreators should only be used for creating the tasks, but not for anything else. These logic are all migrated into TaskManger.

    While doing 2) I noticed we have a couple of minor issues in the code where we duplicate the closing logics, so I also cleaned them up in the following way:
    a) When closing a task, we first trigger the corresponding closeClean/Dirty function; then we remove the task from Tasks bookkeeping, and for active task we also remove its task producer if EOS-V1 is used.
    b) For closing dirty, we swallow the exception from close call and the remove task producer call; for closing clean, we store the thrown exception from either close call or the remove task producer, and then rethrow at the end of the caller. The difference though is that, for the exception from close call we need to retry close it dirty; for the exception from the remove task producer we do not need to re-close it dirty.

Reviewer: Bruno Cadonna <cadonna@apache.org>
2022-07-21 15:11:40 +02:00
Walker Carlson b62d8b975c
KAFKA-12699: Override the default handler for stream threads if the stream's handler is used (#12324)
Override the default handler for stream threads if the stream's handler is used. We do no want the java default handler triggering when a thread is replaced.

Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
2022-07-19 13:35:26 -07:00
Guozhang Wang 693e283802
KAFKA-10199: Add RESUME in state updater (#12387)
* Need to check enforceRestoreActive / transitToUpdateStandby when resuming a paused task.
* Do not expose another getResumedTasks since I think its caller only need the getPausedTasks.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-07-19 09:44:10 -07:00
Walker Carlson 188b2bf280
Revert "KAFKA-12887 Skip some RuntimeExceptions from exception handler (#11228)" (#12421)
This reverts commit 4835c64f

Reviewers: Matthias J. Sax <matthias@confluent.io>
2022-07-19 09:17:46 -07:00
Guozhang Wang 309e0f986e
KAFKA-10199: Add PAUSE in state updater (#12386)
* Add pause action to task-updater.
* When removing a task, also check in the paused tasks in addition to removed tasks.
* Also I realized we do not check if tasks with the same id are added, so I add that check in this PR as well.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-07-18 16:42:48 -07:00
Alex Sorokoumov 4eef28018a
KAFKA-13769 Fix version check in SubscriptionJoinForeignProcessorSupplier (#12420)
This commit changes the version check from != to > as the process method
works correctly on both version 1 and 2. != incorrectly throws on v1
records.

Reviewers: Matthias J. Sax <matthias@confluent.io>
2022-07-18 14:10:02 -07:00
Levani Kokhreidze edad31811c
MINOR: Fix QueryResult Javadocs (#12404)
Fixes the QueryResult javadocs.

Reviewer: Bruno Cadonna <cadonna@apache.org>
2022-07-18 13:39:34 +02:00
Sanjana Kaundinya beac86f049
KAFKA-13043: Implement Admin APIs for offsetFetch batching (#10964)
This implements the AdminAPI portion of KIP-709: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=173084258. The request/response protocol changes were implemented in 3.0.0. A new batched API has been introduced to list consumer offsets for different groups. For brokers older than 3.0.0, separate requests are sent for each group.

Co-authored-by: Rajini Sivaram <rajinisivaram@googlemail.com>
Co-authored-by: David Jacot <djacot@confluent.io>

Reviewers: David Jacot <djacot@confluent.io>,  Rajini Sivaram <rajinisivaram@googlemail.com>
2022-07-14 13:47:34 +01:00
Hao Li b5d4fa7645
KAFKA-13785: [10/N][emit final] more unit test for session store and disable cache for emit final sliding window (#12370)
1. Added more unit test for RocksDBTimeOrderedSessionStore and RocksDBTimeOrderedSessionSegmentedBytesStore
2. Disable cache for sliding window if emit strategy is ON_WINDOW_CLOSE

Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
2022-07-12 10:57:11 -07:00
Matthias J. Sax 38b08dfd33
MINOR: revert KIP-770 (#12383)
KIP-770 introduced a performance regression and needs some re-design.

Needed to resolve some conflict while reverting.

This reverts commits 1317f3f77a and 0924fd3f9f.

Reviewers:  Sagar Rao <sagarmeansocean@gmail.com>, Guozhang Wang <guozhang@confluent.io>
2022-07-07 11:19:37 -07:00
Guozhang Wang 915c781243
KAFKA-10199: Remove main consumer from store changelog reader (#12337)
When store changelog reader is called by a different thread than the stream thread, it can no longer use the main consumer to get committed offsets since consumer is not thread-safe. Instead, we would remove main consumer and leverage on the existing admin client to get committed offsets.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-07-06 17:23:18 -07:00
Bruno Cadonna 00f395bb88
KAFKA-10199: Remove call to Task#completeRestoration from state updater (#12379)
The call to Task#completeRestoration calls methods on the main consumer.
The state updater thread should not access the main consumer since the
main consumer is not thread-safe. Additionally, Task#completeRestoration
changed the state of active tasks, but we decided to keep task life cycle
management outside of the state updater.

Task#completeRestoration should be called by the stream thread on
restored active tasks returned by the state udpater.

Reviewer: Guozhang Wang <guozhang@apache.org>
2022-07-06 12:36:15 +02:00
Matthew de Detrich 4e6326f889
KAFKA-13957: Fix flaky shouldQuerySpecificActivePartitionStores test (#12289)
Currently the tests fail because there is a missing predicate in the retrievableException which causes the test to fail, i.e. the current predicates

containsString("Cannot get state store source-table because the stream thread is PARTITIONS_ASSIGNED, not RUNNING"),
containsString("The state store, source-table, may have migrated to another instance"),
containsString("Cannot get state store source-table because the stream thread is STARTING, not RUNNING")

wasn't complete. Another one needed to be added, namely "The specified partition 1 for store source-table does not exist.". This is because its possible for

assertThat(getStore(kafkaStreams2, storeQueryParam2).get(key), is(nullValue()));

or

assertThat(getStore(kafkaStreams1, storeQueryParam2).get(key), is(nullValue()));

(depending on which branch) to be thrown, i.e. see

org.apache.kafka.streams.errors.InvalidStateStorePartitionException: The specified partition 1 for store source-table does not exist.

	at org.apache.kafka.streams.state.internals.WrappingStoreProvider.stores(WrappingStoreProvider.java:63)
	at org.apache.kafka.streams.state.internals.CompositeReadOnlyKeyValueStore.get(CompositeReadOnlyKeyValueStore.java:53)
	at org.apache.kafka.streams.integration.StoreQueryIntegrationTest.lambda$shouldQuerySpecificActivePartitionStores$5(StoreQueryIntegrationTest.java:223)
	at org.apache.kafka.streams.integration.StoreQueryIntegrationTest.retryUntil(StoreQueryIntegrationTest.java:579)
	at org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQuerySpecificActivePartitionStores(StoreQueryIntegrationTest.java:186)

This happens when the stream hasn't been initialized yet. I have run the test around 12k times using Intellij's JUnit testing framework without any flaky failures. The PR also does some minor refactoring regarding moving the list of predicates into their own functions.

Co-authored-by: Bruno Cadonna <cadonna@apache.org>

Reviewer: Bruno Cadonna <cadonna@apache.org>
2022-07-04 20:26:53 +02:00
Guozhang Wang ae570f5953
HOTFIX: Correct ordering of input buffer and enforced processing sensors (#12363)
1. As titled, fix the right constructor param ordering.
2. Also added a few more loglines.

Reviewers: Matthias J. Sax <matthias@confluent.io>, Sagar Rao <sagarmeansocean@gmail.com>, Hao Li <1127478+lihaosky@users.noreply.github.com>
2022-07-03 10:02:59 -07:00
Bruno Cadonna a82a8e02ce
MINOR: Fix static mock usage in TaskMetricsTest (#12373)
Before this PR the calls to the static methods on StreamsMetricsImpl were just calls and not a verification on the mock. This miss happened during the switch from EasyMock to Mockito.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-07-02 18:48:07 -07:00
Guozhang Wang 3faa6cf6d0
MINOR: Use mock time in DefaultStateUpdaterTest (#12344)
For most tests we would need an auto-ticking mock timer to work with draining-with-timeout functions.
For tests that check for never checkpoint we need no auto-ticking timer to control exactly how much time elapsed.

Reviewers: Bruno Cadonna <cadonna@apache.org>
2022-06-29 12:33:00 -07:00
Guozhang Wang ababc4261b
[9/N][Emit final] Emit final for session window aggregations (#12204)
* Add a new API for session windows to range query session window by end time (KIP related).
* Augment session window aggregator with emit strategy.
* Minor: consolidated some dup classes.
* Test: unit test on session window aggregator.

Reviewers: Guozhang Wang <wangguoz@gmail.com>
2022-06-29 09:22:37 -07:00
Tom Kaszuba 025e47b833
KAFKA-13963: Clarified TopologyDescription JavaDoc for Processors API forward() calls (#12293)
Reviewers: Matthias J. Sax <matthias@confluent.io>
2022-06-27 18:50:34 -07:00