diff --git a/docs/ops.html b/docs/ops.html index 4b2058cafef..8e1395e6618 100644 --- a/docs/ops.html +++ b/docs/ops.html @@ -3161,9 +3161,11 @@ active-process-ratio metrics which have a recording level of info:
State Store Metrics
-All of the following metrics have a recording level of debug, except for the record-e2e-latency-* metrics which have a recording level trace. -Note that the store-scope value is specified in StoreSupplier#metricsScope() for user's customized state stores; -for built-in state stores, currently we have: +All the following metrics have a recording level of debug, except for the record-e2e-latency-* +metrics which have a recording level trace and num-open-iterators which has recording level +info. +Note that the store-scope value is specified in StoreSupplier#metricsScope() for user's +customized state stores; for built-in state stores, currently we have: @@ -333,7 +334,6 @@ streamsSettings.put(StreamsConfig.NUM_STANDBY_REPLICAS_CONFIG, 1); the org.apache.kafka.streams.state.DslStoreSuppliers interface. BuiltInDslStoreSuppliers.RocksDBDslStoreSuppliers - null max.task.idle.ms Medium @@ -439,28 +439,33 @@ streamsSettings.put(StreamsConfig.NUM_STANDBY_REPLICAS_CONFIG, 1); Directory location for state stores. /${java.io.tmpdir}/kafka-streams - task.timeout.ms + task.assignor.class + Medium + A task assignor class or class name implementing the TaskAssignor interface. + The high-availability task assignor. + + task.timeout.ms Medium The maximum amount of time in milliseconds a task might stall due to internal errors and retries until an error is raised. For a timeout of 0 ms, a task would raise an error for the first internal error. For any timeout larger than 0 ms, a task will retry at least once before an error is raised. 300000 milliseconds (5 minutes) - topology.optimization + topology.optimization Medium A configuration telling Kafka Streams if it should optimize the topology and what optimizations to apply. Acceptable values are: StreamsConfig.NO_OPTIMIZATION (none), StreamsConfig.OPTIMIZE (all) or a comma separated list of specific optimizations: StreamsConfig.REUSE_KTABLE_SOURCE_TOPICS (reuse.ktable.source.topics), StreamsConfig.MERGE_REPARTITION_TOPICS (merge.repartition.topics), StreamsConfig.SINGLE_STORE_SELF_JOIN (single.store.self.join). - NO_OPTIMIZATION + NO_OPTIMIZATION - upgrade.from + upgrade.from Medium The version you are upgrading from during a rolling upgrade. See Upgrade From - windowstore.changelog.additional.retention.ms + windowstore.changelog.additional.retention.ms Low Added to a windows maintainMs to ensure data is not deleted from the log prematurely. Allows for clock drift. 86400000 milliseconds (1 day) - window.size.ms + window.size.ms Low Sets window size for the deserializer in order to calculate window end times. null @@ -1017,6 +1022,18 @@ streamsConfig.put(StreamsConfig.ROCKSDB_CONFIG_SETTER_CLASS_CONFIG, CustomRocksD this path must be unique for each such instance. +
+

task.assignor.class

+
+
A task assignor class or class name implementing the + org.apache.kafka.streams.processor.assignment.TaskAssignor interface. Defaults to the + high-availability task assignor. One possible alternative implementation provided in Apache Kafka is the + org.apache.kafka.streams.processor.assignment.assignors.StickyTaskAssignor, which was the default task + assignor before KIP-441 and minimizes task movement at the cost of stateful task availability. Alternative implementations of + the task assignment algorithm can be plugged into the application by implementing a custom TaskAssignor and setting this config to the name of the custom task assignor class. +
+
+

topology.optimization

diff --git a/docs/streams/upgrade-guide.html b/docs/streams/upgrade-guide.html index 127cb99c3b1..f24edfbd42e 100644 --- a/docs/streams/upgrade-guide.html +++ b/docs/streams/upgrade-guide.html @@ -133,12 +133,44 @@ More details about the new config StreamsConfig#TOPOLOGY_OPTIMIZATION can be found in KIP-295.

-

Streams API changes in 3.8.0

+

Streams API changes in 3.8.0

+ +

+ Kafka Streams now supports customizable task assignment strategies via the task.assignor.class + configuration. The configuration can be set to the fully qualified class name of a custom task assignor + implementation that has to extend the new + org.apache.kafka.streams.processor.assignment.TaskAssignor interface. + + The new configuration also allows users to bring back the behavior of the old task assignor + StickyTaskAssignor that was used before the introduction of the + HighAvailabilityTaskAssignor. If no custom task assignor is configured, the default task assignor + HighAvailabilityTaskAssignor is used. + + If you were using the internal.task.assignor.class config, you should switch to using the new + task.assignor.class config instead, as the internal config will be removed in a future release. + If you were previously plugging in the StickyTaskAssignor via the legacy + internal.task.assignor.class config, you will need to make sure that you are importing + the new org.apache.kafka.streams.processor.assignment.StickTaskAssignor when you switch + over to the new task.assignor.class config, which is a version of the StickyTaskAssignor + that implements the new public TaskAssignor interface. + + For more details, see the public interface section of + KIP-924. +

+

The Processor API now support so-called read-only state stores, added via KIP-813. - These stores don't have a dedicated changelog topic, but use their source topic for fault-tolerance, - simlar to KTables with source-topic optimization enabled. + These stores don't have a dedicated changelog topic, but use their source topic for fault-tolerance, + similar to KTables with source-topic optimization enabled. +

+ +

+ To improve detection of leaked state store iterators, we added new store-level metrics to track the number and + age of open iterators. The new metrics are num-open-iterators, iterator-duration-avg, + iterator-duration-max and oldest-iterator-open-since-ms. These metrics are available + for all state stores, including RocksDB, in-memory, and custom stores. More details can be found in + KIP-989.

Streams API changes in 3.7.0