root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity

MINOR: Clarify docs for Streams config max.warmup.replicas. (#13082) 

Documentation only—Minor clarification on how max.warmup.replicas works; specifically, that one "warmup replica" corresponds to a Task that is restoring its state. Also clarifies how max.warmup.replicas interacts with probing.rebalance.interval.ms.

Reviewers: Lucas Brutschy <lbrutschy@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
This commit is contained in:
littlehorse-eng 2023-03-03 11:00:51 -08:00 committed by GitHub
parent 38c409cf33
commit a6d8988179
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 19 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
docs/streams/developer-guide/config-streams.html
View File

@ -778,10 +778,20 @@ rack.aware.assignment.tags: zone,cluster | rack.aware.assignment.tags: zone,cl
<span id="streams-developer-guide-max-warmup-replicas"></span><h4><a class="toc-backref" href="#id29">max.warmup.replicas</a><a class="headerlink" href="#max-warmup-replicas" title="Permalink to this headline"></a></h4>
<blockquote>
<div>
The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping
the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
<p>
The maximum number of warmup replicas (extra standbys beyond the configured <code class="docutils literal"><span class="pre">num.standbys</span></code>) that can be assigned at once for the purpose of keeping
the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker
traffic and cluster state can be used for high availability. Increasing this will allow Streams to warm up more tasks at once, speeding up the time
for the reassigned warmups to restore sufficient state for them to be transitioned to active tasks. Must be at least 1.
</p>
<p>
Note that one warmup replica corresponds to one <a href="https://kafka.apache.org/34/documentation/streams/architecture#streams_architecture_tasks">Stream Task</a>. Furthermore, note that each warmup task can only be promoted to an active task during
a rebalance (normally during a so-called probing rebalance, which occur at a frequency specified by the
<code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code> config). This means that the
maximum rate at which active tasks can be migrated from one Kafka Streams instance to another instance can be determined by
(<code class="docutils literal"><span class="pre">max.warmup.replicas</span></code> /
<code class="docutils literal"><span class="pre">probing.rebalance.interval.ms</span></code>).
</p>
</div>
</blockquote>
</div>

6
streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java
View File

@ -585,7 +585,11 @@ public class StreamsConfig extends AbstractConfig {
public static final String MAX_WARMUP_REPLICAS_CONFIG = "max.warmup.replicas";
private static final String MAX_WARMUP_REPLICAS_DOC = "The maximum number of warmup replicas (extra standbys beyond the configured num.standbys) that can be assigned at once for the purpose of keeping " +
" the task available on one instance while it is warming up on another instance it has been reassigned to. Used to throttle how much extra broker " +
" traffic and cluster state can be used for high availability. Must be at least 1.";
" traffic and cluster state can be used for high availability. Must be at least 1." +
"Note that one warmup replica corresponds to one Stream Task. Furthermore, note that each warmup replica can only be promoted to an active task " +
"during a rebalance (normally during a so-called probing rebalance, which occur at a frequency specified by the `probing.rebalance.interval.ms` config). This means " +
"that the maximum rate at which active tasks can be migrated from one Kafka Streams Instance to another instance can be determined by " +
"(`max.warmup.replicas` / `probing.rebalance.interval.ms`).";
/** {@code metadata.max.age.ms} */
@SuppressWarnings("WeakerAccess")