MINOR: Add docs for replica.alter.log.dirs.io.max.bytes.per.second config (#16386)

Reviewers: Luke Chen <showuon@gmail.com>
This commit is contained in:
Mickael Maison 2024-06-19 11:19:59 +02:00 committed by GitHub
parent a0bfc64470
commit 2b491f9611
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 16 additions and 15 deletions

View File

@ -351,16 +351,17 @@ Topic:foo PartitionCount:1 ReplicationFactor:3 Configs:
Topic: foo Partition: 0 Leader: 5 Replicas: 5,6,7 Isr: 5,6,7</code></pre> Topic: foo Partition: 0 Leader: 5 Replicas: 5,6,7 Isr: 5,6,7</code></pre>
<h4 class="anchor-heading"><a id="rep-throttle" class="anchor-link"></a><a href="#rep-throttle">Limiting Bandwidth Usage during Data Migration</a></h4> <h4 class="anchor-heading"><a id="rep-throttle" class="anchor-link"></a><a href="#rep-throttle">Limiting Bandwidth Usage during Data Migration</a></h4>
Kafka lets you apply a throttle to replication traffic, setting an upper bound on the bandwidth used to move replicas from machine to machine. This is useful when rebalancing a cluster, bootstrapping a new broker or adding or removing brokers, as it limits the impact these data-intensive operations will have on users. Kafka lets you apply a throttle to replication traffic, setting an upper bound on the bandwidth used to move replicas from machine to machine and from disk to disk. This is useful when rebalancing a cluster, adding or removing brokers or adding or removing disks, as it limits the impact these data-intensive operations will have on users.
<p></p> <p></p>
There are two interfaces that can be used to engage a throttle. The simplest, and safest, is to apply a throttle when invoking the kafka-reassign-partitions.sh, but kafka-configs.sh can also be used to view and alter the throttle values directly. There are two interfaces that can be used to engage a throttle. The simplest, and safest, is to apply a throttle when invoking the kafka-reassign-partitions.sh, but kafka-configs.sh can also be used to view and alter the throttle values directly.
<p></p> <p></p>
So for example, if you were to execute a rebalance, with the below command, it would move partitions at no more than 50MB/s. So for example, if you were to execute a rebalance, with the below command, it would move partitions at no more than 50MB/s between brokers, and at no more than 100MB/s between disks on a broker.
<pre class="language-bash">$ bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --execute --reassignment-json-file bigger-cluster.json --throttle 50000000</pre> <pre class="language-bash">$ bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --execute --reassignment-json-file bigger-cluster.json --throttle 50000000 --replica-alter-log-dirs-throttle 100000000</pre>
When you execute this script you will see the throttle engage: When you execute this script you will see the throttle engage:
<pre><code class="language-bash">The inter-broker throttle limit was set to 50000000 B/s <pre><code class="language-text">The inter-broker throttle limit was set to 50000000 B/s
The replica-alter-dir throttle limit was set to 100000000 B/s
Successfully started partition reassignment for foo1-0</code></pre> Successfully started partition reassignment for foo1-0</code></pre>
<p>Should you wish to alter the throttle, during a rebalance, say to increase the throughput so it completes quicker, you can do this by re-running the execute command with the --additional option passing the same reassignment-json-file:</p> <p>Should you wish to alter the throttle, during a rebalance, say to increase the inter-broker throughput so it completes quicker, you can do this by re-running the execute command with the --additional option passing the same reassignment-json-file:</p>
<pre class="language-bash">$ bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --additional --execute --reassignment-json-file bigger-cluster.json --throttle 700000000 <pre class="language-bash">$ bin/kafka-reassign-partitions.sh --bootstrap-server localhost:9092 --additional --execute --reassignment-json-file bigger-cluster.json --throttle 700000000
The inter-broker throttle limit was set to 700000000 B/s</pre> The inter-broker throttle limit was set to 700000000 B/s</pre>
@ -378,12 +379,13 @@ Reassignment of partition [my-topic,0] is completed
Clearing broker-level throttles on brokers 1,2,3 Clearing broker-level throttles on brokers 1,2,3
Clearing topic-level throttles on topic my-topic</code></pre> Clearing topic-level throttles on topic my-topic</code></pre>
<p>The administrator can also validate the assigned configs using the kafka-configs.sh. There are two pairs of throttle <p>The administrator can also validate the assigned configs using the kafka-configs.sh. There are two sets of throttle
configuration used to manage the throttling process. First pair refers to the throttle value itself. This is configured, at a broker configuration used to manage the throttling process. First set refers to the throttle value itself. This is configured, at a broker
level, using the dynamic properties: </p> level, using the dynamic properties: </p>
<pre><code class="language-text">leader.replication.throttled.rate <pre><code class="language-text">leader.replication.throttled.rate
follower.replication.throttled.rate</code></pre> follower.replication.throttled.rate
replica.alter.log.dirs.io.max.bytes.per.second</code></pre>
<p>Then there is the configuration pair of enumerated sets of throttled replicas: </p> <p>Then there is the configuration pair of enumerated sets of throttled replicas: </p>
@ -392,16 +394,16 @@ follower.replication.throttled.replicas</code></pre>
<p>Which are configured per topic. </p> <p>Which are configured per topic. </p>
<p>All four config values are automatically assigned by kafka-reassign-partitions.sh (discussed below).</p> <p>All five config values are automatically assigned by kafka-reassign-partitions.sh (discussed below).</p>
<p>To view the throttle limit configuration:</p> <p>To view the throttle limit configuration:</p>
<pre><code class="language-bash">$ bin/kafka-configs.sh --describe --bootstrap-server localhost:9092 --entity-type brokers <pre><code class="language-bash">$ bin/kafka-configs.sh --describe --bootstrap-server localhost:9092 --entity-type brokers
Configs for brokers '2' are leader.replication.throttled.rate=700000000,follower.replication.throttled.rate=700000000 Configs for brokers '2' are leader.replication.throttled.rate=700000000,follower.replication.throttled.rate=700000000,replica.alter.log.dirs.io.max.bytes.per.second=1000000000
Configs for brokers '1' are leader.replication.throttled.rate=700000000,follower.replication.throttled.rate=700000000</code></pre> Configs for brokers '1' are leader.replication.throttled.rate=700000000,follower.replication.throttled.rate=700000000,replica.alter.log.dirs.io.max.bytes.per.second=1000000000</code></pre>
<p>This shows the throttle applied to both leader and follower side of the replication protocol. By default both sides <p>This shows the throttle applied to both leader and follower side of the replication protocol (by default both sides
are assigned the same throttled throughput value. </p> are assigned the same throttled throughput value), as well as the disk throttle.</p>
<p>To view the list of throttled replicas:</p> <p>To view the list of throttled replicas:</p>
@ -441,8 +443,7 @@ Configs for topic 'my-topic' are leader.replication.throttled.replicas=1:102,0:1
<pre>kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=([-.\w]+),topic=([-.\w]+),partition=([0-9]+)</pre> <pre>kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=([-.\w]+),topic=([-.\w]+),partition=([0-9]+)</pre>
<p>The lag should constantly decrease during replication. If the metric does not decrease the administrator should <p>The lag should constantly decrease during replication. If the metric does not decrease the administrator should
increase the increase the throttle throughput as described above. </p>
throttle throughput as described above. </p>
<h4 class="anchor-heading"><a id="quotas" class="anchor-link"></a><a href="#quotas">Setting quotas</a></h4> <h4 class="anchor-heading"><a id="quotas" class="anchor-link"></a><a href="#quotas">Setting quotas</a></h4>