root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity

MINOR: document increased network bandwidth of 0.10 under replication 

If you're pushing close to the network capacity, 0.10's additional 8 bytes per message can lead to overload of your network. We (Heroku Kafka) ran into this issue whilst benchmarking 0.10 RC and the ijuma suggested it belonged in the update note.

Comments/suggestions welcome.

Author: Tom Crayford <tcrayford@googlemail.com>

Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>

Closes #1389 from tcrayford/upgrade_note_about_increased_network_bandwidth
This commit is contained in:
Tom Crayford 2016-05-16 13:39:33 +01:00 committed by Ismael Juma
parent 7ded19a29e
commit 62985f313f
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/upgrade.html
View File

@ -65,7 +65,10 @@ work with 0.10.0.x brokers. Therefore, 0.9.0.0 clients should be upgraded to 0.9
</p>
<p>
<b>Note:</b> Due to the additional timestamp introduced in each message, producers sending small messages may see a
message throughput degradation because of the increased overhead. When receiving compressed messages, 0.10.0
message throughput degradation because of the increased overhead.
Likewise, replication now transmits an additional 8 bytes per message.
If you're running close to the network capacity of your cluster, it's possible that you'll overwhelm the network cards
and see failures and performance issues due to the overload. When receiving compressed messages, 0.10.0
brokers avoid recompressing the messages, which in general reduces the latency and improves the throughput. In
certain cases, this may reduce the batching size on the producer, which could lead to worse throughput. If this
happens, users can tune linger.ms and batch.size of the producer for better throughput.