mirror of https://github.com/apache/kafka.git
30bc21ca35
This patch rewrites `ProduceRequest` and `ProduceResponse` using the generated protocols. We have also added several new benchmarks to verify no regression in performance. A summary of results is included below:
### Benchmark
1. loop **30** times
1. calculate average
#### kafkatest.benchmarks.core.benchmark_test.Benchmark.test_producer_throughput
> @cluster(num_nodes=5)
> @parametrize(acks=-1, topic=TOPIC_REP_THREE)
- +0.3144915325 %
- 28.08766667 -> 28.1715625 (mb_per_sec)
> @cluster(num_nodes=5)
> @matrix(acks=[1], topic=[TOPIC_REP_THREE], message_size=[100000],compression_type=["none"], security_protocol=['PLAINTEXT'])
- +4.220730323 %
- 157.145 -> 163.7776667 (mb_per_sec)
> @cluster(num_nodes=7)
> @parametrize(acks=1, topic=TOPIC_REP_THREE, num_producers=3)
- +5.996241145%
- 57.64166667 -> 61.098 (mb_per_sec)
> @cluster(num_nodes=5)
> @parametrize(acks=1, topic=TOPIC_REP_THREE)
- +0.3979572536%
- 44.05833333 -> 44.23366667 (mb_per_sec)
> @cluster(num_nodes=5)
> @parametrize(acks=1, topic= TOPIC_REP_ONE)
- +2.228235226%
- 69.23266667 -> 70.77533333 (mb_per_sec)
### JMH results
In short, most ops performance are regression since we have to convert data to protocol data. The cost is inevitable (like other request/response) before we use protocol data directly.
### JMH for ProduceRequest
1. construction regression:
- 281.474 -> 454.935 ns/op
- 296.000 -> 1888.000 B/op
1. toErrorResponse regression:
- 41.942 -> 107.528 ns/op
- 1216.000 -> 1616.000 B/op
1. toStruct improvement:
- 255.185 -> 90.728 ns/op
- 864.000 -> 304.000 B/op
**BEFORE**
```
Benchmark Mode Cnt Score Error Units
ProducerRequestBenchmark.constructorErrorResponse avgt 15 41.942 ± 0.036 ns/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.alloc.rate avgt 15 6409.263 ± 5.478 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.alloc.rate.norm avgt 15 296.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Eden_Space avgt 15 6416.420 ± 76.071 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Eden_Space.norm avgt 15 296.331 ± 3.539 B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Old_Gen avgt 15 0.002 ± 0.002 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Old_Gen.norm avgt 15 ≈ 10⁻⁴ B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.count avgt 15 698.000 counts
ProducerRequestBenchmark.constructorErrorResponse:·gc.time avgt 15 378.000 ms
ProducerRequestBenchmark.constructorProduceRequest avgt 15 281.474 ± 3.286 ns/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.alloc.rate avgt 15 3923.868 ± 46.303 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.alloc.rate.norm avgt 15 1216.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Eden_Space avgt 15 3923.375 ± 59.568 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Eden_Space.norm avgt 15 1215.844 ± 11.184 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Old_Gen avgt 15 0.004 ± 0.001 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Old_Gen.norm avgt 15 0.001 ± 0.001 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.count avgt 15 515.000 counts
ProducerRequestBenchmark.constructorProduceRequest:·gc.time avgt 15 279.000 ms
ProducerRequestBenchmark.constructorStruct avgt 15 255.185 ± 0.069 ns/op
ProducerRequestBenchmark.constructorStruct:·gc.alloc.rate avgt 15 3074.889 ± 0.823 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.alloc.rate.norm avgt 15 864.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Eden_Space avgt 15 3077.737 ± 31.537 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Eden_Space.norm avgt 15 864.800 ± 8.823 B/op
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Old_Gen avgt 15 0.003 ± 0.001 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Old_Gen.norm avgt 15 0.001 ± 0.001 B/op
ProducerRequestBenchmark.constructorStruct:·gc.count avgt 15 404.000 counts
ProducerRequestBenchmark.constructorStruct:·gc.time avgt 15 214.000 ms
```
**AFTER**
```
Benchmark Mode Cnt Score Error Units
ProducerRequestBenchmark.constructorErrorResponse avgt 15 107.528 ± 0.270 ns/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.alloc.rate avgt 15 4864.899 ± 12.132 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.alloc.rate.norm avgt 15 576.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Eden_Space avgt 15 4868.023 ± 61.943 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Eden_Space.norm avgt 15 576.371 ± 7.331 B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Old_Gen avgt 15 0.005 ± 0.001 MB/sec
ProducerRequestBenchmark.constructorErrorResponse:·gc.churn.G1_Old_Gen.norm avgt 15 0.001 ± 0.001 B/op
ProducerRequestBenchmark.constructorErrorResponse:·gc.count avgt 15 639.000 counts
ProducerRequestBenchmark.constructorErrorResponse:·gc.time avgt 15 339.000 ms
ProducerRequestBenchmark.constructorProduceRequest avgt 15 454.935 ± 0.332 ns/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.alloc.rate avgt 15 3769.014 ± 2.767 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.alloc.rate.norm avgt 15 1888.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Eden_Space avgt 15 3763.407 ± 31.530 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Eden_Space.norm avgt 15 1885.190 ± 15.594 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Old_Gen avgt 15 0.004 ± 0.001 MB/sec
ProducerRequestBenchmark.constructorProduceRequest:·gc.churn.G1_Old_Gen.norm avgt 15 0.002 ± 0.001 B/op
ProducerRequestBenchmark.constructorProduceRequest:·gc.count avgt 15 494.000 counts
ProducerRequestBenchmark.constructorProduceRequest:·gc.time avgt 15 264.000 ms
ProducerRequestBenchmark.constructorStruct avgt 15 90.728 ± 0.695 ns/op
ProducerRequestBenchmark.constructorStruct:·gc.alloc.rate avgt 15 3043.140 ± 23.246 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.alloc.rate.norm avgt 15 304.000 ± 0.001 B/op
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Eden_Space avgt 15 3047.251 ± 59.638 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Eden_Space.norm avgt 15 304.404 ± 5.034 B/op
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Old_Gen avgt 15 0.003 ± 0.001 MB/sec
ProducerRequestBenchmark.constructorStruct:·gc.churn.G1_Old_Gen.norm avgt 15 ≈ 10⁻⁴ B/op
ProducerRequestBenchmark.constructorStruct:·gc.count avgt 15 400.000 counts
ProducerRequestBenchmark.constructorStruct:·gc.time avgt 15 205.000 ms
```
### JMH for ProduceResponse
1. construction regression:
- 3.293 -> 303.226 ns/op
- 24.000 -> 1848.000 B/op
1. toStruct improvement:
- 825.889 -> 311.725 ns/op
- 2208.000 -> 896.000 B/op
**BEFORE**
```
Benchmark Mode Cnt Score Error Units
ProducerResponseBenchmark.constructorProduceResponse avgt 15 3.293 ± 0.004 ns/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.alloc.rate avgt 15 6619.731 ± 9.075 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.alloc.rate.norm avgt 15 24.000 ± 0.001 B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Eden_Space avgt 15 6618.648 ± 0.153 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Eden_Space.norm avgt 15 23.996 ± 0.033 B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Old_Gen avgt 15 0.003 ± 0.002 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Old_Gen.norm avgt 15 ≈ 10⁻⁵ B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.count avgt 15 720.000 counts
ProducerResponseBenchmark.constructorProduceResponse:·gc.time avgt 15 383.000 ms
ProducerResponseBenchmark.constructorStruct avgt 15 825.889 ± 0.638 ns/op
ProducerResponseBenchmark.constructorStruct:·gc.alloc.rate avgt 15 2428.000 ± 1.899 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.alloc.rate.norm avgt 15 2208.000 ± 0.001 B/op
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Eden_Space avgt 15 2430.196 ± 55.894 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Eden_Space.norm avgt 15 2210.001 ± 51.009 B/op
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Old_Gen avgt 15 0.003 ± 0.001 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Old_Gen.norm avgt 15 0.002 ± 0.001 B/op
ProducerResponseBenchmark.constructorStruct:·gc.count avgt 15 319.000 counts
ProducerResponseBenchmark.constructorStruct:·gc.time avgt 15 166.000 ms
```
**AFTER**
```
Benchmark Mode Cnt Score Error Units
ProducerResponseBenchmark.constructorProduceResponse avgt 15 303.226 ± 0.517 ns/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.alloc.rate avgt 15 5534.940 ± 9.439 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.alloc.rate.norm avgt 15 1848.000 ± 0.001 B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Eden_Space avgt 15 5534.046 ± 51.849 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Eden_Space.norm avgt 15 1847.710 ± 18.105 B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Old_Gen avgt 15 0.007 ± 0.001 MB/sec
ProducerResponseBenchmark.constructorProduceResponse:·gc.churn.G1_Old_Gen.norm avgt 15 0.002 ± 0.001 B/op
ProducerResponseBenchmark.constructorProduceResponse:·gc.count avgt 15 602.000 counts
ProducerResponseBenchmark.constructorProduceResponse:·gc.time avgt 15 318.000 ms
ProducerResponseBenchmark.constructorStruct avgt 15 311.725 ± 3.132 ns/op
ProducerResponseBenchmark.constructorStruct:·gc.alloc.rate avgt 15 2610.602 ± 25.964 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.alloc.rate.norm avgt 15 896.000 ± 0.001 B/op
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Eden_Space avgt 15 2613.021 ± 42.965 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Eden_Space.norm avgt 15 896.824 ± 11.331 B/op
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Old_Gen avgt 15 0.003 ± 0.001 MB/sec
ProducerResponseBenchmark.constructorStruct:·gc.churn.G1_Old_Gen.norm avgt 15 0.001 ± 0.001 B/op
ProducerResponseBenchmark.constructorStruct:·gc.count avgt 15 343.000 counts
ProducerResponseBenchmark.constructorStruct:·gc.time avgt 15 194.000 ms
```
Reviewers: David Jacot <djacot@confluent.io>, Jason Gustafson <jason@confluent.io>
|
||
|---|---|---|
| .. | ||
| src | ||
| .gitignore | ||