root
/
kafka
mirror of https://github.com/apache/kafka.git
1
0
Fork 0
Code Issues Actions 2 Packages Projects Releases Wiki Activity

KAFKA-17227: Update zstd-jni lib (#16763) 

* KAFKA-17227: Update zstd-jni lib
* Add note in upgrade docs
* Change zstd-jni version in docker native file and add warning in dependencies.gradle file
* Add reference to snappy in upgrade

Reviewers:  Chia-Ping Tsai <chia7712@gmail.com>,  Mickael Maison <mickael.maison@gmail.com>
This commit is contained in:
Josep Prat 2024-08-05 09:51:46 +02:00 committed by GitHub
parent 0b78d8459c
commit a4277c6617
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 16 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
LICENSE-binary
View File

@ -327,7 +327,7 @@ pcollections-4.0.1, see: licenses/pcollections-MIT
---------------------------------------
BSD 2-Clause
zstd-jni-1.5.6-3 see: licenses/zstd-jni-BSD-2-clause
zstd-jni-1.5.6-4 see: licenses/zstd-jni-BSD-2-clause
---------------------------------------
BSD 3-Clause

4
docker/native/native-image-configs/resource-config.json
View File

@ -25,9 +25,9 @@
}, {
"pattern":"\\Qkafka/kafka-version.properties\\E"
}, {
"pattern":"\\Qlinux/amd64/libzstd-jni-1.5.6-3.so\\E"
"pattern":"\\Qlinux/amd64/libzstd-jni-1.5.6-4.so\\E"
}, {
"pattern":"\\Qlinux/aarch64/libzstd-jni-1.5.6-3.so\\E"
"pattern":"\\Qlinux/aarch64/libzstd-jni-1.5.6-4.so\\E"
}, {
"pattern":"\\Qnet/jpountz/util/linux/amd64/liblz4-java.so\\E"
}, {

11
docs/upgrade.html
View File

@ -19,6 +19,17 @@
<script id="upgrade-template" type="text/x-handlebars-template">
<h4><a id="upgrade_3_9_0" href="#upgrade_3_9_0">Upgrading to 3.9.0 from any version 0.8.x through 3.8.x</a></h4>
<h5><a id="upgrade_390_notable" href="#upgrade_390_notable">Notable changes in 3.9.0</a></h5>
<ul>
<li>In case you run your Kafka clusters with no execution permission for the <code>/tmp</code> partition, Kafka will not work properly. It might either refuse to start or fail
when producing and consuming messages. This is due to the compression libraries <code>zstd-jni</code> and <code>snappy</code>.
To remediate this problem you need to pass the following JVM flags to Kafka <code>ZstdTempFolder</code> and <code>org.xerial.snappy.tempdir</code> pointing to a directory with execution permissions.
For example, this could be done via the <code>KAFKA_OPTS</code> environment variable like follows: <code>export KAFKA_OPTS="-DZstdTempFolder=/opt/kafka/tmp -Dorg.xerial.snappy.tempdir=/opt/kafka/tmp"</code>.
This is a known issue for version 3.8.0 as well.
</li>
</ul>
<h4><a id="upgrade_3_8_0" href="#upgrade_3_8_0">Upgrading to 3.8.0 from any version 0.8.x through 3.7.x</a></h4>
<h5><a id="upgrade_380_notable" href="#upgrade_380_notable">Notable changes in 3.8.0</a></h5>

3
gradle/dependencies.gradle
View File

@ -157,7 +157,8 @@ versions += [
spotbugs: "4.8.0",
zinc: "1.9.2",
zookeeper: "3.8.4",
zstd: "1.5.6-3",
// When updating the zstd version, please do as well in docker/native/native-image-configs/resource-config.json
zstd: "1.5.6-4",
junitPlatform: "1.10.2"
]