* KAFKA-17227: Update zstd-jni lib
* Add note in upgrade docs
* Change zstd-jni version in docker native file and add warning in dependencies.gradle file
* Add reference to snappy in upgrade
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Mickael Maison <mickael.maison@gmail.com>
This PR contains the the following documentation changes for the native docker image:
in the docker/README.md: How to build, release and promote the native docker image.
in the tests/README.md: How to run system tests by bringing up kafka in the native mode.
added docker/native/README.md
added html changes for the kafka-site
added native docker image support in the docker compose files examples.
Testing:
Tested all the docker compose files with both the docker images - jvm and native
Tested the html changes locally with the kafka-site
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Vedarth Sharma <vesharma@confluent.io>
This PR aims to add the static Dockerfile and scripts for AK 3.7.0 version. As mentioned in KIP-1028 this PR aims to start the release of the kafka:3.7.0 Docker Official image. This will also help us validate the process and allow us to address any changes suggested by Dockerhub before the 3.8.0 release.
The static Dockerfile and scripts have been generated via the github actions workflows and scripts added as part of https://github.com/apache/kafka/pull/16027. The reports of build and testing the 3.7.0 Docker official image are below.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Vedarth Sharma <vesharma@confluent.io>
This PR aims to add the replace the dockerfile comments under the jvm/dockerfile Dockerfile, with updated url.
The original comment read
# Get kafka from https://archive.apache.org/dist/kafka and pass the url through build arguments . For DOI dockerfiles, we replace this with
# Get Kafka from https://downloads.apache.org/kafka, url passed as env var, for version {kafka_version} .
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Vedarth Sharma <vesharma@confluent.io>
This PR aims to modify the README file under /docker, to include the steps to release the Docker Official Images.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Vedarth Sharma <vesharma@confluent.io>
This PR does following things
System tests should bring up Kafka broker in the native mode
System tests should run on Kafka broker in native mode
Extract out native build command so that it can be reused.
Allow system tests to run on Native Kafka broker using Docker mechanism
To run system tests by bringing up Kafka in native mode:
Pass kafka_mode as native in the ducktape globals:--globals '{\"kafka_mode\":\"native\"}'
Running system tests by bringing up kafka in native mode via docker mechanism
_DUCKTAPE_OPTIONS="--globals '{\"kafka_mode\":\"native\"}'" TC_PATHS="tests/kafkatest/tests/" bash tests/docker/run_tests.sh
To only bring up ducker nodes to cater native kafka
bash tests/docker/ducker-ak up -m native
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
This PR aims to add JVM based Docker Official Image for Apache Kafka as per the following KIP - https://cwiki.apache.org/confluence/display/KAFKA/KIP-1028%3A+Docker+Official+Image+for+Apache+Kafka
This PR adds the following functionalities:
Introduces support for Apache Kafka Docker Official Images via:
GitHub Workflows:
- Workflow to prepare static source files for Docker images
- Workflow to build and test Docker official images
- Scripts to prepare source files and perform Docker image builds and tests
A new directory for Docker official images, named docker/docker_official_images. This is the new directory to house all Docker Official Image assets.
Co-authored-by: Vedarth Sharma <vesharma@confluent.io>
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Vedarth Sharma <vesharma@confluent.io>
This PR aims to add Docker Image for GraalVM based Native Kafka Broker as per the following KIP - https://cwiki.apache.org/confluence/display/KAFKA/KIP-974%3A+Docker+Image+for+GraalVM+based+Native+Kafka+Broker
This PR adds the following functionalities:
Ability to build the docker image for Native Apache Kafka
- Dockerfile
- Launch script
- metadata configs required by graalVM native-image: link
Add Kafka startup ability in the KafkaDockerWrapper.scala
Ability to build and test the image - integrated with the existing JVM docker image framework.
Due to difference in packages present when jsa files were generated and when docker image is generated, there is a log on starting docker image.
[0.001s][warning][cds] The shared archive file has a bad magic number: 0
There is no functionality impact, only startup time is higher because of this issue.
This PR fixes the warning and improves startup performance of the docker image
Reviewers: Manikumar Reddy <anikumar.reddy@gmail.co
The base eclipse-temuring:21-jre-alpine image got modified and had `bash` removed from it. This broke our build, since downstream steps utilizing bash scripts depended on it. This patch explicitly installs bash
Migrates functionality provided by utility to Kafka core. This wrapper will be used to generate property files and format storage when invoked from docker container.
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Viktor Somogyi-Vass <viktorsomogyi@gmail.com>, Manikumar Reddy <manikumar.reddy@gmail.com>
Update the port mapping in test compose files to align with the recommendations in example files
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
Josep Prat
Krishna Agarwal
KrishVora01
Vedarth Sharma
Lokesh Kumar