A prior commit introduced checking for the version of a node related to move to log4j2 but it was causing an error
AttributeError("'ClusterNode' object has no attribute 'version'") This PR uses the get_version method from version.py which checks if the Node has a version attribute preventing an error.
Reviewers: Matthias Sax <mjsax@apache.org>
Due to an issue with handling folders in Kafka version 3.3.2 (see https://github.com/apache/kafka/pull/13130), this end-to-end test requires using a single folder for upgrade/downgrade scenarios involving 3.3.2.
Reviewers: Ismael Juma <ismael@juma.me.uk>, Chia-Ping Tsai <chia7712@gmail.com>
Related to KAFKA-18343,
Currently, there is an issue that the ps ax output is truncated, which causes the Kafka process ID to be unavailable. This issue can be mitigated by replacing ps ax with jcmd (i.e. using java_pids in ducktape), as it does not suffer from the truncation problem.
Reviewers: Justine Olshan <jolshan@confluent.io>
Added transaction version 2 to some of the system tests. Also marking TV2 as production ready.
Also fixes the defaultVersion test.
Reviewers: Jun Rao <jun@confluent.io>
This pull request replaces Log4j with Log4j2 across the entire project, including dependencies, configurations, and code. The notable changes are listed below:
1. Introduce Log4j2 Instead of Log4j
2. Change Configuration File Format from Properties to YAML
3. Adds warnings to notify users if they are still using Log4j properties, encouraging them to transition to Log4j2 configurations
Co-authored-by: Lee Dongjin <dongjin@apache.org>
Reviewers: Luke Chen <showuon@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Added ShareConsumeBenchSpec and ShareConsumeBenchWorker similar to ConsumeBenchSpec/ConsumeBenchWorker. This will help us run trogdor workloads for share consumers as well.
Added a sample json workload running 5 share consumers.
Reviewers: Andrew Schofield <aschofield@confluent.io>, Manikumar Reddy <manikumar.reddy@gmail.com>
The org.reflections is removed, so the initial logger of worker is only "root". However, the e2e needs a non-root logger to verify dynamic logger
We can add a logger to connect_log4j.properties to fix this e2e. For example:
log4j.logger.org.apache.kafka.clients.consumer.ConsumerConfig=ERROR
this can make admin/logger return two logger - org.apache.kafka.clients.consumer.ConsumerConfig and root
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
- Remove some unused Zookeeper code
- Migrate group mode transactions, security rolling upgrade, and throttling tests to using KRaft
- Add KRaft downgrade tests to kraft_upgrade_test.py
Reviewers: Colin P. McCabe <cmccabe@apache.org>
This commit adds AK 3.9 to the system tests on trunk.
Follow-up of #17797
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Bruno Cadonna <cadonna@apache.org>
We plan to remove all ZooKeeper-related code in version 4.0. However, some old brokers in the end-to-end tests still require ZooKeeper service, so we need to run the ZooKeeper service using the 3.x release instead of the dev branch.
Since version 3.9 is not available in the https://s3-us-west-2.amazonaws.com/kafka-packages repo, we can use version 3.8 for now.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Added ShareRoundTripWorker.java similar to RoundTripWorker.java. This will start a producer and a share consumer on a single node. The share consumer reads back the messages produced by the producer.
Reviewers: Andrew Schofield <aschofield@confluent.io>, Manikumar Reddy <manikumar.reddy@gmail.com>
Migrates existing connect tests that were using Zookeeper to use KRaft
instead, and cleans up some dead ZK code. For broker compatibility tests,
tests for versions 2.1-2.3 still need to use ZK.
Reviewers: Colin P. McCabe <cmccabe@apache.org>
This is part one of a multi-pr effort to convert Kafka Streams system tests to KRaft. I decided to break down the changes into multiple PRs to reduce the review load
Reviewers: Matthias Sax <mjsax@apache.org>
This PR removes ZK test parameterizations from ducktape by:
- Removing zk from quorum.all_non_upgrade
- Removing quorum.zk from @matrix and @parametrize annotations
- Changing usages of quorum.all to quorum.all_kraft
- Deleting message_format_change_test.py
The default metadata_quorum value still needs to be changed to KRaft rather than ZK, but this will be done in a follow-up PR.
Reviewers: Kirk True <kirk@kirktrue.pro>, Colin P. McCabe <cmccabe@apache.org>
Bill Bejeck
TengYao Chi
Chirag Wadhwa
TaiJuWu
Kirk True
kevin-wu24
Jhen-Yung Hsu
Kuan-Po Tseng
Justine Olshan
David Jacot
Jhen-Yung Hsu
ClarkChen
Nick Guo
Ken Huang
ShivsundarR
mingdaoy
Peter Lee
PoAn Yang
Josep Prat
Mahsa Seifikar