Commit Graph

246 Commits

Author SHA1 Message Date
Gaurav Narula 8f849f91f1
MINOR: Update Scala to v2.13.16 to support Java 24 (#19782)
Refer

https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html#scala-compatibility-table

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2025-05-23 00:54:51 +08:00
Manikumar Reddy bb7d8eb2c8
MINOR: Supress stdout when checking Log4j 1.x configuration compatibility mode (#19502)
when using log41 config, we are printing addtional line like below. This
PR is to fix that.
2025-04-17 14:29:21 +05:30
Alieh Saeedi fa62bce63d
KAFKA-18287: Add support for kafka-streams-groups.sh --list (#19422)
Implement the core of kafka-streams-groups.sh for `KIP-1071`
- Implement `--list` and its options: (only `--state`)

Reviewers: Bruno Cadonna <cadonna@apache.org>
2025-04-09 17:57:34 +02:00
Chirag Wadhwa 8cc560e954
Infrastructure for system tests for the new share consumer client (#18209)
Reviewers: Apoorv Mittal <apoorvmittal10@gmail.com>, Andrew Schofield <aschofield@confluent.io>
2025-01-17 12:03:32 +00:00
Abhinav Dixit b51b31ed9c
KAFKA-18428: Measure share consumers performance (#18415)
Reviewers: Andrew Schofield <aschofield@confluent.io>
2025-01-08 14:33:13 +00:00
TengYao Chi 5ae4b62876
KAFKA-18347 Add tools-log4j2.yaml to config and remove unsed tools-log4j.properties from config (#18351)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2025-01-01 21:06:01 +08:00
TengYao Chi 585b7db482
MINOR: log4j2 upgrade follow-up (#18290)
Reviewers: PoAn Yang <payang@apache.org>, Chia-Ping Tsai <chia7712@gmail.com>
2024-12-30 20:38:58 +08:00
Mickael Maison 57eb5fd7dc
KAFKA-14587: Move AclCommand to tools (#17880)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-14 20:05:46 +01:00
TengYao Chi b37b89c668
KAFKA-9366 Upgrade log4j to log4j2 (#17373)
This pull request replaces Log4j with Log4j2 across the entire project, including dependencies, configurations, and code. The notable changes are listed below:

1. Introduce Log4j2 Instead of Log4j
2. Change Configuration File Format from Properties to YAML
3. Adds warnings to notify users if they are still using Log4j properties, encouraging them to transition to Log4j2 configurations

Co-authored-by: Lee Dongjin <dongjin@apache.org>

Reviewers: Luke Chen <showuon@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-12-14 01:14:31 +08:00
mingdaoy 4603f7495e
KAFKA-18030 Remove old upgrade-system-tests modules (#17843)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-10 11:19:14 +08:00
Mickael Maison 7ef56a9313
KAFKA-17987 Remove ZooKeeper related windows scripts (#17811)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-18 02:21:24 +08:00
Colin Patrick McCabe 085b27ec6e
KAFKA-17987 Remove assorted ZK-related files (#17768)
Remove zookeeper files in bin:
- bin/zookeeper-security-migration.sh
- bin/zookeeper-server-start.sh
- bin/zookeeper-server-stop.sh
- bin/zookeeper-shell.sh

Remove files used to configure Kafka in zookeeper mode in config:
- config/server.properties
- config/zookeeper.properties

Remove ZK references from all remaining Kafka configuration files.

Remove ZK references from all log4j.properties files.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-13 20:32:18 +08:00
Andrew Schofield 3d9f88daf3
KAFKA-17546 Admin.listGroups and kafka-groups.sh (#17626)
This implements the kafka-groups.sh tool and Admin.listGroups method defined in KIP-1043.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-02 05:37:04 +08:00
Dmitry Werner 672c617233
KAFKA-14577: Move ConsoleProducer to tools module (#17157)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Federico Valeri <fedevaleri@gmail.com>
2024-10-07 11:19:59 +02:00
Ken Huang 9cb49092f0
KAFKA-17653 Update Scala version to 2.13.15 for support Java 23 (#17314)
Scala compatibility table: https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html#scala-compatibility-table

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-09-30 02:08:51 +08:00
abhi-ksolves fb19b3f7e7
KAFKA-14262 Deletion of MirrorMaker v1 deprecated classes & tests (#16879)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-08-28 09:49:56 +08:00
PoAn Yang 28cd7136fb
KAFKA-16324 Move BrokerApiVersionsCommand to tools (#16850)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-08-22 09:22:20 +08:00
ShivsundarR 8cfd631264
KAFKA-16723 : Added kafka-console-share-consumer.sh tool. (#16860)
Added kafka-console-share-consumer.sh which will start a share consumer on a share group.
This tool helps to read data from Kafka topics using share groups and outputs it to standard output.

Reviewers:  Andrew Schofield <aschofield@confluent.io>, Apoorv Mittal <apoorvmittal10@gmail.com>,  Manikumar Reddy <manikumar.reddy@gmail.com>
2024-08-16 15:26:54 +05:30
Andrew Schofield d64f4b9cd0
KAFKA-16714: kafka-share-groups.sh list and describe (#16835)
Introduces kafka-share-groups.sh for listing and describing share groups. The PR also contains the remaining options in the command parser and usage message in preparation of their implementation.

Reviewers:  Manikumar Reddy <manikumar.reddy@gmail.com>, Apoorv Mittal <apoorvmittal10@gmail.com>
2024-08-14 16:22:26 +05:30
Krishna Agarwal bb6a042e99
KAFKA-16827: Integrate kafka native-image with system tests (#16046)
This PR does following things

System tests should bring up Kafka broker in the native mode
System tests should run on Kafka broker in native mode
Extract out native build command so that it can be reused.
Allow system tests to run on Native Kafka broker using Docker mechanism

To run system tests by bringing up Kafka in native mode:
Pass kafka_mode as native in the ducktape globals:--globals '{\"kafka_mode\":\"native\"}'

Running system tests by bringing up kafka in native mode via docker mechanism
_DUCKTAPE_OPTIONS="--globals '{\"kafka_mode\":\"native\"}'" TC_PATHS="tests/kafkatest/tests/"  bash tests/docker/run_tests.sh

To only bring up ducker nodes to cater native kafka
bash tests/docker/ducker-ak up -m native

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2024-05-30 22:24:23 +05:30
Ken Huang 7fea279ff9
KAFKA-16763 Upgrade to scala 2.12.19 and scala 2.13.14 (#15958)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-18 00:36:38 +08:00
Nikolay b6183a4134
KAFKA-14589 ConsumerGroupCommand rewritten in java (#14471)
This PR contains changes to rewrite ConsumerGroupCommand in java and transfer it to tools module

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-03-20 15:34:45 +08:00
Mickael Maison 0bf830fc9c
KAFKA-14576: Move ConsoleConsumer to tools (#15274)
Reviewers: Josep Prat <josep.prat@aiven.io>, Omnia Ibrahim <o.g.h.ibrahim@gmail.com>
2024-02-13 19:24:07 +01:00
Hailey Ni 40e817eeff
KAFKA-15471 [MINOR]: Fix backward-compatibility bug (#14996)
Fixes a backwards-compatibility issue in the KIP-979 change to the kafka-server-stop.sh script where it would not stop processes if run from a directory that differed from the directory where the processes were started form and the config file specified at start time was a relative path.

Reviewers: Ron Dagostino <rdagostino@confluent.io>
2023-12-14 10:32:16 -05:00
Hailey Ni ad1d621ca4
KAFKA-15471: Allow independently stop KRaft controllers or brokers (#14400)
Reviewers: Ron Dagostino <rdagostino@confluent.io>
2023-12-12 03:20:43 -05:00
Andrew Schofield 46852eea1c
KAFKA-15871: kafka-client-metrics.sh (#14926)
Initial implementation of kafka-client-metrics.sh tools for KIP-714 and KIP-1000.

Reviewers: Igor Soarez <soarez@apple.com>, Jun Rao <junrao@gmail.com>
2023-12-06 10:10:10 -08:00
Yash Mayya 1bc4de75a2
KAFKA-15470: Allow creating connectors in a stopped state (#14704)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-11-15 11:37:50 +05:30
Nikolay 76b1b50b64
KAFKA-14595 Move ReassignPartitionsCommand to java (#13247)
This PR contains changes required to move PartitionReassignmentState class to java code.

Reviewers: Mickael Maison <mickael.maison@gmail.com>, Justine Olshan <jolshan@confluent.io>, Federico Valeri <fedevaleri@gmail.com>, Taras Ledkov Taras Ledkov <tledkov@apache.org>, Kamal Chandraprakash<kamal.chandraprakash@gmail.com>,
2023-10-31 17:29:05 -07:00
maniekes 987609404c
KAFKA-15685: Add support for MinGW and MSYS2 (windows OS) (#13321)
Kafka class runner does not work with MINGW/Git Bash on Windows. This commit adds support for MinGW and MSYS2 development environments.

Reviewers: Divij Vaidya <diviv@amazon.com>
2023-10-26 11:29:39 +02:00
Omnia G.H Ibrahim 9af1e74b5e
KAFKA-14596: Move TopicCommand to tools (#13201)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Federico Valeri <fedevaleri@gmail.com>
2023-10-17 11:40:15 +02:00
Omnia G.H Ibrahim 7553d3f562
KAFKA-14593: Move LeaderElectionCommand to tools (#13204)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Federico Valeri <fedevaleri@gmail.com>
2023-10-03 11:59:56 +02:00
Ismael Juma 7ba6d7a0b4
MINOR: Update to Scala 2.13.12 (#14430)
It offers a quickfix action for certain errors, includes a number of bug fixes and it
introduces a new warning by default (https://github.com/scala/scala/pull/10462).

In addition to the scala version bump, we also fix the new compiler warnings and
bump the scalafmt version (the previous version failed with the new scala version).

Release notes: https://github.com/scala/scala/releases/tag/v2.13.12

Reviewers: Divij Vaidya <diviv@amazon.com>, Satish Duggana <satishd@apache.org>
2023-09-24 06:05:12 -07:00
Ruslan Krivoshein b72d92919f
KAFKA-14581: Moving GetOffsetShell to tools (#13562)
This PR moves GetOffsetShell from core module to tools module with rewriting from Scala to Java.

Reviewers: Federico Valeri fedevaleri@gmail.com, Ziming Deng dengziming1993@gmail.com, Mickael Maison mimaison@apache.org.
2023-09-11 10:30:22 +08:00
Greg Harris f5655d31d3
KAFKA-15030: Add connect-plugin-path command-line tool (#14064)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-11 12:05:51 -07:00
Federico Valeri bb677c4959
KAFKA-14583: Move ReplicaVerificationTool to tools (#14059)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-07-26 12:04:34 +02:00
Nikolay 4bba2c8a32
KAFKA-14591: Move DeleteRecordsCommand to tools (#13278)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Federico Valeri <fedevaleri@gmail.com>
2023-07-21 17:30:28 +02:00
Ismael Juma dfaae317b8
MINOR: Upgrade Scala for Java 20/21 support (#13840)
Upgrade to Scala 2.13.11 and Scala 2.12.18. A minor test change was required
to fix compilation with Scala 2.13.11.

Scala 2.13 release notes:
* https://github.com/scala/scala/releases/tag/v2.13.11

Scala 2.12 release notes:
* https://github.com/scala/scala/releases/tag/v2.12.16
* https://github.com/scala/scala/releases/tag/v2.12.17
* https://github.com/scala/scala/releases/tag/v2.12.18

Reviewers: Justine Olshan <jolshan@confluent.io>, Josep Prat <josep.prat@aiven.io>
2023-06-20 10:29:23 -07:00
Christo Lolov dc7819d7f1
KAFKA-14594: Move LogDirsCommand to tools module (#13122)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-05-04 12:00:33 +02:00
Gantigmaa Selenge ea540fa400
KAFKA-14592: Move FeatureCommand to tools (#13459)
KAFKA-14592: Move FeatureCommand to tools

Reviewers: Luke Chen <showuon@gmail.com>
2023-04-25 20:28:37 +08:00
Federico Valeri 2c1cf03a89
MINOR: Add ClusterTool wrapper scripts and redirection (KIP-906) (#13560)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-04-17 16:09:56 +02:00
Federico Valeri ac38fd9db9
MINOR: Add EndToEndLatency wrapper scripts and redirection (KIP-906) (#13556)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-04-17 15:31:50 +02:00
Federico Valeri 57d6a881ca
MINOR: Add JmxTool wrapper scripts and redirection (KIP-906) (#13195)
Reviewers: Mickael Maison <mickael.maison@gmail.com>

, 
David Jacot <djacot@confluent.io>, Christo Lolov <christololov@gmail.com>, Alexandre Dupriez <alexandre.dupriez@gmail.com>
2023-04-17 14:59:35 +02:00
vamossagar12 c14f56b484
KAFKA-14586: Moving StreamResetter to tools (#13127)
Moves StreamResetter to tools project.

Reviewers: Federico Valeri <fedevaleri@gmail.com>, Christo Lolov <lolovc@amazon.com>, Bruno Cadonna <cadonna@apache.org>
2023-03-28 14:43:22 +02:00
Chia-Ping Tsai a375777d80
MINOR: correct the class name for delegation token script (#13411)
Reviewers: David Jacot <djacot@confluent.io>
2023-03-18 18:53:22 +08:00
José Armando García Sancio df86228aaf
MINOR; Fix command name kafka-metadata-quorum (#13381)
The name of the command should be kafka-metadata-quorum not
kafka-metatada-quorum.

Reviewers: Ron Dagostino <rdagostino@confluent.io>, Divij Vaidya <diviv@amazon.com>
2023-03-13 10:51:32 -07:00
Federico Valeri 07e2f6cd4d
KAFKA-14578: Move ConsumerPerformance to tools (#13215)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Alexandre Dupriez <alexandre.dupriez@gmail.com>
2023-03-06 18:16:55 +01:00
Mickael Maison 8b44237655
KAFKA-14575: Move ClusterTool to tools module (#13080)
Reviewers: dengziming <dengziming1993@gmail.com>, Federico Valeri  <fedevaleri@gmail.com>
2023-01-22 12:50:43 +01:00
Ismael Juma 88725669e7
MINOR: Move MetadataQuorumCommand from `core` to `tools` (#12951)
`core` should only be  used for legacy cli tools and tools that require
access to `core` classes instead of communicating via the kafka protocol
(typically by using the client classes).

Summary of changes:
1. Convert the command implementation and tests to Java and move it to
    the `tools` module.
2. Introduce mechanism to capture stdout and stderr from tests.
3. Change `kafka-metadata-quorum.sh` to point to the new command class.
4. Adjusted the test classpath of the `tools` module so that it supports tests
    that rely on the `@ClusterTests` annotation.
5. Improved error handling when an exception different from `TerseFailure` is
    thrown.
6. Changed `ToolsUtils` to avoid usage of arrays in favor of `List`.

Reviewers: dengziming <dengziming1993@gmail.com>
2022-12-09 09:22:58 -08:00
Matthew de Detrich 6ae08c4ee8
KAFKA-14256: Upgrade from Scala 2.13.8 to 2.13.10 (#12675)
In addition to the version bump, we also had to:
* Update the zinc version
* Workaround compiler warnings via suppression (proper fix in a follow up)
* Adjust `testDeleteTopicDoesNotRetryThrottlingQuotaExceededException` to fix a test failure

Release notes:
* https://github.com/scala/scala/releases/tag/v2.13.9
* https://github.com/scala/scala/releases/tag/v2.13.10

Reviewers: Ismael Juma <ismael@juma.me.uk>
2022-12-04 21:02:43 -08:00
FUNKYE 41d075ba14
KAFKA-14430: Specify JMX RMI port system property when not already set (#12898)
Reviewers: Andriy Redko <andriy.redko@aiven.io>, Chris Egerton <chrise@aiven.io>
2022-12-01 22:46:37 -05:00