Commit Graph

1139 Commits

Author SHA1 Message Date
Patrik Marton 0bbed823e8
KAFKA-17200: Allow the replication of user internal topics (#17815)
Reviewers: Viktor Somogyi-Vass <viktorsomogyi@gmail.com>
2024-12-06 15:23:58 +01:00
Andrew Schofield e7d986e48c
KAFKA-17550: DescribeGroups v6 exploitation (#17706)
This PR introduces the DescribeGroups v6 API as part of KIP-1043. This adds an error message for the described groups so that it is possible to get some context on the error. It also changes the behaviour for when the group ID cannot be found but returning error code GROUP_ID_NOT_FOUND rather than NONE.

Reviewers: David Jacot <djacot@confluent.io>
2024-12-05 23:12:24 -08:00
Nick Guo 970d8930ac
KAFKA-18159 Remove onPartitionsRevoked and onPartitionsAssigned from SinkTask (#18049)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-06 04:28:34 +08:00
Ken Huang 2b43c49f51
KAFKA-18050 Upgrade the checkstyle version to 10.20.2 (#17999)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-05 10:59:18 +08:00
TengYao Chi 6fd951a9c0
KAFKA-17610 Drop alterConfigs (#18002)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-02 23:26:06 +08:00
TengYao Chi 056a76e2b9
KAFKA-17811 Separate modules to use different JDKs (#17522)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-26 23:30:41 +08:00
Manikumar Reddy 3268435fd6
KAFKA-18013: Add AutoOffsetResetStrategy internal class (#17858)
- Deprecates OffsetResetStrategy enum
- Adds new internal class AutoOffsetResetStrategy
- Replaces all OffsetResetStrategy enum usages with AutoOffsetResetStrategy
- Deprecate old/Add new constructors to MockConsumer

 Reviewers: Andrew Schofield <aschofield@confluent.io>, Matthias J. Sax <matthias@confluent.io>
2024-11-25 19:11:12 +05:30
Logan Zhu 337e2e1506
KAFKA-18037 Remove deprecated methods from EmbeddedConnect and EmbeddedConnectCluster (#17857)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-23 23:16:41 +08:00
Yash Mayya 4f1688742e
KAFKA-15387: Remove Connect's deprecated task configurations endpoint (#17412)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-11-21 19:43:54 +05:30
Joao Pedro Fonseca Dantas e9ccc2d6f5
KAFKA-16041: Replace Afterburn module with Blackbird (#17884)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-11-21 14:52:45 +01:00
Andrew Schofield 32c887b05e
KAFKA-17949: Introduce GroupState and replace ShareGroupState (#17763)
This PR introduces the unified GroupState enum for all group types from KIP-1043. This PR also removes ShareGroupState and begins the work to replace Admin.listShareGroups with Admin.listGroups. That will complete in a future PR.

Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
2024-11-19 21:17:12 +05:30
Mickael Maison 624cd4f7d0
MINOR: Various cleanups in connect:runtime tests (#17827)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-19 15:55:54 +08:00
Rajini Sivaram 52d2fa5c8b
KAFKA-17885: Enable clients to rebootstrap based on timeout or error code (KIP-1102) (#17720)
Implementation of https://cwiki.apache.org/confluence/display/KAFKA/KIP-1102%3A+Enable+clients+to+rebootstrap+based+on+timeout+or+error+code
- Introduces rebootstrap trigger interval config metadata.recovery.rebootstrap.trigger.ms, set to 5 minutes by default
- Makes rebootstrap the default for metadata.recovery.strategy
- Adds new error code REBOOTSTRAP_REQUIRED, introduces top-level error code in metadata response. On this error, clients rebootstrap.
- Configs apply to producers, consumers, share consumers, admin clients, Connect and KStreams clients.

Reviewers: Apoorv Mittal <apoorvmittal10@gmail.com>,  Manikumar Reddy <manikumar.reddy@gmail.com>
2024-11-13 13:01:08 +00:00
Linsiyuan9 af53758746
KAFKA-17814 Use `final` declaration to replace the suppression `this-escape` (#17613)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-11-03 15:00:02 +08:00
David Arthur 4fadec527d
MINOR: Quarantine the worst flaky tests (#17639)
Using the last 7 days of data on Oct 30 2024, this patch marks all flaky tests with more than 10% flakiness on trunk.

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-30 17:02:04 -04:00
Ken Huang 09d76f917c
KAFKA-16564 Apply `Xlint` to java code in core module (#16965)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-26 01:07:56 +08:00
Joao Pedro Fonseca Dantas 3856644cd6
KAFKA-17233: MirrorCheckpointConnector should use batched listConsumerGroupOffsets (#17038)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Greg Harris <gharris1727@gmail.com>
2024-10-24 09:38:44 +02:00
Alieh Saeedi 14a098b289
KAFKA-17600: Add nextOffsets to the ConsumerRecords (#17414)
This PR implements KIP-1094.

Reviewers: Andrew Schofield <aschofield@confluent.io>, Kirk True <ktrue@confluent.io>, Lucas Brutschy <lbrutschy@confluent.io>
2024-10-23 16:25:50 +02:00
Dmitry Werner eb897c6ad5
MINOR: Fix potential NPE (#17541)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-19 01:05:45 +08:00
Kuan-Po Tseng f7498a40f2
MINOR: fix broken link in javadoc (#17526)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-18 00:50:27 +08:00
Anshul Goyal 2353a7c508
KAFKA-17723 Fix "this-escape" compiler warnings (MultiThreadedEventProcessor and DistributedHerder) for JDK 23 (#17417)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-11 21:28:27 +08:00
Gaurav Narula b03fe66cfe
KAFKA-17759 Remove Utils.mkSet (#17460)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-11 21:20:43 +08:00
Dániel Urbán 6d650ff9e5
KAFKA-17534: Add configuration to disable the heartbeats topic replication (#17413)
Introducing heartbeats.replication.enabled to explicitly disable the default heartbeats topic replication.
This change implements KIP-1089.

Reviewers: Viktor Somogyi-Vass <viktorsomogyi@gmail.com>
2024-10-10 14:21:36 +02:00
Mickael Maison 3815339e05
KAFKA-14158: Remove auto.include.jmx.reporter configuration (#17360)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-08 16:17:07 +02:00
Mickael Maison 5624bc7c7e
MINOR: Remove hamcrest from connect:runtime, raft and server-common (#17394)
Reviewers: David Arthur <mumrah@gmail.com>
2024-10-08 14:22:45 +02:00
Chia-Chuan Yu 9e809be4ce
KAFKA-17529 Remove blacklist from MM2 (#17202)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-07 13:21:31 +08:00
David Arthur d38a90df2b
KAFKA-17672 Run quarantined tests separately (#17329)
Introduce new quarantinedTest that excludes tests tagged with "flaky". Also introduce two new build parameters "maxQuarantineTestRetries" and "maxQuarantineTestRetryFailures".

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-06 14:09:24 +08:00
TaiJuWu 3bb408c4de
MINOR: rename ConfigName.RENAME to ConfigName.RENAMES for replaceFiled (#17369)
Reviewers: Andrew Schofield <aschofield@confluent.io>, TengYao Chi <kitingiao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-10-05 05:50:48 +08:00
Chia-Ping Tsai 979740b49d
KAFKA-17589 Move JUnit extensions to test-common module (#17318)
This patch completely removes the compile-time dependency on core for both test and main sources by introducing two new modules.

1) `test-common` include all the common test implementation code (including dependency on :core for BrokerServer, ControllerServer, etc)
2) `test-common:api` new sub-module that just includes interfaces including our junit extension

Reviewers: David Arthur <mumrah@gmail.com>
2024-10-03 10:28:37 +08:00
Mickael Maison 7fb25a2b06
KAFKA-16769 Remove add.source.alias.to.metrics configuration (#17323)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-02 02:03:02 +08:00
Greg Harris 818ee8a581
KAFKA-17078: Add SecurityManagerCompatibility shim (#16522)
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: José Armando García Sancio <jsancio@apache.org>, Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Chris Egerton <fearthecellos@gmail.com>, Luke Chen <showuon@gmail.com>, David Arthur <mumrah@gmail.com>
2024-09-30 08:06:14 -07:00
TaiJuWu db56553fd3
KAFKA-17530 Remove blacklist/whitelist from ReplaceField (#17237)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-09-25 01:02:28 +08:00
David Jacot 70f7418660
MINOR: Increase logging verbosity in Connect integration tests (#17189)
This patch increases the verbosity of the logging in Connect's integration tests. This is to better understand the causes of the flaky tests described in KAFKA-17493.

Reviewers: Chris Egerton <chrise@aiven.io>
2024-09-14 00:04:46 -07:00
Xuan-Zhang Gong 9242723e4c
KAFKA-17435 remove use.incremental.alter.configs (#17027)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-09-11 14:34:05 +08:00
Chris Egerton 50e7022a1b
MINOR: Improve error message when Connect's EmbeddedKafkaCluster::verifyClusterReadiness method fails (#16918)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-09-07 10:23:59 -04:00
xijiu a39037e55c
KAFKA-17399 Apply LambdaValidator to code base (#16980)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-08-26 21:53:49 +08:00
Greg Harris b40b5a24f4
KAFKA-17369: Remove Reflections from logging and update licenses (#16924)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-08-19 16:27:36 -07:00
Steven Xu e24354a21d
KAFKA-15203: Use Classgraph since org.reflections is no longer under maintenance (#16604)
Reviewers: Liam Miller-Cushon <cushon@google.com>, Greg Harris <greg.harris@aiven.io>
2024-08-19 10:46:24 -07:00
Dmitry Werner 4763d0fc30
KAFKA-17242: Do not log spurious timeout message for MirrorCheckpointTask sync store startup (#16773)
Reviewers: Chris Egerton <chrise@aiven.io>
2024-08-08 10:03:42 -04:00
TengYao Chi 0dc74c5556
KAFKA-17232: Do not generate task configs in MirrorCheckpointConnector if initial consumer group load times out (#16767)
Reviewers: Hongten <hongtenzone@foxmail.com>, Chris Egerton <chrise@aiven.io>
2024-08-08 09:58:11 -04:00
Chris Egerton 8152ee6519
MINOR: Increase test producer max.block.ms in MirrorConnectorsIntegrationTransactionsTest (#16785)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-05 14:37:17 -04:00
Chris Egerton 3f3e376797
MINOR: Fix stage recording in DistributedHerder::addRequest (#16784)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-05 14:36:39 -04:00
Chris Egerton 39b1586f93
MINOR: Fix ConnectWorkerIntegrationTest::testRequestTimeouts (#16772)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-02 16:41:34 -04:00
Chris Egerton 5afdb17092
MINOR: Fix consumer group warmup in MirroMaker 2 integration tests (#16771)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-02 13:57:05 -04:00
Chris Egerton a524609f4c
MINOR: Add extended cluster readiness check for Connect's embedded Kafka cluster (#16757)
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-01 16:12:57 -04:00
Kondrat Bertalan 8d8c367066
KAFKA-17192 Fix MirrorMaker2 worker config does not pass config.provi… (#16678)
Reviewers: Chris Egerton <chrise@aiven.io>
2024-07-31 16:50:22 -04:00
Chris Egerton 342bd2844f
MINOR: Unify Log4j levels across Connect runtime and MirrorMaker 2 integration tests (#16733)
Reviewers: Yash Mayya <yash.mayya@gmail.com>
2024-07-31 12:07:38 +02:00
Chung, Ming-Yen 7c0a96d08d
KAFKA-17185 Declare Loggers as static to prevent multiple logger instances (#16680)
As discussed in #16657 (comment) , we should make logger as static to avoid creating multiple logger instances.
I use the regex private.*Logger.*LoggerFactory to search and check all the results if certain logs need to be static.

There are some exceptions that loggers don't need to be static:
1) The logger in the inner class. Since java8 doesn't support static field in the inner class.
        https://github.com/apache/kafka/blob/trunk/clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java#L3676

2) Custom loggers for each instance (non-static + non-final). In this case, multiple logger instances is actually really needed.
        https://github.com/apache/kafka/blob/trunk/storage/src/test/java/org/apache/kafka/server/log/remote/storage/LocalTieredStorage.java#L166

3) The logger is initialized in constructor by LogContext. Many non-static but with final modifier loggers are in this category, that's why I use .*LoggerFactory to only check the loggers that are assigned initial value when declaration.
    
4) protected final Logger log = Logger.getLogger(getClass())
    This is for subclass can do logging with subclass name instead of superclass name.
    But in this case, if the log access modifier is private, the purpose cannot be achieved since subclass cannot access the log defined in superclass. So if access modifier is private, we can replace getClass() with <className>.class

Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-31 02:37:36 +08:00
Chris Egerton 61f61d6240
KAFKA-14569: Migrate Connect's integration test EmbeddedKafkaCluster from ZK to KRaft mode (#16599)
Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-07-29 10:43:55 -04:00
PaulRMellor 738d8cc91e
MINOR: Update bootstrap.servers doc string (#16655)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-07-26 15:08:01 +02:00