b40b5a24f4
KAFKA-17369: Remove Reflections from logging and update licenses ( #16924 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-08-19 16:27:36 -07:00
e24354a21d
KAFKA-15203: Use Classgraph since org.reflections is no longer under maintenance ( #16604 )
...
Reviewers: Liam Miller-Cushon <cushon@google.com>, Greg Harris <greg.harris@aiven.io>
2024-08-19 10:46:24 -07:00
4763d0fc30
KAFKA-17242: Do not log spurious timeout message for MirrorCheckpointTask sync store startup ( #16773 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-08-08 10:03:42 -04:00
0dc74c5556
KAFKA-17232: Do not generate task configs in MirrorCheckpointConnector if initial consumer group load times out ( #16767 )
...
Reviewers: Hongten <hongtenzone@foxmail.com>, Chris Egerton <chrise@aiven.io>
2024-08-08 09:58:11 -04:00
8152ee6519
MINOR: Increase test producer max.block.ms in MirrorConnectorsIntegrationTransactionsTest ( #16785 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-05 14:37:17 -04:00
3f3e376797
MINOR: Fix stage recording in DistributedHerder::addRequest ( #16784 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-05 14:36:39 -04:00
39b1586f93
MINOR: Fix ConnectWorkerIntegrationTest::testRequestTimeouts ( #16772 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-02 16:41:34 -04:00
5afdb17092
MINOR: Fix consumer group warmup in MirroMaker 2 integration tests ( #16771 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-02 13:57:05 -04:00
a524609f4c
MINOR: Add extended cluster readiness check for Connect's embedded Kafka cluster ( #16757 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-08-01 16:12:57 -04:00
8d8c367066
KAFKA-17192 Fix MirrorMaker2 worker config does not pass config.provi… ( #16678 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-07-31 16:50:22 -04:00
342bd2844f
MINOR: Unify Log4j levels across Connect runtime and MirrorMaker 2 integration tests ( #16733 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>
2024-07-31 12:07:38 +02:00
7c0a96d08d
KAFKA-17185 Declare Loggers as static to prevent multiple logger instances ( #16680 )
...
As discussed in #16657 (comment) , we should make logger as static to avoid creating multiple logger instances.
I use the regex private.*Logger.*LoggerFactory to search and check all the results if certain logs need to be static.
There are some exceptions that loggers don't need to be static:
1) The logger in the inner class. Since java8 doesn't support static field in the inner class.
https://github.com/apache/kafka/blob/trunk/clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetchRequestManagerTest.java#L3676
2) Custom loggers for each instance (non-static + non-final). In this case, multiple logger instances is actually really needed.
https://github.com/apache/kafka/blob/trunk/storage/src/test/java/org/apache/kafka/server/log/remote/storage/LocalTieredStorage.java#L166
3) The logger is initialized in constructor by LogContext. Many non-static but with final modifier loggers are in this category, that's why I use .*LoggerFactory to only check the loggers that are assigned initial value when declaration.
4) protected final Logger log = Logger.getLogger(getClass())
This is for subclass can do logging with subclass name instead of superclass name.
But in this case, if the log access modifier is private, the purpose cannot be achieved since subclass cannot access the log defined in superclass. So if access modifier is private, we can replace getClass() with <className>.class
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-31 02:37:36 +08:00
61f61d6240
KAFKA-14569: Migrate Connect's integration test EmbeddedKafkaCluster from ZK to KRaft mode ( #16599 )
...
Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-07-29 10:43:55 -04:00
738d8cc91e
MINOR: Update bootstrap.servers doc string ( #16655 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-07-26 15:08:01 +02:00
956d740c46
KAFKA-16068: Use TestPlugins mechanism in ConnectorValidationIntegrationTest to prevent ERROR-level log spam in unrelated test suites ( #16647 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-07-23 13:15:03 -04:00
defcbb51ee
KAFKA-17082 replace kafka.utils.LogCaptureAppender with org.apache.kafka.common.utils.LogCaptureAppender ( #16601 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-21 18:22:05 +08:00
208bb1fbc2
MINOR: Add integration test for plugin aliases ( #16621 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-19 12:47:41 -04:00
c97421c100
KAFKA-17150: Use Utils.loadClass instead of Class.forName to resolve aliases correctly ( #16608 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>, Chia-Ping Tsai <chia7712@gmail.com>, Josep Prat <josep.prat@aiven.io>
2024-07-17 16:00:45 -07:00
53ec055394
KAFKA-17105: Prevent redundant restarts for newly-created connectors ( #16581 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-07-17 14:33:47 -04:00
177b38ad66
KAFKA-16383: Ensure tasks have already polled their consumers before producing verified records in MirrorConnectorsIntegrationBaseTest::testReplicateFromLatest ( #16598 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-07-16 19:42:31 -04:00
6cffed56f6
KAFKA-16943: Synchronously verify Connect worker startup failure in InternalTopicsIntegrationTest ( #16451 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-07-16 14:20:44 -04:00
808498e939
KAFKA-14401: Fail kafka log read end requests if underneath work thread fails ( #14372 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-07-15 20:55:52 -04:00
6e9c039eea
KAFKA-17097 Add replace.null.with.default configuration to ValueToKey and ReplaceField ( #16571 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-15 18:12:59 +08:00
83081652ab
MINOR: Fix ConnectWorkerIntegrationTest::testBrokerCoordinator ( #16585 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-07-13 03:48:19 -04:00
3f43d27483
KAFKA-17111: explicitly register Afterburner module in JsonSerializer and JsonDeserializer ( #16565 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Josep Prat <josep.prat@aiven.io>, Greg Harris <greg.harris@aiven.io>
2024-07-11 14:17:27 +02:00
0781151794
KAFKA-16254: Allow MM2 to fully disable offset sync feature ( #15999 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chris Egerton <fearthecellos@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>, Igor Soarez <soarez@apple.com>
2024-07-09 12:38:03 +02:00
515cdbb707
MINOR: Fix retry logic in DedicatedMirrorIntegrationTest::awaitTaskConfigurations ( #16525 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-07-08 14:51:23 -04:00
e2ec389a7c
KAFKA-15838: KIP-1040: Add replace.null.with.default to InsertField, ExtractField, HeaderFrom, Cast, SetSchemaMetadata, TimestampConverter, MaskField transforms ( #15756 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-07-08 11:09:54 -07:00
27220d146c
KAFKA-10816: Add health check endpoint for Kafka Connect ( #16477 )
...
Reviewers: Greg Harris <gharris1727@gmail.com>
2024-07-03 14:15:15 -04:00
4550550c7c
MINOR: Cleanup TestPlugins and normalize TestPlugin enum ( #13333 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-07-03 09:44:09 -07:00
e93242283d
KAFKA-15917: Wait for zombie sink tasks' consumers to commit offsets before trying to modify their offsets in integration tests ( #15302 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-07-03 10:25:23 -04:00
6897b06b03
KAFKA-3346 Rename Mode to ConnectionMode ( #16403 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-07-03 02:46:04 +08:00
645eefe95a
KAFKA-17007 Fix SourceAndTarget#equal ( #16416 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-21 20:52:07 +08:00
0772144e51
MINOR: Add javadoc for Connect public packages/classes ( #16404 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-21 10:23:35 +02:00
8d92535382
KAFKA-10787 apply spotless to connect:runtime module ( #16412 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-21 16:04:51 +08:00
4de6a90801
KAFKA-16921 [6/N] Remove junit 4 dependency from connect:runtime module ( #16383 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-19 14:39:16 +08:00
d5592d8fe6
KAFKA-16921 [5/N] Migrate test of connect module to Junit5 (Runtime subpackage) ( #16350 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-18 18:20:14 +08:00
3a3f9ce48e
KAFKA-16921 [4/N] Migrate test of connect module to Junit5 (Runtime direct) ( #16351 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-18 18:09:40 +08:00
fc6f8b6591
KAFKA-16921 [3/N] migrate connect module to junit 5 ( #16330 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-15 00:37:59 +08:00
8682334b6a
KAFKA-16921 [2/N] migrate connect module to junit 5 ( #16328 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-15 00:25:41 +08:00
133f2b0f31
KAFKA-16879 SystemTime should use singleton mode ( #16266 )
...
Reviewers: Greg Harris <gharris1727@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-06-14 08:49:19 +08:00
6af937639d
KAFKA-16076 Fix Missing thread interrupt call in RestClient class ( #16282 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-14 08:07:13 +08:00
ea60666513
KAFKA-16921 [1/N] Migrate all junit 4 code to junit 5 for connect module ( #16253 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-13 16:11:37 +08:00
596b945072
KAFKA-16643 Add ModifierOrder checkstyle rule ( #15890 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-13 15:39:32 +08:00
9ddd58bd6c
MINOR: Add readiness check for connector and separate Kafka cluster in ExactlyOnceSourceIntegrationTest::testSeparateOffsetsTopic ( #16306 )
...
Reviewers: Greg Harris <gharris1727@gmail.com>
2024-06-12 23:43:33 -04:00
e76e1da5ea
KAFKA-16935: Automatically wait for cluster startup in embedded Connect integration tests ( #16288 )
...
Reviewers: Greg Harris <gharris1727@gmail.com>
2024-06-12 20:18:23 -04:00
dd755b7ea9
KAFKA-8206: KIP-899: Allow client to rebootstrap ( #13277 )
...
This commit implements KIP-899: Allow producer and consumer clients to rebootstrap. It introduces the new setting `metadata.recovery.strategy`, applicable to all the types of clients.
Reviewers: Greg Harris <gharris1727@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com>
2024-06-12 20:48:32 +01:00
79b9c44acd
KAFKA-10787 Apply spotless to connect module ( #16299 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-12 22:19:47 +08:00
2fa2c72581
MINOR: Wait for embedded clusters to start before using them in Connect OffsetsApiIntegrationTest ( #16286 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-06-11 17:15:07 -04:00
ac2a642ba9
MINOR: Fix flaky test ConnectWorkerIntegrationTest::testReconfigureConnectorWithFailingTaskConfigs ( #16273 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-06-11 15:13:35 -04:00
eec8fd6a98
KAFKA-9228: Restart tasks on runtime-only connector config changes ( #16053 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-06-10 17:02:08 -04:00
271a1b42cd
KAFKA-16223 Migrate KafkaConfigBackingStoreTest to Mockito ( #16126 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-07 07:02:14 +08:00
52514a89e3
KAFKA-16858: Throw DataException from validateValue on array and map schemas without inner schemas ( #16161 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-06-05 11:35:32 -07:00
0409003c43
KAFKA-16837, KAFKA-16838: Ignore task configs for deleted connectors, and compare raw task configs before publishing them ( #16122 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-06-04 09:36:24 -04:00
5cbc1d616a
KAFKA-16881: InitialState type leaks into the Connect REST API OpenAPI spec ( #16175 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-06-03 13:34:58 -04:00
9eb05fc729
KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest ( #16164 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-02 20:32:54 +08:00
966f2eb3ef
Minor: Add URL to log for Connect RestClient ( #16166 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-06-01 16:25:16 -04:00
e974914ca5
MINOR: Code Cleanup - Connect Module ( #16066 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-31 04:55:00 +08:00
32b2b73f67
KAFKA-16844: Add ByteBuffer support for Connect ByteArrayConverter ( #16101 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-05-30 11:26:25 -04:00
3f3f3ac155
MINOR: Delete KafkaSecurityConfigs class ( #16113 )
...
Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-05-30 05:55:24 +08:00
64f699aeea
KAFKA-15853: Move general configs out of KafkaConfig ( #16040 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-05-28 16:22:54 +02:00
11ad5e8bca
MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions ( #15469 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-05-23 13:23:18 -07:00
0ba15ad4d1
KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently i… ( #15910 )
...
* KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently interrupt offset translation
MirrorCheckpointTask reloads the last checkpoint at start,
OffsetSyncStore stores OffsetSyncs before reading till end.
If CheckpointTask cannot read checkpoints at startup,
use previous OffsetSyncStore load logic, with
warning log message about degraded offset translation.
Also addresses KAFKA-16622 Mirromaker2 first Checkpoint not emitted until
consumer group fully catches up once because the OffsetSyncStore store
is populated before reading to log end.
Co-Authored-By: Adrian Preston <prestona@uk.ibm.com>
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-05-23 17:17:56 +01:00
476d323f5a
KAFKA-16197: Print Connect worker specific logs on poll timeout expiry ( #15305 )
...
Reviewers: Luke Chen <showuon@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-05-20 17:00:04 -07:00
6aac009a2d
MINOR: Remove deprecated constructors from Connect's Kafka*BackingStore classes ( #15865 )
...
- These constructors were deprecated over 3 years ago in KAFKA-10021: Changed Kafka backing stores to use shared admin client to get end offsets and create topics #9780 .
- While these classes are not a part of Connect's public API, deprecation was still introduced instead of outright removal because they are useful utility classes that might've been used outside of Connect.
- The KafkaOffsetBackingStore's deprecated constructor was removed in KAFKA-14785: Connect offset read REST API #13434 .
- This patch removes the deprecated constructors for KafkaConfigBackingStore and KafkaStatusBackingStore.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-18 04:09:30 +08:00
93a5efc4b7
KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest ( #15933 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-18 04:01:27 +08:00
7453549314
KAFKA-16671 enable test for ensureInternalEndpointIsSecured ( #15868 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-15 12:49:36 +08:00
e1771448cf
MINOR: Add coverage for MM2 heartbeats topic detection ( #15932 )
...
Reviewers: Greg Harris <gharris1727@gmail.com>
2024-05-14 10:11:34 +02:00
0cfc2983a5
KAFKA-16684: Fix flaky DedicatedMirrorIntegrationTest ( #15906 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-05-10 14:42:26 -04:00
5a9ccb6b77
KAFKA-16445: Add PATCH method for connector config ( #6934 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-05-09 21:59:09 +02:00
5f933ac840
MINOR: Correct connector scheduled rebalance logs ( #15875 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-08 12:32:47 +08:00
525b9b1d76
KAFKA-15018: Write connector tombstone offsets to secondary store before primary store ( #13801 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-05-07 14:29:02 -04:00
a4c6cefd10
KAFKA-14226: Introduce FieldPath abstraction and nested path support for ExtractField SMT ( #15379 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-05-07 14:07:18 -04:00
05df10449e
KAFKA-13328, KAFKA-13329 (2): Add custom preflight validation support for connector header, key, and value converters ( #14309 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-05-07 11:30:57 -04:00
55a00be4e9
MINOR: Replaced Utils.join() with JDK API. ( #15823 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-06 15:13:01 +08:00
41f5bf844d
KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (2/3) ( #15841 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-06 08:47:17 +08:00
b8b2415d5e
KAFKA-16592 Add a new constructor which invokes the existing constructor with default value for alternativeString ( #15762 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-24 20:58:29 +08:00
1b301b3020
KAFKA-15853 Move socket configs into org.apache.kafka.network.SocketServerConfigs ( #15772 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-23 17:39:36 +08:00
ecb2dd4cdc
KAFKA-15853 Move KafkaConfig log properties and docs out of core ( #15569 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Nikolay <nizhikov@apache.org>, Federico Valeri <fvaleri@redhat.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-04-20 04:14:23 +08:00
53ff1a5a58
KAFKA-15585: DescribeTopicPartitions client side change. ( #15470 )
...
Add the support for DescribeTopicPartitions API to AdminClient. For this initial implementation, we are simply loading all of the results into memory on the client side.
Reviewers: Andrew Schofield <aschofield@confluent.io>, Kirk True <ktrue@confluent.io>, David Jacot <djacot@confluent.io>, Artem Livshits <alivshits@confluent.io>, David Arthur <mumrah@gmail.com>
2024-04-18 12:09:14 -04:00
aee9724ee1
MINOR: Remove unneeded explicit type arguments ( #15736 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-17 21:55:58 +02:00
7b1c777b33
MINOR: Various cleanups in connect ( #15734 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-17 20:47:25 +08:00
363f4d2847
KAFKA-15853 Move consumer group and group coordinator configs out of core ( #15684 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-17 20:41:22 +08:00
a1ca788c99
KAFKA-15517: Improve MirrorMaker logging in case of authorization errors ( #15558 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-04-16 12:09:14 +02:00
8c0458861c
KAFKA-15853 Move KafkaConfig Replication properties and docs out of … ( #15575 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-04-16 15:28:35 +08:00
2118d85805
KAFKA-16223 Replace EasyMock/PowerMock with Mockito for KafkaConfigBackingStoreTest (1/3) ( #15520 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-03-29 15:39:36 -07:00
71bcac3b6a
MINOR: AbstractConfig cleanup ( #15597 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>, Mickael Maison <mickael.maison@gmail.com>, Omnia G H Ibrahim <o.g.h.ibrahim@gmail.com>, Matthias J. Sax <matthias@confluent.io>
2024-03-28 13:27:41 -07:00
355873aa54
MINOR: Use CONFIG suffix in ZkConfigs ( #15614 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>, Omnia Ibrahim <o.g.h.ibrahim@gmail.com>
Co-authored-by: n.izhikov <n.izhikov@vk.team>
2024-03-28 15:52:34 +01:00
6f38fe5e0a
KAFKA-14588 ZK configuration moved to ZkConfig ( #15075 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-03-27 22:37:01 +08:00
4099774da9
KAFKA-16084: Simplify and deduplicate standalone herder test mocking ( #15389 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-03-26 14:24:37 -07:00
f1d741a9c1
KAFKA-16392: Stop emitting warning log message when parsing source connector offsets with null partitions ( #15562 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>
2024-03-20 15:54:22 +00:00
12a1d85362
KAFKA-12187 replace assertTrue(obj instanceof X) with assertInstanceOf ( #15512 )
...
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-03-20 10:36:25 +08:00
7b9f31e35c
KAFKA-16358: Sort transformations by name in documentation; add missing transformations to documentation; add hyperlinks ( #15499 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>
2024-03-15 16:26:36 +05:30
178761eb36
KAFKA-14683 Cleanup WorkerSinkTaskTest ( #15506 )
...
1) Rename WorkerSinkTaskMockitoTest back to WorkerSinkTaskTest
2) Tidy up the code a bit
3) rewrite "fail" by "assertThrow"
Reviewers: Omnia Ibrahim <o.g.h.ibrahim@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-03-15 03:50:57 +08:00
aa7bef414e
MINOR: Resolve SSLContextFactory.getNeedClientAuth deprecation ( #15468 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-03-13 11:45:38 -07:00
62998b7264
KAFKA-14683: Migrate WorkerSinkTaskTest to Mockito (3/3) ( #15316 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-03-06 10:31:33 -08:00
eea369af94
KAFKA-14588 Log cleaner configuration move to CleanerConfig ( #15387 )
...
In order to move ConfigCommand to tools we must move all it's dependencies which includes KafkaConfig and other core classes to java. This PR moves log cleaner configuration to CleanerConfig class of storage module.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-03-05 18:11:56 +08:00
99e511c706
KAFKA-16288, KAFKA-16289: Fix Values convertToDecimal exception and parseString corruption ( #15399 )
...
* KAFKA-16288: Prevent ClassCastExceptions for strings in Values.convertToDecimal
* KAFKA-16289: Values inferred schemas for map and arrays should ignore element order
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-03-04 15:59:47 -08:00
553f45bca8
MINOR: Fix toString method of IsolationLevel ( #14782 )
...
Reviewers: Matthias J. Sax <matthias@confluent.io>, Ashwin Pankaj <apankaj@confluent.io>
2024-02-15 19:07:18 -08:00
794c52802c
KAFKA-14683: Migrate WorkerSinkTaskTest to Mockito (2/3) ( #15313 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-02-12 11:15:27 -08:00
d233eb98f7
KAFKA-14957: Update-Description-String ( #13909 )
...
HTML code for configs is auto-generated and for Kafka Streams config `state.dir` produces a confusing default value.
This PR adds a new property `alternativeString` to set a "default" value which should be rendered in HTML instead of the actual default value.
Reviewers: Manyanda Chitimbo <manyanda.chitimbo@gmail.com>, @eziosudo <eziosudo@gmail.com>, Matthias J. Sax <matthias@confluent.io>
2024-02-10 12:46:51 -08:00
24a79c2613
KAFKA-14683: Migrate WorkerSinkTaskTest to Mockito (1/3) ( #14663 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2024-02-05 10:55:57 -08:00
4f0a405908
KAFKA-15575: Begin enforcing 'tasks.max' property for connectors ( #15180 )
...
Reviewers: Ashwin Pankaj <apankaj@confluent.io>, Greg Harris <greg.harris@aiven.io>
2024-02-01 11:33:04 -05:00
7bc4afee11
KAFKA-15675: Improve worker liveness check during Connect integration tests ( #15249 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>, Yash Mayya <yash.mayya@gmail.com>
2024-02-01 09:20:24 -05:00
4e6920cc5f
KAFKA-16196: Handle invalid whole value casts in the Cast transform gracefully ( #15267 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2024-01-26 19:04:39 +05:30
d1d6b5096f
KAFKA-16166: Generify RetryWithToleranceOperator, ErrorReporter, and WorkerTask ( #15233 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-01-22 16:56:52 -08:00
93971465b6
KAFKA-15827: Prevent KafkaBasedLog subclasses from leaking passed-in clients ( #14763 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-01-19 12:50:01 -08:00
e45e032b8d
KAFKA-16087: Avoid data race which drops wrong record when errors.tolerance=all ( #15154 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-01-18 13:26:26 -08:00
72b70288eb
KAFKA-5863: Avoid NPE when RestClient calls expecting no-content receive content. ( #13294 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Hector Geraldino <hgeraldino@gmail.com>, Yash Mayya <yash.mayya@gmail.com>
2024-01-18 11:41:27 -08:00
a989329ee3
MINOR: Stop leaking threads in BlockingConnectorTest ( #12290 )
...
Reviewers: Kvicii <kvicii.yu@gmail.com>, Viktor Somogyi-Vass <viktorsomogyi@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-01-18 11:11:32 -05:00
988aea3f58
KAFKA-16051: Fixed deadlock in StandaloneHerder ( #15080 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>, Sagar Rao <sagarmeansocean@gmail.com>
2024-01-12 15:46:17 -08:00
65424ab484
MINOR: New year code cleanup - include final keyword ( #15072 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Sagar Rao <sagarmeansocean@gmail.com>
2024-01-11 17:53:35 +01:00
dbf00bcf45
KAFKA-16093: Fix spurious REST-related warnings on Connect startup ( #15149 )
...
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-01-10 09:03:23 -05:00
c7e1fdca64
KAFKA-15988: Reuse embedded clusters across test cases in Connect OffsetsApiIntegrationTest suite ( #14966 )
...
Reviewers: Sudesh Wasnik <swasnik@confluent.io>, Sagar Rao <sagarmeansocean@gmail.com>, Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2024-01-09 10:32:39 -05:00
eccfb03e9a
MINOR: Small cleanups in Connect ( #15128 )
...
Reviewers: Divij Vaidya <diviv@amazon.com>
2024-01-05 11:00:10 +01:00
cd1eb634c5
KAFKA-15816: Fix leaked sockets in runtime tests ( #14764 )
...
This commit fixes to prevent socket leaks, mostly through clients that are left open.
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-01-01 11:38:11 +01:00
314de9f23c
KAFKA-15996: Improve JsonConverter performance ( #14992 )
...
Improve JsonConverter performance by using afterBurnModule of Jackson library.
Reviewers: Divij Vaidya <diviv@amazon.com>, Mickael Maison <mickael.maison@gmail.com>
2023-12-24 21:47:12 +01:00
05014badf2
KAFKA-15208: Upgrade Jackson dependencies to version 2.16.0 ( #13662 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Divij Vaidya <diviv@amazon.com>
2023-12-19 15:31:24 +01:00
3f7eadaa00
KAFKA-15372: Reconfigure dedicated MM2 connectors after leadership change ( #14293 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2023-12-12 13:57:56 -08:00
75ad884161
KAFKA-15906: Emit latest MM2 offset syncs every offset.flush.interval.ms ( #14967 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2023-12-11 16:39:54 -08:00
2a5fbf2882
KAFKA-15563: Provide informative error messages when Connect REST requests time out ( #14562 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-12-11 16:48:16 -05:00
dc857fb6bf
KAFKA-13328, KAFKA-13329 (1): Add preflight validations for key, value, and header converter classes ( #14304 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-12-08 15:00:44 -05:00
964e73178b
KAFKA-14132; Replace EasyMock with Mockito in StandaloneHerderTest ( #12728 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-12-07 10:01:17 -05:00
a2d596c428
KAFKA-15816: Fix leaked sockets in mirror tests ( #14761 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-12-06 10:44:32 -08:00
a83bc2d977
KAFKA-13988: Enable replicating from latest offset with MirrorMaker 2 ( #14567 )
...
Reviewers: hudeqi <1217150961@qq.com>, Federico Valeri <fedevaleri@gmail.com>, Greg Harris <gharris1727@gmail.com>
2023-12-04 16:37:37 -05:00
e8edc279f4
MINOR: Correcting Javadoc for ConnectAssertions ( #14827 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>
---------
Co-authored-by: Yash Mayya <yash.mayya@gmail.com>
2023-11-27 15:29:08 +05:30
ebf435e718
KAFKA-15888: DistributedHerder log context should not use the same client ID for each Connect worker by default ( #14825 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-11-23 15:31:39 +05:30
5d717f2343
MINOR: Fix broken method link in DistributedHerder::writeToConfigTopicAsLeader Javadoc ( #14824 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-11-23 03:22:56 +05:30
1bc4de75a2
KAFKA-15470: Allow creating connectors in a stopped state ( #14704 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-11-15 11:37:50 +05:30
989618a56e
KAFKA-15800: Prevent DataExceptions from corrupting KafkaOffsetBackingStore ( #14718 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Yash Mayya <yash.mayya@gmail.com>
2023-11-10 08:40:16 -08:00
39c6170aa9
KAFKA-15693: Immediately reassign lost connectors and tasks when scheduled rebalance delay is disabled ( #14647 )
...
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Yash Mayya <yash.mayya@gmail.com>
2023-11-09 10:48:43 -05:00
ca05da95c3
KAFKA-15680: Fix sink task partition-count metric when cooperative consumer protocol is used ( #14630 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-11-06 11:32:05 -05:00
4083cd627e
KAFKA-15607: Fix NPE in MirrorCheckpointTask::syncGroupOffset ( #14587 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-10-20 12:17:51 -04:00
091eb9b349
KAFKA-15428: Cluster-wide dynamic log adjustments for Connect ( #14538 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>, Yang Yang <yayang@uber.com>, Yash Mayya <yash.mayya@gmail.com>
2023-10-20 09:52:37 -04:00
9b468fb278
MINOR: Do not end Javadoc comments with `**/` ( #14540 )
...
Reviewers: Bruno Cadonna <bruno@confluent.io>, Bill Bejeck <bill@confluent.io>, Hao Li <hli@confluent.io>, Josep Prat <josep.prat@aiven.io>
2023-10-17 21:11:04 -07:00
4150595b0a
KAFKA-14684: Replace EasyMock/PowerMock with Mockito in WorkerSinkTaskThreadedTest ( #14505 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Christo Lolov <christololov@gmail.com>
2023-10-16 11:24:52 +02:00
1c8bb61a43
KAFKA-15387: Deprecate Connect's redundant task configurations endpoint ( #14361 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>
2023-10-14 14:46:50 +05:30
4bad90835b
KAFKA-15465: Don't throw if MirrorMaker not authorized to create internal topics. ( #14388 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Ahmed Hibot
2023-10-13 12:53:09 +02:00
41c0d44402
KAFKA-15570: Add unit tests for MemoryConfigBackingStore ( #14518 )
...
Reviewers: Chris Egerton <chrise@aiven.io>, Kalpesh Patel <kpatel@confluent.io>
2023-10-12 14:25:40 +05:30
fbc39bcdbe
KAFKA-15547: Fix thread leak in MirrorMakerConfigTest ( #14487 )
...
Reviewers: Akhilesh Chaganti <akhileshchg@users.noreply.github.com>, Rajini Sivaram <rajinisivaram@googlemail.com>, Yash Mayya <yash.mayya@gmail.com>
2023-10-05 10:19:15 +05:30
9c2e5daf60
MINOR: Revert log level changes in LogCaptureAppender ( #14436 )
...
LogCaptureAppender sets the log level in various tests to check if a certain log message is produced. The log level is however never reverted, changing the log level across the board and introducing flakiness due to non-determinism since the log level depends on execution order. Some log messages change the timing inside tests significantly.
Reviewer: Bruno Cadonna <cadonna@apache.org>
2023-09-26 10:49:41 +02:00
7107a750ba
KAFKA-15248 (KIP-959): Add BooleanConverter to Kafka Connect ( #14093 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>, Qichao Chu <5326144+ex172000@users.noreply.github.com>
2023-09-26 12:42:16 +05:30
98febb989a
KAFKA-15485: Fix "this-escape" compiler warnings introduced by JDK 21 (1/N) ( #14427 )
...
This is one of the steps required for kafka to compile with Java 21.
For each case, one of the following fixes were applied:
1. Suppress warning if fixing would potentially result in an incompatible change (for public classes)
2. Add final to one or more methods so that the escape is not possible
3. Replace method calls with direct field access.
In addition, we also fix a couple of compiler warnings related to deprecated references in the `core` module.
See the following for more details regarding the new lint warning:
https://www.oracle.com/java/technologies/javase/21-relnote-issues.html#JDK-8015831
Reviewers: Divij Vaidya <diviv@amazon.com>, Satish Duggana <satishd@apache.org>, Chris Egerton <chrise@aiven.io>
2023-09-24 05:59:29 -07:00
b088307612
KAFKA-15473: Hide duplicate plugins in /connector-plugins ( #14398 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>, Hector Geraldino <hgeraldino@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-09-19 12:37:21 -04:00
7872a1ff5b
KAFKA-14855: Harden integration testing logic for asserting that a connector is deleted ( #14371 )
...
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-09-19 11:39:39 -04:00
54ab5b29e4
KAFKA-15416: Fix flaky TopicAdminTest::retryEndOffsetsShouldRetryWhenTopicNotFound test case ( #14313 )
...
Reviewers: Philip Nee <pnee@confluent.io>, Greg Harris <greg.harris@aiven.io>
2023-09-07 19:24:17 -04:00
88b554fdbd
KAFKA-15179: Add integration tests for the file sink and source connectors ( #14279 )
...
Reviewers: Ashwin Pankaj <apankaj@confluent.io>, Chris Egerton <chrise@aiven.io>
2023-09-07 12:24:13 -04:00
b49013b73e
KAFKA-9800: Exponential backoff for Kafka clients - KIP-580 ( #14111 )
...
Implementation of KIP-580 to add exponential back-off to situations in which retry.backoff.ms
is used to delay backoff attempts. This KIP adds exponential backoff behavior with a maximum
controlled by a new config retry.backoff.max.ms, together with a +/- 20% of jitter to spread the
retry attempts of the client fleet.
Reviewers: Mayank Shekhar Narula <mayanks.narula@gmail.com>, Milind Luthra <i.milind.luthra@gmail.com>, Kirk True <kirk@mustardgrain.com>, Jun Rao<junrao@gmail.com>
2023-09-05 11:57:51 -07:00
1c5020e142
KAFKA-13327: Gracefully report connector validation errors instead of returning 500 responses ( #14303 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-08-29 15:44:22 -04:00
8611d28b2e
KAFKA-15392: Prevent shadowing RestServer shutdown exceptions ( #14277 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-29 12:25:30 -07:00
Greg Harris
Steven Xu
Dmitry Werner
TengYao Chi
Chris Egerton
Kondrat Bertalan
Chung, Ming-Yen
PaulRMellor
PoAn Yang
Zhengke Zhou
vamossagar12
Vikas Balani
Omnia Ibrahim
Fiore Mario Vitale
abhi-ksolves
Mickael Maison
gongxuanzhang
Ken Huang
Ken Huang
dujian0068
Ivan Yurchenko
Johnny Hsu
Chia Chuan Yu
Farbod Ahmadian
Sanskar Jhajharia
Fan Yang
Edoardo Comar
Yash Mayya
Ivan Yurchenko
yuz10
Jorge Esteban Quilcate Otoya
Hector Geraldino
Calvin Liu
Nikolay
Ahmed Sobeh
Kuan-Po (Cooper) Tseng
Minha, Jeong
Owen Leung
Octavian Ciubotaru
Divij Vaidya
Fiore Mario Vitale
Said Boudjelda
Matthew de Detrich
kumarpritam863
hudeqi
Matthias J. Sax
kpatelatwork
Lucas Brutschy
Ismael Juma
Andrew Schofield