Commit Graph

971 Commits

Author SHA1 Message Date
hudeqi 4083cd627e
KAFKA-15607: Fix NPE in MirrorCheckpointTask::syncGroupOffset (#14587)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-10-20 12:17:51 -04:00
Chris Egerton 091eb9b349
KAFKA-15428: Cluster-wide dynamic log adjustments for Connect (#14538)
Reviewers: Greg Harris <greg.harris@aiven.io>, Yang Yang <yayang@uber.com>, Yash Mayya <yash.mayya@gmail.com>
2023-10-20 09:52:37 -04:00
Matthias J. Sax 9b468fb278
MINOR: Do not end Javadoc comments with `**/` (#14540)
Reviewers: Bruno Cadonna <bruno@confluent.io>, Bill Bejeck <bill@confluent.io>, Hao Li <hli@confluent.io>, Josep Prat <josep.prat@aiven.io>
2023-10-17 21:11:04 -07:00
Hector Geraldino 4150595b0a
KAFKA-14684: Replace EasyMock/PowerMock with Mockito in WorkerSinkTaskThreadedTest (#14505)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Christo Lolov <christololov@gmail.com>
2023-10-16 11:24:52 +02:00
Yash Mayya 1c8bb61a43
KAFKA-15387: Deprecate Connect's redundant task configurations endpoint (#14361)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>
2023-10-14 14:46:50 +05:30
Omnia G.H Ibrahim 4bad90835b
KAFKA-15465: Don't throw if MirrorMaker not authorized to create internal topics. (#14388)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Ahmed Hibot
2023-10-13 12:53:09 +02:00
Yash Mayya 41c0d44402
KAFKA-15570: Add unit tests for MemoryConfigBackingStore (#14518)
Reviewers: Chris Egerton <chrise@aiven.io>, Kalpesh Patel <kpatel@confluent.io>
2023-10-12 14:25:40 +05:30
kpatelatwork fbc39bcdbe
KAFKA-15547: Fix thread leak in MirrorMakerConfigTest (#14487)
Reviewers: Akhilesh Chaganti <akhileshchg@users.noreply.github.com>, Rajini Sivaram <rajinisivaram@googlemail.com>, Yash Mayya <yash.mayya@gmail.com>
2023-10-05 10:19:15 +05:30
Lucas Brutschy 9c2e5daf60
MINOR: Revert log level changes in LogCaptureAppender (#14436)
LogCaptureAppender sets the log level in various tests to check if a certain log message is produced. The log level is however never reverted, changing the log level across the board and introducing flakiness due to non-determinism since the log level depends on execution order. Some log messages change the timing inside tests significantly.

Reviewer: Bruno Cadonna <cadonna@apache.org>
2023-09-26 10:49:41 +02:00
Hector Geraldino 7107a750ba
KAFKA-15248 (KIP-959): Add BooleanConverter to Kafka Connect (#14093)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>, Qichao Chu <5326144+ex172000@users.noreply.github.com>
2023-09-26 12:42:16 +05:30
Ismael Juma 98febb989a
KAFKA-15485: Fix "this-escape" compiler warnings introduced by JDK 21 (1/N) (#14427)
This is one of the steps required for kafka to compile with Java 21.

For each case, one of the following fixes were applied:
1. Suppress warning if fixing would potentially result in an incompatible change (for public classes)
2. Add final to one or more methods so that the escape is not possible
3. Replace method calls with direct field access.

In addition, we also fix a couple of compiler warnings related to deprecated references in the `core` module.

See the following for more details regarding the new lint warning:
https://www.oracle.com/java/technologies/javase/21-relnote-issues.html#JDK-8015831

Reviewers: Divij Vaidya <diviv@amazon.com>, Satish Duggana <satishd@apache.org>, Chris Egerton <chrise@aiven.io>
2023-09-24 05:59:29 -07:00
Greg Harris b088307612
KAFKA-15473: Hide duplicate plugins in /connector-plugins (#14398)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>, Hector Geraldino <hgeraldino@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-09-19 12:37:21 -04:00
Yash Mayya 7872a1ff5b
KAFKA-14855: Harden integration testing logic for asserting that a connector is deleted (#14371)
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-09-19 11:39:39 -04:00
Chris Egerton 54ab5b29e4
KAFKA-15416: Fix flaky TopicAdminTest::retryEndOffsetsShouldRetryWhenTopicNotFound test case (#14313)
Reviewers: Philip Nee <pnee@confluent.io>, Greg Harris <greg.harris@aiven.io>
2023-09-07 19:24:17 -04:00
Yash Mayya 88b554fdbd
KAFKA-15179: Add integration tests for the file sink and source connectors (#14279)
Reviewers: Ashwin Pankaj <apankaj@confluent.io>, Chris Egerton <chrise@aiven.io>
2023-09-07 12:24:13 -04:00
Andrew Schofield b49013b73e
KAFKA-9800: Exponential backoff for Kafka clients - KIP-580 (#14111)
Implementation of KIP-580 to add exponential back-off to situations in which retry.backoff.ms
is used to delay backoff attempts. This KIP adds exponential backoff behavior with a maximum
controlled by a new config retry.backoff.max.ms, together with a +/- 20% of jitter to spread the
retry attempts of the client fleet.

Reviewers: Mayank Shekhar Narula <mayanks.narula@gmail.com>, Milind Luthra <i.milind.luthra@gmail.com>, Kirk True <kirk@mustardgrain.com>, Jun Rao<junrao@gmail.com>
2023-09-05 11:57:51 -07:00
Chris Egerton 1c5020e142
KAFKA-13327: Gracefully report connector validation errors instead of returning 500 responses (#14303)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-08-29 15:44:22 -04:00
Greg Harris 8611d28b2e
KAFKA-15392: Prevent shadowing RestServer shutdown exceptions (#14277)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-29 12:25:30 -07:00
Yash Mayya 880288879e
KAFKA-15377: Don't expose externalized secret values in tasks-config API endpoint (#14244)
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-08-24 15:44:53 -07:00
Greg Harris a3303b6112
KAFKA-15393: Improve shutdown behavior in MM2 integration tests (#14278)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-08-24 12:24:34 -07:00
Yash Mayya 064a14795c
MINOR: Update OpenAPI summary documentation for Connect's /tasks endpoint (#14267)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-08-22 18:26:29 +02:00
Yash Mayya 7802c264c9
MINOR: Allow writing tombstone offsets for arbitrary partitions in the FileStreamSourceConnector (#14234)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-17 14:13:53 -04:00
Chris Egerton de409b389d
KAFKA-15177: Implement KIP-875 SourceConnector::alterOffset API in MirrorMaker 2 connectors (#14005)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-08-17 09:33:59 -04:00
Greg Harris 6bd17419b7
KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898) (#14195)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-16 11:37:33 -07:00
Greg Harris a9efca0bf6
KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module (#13302)
Reviewers: Hector Geraldino <hgeraldino@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-08-16 10:30:24 -07:00
Omnia G.H Ibrahim 35e925f353
KAFKA-15102: Add replication.policy.internal.topic.separator.enabled property to MirrorMaker 2 (KIP-949) (#14082)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 19:58:52 -04:00
Greg Harris 28858f3a3e
MINOR: Fix SynchronizationTest classloaders sometimes not being parallel capable (#14177)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 14:24:48 -07:00
Greg Harris 1a001c1e88
KAFKA-15336: Add ServiceLoader Javadocs for Connect plugins (#14194)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 13:21:45 -07:00
Greg Harris f5655d31d3
KAFKA-15030: Add connect-plugin-path command-line tool (#14064)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-11 12:05:51 -07:00
bachmanity1 f137da04fa
KAFKA-14132: Replace Easymock & Powermock with Mockito in KafkaBasedLogTest (#14153)
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Divij Vaidya <diviv@amazon.com>
2023-08-11 10:50:37 +02:00
Federico Valeri 8de3e0436a
KAFKA-15239: Fix system tests using producer performance service (#14092)
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-08-10 14:23:43 -07:00
Yash Mayya f2ebd333e8
KAFKA-13187: Replace EasyMock / PowerMock with Mockito in DistributedHerderTest (#14102)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-10 16:47:30 -04:00
Greg Harris 0ee26640e5
KAFKA-15202: Fix MM2 offset translation when syncs are variably spaced (#14156)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-10 13:25:50 -07:00
aindriu-aiven db34f8b9a1
KAFKA-15291: Connect plugins should declare a version (#14159)
Signed-off-by: Aindriu Lavelle <aindriu.lavelle@aiven.io>
Reviewers: Andrew Schofield, Greg Harris <greg.harris@aiven.io>
2023-08-10 10:45:12 -07:00
Yash Mayya f23394336a
MINOR: Use Mockito's strict stubs functionality for Connect tests and cleanup unused stubs (#14166)
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-08-09 11:13:28 +02:00
Greg Harris ff4fed5cbe
KAFKA-15031: Add plugin.discovery to Connect worker configuration (KIP-898) (#14055)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-08 10:06:35 -07:00
Yash Mayya b9936d6292
KAFKA-7438: Replace PowerMockRunner with MockitoJUnitRunner in RetryUtilTest (#14143)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-03 13:07:35 -04:00
Greg Harris b9a45546a7
KAFKA-15244: Remove PluginType.from(Class) (#14089)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-01 10:05:46 -07:00
Yash Mayya 3ba718e159
MINOR: Remove duplicate instantiation of MockConnectMetrics in AbstractWorkerSourceTaskTest (#14091)
Reviewers: Christo Lolov <christololov@gmail.com>, Manyanda Chitimbo <manyanda.chitimbo@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-07-28 11:35:42 -07:00
Yash Mayya 08b3820d5e
KAFKA-15238: Move DLQ reporter setup from the DistributedHerder's tick thread to the sink task thread (#14079)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-25 09:03:29 -04:00
Chris Egerton 58b8c5c7b1
MINOR: Downgrade log level for conflicting Connect plugin aliases (#14081)
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-07-25 08:12:46 -04:00
Yash Mayya 4daeb2714c
KAFKA-13431 (KIP-793): Expose the original pre-transform topic partition and offset in sink records (#14024)
Reviewers: Greg Harris <greg.harris@aiven.io>, Chris Egerton <chrise@aiven.io>
2023-07-21 12:06:01 -04:00
Greg Harris 125dbb9286
KAFKA-14760: Move ThroughputThrottler from tools to clients, remove tools dependency from connect-runtime (#13313)
Reviewers: Ismael Juma <ismael@juma.me.uk>
2023-07-20 12:58:48 -07:00
Yash Mayya ea6e100ec3
KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers (#14044)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-20 10:22:35 -04:00
Greg Harris f6e7aa3763
KAFKA-15150: Add ServiceLoaderScanner implementation (#13971)
Reviewers: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 16:21:58 -07:00
Greg Harris 844469311a
KAFKA-10579: Upgrade reflections from 0.9.12 to 0.10.2 (#14029)
Reviewers: Divij Vaidya <diviv@amazon.com>, Chris Egerton <chris.egerton@aiven.io>
2023-07-19 14:00:15 -07:00
Greg Harris d5a00cca74
KAFKA-15162: Reflectively find plugins in parent ClassLoaders that aren't on the classpath (#13977)
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewed-by: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 12:49:25 -07:00
Yash Mayya a68281bf3c
MINOR: Fix Connect compatibility javadocs to use NoSuchMethodError instead of NoSuchMethodException (#14036)
Reviewed-by: Greg Harris <greg.harris@aiven.io>
2023-07-19 10:23:02 -07:00
Greg Harris 1f7f6ceebf
KAFKA-15211: Mock InvalidParameterException in DistributedConfigTest (#14039)
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewed-by: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 09:02:44 -07:00
Yash Mayya d6aaddf6ee
KAFKA-15091: Fix misleading Javadoc for SourceTask::commit (#13948)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-18 16:38:30 -04:00