6bd17419b7
KAFKA-15228: Add sync-manifests command to connect-plugin-path (KIP-898) ( #14195 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-16 11:37:33 -07:00
a9efca0bf6
KAFKA-14759: Move Mock, Schema, and Verifiable connectors to new test-plugins module ( #13302 )
...
Reviewers: Hector Geraldino <hgeraldino@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-08-16 10:30:24 -07:00
35e925f353
KAFKA-15102: Add replication.policy.internal.topic.separator.enabled property to MirrorMaker 2 (KIP-949) ( #14082 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 19:58:52 -04:00
28858f3a3e
MINOR: Fix SynchronizationTest classloaders sometimes not being parallel capable ( #14177 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 14:24:48 -07:00
1a001c1e88
KAFKA-15336: Add ServiceLoader Javadocs for Connect plugins ( #14194 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 13:21:45 -07:00
f5655d31d3
KAFKA-15030: Add connect-plugin-path command-line tool ( #14064 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-11 12:05:51 -07:00
f137da04fa
KAFKA-14132: Replace Easymock & Powermock with Mockito in KafkaBasedLogTest ( #14153 )
...
Reviewers: Yash Mayya <yash.mayya@gmail.com>, Divij Vaidya <diviv@amazon.com>
2023-08-11 10:50:37 +02:00
8de3e0436a
KAFKA-15239: Fix system tests using producer performance service ( #14092 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-08-10 14:23:43 -07:00
f2ebd333e8
KAFKA-13187: Replace EasyMock / PowerMock with Mockito in DistributedHerderTest ( #14102 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-10 16:47:30 -04:00
0ee26640e5
KAFKA-15202: Fix MM2 offset translation when syncs are variably spaced ( #14156 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-10 13:25:50 -07:00
db34f8b9a1
KAFKA-15291: Connect plugins should declare a version ( #14159 )
...
Signed-off-by: Aindriu Lavelle <aindriu.lavelle@aiven.io>
Reviewers: Andrew Schofield, Greg Harris <greg.harris@aiven.io>
2023-08-10 10:45:12 -07:00
f23394336a
MINOR: Use Mockito's strict stubs functionality for Connect tests and cleanup unused stubs ( #14166 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-08-09 11:13:28 +02:00
ff4fed5cbe
KAFKA-15031: Add plugin.discovery to Connect worker configuration (KIP-898) ( #14055 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-08 10:06:35 -07:00
b9936d6292
KAFKA-7438: Replace PowerMockRunner with MockitoJUnitRunner in RetryUtilTest ( #14143 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-03 13:07:35 -04:00
b9a45546a7
KAFKA-15244: Remove PluginType.from(Class) ( #14089 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-01 10:05:46 -07:00
3ba718e159
MINOR: Remove duplicate instantiation of MockConnectMetrics in AbstractWorkerSourceTaskTest ( #14091 )
...
Reviewers: Christo Lolov <christololov@gmail.com>, Manyanda Chitimbo <manyanda.chitimbo@gmail.com>, Greg Harris <greg.harris@aiven.io>
2023-07-28 11:35:42 -07:00
08b3820d5e
KAFKA-15238: Move DLQ reporter setup from the DistributedHerder's tick thread to the sink task thread ( #14079 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-25 09:03:29 -04:00
58b8c5c7b1
MINOR: Downgrade log level for conflicting Connect plugin aliases ( #14081 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>
2023-07-25 08:12:46 -04:00
4daeb2714c
KAFKA-13431 (KIP-793): Expose the original pre-transform topic partition and offset in sink records ( #14024 )
...
Reviewers: Greg Harris <greg.harris@aiven.io>, Chris Egerton <chrise@aiven.io>
2023-07-21 12:06:01 -04:00
125dbb9286
KAFKA-14760: Move ThroughputThrottler from tools to clients, remove tools dependency from connect-runtime ( #13313 )
...
Reviewers: Ismael Juma <ismael@juma.me.uk>
2023-07-20 12:58:48 -07:00
ea6e100ec3
KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers ( #14044 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-20 10:22:35 -04:00
f6e7aa3763
KAFKA-15150: Add ServiceLoaderScanner implementation ( #13971 )
...
Reviewers: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 16:21:58 -07:00
844469311a
KAFKA-10579: Upgrade reflections from 0.9.12 to 0.10.2 ( #14029 )
...
Reviewers: Divij Vaidya <diviv@amazon.com>, Chris Egerton <chris.egerton@aiven.io>
2023-07-19 14:00:15 -07:00
d5a00cca74
KAFKA-15162: Reflectively find plugins in parent ClassLoaders that aren't on the classpath ( #13977 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewed-by: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 12:49:25 -07:00
a68281bf3c
MINOR: Fix Connect compatibility javadocs to use NoSuchMethodError instead of NoSuchMethodException ( #14036 )
...
Reviewed-by: Greg Harris <greg.harris@aiven.io>
2023-07-19 10:23:02 -07:00
1f7f6ceebf
KAFKA-15211: Mock InvalidParameterException in DistributedConfigTest ( #14039 )
...
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewed-by: Chris Egerton <chris.egerton@aiven.io>
2023-07-19 09:02:44 -07:00
d6aaddf6ee
KAFKA-15091: Fix misleading Javadoc for SourceTask::commit ( #13948 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-18 16:38:30 -04:00
5cb2504c49
KAFKA-14669: Use the generated docs for MirrorMaker configs in the doc ( #13658 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-18 11:27:19 -04:00
0b025b7711
KAFKA-15121: Implement the alterOffsets method in the FileStreamSourceConnector and the FileStreamSinkConnector ( #13945 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-18 10:37:51 -04:00
2b19b7325a
KAFKA-15182: Normalize source connector offsets before invoking SourceConnector::alterOffsets ( #14003 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-14 12:14:03 -04:00
8d24716f27
MINOR: Avoid slow Set.removeAll(List) in MirrorSourceConnector ( #13992 )
...
Reviewed-by: Greg Harris <greg.harris@aiven.io>
2023-07-12 12:19:35 -07:00
d114d8e29c
KAFKA-14938: Fixing flaky test testConnectorBoundary ( #13646 )
...
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Yash Mayya <yash.mayya@gmail.com>,
Sudesh Wasnik <swasnik@confluent.io>, Chris Egerton <chrise@aiven.io>
2023-07-12 11:45:49 -04:00
170f5f4ed0
KAFKA-15148: Mark tests correctly as integration tests where they running as unit tests ( #13973 )
...
Reviewers: Divij Vaidya <diviv@amazon.com>
2023-07-12 13:41:58 +02:00
adacfea2d6
KAFKA-12525: Ignoring stale status statuses when reading from Connect status topic ( #13453 )
...
During fast consecutive rebalances where a task is revoked from one worker and assigned to another one, it has been observed that there is a small time window and thus a race condition during which a RUNNING status record in the new generation is produced and is immediately followed by a delayed UNASSIGNED status record belonging to the same or a previous generation before the worker that sends this message reads the RUNNING status record that corresponds to the latest generation.
Although this doesn't inhibit the actual execution of tasks, it reports an incorrect status for those tasks(i.e UNASSIGNED). If the users have setup some kind of monitoring on tasks status then this could lead to false alarms for example.
This fix addresses this problem by checking if a status message is stale after reading it and updates it's status only when it is safe to.
Reviewers: Lucent-Wong <manchesterfans@live.cn>, Chris Egerton <chrise@aiven.io>, Yash Mayya <yash.mayya@gmail.com>, Konstantine Karantasis <k.karantasis@gmail.com>
2023-07-11 08:05:10 -07:00
51bc41031b
KAFKA-15139: Avoid slow Set.removeAll(List) in MirrorCheckpointConnector ( #13946 )
...
Reviewed-by: Greg Harris <greg.harris@aiven.io>
2023-07-10 14:35:46 -07:00
9ee28d1fe6
KAFKA-15145: Don't re-process records filtered out by SMTs on Kafka client retriable exceptions in AbstractWorkerSourceTask ( #13955 )
...
Reviewers: Sagar Rao <sagarmeansocean@gmail.com>, Chris Egerton <chrise@aiven.io>
2023-07-10 13:26:58 -04:00
6368d14a1d
KAFKA-14059 Replace PowerMock with Mockito in WorkerSourceTaskTest ( #13383 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-10 12:58:54 -04:00
d9a3e60dcc
KAFKA-14718: Wait for MirrorMaker to start before executing test ( #13284 )
2023-07-10 12:53:01 -04:00
1b925e9ee7
KAFKA-15069: Refactor plugin scanning logic into ReflectionScanner ( #13821 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-06 13:22:28 -04:00
a1a3ec0bcb
MINOR: Update connector status metric description to include 'stopped' as a potential value ( #13967 )
...
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2023-07-06 15:28:07 +02:00
96e59d7bfd
[MINOR] Correcting few WARN log lines in DistributedHerder#handleRebalance ( #13939 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-30 12:39:37 -04:00
1f4cbc5d53
MINOR: Add JDK 20 CI build and remove some branch builds ( #12948 )
...
It's good for us to add support for Java 20 in preparation for Java 21 - the next LTS.
Given that Scala 2.12 support has been deprecated, a Scala 2.12 variant is not included.
Also remove some branch builds that add load to the CI, but have
low value: JDK 8 & Scala 2.13 (JDK 8 support has been deprecated),
JDK 11 & Scala 2.12 (Scala 2.12 support has been deprecated) and
JDK 17 & Scala 2.12 (Scala 2.12 support has been deprecated).
A newer version of Mockito (4.9.0 -> 4.11.0) is required for Java 20 support, but we
only use it with Scala 2.13+ since it causes compilation errors with Scala 2.12. Similarly,
we upgrade easymock when the Java version is 16 or newer as it's incompatible
with powermock (which doesn't support Java 16 or newer).
Filed KAFKA-15117 for a test that fails with Java 20 (SslTransportLayerTest.testValidEndpointIdentificationCN).
Finally, fixed some lossy conversions that were added after #13582 was submitted.
Reviewers: Ismael Juma <ismael@juma.me.uk>
2023-06-30 01:12:00 -07:00
32bcdac6a1
MINOR: Replace synchronization with atomic update in Connect's StateTracker::changeState method ( #13934 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-29 15:05:06 -04:00
005416879e
KAFKA-15053: Use case insensitive validator for security.protocol config ( #13831 )
...
Fixed a regression described in KAFKA-15053 that security.protocol only allows uppercase values like PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. With this fix, both lower case and upper case values will be supported (e.g. PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL, plaintext, ssl, sasl_plaintext, sasl_ssl)
Reviewers: Chris Egerton <chrise@aiven.io>, Divij Vaidya <diviv@amazon.com>
2023-06-29 10:13:21 +02:00
3978684126
MINOR: Silence error logs for faulty plugins in integration tests ( #13912 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-23 14:10:18 -04:00
6e72986949
KAFKA-14784: Connect offset reset REST API ( #13818 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-23 13:27:46 -04:00
3c059133d3
MINOR: Fix generated client ids for Connect ( #13896 )
...
Reviewers: Chris Egerton <fearthecellos@gmail.com>
2023-06-21 21:44:14 +02:00
3b72b0abb1
MINOR: Optimize runtime of MM2 integration tests by batching transactions ( #13816 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-21 14:51:54 -04:00
d751c13950
MINOR: Fix typos for connect ( #13885 )
...
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-20 16:16:16 -04:00
73dd51e137
Revert "MINOR: Reduce MM2 integration test flakiness due to missing dummy offset commits ( #13838 )" ( #13864 )
...
Reviewers: Josep Prat <josep.prat@aiven.io>
Reverts commit 505c7b6487
2023-06-16 12:10:26 -04:00
Greg Harris
Omnia G.H Ibrahim
bachmanity1
Federico Valeri
Yash Mayya
aindriu-aiven
Chris Egerton
Gantigmaa Selenge
hudeqi
Sambhav Jain
ezio
vamossagar12
Hector Geraldino
Divij Vaidya
Ismael Juma
Bo Gao
Mickael Maison
minjian.cai