Commit Graph

147 Commits

Author SHA1 Message Date
Mickael Maison c13324fc16
KAFKA-15995: Adding KIP-877 support to Connect (#17804)
Reviewers: Greg Harris <gharris1727@gmail.com>
2025-02-11 18:27:15 +01:00
Martin Sillence d001b47093
KAFKA-17792: Efficiently parse decimals with large exponents in Connect Values (#17510)
Reviewers: Greg Harris <greg.harris@aiven.io>, Mickael Maison <mickael.maison@gmail.com>
2025-01-27 09:04:29 -08:00
Ken Huang b4674c002f
KAFKA-18585 Fix fail test ValuesTest#shouldConvertDateValues (#18611)
Reviewers: Divij Vaidya <diviv@amazon.com>, Andrew Schofield <aschofield@confluent.io>
2025-01-21 13:51:39 +00:00
Andrew Schofield 42fd4d1d11
KAFKA-7776: Tests for ISO8601 in Connect value parsing (#18374)
Reviewers: Andrew Schofield <aschofield@confluent.io>
2025-01-16 14:26:03 +00:00
Ismael Juma 409a43eff7
MINOR: Collection/Option usage simplification via methods introduced in Java 9 & 11 (#18305)
Relevant methods:
1. `List.of`, `Set.of`, `Map.of` and similar (introduced in Java 9)
2. Optional: `isEmpty` (introduced in Java 11), `stream` (introduced in Java 9).

Reviewers: Mickael Maison <mimaison@users.noreply.github.com>
2025-01-03 16:13:39 -08:00
Ismael Juma d6f24d3665
Use `instanceof` pattern to avoid explicit cast (#18373)
This feature was introduced in Java 16.

Reviewers: David Arthur <mumrah@gmail.com>, Apoorv Mittal <apoorvmittal10@gmail.com>
2025-01-02 09:32:51 -08:00
Nick Guo 0163fa2d06
KAFKA-18294 Remove deprecated SourceTask#commitRecord (#18260)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-19 01:06:33 +08:00
Christopher L. Shannon bd6d0fbf3d
KAFKA-16437 Upgrade to Jakarta and Jetty 12 (KIP-1032) (#16754)
This commit implements the changes for KIP-1032. This updates Kafka to Jakarta specs, JavaEE 10 and Jetty 12. The changes here primarily effect Kafka Connect and MM2.

Todo/Notes:

1) I bumped the connect modules to JDK 17 but I also had to bump a couple other things that had a dependency on conect. The tools project depends on connect so that had to be bumped, and streams depends on tools so that needed to be bumped. This means we may need to separate some things if we don't want to enforce JDK 17 on streams.

2) There is an issue with a test in DedicatedMirrorIntegrationTest that I had to change for now that involves escaping characters and not quite sure what to do about it yet. The cause is the Servlet 6 spec changing what is allowed in the path. See: Jetty 12: 400: Ambiguous URI path encoding for path <%=FOO%>~1 (encoded: %3C%25%3DFOO%25%3E%7E1) jetty/jetty.project#11890

3) I had to configure the idle timeout in Jetty requests to match our request timeout so tests didn't fail. This was needed to fix the ConnectWorkerIntegrationTest#testPollTimeoutExpiry() test

Testing is being done by just using the existing tests for Connect and MM2 which should be sufficient.

Reviewers: Greg Harris <greg.harris@aiven.io>, David Arthur <mumrah@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
2024-12-11 23:24:14 +08:00
Nick Guo 970d8930ac
KAFKA-18159 Remove onPartitionsRevoked and onPartitionsAssigned from SinkTask (#18049)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-12-06 04:28:34 +08:00
Dmitry Werner eb897c6ad5
MINOR: Fix potential NPE (#17541)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-10-19 01:05:45 +08:00
gongxuanzhang 596b945072
KAFKA-16643 Add ModifierOrder checkstyle rule (#15890)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-13 15:39:32 +08:00
gongxuanzhang 79b9c44acd
KAFKA-10787 Apply spotless to connect module (#16299)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-06-12 22:19:47 +08:00
Greg Harris 52514a89e3
KAFKA-16858: Throw DataException from validateValue on array and map schemas without inner schemas (#16161)
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-06-05 11:35:32 -07:00
Sanskar Jhajharia e974914ca5
MINOR: Code Cleanup - Connect Module (#16066)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-05-31 04:55:00 +08:00
Greg Harris 11ad5e8bca
MINOR: Refactor Values class to fix checkstyle, add benchmark, optimize exceptions (#15469)
Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Mickael Maison <mickael.maison@gmail.com>
2024-05-23 13:23:18 -07:00
Kuan-Po (Cooper) Tseng 12a1d85362
KAFKA-12187 replace assertTrue(obj instanceof X) with assertInstanceOf (#15512)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2024-03-20 10:36:25 +08:00
Greg Harris 99e511c706
KAFKA-16288, KAFKA-16289: Fix Values convertToDecimal exception and parseString corruption (#15399)
* KAFKA-16288: Prevent ClassCastExceptions for strings in Values.convertToDecimal
* KAFKA-16289: Values inferred schemas for map and arrays should ignore element order

Signed-off-by: Greg Harris <greg.harris@aiven.io>
Reviewers: Chris Egerton <chrise@aiven.io>
2024-03-04 15:59:47 -08:00
Divij Vaidya 65424ab484
MINOR: New year code cleanup - include final keyword (#15072)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Sagar Rao <sagarmeansocean@gmail.com>
2024-01-11 17:53:35 +01:00
Matthias J. Sax 9b468fb278
MINOR: Do not end Javadoc comments with `**/` (#14540)
Reviewers: Bruno Cadonna <bruno@confluent.io>, Bill Bejeck <bill@confluent.io>, Hao Li <hli@confluent.io>, Josep Prat <josep.prat@aiven.io>
2023-10-17 21:11:04 -07:00
Greg Harris 1a001c1e88
KAFKA-15336: Add ServiceLoader Javadocs for Connect plugins (#14194)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-15 13:21:45 -07:00
aindriu-aiven db34f8b9a1
KAFKA-15291: Connect plugins should declare a version (#14159)
Signed-off-by: Aindriu Lavelle <aindriu.lavelle@aiven.io>
Reviewers: Andrew Schofield, Greg Harris <greg.harris@aiven.io>
2023-08-10 10:45:12 -07:00
Greg Harris ff4fed5cbe
KAFKA-15031: Add plugin.discovery to Connect worker configuration (KIP-898) (#14055)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-08-08 10:06:35 -07:00
Yash Mayya 4daeb2714c
KAFKA-13431 (KIP-793): Expose the original pre-transform topic partition and offset in sink records (#14024)
Reviewers: Greg Harris <greg.harris@aiven.io>, Chris Egerton <chrise@aiven.io>
2023-07-21 12:06:01 -04:00
Yash Mayya ea6e100ec3
KAFKA-15216: InternalSinkRecord::newRecord should not ignore new headers (#14044)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-20 10:22:35 -04:00
Yash Mayya a68281bf3c
MINOR: Fix Connect compatibility javadocs to use NoSuchMethodError instead of NoSuchMethodException (#14036)
Reviewed-by: Greg Harris <greg.harris@aiven.io>
2023-07-19 10:23:02 -07:00
Yash Mayya d6aaddf6ee
KAFKA-15091: Fix misleading Javadoc for SourceTask::commit (#13948)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-07-18 16:38:30 -04:00
Yash Mayya 6e72986949
KAFKA-14784: Connect offset reset REST API (#13818)
Reviewers: Chris Egerton <chrise@aiven.io>
2023-06-23 13:27:46 -04:00
Yash Mayya 7ff2dbb107
KAFKA-14368: Connect offset write REST API (#13465)
Reviewers: Greg Harris <greg.harris@aiven.io>, Chris Egerton <chrise@aiven.io>
2023-05-26 12:08:06 -04:00
Chris Egerton 404a833df7
KAFKA-14799: Ignore source task requests to abort empty transactions (#13379)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2023-03-14 15:10:29 -04:00
Mickael Maison 8d7d563231
MINOR: Remove duplicate empty string check (#13145)
Reviewers: Chris Egerton <fearthecellos@gmail.com>, Christo Lolov <christololov@gmail.com>
2023-02-27 14:49:12 +01:00
Yash Mayya 8f00782be7
MINOR: Connect Javadocs improvements (#13120)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Greg Harris <gharris1727@gmail.com>
2023-02-06 12:07:53 +01:00
Chris Egerton a382acd31d
KAFKA-13709 (follow-up): Avoid mention of 'exactly-once delivery' or 'delivery guarantees' in Connect (#13106) 2023-01-13 09:19:29 -05:00
Greg Harris 526af63cfe
KAFKA-13881: Add Connect package infos (#12937)
Reviewers: Chris Egerton <chrise@aiven.io>
2022-12-13 09:39:10 -05:00
Mickael Maison d606eb46ef
MINOR: Small javadoc/code cleanups in connect api and transforms (#12558)
Reviewers: Luke Chen <showuon@gmail.com>, Divij Vaidya <diviv@amazon.com>
2022-08-29 09:44:56 +02:00
vamossagar12 0c5f5a7f8b
KAFKA-14007: Close header converters during Connect task shutdown (#12309)
The HeaderConverter interface extends Closeable, but we weren't closing them anywhere before. This change causes header converters to be closed as part of task shutdown.

Reviewers: Kvicii <42023367+Kvicii@users.noreply.github.com>, Chris Egerton <fearthecellos@gmail.com>
2022-07-27 22:31:07 -04:00
Chris Egerton 7268284699
KAFKA-10000: Add all public-facing config properties (#11775)
Reviewers: Luke Chen <showuon@gmail.com>, Tom Bentley <tbentley@redhat.com>, Andrew Eugene Choi <andrew.choi@uwaterloo.ca>
2022-05-12 14:45:53 +08:00
Chris Egerton a586c94af1
KAFKA-10000: Add new source connector APIs related to exactly-once support (KIP-618) (#11773)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Reviewers: Tom Bentley <tbentley@redhat.com>, Hector Geraldino <hgeraldino@bloomberg.net>, Andrew Eugene Choi <andrew.choi@uwaterloo.ca>
2022-05-06 11:13:59 +02:00
Mickael Maison 029a14b530
KAFKA-13510: Connect APIs to list all connector plugins and retrieve their configs (#11572)
Implements KIP-769: https://cwiki.apache.org/confluence/display/KAFKA/KIP-769%3A+Connect+APIs+to+list+all+connector+plugins+and+retrieve+their+configuration+definitions

Reviewers: Tom Bentley <tbentley@redhat.com>, Chris Egerton <fearthecellos@gmail.com>
2022-03-03 14:28:50 +01:00
Knowles Atchison, Jr 9f2f63e3a1
KAFKA-13348: Allow Source Tasks to Handle Producer Exceptions (KIP-779) (#11382)
Reviewers: Mickael Maison <mickael.maison@gmail.com>, Chris Egerton <fearthecellos@gmail.com>
2022-01-27 19:17:59 +01:00
Chris Egerton a9ffabc447
MINOR: Improve decimal scale mismatch error message in Connect (#11384)
Use term Decimal, rather than BigDecimal.

Reviewers: Tom Bentley <tbentley@redhat.com>
2021-10-08 10:15:37 +01:00
Alexander Iskuskov efe6029f9c
KAFKA-10675: Add schema name to ConnectSchema.validateValue() error message (#9541)
The following error message
`org.apache.kafka.connect.errors.DataException: Invalid Java object for schema type INT64: class java.lang.Long for field: "moderate_time"`
can be confusing because java.lang.Long is acceptable type for schema INT64.

In fact, in this case `org.apache.kafka.connect.data.Timestamp` is used but this info is not logged.

Reviewers: Randall Hauch <rhauch@gmail.com>, Chris Egerton <chrise@confluent.io>, Konstantine Karantasis <k.karantasis@gmail.com>
2021-07-09 22:35:02 -07:00
Tom Bentley 862f814cc3
KAFKA-7613: Enable -Xlint:rawtypes for connect, fixing warnings (#8571)
Reviewers: Konstantine Karantasis <konstantine@confluent.io>
2021-07-07 17:24:31 +01:00
CHUN-HAO TANG 954c090ffc
MINOR: apply Utils.isBlank to code base (#10124)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2021-02-20 11:44:29 +08:00
dengziming 3769bc21b5
MINOR: replace hard-coding utf-8 with StandardCharsets.UTF_8 (#10079)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2021-02-09 10:06:01 +08:00
Lev Zemlyanov 07843cfbf4
KAFKA-10834: Remove redundant type casts in Connect (#10053)
Cleanup up to remove redundant type casts in Connect and use the diamond operator when needed 

Reviewers: Konstantine Karantasis <k.karantasis@gmail.com>
2021-02-04 17:08:56 -08:00
Lev Zemlyanov c19a35d1b7
KAFKA-10835: Replace Runnable and Callable overrides with lambdas in Connect (#9867)
Reviewers: Konstantine Karantasis <k.karantasis@gmail.com>
2021-02-04 09:15:49 -08:00
Geordie fb6c7beb29
MINOR: Remove redundant casting and if condition from ConnectSchema (#9959)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
2021-01-27 14:52:33 +08:00
Geordie 462be6da9b
KAFKA-12196: Migrate connect:api module to JUnit 5 (#9909)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>
2021-01-17 19:17:47 -08:00
dengziming 119a2d9127
MINOR: Substitute assertEquals(null) with assertNull (#9852)
Reviewers: David Jacot <djacot@confluent.io>
2021-01-10 20:06:37 +01:00
Chia-Ping Tsai 913a019d6c
MINOR: replace test "expected" parameter by assertThrows (#9520)
This PR includes following changes.

1. @Test(expected = Exception.class) is replaced by assertThrows
2. remove reference to org.scalatest.Assertions
3. change the magic code from 1 to 2 for testAppendAtInvalidOffset to test ZSTD
4. rename testMaybeAddPartitionToTransactionXXXX to testNotReadyForSendXXX
5. increase maxBlockMs from 1s to 3s to avoid unexpected timeout from TransactionsTest#testTimeout

Reviewers: Ismael Juma <ismael@confluent.io>
2021-01-10 20:20:13 +08:00