* Update CI to build with Java 21 instead of Java 20
* Disable spotbugs when building with Java 21 as it doesn't support it yet (filed KAFKA-15492 for
addressing this)
* Disable SslTransportLayerTest.testValidEndpointIdentificationCN with Java 21 (same as Java 20)
Reviewers: Divij Vaidya <diviv@amazon.com>
* KAFKA-14682: Report Mockito unused stubbings during Jenkins build
* DO NOT MERGE: Add test case that should fail during Jenkins build
* Revert "DO NOT MERGE: Add test case that should fail during Jenkins build"
This reverts commit 8418b835ec.
It's good for us to add support for Java 20 in preparation for Java 21 - the next LTS.
Given that Scala 2.12 support has been deprecated, a Scala 2.12 variant is not included.
Also remove some branch builds that add load to the CI, but have
low value: JDK 8 & Scala 2.13 (JDK 8 support has been deprecated),
JDK 11 & Scala 2.12 (Scala 2.12 support has been deprecated) and
JDK 17 & Scala 2.12 (Scala 2.12 support has been deprecated).
A newer version of Mockito (4.9.0 -> 4.11.0) is required for Java 20 support, but we
only use it with Scala 2.13+ since it causes compilation errors with Scala 2.12. Similarly,
we upgrade easymock when the Java version is 16 or newer as it's incompatible
with powermock (which doesn't support Java 16 or newer).
Filed KAFKA-15117 for a test that fails with Java 20 (SslTransportLayerTest.testValidEndpointIdentificationCN).
Finally, fixed some lossy conversions that were added after #13582 was submitted.
Reviewers: Ismael Juma <ismael@juma.me.uk>
Also re-enable it in CI. We do this by adjusting the `Jenkinsfile`
to use a more general task (`./gradlew check -x test`).
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Dejan Stojadinović <dejan2609@users.noreply.github.com>
Blocks #13205.
Rationale:
- build works fine in trunk with Gradle 7.6 and spotless gradle plugin 6.13.0 for all currently used JDK versions (that is: JDK 8 / JDK 11 / JDK 17)
- however, recent spotless gradle plugin versions (6.14.+) support only JDK 11+ versions: https://github.com/diffplug/spotless/blob/main/plugin-gradle/CHANGES.md#6140---2023-01-26
- given a fact that Kafka still needs to support JDK 8 builds (until Kafka version 4.0) it is reasonable to simply remove spotless checks out of Jenkinsfile (and re-introduce them when the time comes).
For even more details see GitHub discussion here: https://github.com/apache/kafka/pull/13205#issuecomment-1431575644
Reviewers: Ismael Juma <ismael@juma.me.uk>
Pull requests' builds are often waiting unnecessarily on getting an ubuntu node in the post step. The post step sends out an email to the dev mailing list but it does so only for regular branches. For pull requests, it does nothing. However, the check to verify if the build is for a pull request is executed when the node is allocated so the build has to wait for it.
With this change, the check is executed before requesting the node so it does not unnecessarily request a node if one is not needed.
Note that I have verified that sending the email still works with this change (see fc74c8375a).
Reviewers: Mickael Maison <mickael.maison@gmail.com>
Originally, we only enabled retries for PR builds to avoid hiding timing
related issues. In practice, however, the results are too noisy without
any retry due to various environmental issues.
Enable 1 retry for all builds and increase the max test retry to 10.
Reviewers: José Armando García Sancio <jsancio@users.noreply.github.com>
When a new commit is pushed to a PR, we should abort the build
and start a new one instead of queuing one build per commit.
This reduces wasted resources and provides the desired feedback
faster (since we only care about the build results for the last
commit in the PR).
Reviewers: Luke Chen <showuon@gmail.com>
We have been seeing builds fail due to errors such as the following:
```
Timeout waiting to lock zinc-1.6.1_2.13.8_17 compiler cache (/home/jenkins/.gradle/caches/7.5.1/zinc-1.6.1_2.13.8_17). It is currently in use by another Gradle instance.
```
This patch includes a workaround: if the compilation fails due to this zinc compile cache error, then we retry it.
Reviewers: Ismael Juma <ismael@juma.me.uk>
Co-authored-by: Lucas Bradstreet <lucasbradstreet@gmail.com>
Also add an additional stage to ARM and PowerPC builds which will fail-fast if the agent is not available.
Reviewers: Jason Gustafson <jason@confluent.io>
Java 17 is at release candidate stage and it will be a LTS release once
it's out (previous LTS release was Java 11).
Details:
* Replace Java 16 with Java 17 in Jenkins and Readme.
* Replace `--illegal-access=permit` (which was removed from Java 17)
with `--add-opens` for the packages we require internal access to.
Filed KAFKA-13275 for updating the tests not to require `--add-opens`
(where possible).
* Update `release.py` to use JDK8. and JDK 17 (instead of JDK 8 and JDK 15).
* Removed all but one Streams test from `testsToExclude`. The
Connect test exclusion list remains the same.
* Add notable change to upgrade.html
* Upgrade to Gradle 7.2 as it's required for proper Java 17 support.
* Upgrade mockito to 3.12.4 for better Java 17 support.
* Adjusted `KafkaRaftClientTest` and `QuorumStateTest` not to require
private access to `jdk.internal.util.random`.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Gradle 7.1 improves Java incremental compilation:
https://docs.gradle.org/7.1.1/release-notes.html
We previously kept the JDK 15 build because some
tests didn't work with JDK 16. Since then, a number
of PRs were submitted to fix this so it's best
to remove the JDK 15 build before we create the
3.0 release branch.
Finally bump `test-retry` gradle plugin version too.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Luke Chen <showuon@gmail.com>
JDK 15 no longer receives updates, so we want to switch from JDK 15 to JDK 16.
However, we have a number of tests that don't yet pass with JDK 16.
Instead of replacing JDK 15 with JDK 16, we have both for now and we either
disable (via annotations) or exclude (via gradle) the tests that don't pass with
JDK 16 yet. The annotations approach is better, but it doesn't work for tests
that rely on the PowerMock JUnit 4 runner.
Also add `--illegal-access=permit` when building with JDK 16 to make MiniKdc
work for now. This has been removed in JDK 17, so we'll have to figure out
another solution when we migrate to that.
Relevant JIRAs for the disabled tests: KAFKA-12790, KAFKA-12941, KAFKA-12942.
Moved some assertions from `testTlsDefaults` to `testUnsupportedTlsVersion`
since the former claims to test the success case while the former tests the failure case.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
We no longer need this since:
1. The PR and branch jobs are configured to `clean before checkout`.
2. The Gradle build outputs the gradle version on start-up.
The description of `clean before checkout` is:
> Clean up the workspace before every checkout by deleting all untracked files and directories, including those which are specified in .gitignore. It also resets all tracked files to their versioned state. This ensures that the workspace is in the same state as if you cloned and checked out in a brand-new empty directory, and ensures that your build is not affected by the files generated by the previous build.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
The `node` block achieves that.
Tested that an email was sent to the mailing list for:
592a0c31d5
Added back the condition not to send emails for PR builds after
such test.
Reviewers: Gwen Shapira <cshapi@gmail.com>
* Run all JDK/Scala version combinations for trunk/release branch builds.
* Only retry failures in PR builds for now (we can remove this distinction if/when
we report flaky failures as described in KAFKA-12216).
* Disable concurrent builds
* Send email to dev list on build failure
* Use triple double quotes in `doValidation` since we use string interpolation
for `SCALA_VERSION`.
* Update release.py to output new `Unit/integration tests` Jenkins link
Reviewers: Gwen Shapira <cshapi@gmail.com>, David Arthur <mumrah@gmail.com>
Gradle 7.0 is required for Java 16 compatibility and it removes a number of
deprecated APIs. Fix most issues preventing the upgrade to Gradle 7.0.
The remaining ones are more complicated and should be handled
in a separate PR. Details of the changes:
* Release tarball no longer includes includes test, sources, javadoc and test sources jars (these
are still published to the Maven Central repository).
* Replace `compile` with `api` or `implementation` - note that `implementation`
dependencies appear with `runtime` scope in the pom file so this is a (positive)
change in behavior
* Add missing dependencies that were uncovered by the usage of `implementation`
* Replace `testCompile` with `testImplementation`
* Replace `runtime` with `runtimeOnly` and `testRuntime` with `testRuntimeOnly`
* Replace `configurations.runtime` with `configurations.runtimeClasspath`
* Replace `configurations.testRuntime` with `configurations.testRuntimeClasspath` (except for
the usage in the `streams` project as that causes a cyclic dependency error)
* Use `java-library` plugin instead of `java`
* Use `maven-publish` plugin instead of deprecated `maven` plugin - this changes the
commands used to publish and to install locally, but task aliases for `install` and
`uploadArchives` were added for backwards compatibility
* Removed `-x signArchives` line from the readme since it was wrong (it was a
no-op before and it fails now, however)
* Replaces `artifacts` block with an approach that works with the `maven-publish` plugin
* Don't publish `jmh-benchmark` module - the shadow jar is pretty large and not
particularly useful (before this PR, we would publish the non shadow jars)
* Replace `version` with `archiveVersion`, `baseName` with `archiveBaseName` and
`classifier` with `archiveClassifier`
* Update Gradle and plugins to the latest stable version (7.0 is not stable yet)
* Use `plugin` DSL to configure plugins
* Updated notable changes for 3.0
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Randall Hauch <rhauch@gmail.com>
Ismael Juma
Chris Egerton
Dejan Stojadinović
David Jacot
Jason Gustafson
David Arthur
Mickael Maison
