Increase the maximum number of flaky tests we tolerate for the main test suite from 3 to 10. This will result in fewer failed builds.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This patch removes the explicit failure of test tasks in Gradle when there is a flaky test. This also fixes a fall-through case in junit.py where we did not recognize an error prior to running the tests (such as the javadoc task).
Additionally, this patch removes usages of ignoreFailures in our CI and changes the XML copy task to a finalizer task instead of doLast closure.
Reviewers: Jun Rao <junrao@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
This patch reorganizes our test infrastructure into three Gradle modules:
":test-common:test-common-internal-api" is now a minimal dependency which exposes interfaces and annotations only. It has one project dependency on server-common to expose commonly used data classes (MetadataVersion, Feature, etc). Since this pulls in server-common, this module is Java 17+. It cannot be used by ":clients" or other Java 11 modules.
":test-common:test-common-util" includes the auto-quarantined JUnit extension. The @Flaky annotation has been moved here. Since this module has no project dependencies, we can add it to the Java 11 list so that ":clients" and others can utilize the @Flaky annotation
":test-common:test-common-runtime" now includes all of the test infrastructure code (TestKitNodes, etc). This module carries heavy dependencies (core, etc) and so it should not normally be included as a compile-time dependency.
In addition to this reorganization, this patch leverages JUnit SPI service discovery so that modules can utilize the integration test framework without depending on ":core". This will allow us to start moving integration tests out of core and into the appropriate sub-module. This is done by adding ":test-common:test-common-runtime" as a testRuntimeOnly dependency rather than as a testImplementation dependency. A trivial example was added to QuorumControllerTest to illustrate this.
Reviewers: Ismael Juma <ismael@juma.me.uk>, Chia-Ping Tsai <chia7712@gmail.com>
This patch includes some maintenance updates for Develocity.
* Publish build scans to develocity.apache.org
* Update Develocity Gradle plugin to to 3.19
* Use `DEVELOCITY_ACCESS_KEY` to authenticate to `develocity.apache.org`
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Arthur <mumrah@gmail.com>
This patch ensures we only analyze trunk builds for the Develocity report. Some other minor fixes are also included
Reviewers: David Arthur <mumrah@gmail.com>
For PRs that have been reviewed (by anyone, not just a committer), remove the "triage" label. This job runs once per night.
Reviewers: Justine Olshan <jolshan@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
Instead of parsing the GE_ACCESS_TOKEN, the patch uses the newly minted DV_API_ACCESS secret for Develocity API access.
Reviewers: Mickael Maison <mickael.maison@gmail.com>
Automatically adds a "triage" label to PRs from the community. After 7 days, if no review has been made and the "triage" label is still present, a "needs-attention" label is added.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Mickael Maison <mickael.maison@gmail.com>
Docker tests rely on docker compose. In recent runs it has been observed that github actions does not provide support for docker compose, so we are installing it explicitly in the workflow.
The Stale PRs workflow is only able to act on a relatively small number of PRs due to the API operations limit. This patch increases the limit from 100 to 500.
Reviewers: Josep Prat <josep.prat@aiven.io>, Chia-Ping Tsai <chia7712@gmail.com>
Fixes another issue introduced in #17725 where the streaming XML parser would skip over tests that followed a SKIPPED test. This caused a large number of tests to be removed from the test catalog e4a5eb8
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Add a new "load-catalog" job to the workflow. This job will checkout the test-catalog branch at 7 days prior and generate a text file of all the tests that were known at that time. This file is then passed down to the two parallel "test" jobs to be used as a source of data for the quarantined test behavior.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This patch adds a CI job to store our test catalog in an orphaned branch named "test-catalog" within this repo.
This data will be used to help determine which tests should be quarantined.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Our "validate" job was running on JDK 21 while the "test" job was running 11 and 23. This patch updates the validate job to 23 and fixes the test catalog step to only run on JDK 23 (instead of 21)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
David Arthur
Clay Johnson
Vadym Zhytkevych
santhoshct
Logan Zhu
Ken Huang
David Jacot
Nick Guo
Matthias J. Sax
Divij Vaidya
Josep Prat
Vedarth Sharma
TengYao Chi
Colin Patrick McCabe