Adds two new workflows to help us enforce some uniform PR structure. The
first workflow runs in an unprivileged context and simply captures the
PR number into a text file and archives it. This is then used by another
workflow that runs in a privileged context using the code in `trunk` to
actually do the validation.
The validation is done using a new Python script. This script fetches a
PR using the GitHub CLI and validates its structure. For now this just
includes the title and body, but could perform other non-code related
checks in the future.
This validation is needed for the up-coming merge queue functionality.
Reviewers: Justine Olshan <jolshan@confluent.io>
Split the JUnit tests into "new", "flaky", and the remainder.
On PR builds, "new" tests are anything that do not exist on trunk. They are run with zero tolerance for flakiness.
On trunk builds, "new" tests are anything added in the last 7 days. They are run with some tolerance for flakiness.
Another change included here is that we will not update the test catalog if any test job fails on a trunk build. We have had difficulty determining if all the tests had or not (due to timeout or failures in upstream Gradle tasks). By requiring green ":test" jobs, we can be sure that the resulting catalog will be valid.
---
The purpose of this change is to discourage contributors from adding flaky tests, but give some leeway for trunk so we have successful builds.
The "quarantinedTest" Gradle target has been consolidated into the regular "test" target. There are now some
runtime properties to control what tests are run.
* kafka.test.catalog.file: path to test catalog
* kafka.test.run.new: include new tests. this selection depends on the age of the loaded test catalog
* kafka.test.run.flaky: include tests marked as `@Flaky` (replaces the `excludeTags 'flaky'` directive)
* kafka.test.verbose: include additional logging from new JUnit classes (enabled by default if re-running GitHub workflow with debug logging enabled)
* maxTestRetries: how many retries to allow via Develocity retry plugin (default 0)
* maxTestRetryFailures: how many failures to allow before stopping retries (default 0)
Thanks to Jun Rao for inspiring the idea.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
This patch includes some maintenance updates for Develocity.
* Publish build scans to develocity.apache.org
* Update Develocity Gradle plugin to to 3.19
* Use `DEVELOCITY_ACCESS_KEY` to authenticate to `develocity.apache.org`
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, David Arthur <mumrah@gmail.com>
For PRs that have been reviewed (by anyone, not just a committer), remove the "triage" label. This job runs once per night.
Reviewers: Justine Olshan <jolshan@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
Instead of parsing the GE_ACCESS_TOKEN, the patch uses the newly minted DV_API_ACCESS secret for Develocity API access.
Reviewers: Mickael Maison <mickael.maison@gmail.com>
Automatically adds a "triage" label to PRs from the community. After 7 days, if no review has been made and the "triage" label is still present, a "needs-attention" label is added.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Mickael Maison <mickael.maison@gmail.com>
Docker tests rely on docker compose. In recent runs it has been observed that github actions does not provide support for docker compose, so we are installing it explicitly in the workflow.
The Stale PRs workflow is only able to act on a relatively small number of PRs due to the API operations limit. This patch increases the limit from 100 to 500.
Reviewers: Josep Prat <josep.prat@aiven.io>, Chia-Ping Tsai <chia7712@gmail.com>
Add a new "load-catalog" job to the workflow. This job will checkout the test-catalog branch at 7 days prior and generate a text file of all the tests that were known at that time. This file is then passed down to the two parallel "test" jobs to be used as a source of data for the quarantined test behavior.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This patch adds a CI job to store our test catalog in an orphaned branch named "test-catalog" within this repo.
This data will be used to help determine which tests should be quarantined.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Our "validate" job was running on JDK 21 while the "test" job was running 11 and 23. This patch updates the validate job to 23 and fixes the test catalog step to only run on JDK 23 (instead of 21)
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Introduce new quarantinedTest that excludes tests tagged with "flaky". Also introduce two new build parameters "maxQuarantineTestRetries" and "maxQuarantineTestRetryFailures".
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This updates the versions of Java we test on from 8 and 21 to 11 and 21. This also removes unnecessary Check and Compile Java variations.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This patch allows workflow runs to be controlled by the ci-approved label on Pull Requests. Rather than manually approving each workflow run explicitly, committers can now add the appropriate label and the new "CI Requested" and "PR Labeled" workflows will auto-approve the requested run.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
On trunk, our CI runs in response to "push" events. The change in #17227 causes the workflow template to be invalid, which prevents the build from starting. This patch fixes that by defaulting to `false`
Reviewers: Justine Olshan <jolshan@confluent.io>
Fix the CI workflow to treat the `is-public-fork` input as a string.
Also add some docs on composite actions.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
David Arthur
Clay Johnson
Vadym Zhytkevych
Ken Huang
David Jacot
Nick Guo
Matthias J. Sax
Vedarth Sharma
TengYao Chi
Colin Patrick McCabe
Josep Prat
TaiJuWu
Logan Zhu