kafka/settings.gradle

132 lines
4.0 KiB
Groovy
Raw Normal View History

// Licensed to the Apache Software Foundation (ASF) under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// The ASF licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
plugins {
id 'com.gradle.develocity' version '3.19'
id 'com.gradle.common-custom-user-data-gradle-plugin' version '2.0.2'
}
def isGithubActions = System.getenv('GITHUB_ACTIONS') != null
KAFKA-18748 Run new tests separately in PRs (#18770) Split the JUnit tests into "new", "flaky", and the remainder. On PR builds, "new" tests are anything that do not exist on trunk. They are run with zero tolerance for flakiness. On trunk builds, "new" tests are anything added in the last 7 days. They are run with some tolerance for flakiness. Another change included here is that we will not update the test catalog if any test job fails on a trunk build. We have had difficulty determining if all the tests had or not (due to timeout or failures in upstream Gradle tasks). By requiring green ":test" jobs, we can be sure that the resulting catalog will be valid. --- The purpose of this change is to discourage contributors from adding flaky tests, but give some leeway for trunk so we have successful builds. The "quarantinedTest" Gradle target has been consolidated into the regular "test" target. There are now some runtime properties to control what tests are run. * kafka.test.catalog.file: path to test catalog * kafka.test.run.new: include new tests. this selection depends on the age of the loaded test catalog * kafka.test.run.flaky: include tests marked as `@Flaky` (replaces the `excludeTags 'flaky'` directive) * kafka.test.verbose: include additional logging from new JUnit classes (enabled by default if re-running GitHub workflow with debug logging enabled) * maxTestRetries: how many retries to allow via Develocity retry plugin (default 0) * maxTestRetryFailures: how many failures to allow before stopping retries (default 0) Thanks to Jun Rao for inspiring the idea. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
2025-02-25 06:08:15 +08:00
def runNew = ext.find("kafka.test.run.new") == "true"
def runFlaky = ext.find("kafka.test.run.flaky") == "true"
def currentJvm = JavaVersion.current()
develocity {
server = "https://develocity.apache.org"
projectId = "kafka"
buildScan {
uploadInBackground = !isGithubActions
publishing.onlyIf { it.authenticated }
obfuscation {
// This obfuscates the IP addresses of the build machine in the build scan.
// Alternatively, the build scan will provide the hostname for troubleshooting host-specific issues.
ipAddresses { addresses -> addresses.collect { address -> "0.0.0.0"} }
}
if (isGithubActions) {
tag "github"
} else {
tag "local"
}
tag "JDK$currentJvm"
KAFKA-18748 Run new tests separately in PRs (#18770) Split the JUnit tests into "new", "flaky", and the remainder. On PR builds, "new" tests are anything that do not exist on trunk. They are run with zero tolerance for flakiness. On trunk builds, "new" tests are anything added in the last 7 days. They are run with some tolerance for flakiness. Another change included here is that we will not update the test catalog if any test job fails on a trunk build. We have had difficulty determining if all the tests had or not (due to timeout or failures in upstream Gradle tasks). By requiring green ":test" jobs, we can be sure that the resulting catalog will be valid. --- The purpose of this change is to discourage contributors from adding flaky tests, but give some leeway for trunk so we have successful builds. The "quarantinedTest" Gradle target has been consolidated into the regular "test" target. There are now some runtime properties to control what tests are run. * kafka.test.catalog.file: path to test catalog * kafka.test.run.new: include new tests. this selection depends on the age of the loaded test catalog * kafka.test.run.flaky: include tests marked as `@Flaky` (replaces the `excludeTags 'flaky'` directive) * kafka.test.verbose: include additional logging from new JUnit classes (enabled by default if re-running GitHub workflow with debug logging enabled) * maxTestRetries: how many retries to allow via Develocity retry plugin (default 0) * maxTestRetryFailures: how many failures to allow before stopping retries (default 0) Thanks to Jun Rao for inspiring the idea. Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
2025-02-25 06:08:15 +08:00
if (runFlaky) {
tag "flaky"
}
if (runNew) {
tag "new"
}
}
}
buildCache {
local {
// This allows the build cache to be used locally or on GitHub Actions.
// Using the cache on GH should be safe since each job is run on a new VM
enabled = true
}
remote(develocity.buildCache) {
enabled = false
}
}
include 'clients',
'clients:clients-integration-tests',
'connect:api',
'connect:basic-auth-extension',
'connect:file',
'connect:json',
'connect:mirror',
'connect:mirror-client',
'connect:runtime',
'connect:test-plugins',
'connect:transforms',
'coordinator-common',
'core',
'examples',
'generator',
'group-coordinator',
'group-coordinator:group-coordinator-api',
'jmh-benchmarks',
'metadata',
'raft',
'server',
'server-common',
'share-coordinator',
'shell',
'storage',
'storage:api',
'streams',
'streams:examples',
'streams:integration-tests',
'streams:streams-scala',
'streams:test-utils',
'streams:upgrade-system-tests-0110',
'streams:upgrade-system-tests-10',
'streams:upgrade-system-tests-11',
'streams:upgrade-system-tests-20',
'streams:upgrade-system-tests-21',
'streams:upgrade-system-tests-22',
'streams:upgrade-system-tests-23',
'streams:upgrade-system-tests-24',
'streams:upgrade-system-tests-25',
'streams:upgrade-system-tests-26',
'streams:upgrade-system-tests-27',
'streams:upgrade-system-tests-28',
'streams:upgrade-system-tests-30',
'streams:upgrade-system-tests-31',
'streams:upgrade-system-tests-32',
'streams:upgrade-system-tests-33',
'streams:upgrade-system-tests-34',
'streams:upgrade-system-tests-35',
'streams:upgrade-system-tests-36',
'streams:upgrade-system-tests-37',
'streams:upgrade-system-tests-38',
'streams:upgrade-system-tests-39',
'streams:upgrade-system-tests-40',
'streams:upgrade-system-tests-41',
'tools',
'tools:tools-api',
'transaction-coordinator',
'trogdor',
KAFKA-17587 Refactor test infrastructure (#18602) This patch reorganizes our test infrastructure into three Gradle modules: ":test-common:test-common-internal-api" is now a minimal dependency which exposes interfaces and annotations only. It has one project dependency on server-common to expose commonly used data classes (MetadataVersion, Feature, etc). Since this pulls in server-common, this module is Java 17+. It cannot be used by ":clients" or other Java 11 modules. ":test-common:test-common-util" includes the auto-quarantined JUnit extension. The @Flaky annotation has been moved here. Since this module has no project dependencies, we can add it to the Java 11 list so that ":clients" and others can utilize the @Flaky annotation ":test-common:test-common-runtime" now includes all of the test infrastructure code (TestKitNodes, etc). This module carries heavy dependencies (core, etc) and so it should not normally be included as a compile-time dependency. In addition to this reorganization, this patch leverages JUnit SPI service discovery so that modules can utilize the integration test framework without depending on ":core". This will allow us to start moving integration tests out of core and into the appropriate sub-module. This is done by adding ":test-common:test-common-runtime" as a testRuntimeOnly dependency rather than as a testImplementation dependency. A trivial example was added to QuorumControllerTest to illustrate this. Reviewers: Ismael Juma <ismael@juma.me.uk>, Chia-Ping Tsai <chia7712@gmail.com>
2025-01-24 22:03:43 +08:00
'test-common:test-common-internal-api',
'test-common:test-common-util',
'test-common:test-common-runtime'
project(":storage:api").name = "storage-api"
rootProject.name = 'kafka'