Gradle now fully supports compiling, testing and running on Java 20.
Among other general performance improvements this release introduces --test-dry-run command line option that allows checking if tests are filtered or not by gradle.
Required updating nebula ospackage plugin as setuid was broken in gradle 8.3.
Lots of spots where we did weird things around streams like redundant stream creation, redundant collecting
before adding all the collected elements to another collection or so, redundant streams for joining strings
and using less efficient `Collectors.toList` and in a few cases also incorrectly relying on the result being mutable.
Using gradle toolchain support in gradle requires refactoring how the composite build is composed.
We added three toolchain resolver
1. Resolver for resolving defined bundled version from oracle as openjdk
2. Resolve all available jdks from Adoption
3. Resolve archived Oracle jdk distributions.
We should be able to remove the JdkDownloadPlugin altogether without having that in place, but we'll do that in a separate effort.
Fixes#95094
Version.java currently contains mappings to the Lucene version for each
Elasticsearch version. The only use of this in the build logic is for
filtering based on index compatibility. However, that compatibility can
be inferred based on the Elasticsearch major version since there is a
one to one mapping between Elasticsearch major and Lucene major. This
commit removes Lucene version extraction from the build bwc logic.
Fixes#82794. Upgrade the spotless plugin, which addresses the issue
around formatting `instanceof` expressions. Formatting of statements
including lambdas seems to have improved too.
- Remove custom checksum build logic in wrapper task
- Adjust jdk home handling adjusting the change in behaviour in gradle. Requires providing canonical paths for provisioned jdk homes.
- Fix test by add workaround to bug in configuration cache
This updates the gradle wrapper to 7.6.1. This patch release contains a
fix for incremental compilation of java modules we have raised against
gradle 7.6
see https://github.com/gradle/gradle/issues/23067
in example plugins a pluginApiVersion is more appropriate than elasticsearchVersion as it better implies that it can be different then the version of ES cluster
The previous model relied on treating the server's certificate
configuration as a trust anchor. This isn't guaranteed to work,
which lead to needing to support "certificate_authorities" as an
alternative, which in turn polluted the node's config with settings
that only existed to enable tests to run.
The new model ties the "wait-for-health" HTTP client to the leaf
certificates themselves. This means that it will always connect to a
node that has the exact certificates it expects, and doesn't rely on
knowing the issuer of the node's certificate.
This PR adds settings and infrastructure to support a new Remote Cluster port,
to be used in Remote Cluster Security 2.0. Specifically, this commit adds new
settings that allow opening a new Remote Cluster port, which will eventually
exclusively support the new cross-cluster authentication method. Since support
for that new authentication method is still under construction, these settings
are hidden behind a feature flag.
The new settings cover all Transport profile settings, to ensure that users will
not have to be exposed to Transport Profiles directly to make use of RCS2.0
functionality. This includes all core settings, as well as IP filter and SSL
configuration.
Co-authored-by: Yang Wang <yang.wang@elastic.co>
Refactoring that drops the api suffix from package name
This will have to be followed up by a plugins/examples fix in imports
Also set an artifact group name to `org.elasticsearch.plugin` in the plugin-api and plugin-analysis-api
new stable plugins require generated named_components.json file which contains all analysis components implemented by this plugin. The generation is currently done in build-tools by elasticsearch.stable-esplugin
However this makes the generation only available for plugins using gradle. Plugin developers using maven or other building tooling will not be able to use it.
This commits refactors the scanning logic into libs:plugin-scanner which will allow for plugin install command to perform the scanning too.
relates #88980
This commit adds a new test framework for configuring and orchestrating
test clusters for both Java and YAML REST testing. This will eventually
replace the existing "test-clusters" Gradle plugin and the build-time
cluster orchestration.
* Fix TestResultProcessor api changes
* Fix inputs for generateProviderManifest
* Ignore tests for now until gradle has fixed reporting issue
* Fix dependency substitution in example plugins build
* Use right java bin path on windows
* Add hint to task onlyif when no docker is available
This commit no longer explicitly sets the default configuration for FIPS tests.
This allows each project's tests to run in FIPS mode with out deviation (other
than the FIPS mode).
A side product of this change is that any REST test
can now enable security if they so choose without needing to use the default
distribution. This allows for additional usage of the integ_test distribution
which can help with testing modularization.
This only possible now that the security plugin is always included
with the integ_test distribution via #77632fixes: #70005
related: #77632
Stable plugins do not use className, extendedPlugins and hasNativeController properties. They also don't necessarily have moduleName.
These properties should not be present in stable-plugin-descriptor, even if they are empty because the parsing will fail complaining about unused properties.
Old plugins can use these properties, but they use defaults when the property is not present These defaults are:
has.native.controller = false
extended.plugins = emptyList
moduleName = null
classname is required and will throw exception when old plugin is not configuring this property
relates #88980
With the new StableBuildPlugin we do not add default dependencies to the plugin classpath.
That exposed an issue with the JarHellPlugin not really taking care of configuring the
jarHell configuration.
The JarHell Plugin only worked so far as the plugin build plugin has added a transitive dependency
to elasticsearch-core and therefore kind of only worked by accident-.
This adds a default configuration of the jarHell configuration that can be overridden manually.
Furthermore we clear some inter plugin dependencies explicit and add proper functional
test coverage
New ES stable plugins when built should have a stable-plugin-descriptor.properties file instead of plugin-descriptor.properties.
New plugins also do not use classname property in the plugin descriptor
new plugin will also scan classes and libraries for @NamedComponents and will create named_components.json file. That file contains a map of Extensible interface (like TokenizerFactory) to a map of "component name" to "className"
This commit extracts common logic from PluginBuildPlugin into BasePluginBuildPLugin so that it can also be used by StableBuildPlugin
the differences are:
classaname - used in old plugin, but not in the new one (stable)
the plugin descriptor file name - the new one has stable-plugin-descriptor.properties
dependencies - the new plugin does not need elasticsearch as a dependency.
We might want to consider if we want to add test framework dependency in the future.
relates #88980
When including a build which applies the global build info plugin in a
composite we don't want to print this header out. It's like that a) it's
non-applicable to the outer build, or b) the outer build will also apply
this plugin resulting in duplicate output.
This commit adds TLS for the transport layer and optional support
for https for `./gradlew run` and `./gradlew run-ccs` tasks.
A new system property --https can be passed to expose https.
Transport layer TLS is always enabled which is always applicable
for `./gradlew run-css`, but only applicable for `./gradlew run` when multiple nodes are configured.
Additional certificates and instructions for use are also included in the commit.
This commit reverts the changes introduced via https://github.com/elastic/elasticsearch/pull/89442 / 20ed7e3fd9
and re-implements the same goals but with fewer changes. Also included in this commit
is the ability to change the proxyMode via system property and mention of this
new run task in the testing documentation.
This commit makes minor changes to support multiple nodes
via ./gradlew run. The specific desired use case, where there 2 nodes
in the cluster where one is a coordinating only node is used as
the example configuration.
This commit introduces a `./gradlew run-ccs` task with the following goals:
* mirror the ease of use of `./gradlew run` for manual cross cluster search development
* same credentials
* same well known ports
* uses ccs specific naming
* enable debugging across both clusters
This is admittedly kinda hacky. Test clusters have support multi-cluster and are in use for
for automated testing. There are some nuances that make that setup (and this setup)
a bit cumbersome..specifically needing to read one cluster's config to configure another cluster.
The run task adds a bit more config (well defined ports, etc.) than the tests need to
so that also complicates this abit more. I found that without the additions here I was unable to get both
sharing of cluster configuration (like in the [tests](https://github.com/elastic/elasticsearch/blob/main/qa/ccs-common-rest/build.gradle#L55))
and the run task's hard coded config to work well together. Hopefully the additions to the run task are not
too hacky as I could not find any other way.
- keep guh separated from test project dir
- unify folder handling
as a side effect when keeping the project dir for debugging failed tests we do not copy the whole GUH which usually isn't providing any additional help for debugging those
This simplifies our debugging logic we have in place to keep the test
build environment for failed tests, making debugging easier without
dealing with magic flags
Add retry logic for cleanup / deletion in testcluster's
ElasticsearchNode, to tolerate the asynchronous nature of deletions
on the Windows file-system.
We only rely on the checkstyle version in the buildLibs.toml gradle version catalogue with this change.
Also added some hints for gradle best practices.
This is an aftermath of #88283
Przemyslaw Gomulka
Rene Groeschke
Brian Seeders
David Turner
Armin Braun
Ryan Ernst
Rory Hunter
Mark Vieira
Tim Vernum
Gordon Brown
Jake Landis
Chris Hegarty