This commit updates two task names:
```
yamlRestCompatTest -> yamlRestTestV7CompatTest
transformV7RestTests -> yamlRestTestV7CompatTransform
```
`7` is the N-1 version and calculated, such that when `8` is
N-1 version the task names will be `yamlRestTestV8CompatTest` and
`yamlRestTestV8CompatTransform`
The motivation for `yamlRestCompatTest -> yamlRestTestV7CompatTest` is that
many projects have configured `yamlRestCompatTest`
but that configuration is specific to the N-1 version. For example,
if we blacklist tests when running compatibility with v7, we don't also
want to blacklist those tests when running compatibility with v8.
By introducing a version-specific identifier in the name, the task will not
even exist when bumping the version creating the need to (correctly) remove
the version-specific condition.
The motivation for `transformV7RestTests -> yamlRestTestV7CompatTransform`
is to provide more consistent naming.
The idea behind the naming is the main task people
are likely familiar with is :
`yamlRestTest` so we will use that as a base.
`yamlRestTestV7CompatTest` to run the version-specific compat tests
`yamlRestTestV7CompatTransform` to run the version-specific transformations for the compat tests
CI should be un-effected since since we introduced a lifecycle task
name `checkRestCompat` which is what CI should be configured to use.
`AbstractFilteringTestCase` had a ton of carefully formatted
`XContentBuilder` calls that would have been totally unreadable after
the formatter worked its magic. So I formatted a bunch of them by hand
and extracted the rest to json files.
In a number of places, we read and write binary data into byte arrays using lucene's
DataInput and DataOutput abstractions. In lucene 9 these abstractions are changing
the endianness of their read/writeInt methods. To avoid dealing with this formatting
change, this commit changes things to use elasticsearch StreamInput/StreamOutput
abstractions instead, which have basically the same API but will preserve endianness.
Relates to #73324
On Windows, rendered Groovy templates contain carriage returns, which
breaks the unit tests and results in them sneaking into the output. Fix
this by rendering into a string and removing the carriage returns.
The changelog generation process currently relies on version
information being present in the changelog YAML descriptors. However,
this makes them difficult to update in some scenarios. For example,
if a PR is merged and subsequently labelled for backporting, our
automation won't update the versions in the changelog YAML.
We can make the process more flexible by removing version data from
the changelog YAML files, and instead inferring the versions from
each changelog YAML file's existence in the git tree at each tag
in the minor series.
This change makes the process more ergonomic for developers, but
harder to test, since I can't simply concoct YAML data for a range
of versions. Instead, I've added a number of unit tests, and tried
to exercise all the relevant parts.
It is now an error to include `versions` the YAML file.
This introduces a basic public yaml rest test plugin that is supposed to be used by external
elasticsearch plugin authors. This is driven by #76215
- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins
Co-authored-by: Mark Vieira <portugee@gmail.com>
* [TEST] Implement HotThreads unit tests
Add unit tests for the internal HotThreads logic for calculating and
sorting threads by CPU, Wait and Blocked "hotness". Also adds tests
for identifying certain threads as idle, as well as supported report
types (e.g. cpu, wait, blocked).
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Closes#74795.
Introduce two Docker image variants for Cloud. The first bundles
(actually installs) the S3, Azure and GCS repository plugins. The
second bundles all official plugins, but only installs the repository
plugins.
Both images also bundle Filebeat and Metricbeat.
The testing utils have been refactored to introduce a `docker`
sub-package. This allows the static `Docker.containerId` to be
shared without needing all the code in one big class. The code for
checking file ownership / permissions has also been refactored to
a more Hamcrest style, using a custom Docker file matcher.
An attempt to apply the spotless plugin everywhere and then disable
where it wasn't appropriate didn't work, and instead everything was
formatted. Revert how we apply the pluing, and use a different approach
to applying extra configuration in build files.
* Reformatting to keep Checkstyle after formatting
* Configure spotless everywhere, and disable the tasks if necessary
* Add XContentBuilder helpers, fix test
* Tweaks
* Add a TODO
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* Update Gradle wrapper to 7.2-rc-1
Fix deprecation warnings on the go
* Remove deprecated lambda based Gradle task actions
* Remove usage of deprecated BasePluginConvention
* Update wrapper to 7.2-rc-2
* Update gradle wrapper to 7.2-rc-3
* Update gradle wrapper to 7.2
The quota aware filesystem was added as a means of allowing
Elasticsearch to track the used space of the underlying filesystem in
virtualized environments. However, the need for it was due to a bug in a
much earlier version of Elasticsearch that always found the underlying
mount and checked it directely for usage. That bug has already been
fixed, so the there is no longer a need for this plugin. This commit
removes the plugin. We should consider separately whether there is still
a need for bootstrap plugins.
closes#70309
* Flip node shutdown feature flag to default to true on snapshot builds
It previously defaulted to false. The setting can still only be set to 'true' on a
non-release (snapshot) build of Elasticsearch.
Relates to #70338
* Handle case where operator privileges are enabled
Part of #67335.
Add tasks for generating release notes, using information stored in files
in the repository:
* `generateReleaseNotes` - generates new release notes, release
highlights and breaking changes
* `validateChangelogs` - validates that all the changelog YAML files are
well-formed (confirm to schema, have required fields depending on the
`type` value)
I also changed `Version` to allow a `v` prefix in relaxed mode
in #50067 for _license the accept_enterprise = false was no longer supported. This
commit allows it under rest compatibility and is showing enterprise licenses as platinum
The same change had to be applied to _xpack endpoint #58217
in #50735 max_resource_units was introduced to be more accurate. For v7 requests, which do not know about enterprise license we will return max_node which will be set from max_resource_units (it assumes that one resource unit is 32GB - which corresponds to 1 node)
relates #51816
This introduces a new cmd line tool that generates the security configuration
for a new node in a new cluster (as opposed to joining an existing cluster).
The security configuration consists of TLS key and certificates, which
are stored in a directory inside the config path, as well as settings appended
to the elasticsearch.yml referencing the aforementioned certs.
Warning related transformations missed the possibility to apply per single test only.
Also a warning changed in #67158 for indices.close so this PR also applies the transformation for 7.x test
relates #51816
This commit introduces replaceValueTextByKeyValue which allows
replacements based on a given key/value pair. This is useful
for REST test transforms that that may not have a well known structure.
This commit also implements this new transform to fix a Watcher compat test.
This commit also fixes a latent bug that would prevent the traversal when multiple
differing transforms matched.
adding a conditions that has to be satisfied in order to apply headers in InjectHeaders.
this will allow to skip the transformation for cat operations
relates #51816
This fixes a regression introduced with #74670.
Vanilla test tasks (of plain type Test) must also dependOn `fipsResources`
when build is run in FIPS mode. Otherwise we see the plain test task failing with
"SHA MessageDigest not available"
Fixes#74922
- avoid eagerly created test cluster
- remove duplicate superflous configuration
- resolve system properties via provider factory
- move common test configuration / setup into rest test base plugin
With the overall theme of trying to configure and add less to the build instead of just disabling it later,
we're replacing standalone-test by standalone-rest tasks avoids creating the
unused test tasks.
Standalone rest test plugin and the other rest test plugins behave a little bit different in the sense how source sets and test tasks are wired.
The standalone rest test plugin assumes that all RestTestTasks are using the same sourceSet (test). The yaml, java Rest test plugins use one dedicated sourceSet per test task.
In the long run we probably will migrate standalone-rest-test usages to one of the other plugins and deprecate standalone-rest-test
The fips specific cluster specification depends on files that are declared
as output of the fips-resources task. So far there has not been an explicit
dependency of all TestClustersAware tasks (so far only for Test tasks) to this
resources task in our build and.
implicit usage of task outputs without any explicit task dependency has been
deprecated in Gradle as the build tool detects that these file are outputs of
this task and urges users to have this explicit dependency declared in a build.
RestIntegTestTask had this dependencies explicitly declared by us having defined a dependency
by declaring
`tasks.withType(Test) { dependsOn 'fipsResources'}`.
This has be replaced by with this PR by
`tasks.withType(TestClustersAware) { dependsOn 'fipsResources'}`
which also covers usages of `DefaultTestClustersTask` as used in the
`:x-pack:plugin:eql:qa:multi-cluster-with-security` project which
fails in #74664fixes#74664
- No manual show task dependencies logic needed. we can use build scans instead for that.
- Move common plugin configuration into according plugins
- Reduce overall callbacks in root script, remove unneeded logic in afterEvaluate hooks
We only need the javaRestTest sourceSet and can avoid main and test sourceSet by
just using the new introduced ElasticsearchJavaBase instead of ElasticsearchJava plugin
Jake Landis
Nik Everett
Rene Groeschke
Armin Braun
Alan Woodward
Rory Hunter
Mark Tozzi
Mark Vieira
Nikola Grcevski
Ryan Ernst
Joe Gallo
Lee Hinman
Przemyslaw Gomulka
Przemko Robakowski
Albert Zaharovits
Nhat Nguyen