Type query support was removed in #47207. This query will throw an exception in v7 rest api compatibility indicating that the support was removed + deprecation warnings.
In v8 it will not be available and error about type query being not found will be returned.
relates main meta issue #51816
relates types removal meta #54160
synced flush is going to be replaced by flush. This commit allows to synced_flush api only in v7 compatibility mode.
Worth noting - sync_id is gone and won't be available in v7 responses from indices.stats
relates removal pr #50882
relates #51816
the exception message has changed in #55291. This is not covered by rest
api compatibility, so no need to return the old message for v7 requests.
This commit adds a transformation to allow for the 7.x test to pass with
a new exception message
relates #51816
Previously the compatibility layer was always returning an _doc in mappings for get
template.
This commit does not return _doc in empty mappings.
Returning just {} empty object (v7 and v8 behaviour)
also moving term lookups tests which are already fixed (relates #74544)
relates #70966
relates main meta issue #51816
relates types removal meta #54160
previously disallowed in #70209. However since now the recommendation is
to not set the field at all - and rely on a default - this would be a
shape change for users prefering to set -1 so far.
With request compatible with v7 they will still be allowed to do this
and will get a warning.
relates #69548
relates #51816
Warning related transformations missed the possibility to apply per single test only.
Also a warning changed in #67158 for indices.close so this PR also applies the transformation for 7.x test
relates #51816
previously removed in #38514, deprecated in 7.x and defaulted to true.
With rest api compatibility and when value is true it is ignored and warning is emitted
when value is false, an exception is thrown
relates #51816
the adjust_pure_negative value used to be always present in to_xcontent
response, but it was changed in #49543 to only serialise this field when
the value is false
relates #51816
indices upgrade api (/_upgrade or /{index}/_upgrade) was removed and _reindex is suggested to be used instead.
There is no easy way to translate _upgrade request to _reindex requests. The dummy Upgrade action will return an exception to a user with a message indicating that _reindex should be used.
upgrade api removal #64732
relates #51816
this PR removes tests which are not meant to be fixed (ml/, vectors/) to a separate "not to be fixed list" so that we can see which compatible changes are meant to be implemented.
relates #51816
With types removal changes being available under rest api compatibility I have removed the block entries for tests which are already fixed
relates #51816
Previously removed in #46943
parsing type field in term lookup is now possible with rest
compatible api. The type field is ignored
relates main meta issue #51816
relates type removal meta issue #54160
This PR returns the get snapshots API to the 7.x format (and transport client behavior) and enhances it for requests that ask for multiple repositories.
The changes for requests that target multiple repositories are:
* Add `repository` field to `SnapshotInfo` and REST response
* Add `failures` map alongside `snapshots` list instead of returning just an exception response as done for single repo requests
* Pagination now works across repositories instead of being per repository for multi-repository requests
closes#69108closes#43462
This disables the filter-by-filter aggregation optimization used by
`terms`, `range`, `date_histogram`, and `date_range` aggregations unless
we're *sure* that its faster than the "native" implementation. Mostly this
is when the top level query is empty or we can merge it into the filter
generated by the agg rewrite process.
Now that we have hard and fast rules we can drop the cost estimation
framework without too much fear. So we remove it in this change. It
stomps a bunch of complexity. Sadly, without the cost estimation stuff
we have to add a separate mechanism for blocking the optimization
against runtime fields for which it'd be kind of garbage. For that I
added another rule preventing the filter-by-filter aggregation from
running against the queries made by runtime fields. Its not fool-proof,
but we have control over what queries we pass as a filter so its not
wide open.
I spent a lot of time working on an alternative to this that preserved
that fancy filter-by-filter collection mechanism and was much more kind
to the query cache. It detected cases where going full filter-by-filter
was bad and grouped those filters together to collect in one pass with a
funny ORing collector. It *worked*. And, if we were super concerned with
the performance of the `filters` aggregation it'd be the way to go. But
it was very complex and it was actually slower than using the native
aggregation for things like `terms` and `date_histogram`. It was
glorious. But it was wrong for us. Too complex and optimized the wrong
things.
So here we are. Hopefully this is a fairly simple solution to a sneaky
problem.
previously removed in #55622 use_field_mapping option can be used on doc
value format under rest api compatibility.
The value itself is ignored (replaced with null) as it is a default
behaviour to use field mapping format.
relates https://github.com/elastic/elasticsearch/issues/51816
the per type indexing stats is simplified and when _types is requested
it will return total stats for the index repeated under types/_doc/
the removal #47203
relates main meta issue #51816
relates types removal issue #54160
block list at the moment contains a lot of tests that we already identified that won't be fixed.
This commit moves them out of the main block list so that it is easier to count progress.
This moves the public build api and plugins into a separete included build called 'build-tools'
and we removed the duplication of included buildSrc twice (2nd import as build-tools).
The elasticsearch internal build logic is kept in build-tools-internal as included build which allows us better handling of this project that its just being an buildSrc project (e.g. we can reference tasks directly from the root build etc.)
Convention logic applied to both projects will live in a new build-conventions project.
This change adds support for using `search_after` with field collapsing. When
using these in conjunction, the same field must be used for both sorting and
field collapsing. This helps keep the behavior simple and predictable.
Otherwise it would be possible for a group to appear on multiple pages of
results.
Currently search after is handled directly in `CollapsingTopDocsCollector`. As
a follow-up, we could generalize the logic and move support to the Lucene
grouping framework.
Closes#53115.
Implements a V7 compatible typed endpoints for REST for search related apis
retrofits the REST layer change removed in #41640
relates main meta issue #51816
relates types removal issue #54160
Implements a V7 compatible typed endpoints for REST put and get mapping endpoints. Also for Get Field Mappings endpoints.
retrofits the REST layer change removed in #41676
relates main meta issue #51816
relates types removal issue #54160
This commit allows to use the include_type_name parameter with the compatible rest api.
The support for include_type_name was previously removed in #48632
relates #51816
types removal meta issue #54160
This commit reenables the BWC tests after the ML roles were migrated to
server. During the course of that work, the BWC tests were disabled
pending that work being backported to 7.x. Now that that work is not
going to be backported to 7.x, instead we apply some permanent
transformations to the 7.x assertions run against ES in the REST
compatibility tests.
The types removal effort has removed the type from Index API in #47671 and from Get API in #46587
This commit allows to use 'typed' endpoints for the both Index and Get APIs
relates compatible types-removal meta issue #54160
We used to treat setting size to -1 in search request bodies or as a rest
parameter as a no-op, using the default search size of 10 in this case. This
lenient behaviour was deprecated in #69548 and is removed with this PR in 8.0.
Relates to #69548
A #68808 introduced a possibility to declare fields which will be only available to parsing when a compatible API was used.
This commit replaces deprecated log with compatible logging when a 'compatible only' field was used. Also includes a refactoring of LoggingDeprecationHandler method names
relates #51816
This allows many of the optimizations added in #63643 and #68871 to run
on aggregations with sub-aggregations. This should:
* Speed up `terms` aggregations on fields with less than 1000 values that
also have sub-aggregations. Locally I see 2 second searches run in 1.2
seconds.
* Applies that same speedup to `range` and `date_histogram` aggregations but
it feels less impressive because the point range queries are a little
slower to get up and go.
* Massively speed up `filters` aggregations with sub-aggregations that
don't have a `parent` aggregation or collect "other" buckets. Also
save a ton of memory while collecting them.
The endpoint `_snapshottable_features` is long and implies incorrect
things about this API - it is used not just for snapshots, but also for
the upcoming reset API. Following discussions on the team, this commit
changes the endpoint to `_features` and removes the connection between
this API and snapshots, as snapshots are not the only use for the output
of this API.
This ports the majority of the rest integ tests tasks to use the task avoidance api.
- There are some edge cases left that we need to investigate, but we can do that separately.
This PR implements value fetching for the following field types:
* `text` phrase and prefix subfields
* `search_as_you_type`, plus its subfields
* `token_count`, which is implemented by fetching doc values
Supporting these types helps ensure that retrieving all fields through
`"fields": ["*"]` doesn't fail because of unsupported value fetchers.
The build plugin is still necessary to generate the POM file.
This commit adds the build plugin plugin back to the rest-api-spec
project. It was recently removed as it was thought to be
unnecessary.
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.
The remaining cases in modules, plugins, and x-pack will be handled in followups.
This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.
The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.
Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).
As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.
Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
This is another part of the breakup of the massive BuildPlugin. This PR
moves the code for configuring publications to a separate plugin. Most
of the time these publications are jar files, but this also supports the
zip publication we have for integ tests.
The rest spec and documentation for _cat/threadpool supports a "size" parameter.
However, the "size" parameter will have no impact since there are no values
of type "SizeValue" of the return value of this _cat api.
This commit removes the "size" param from the spec and documentation.
This commit also adds support for the "time" param since and support to
format the time param for the "keep_alive" column. By default, the output
should not change since the "TimeValue" rendered default (via RestTable)
is toString(), and the code prior to this also called toString().
closes#54478
A JSON schema was recently introduced for the REST API specification. #54252
This PR introduces a 3rd party validation tool to ensure that the
REST specification conforms to the schema.
The task is applied to the 3 projects that contain REST API specifications.
The plugin wires this task into the precommit commit task, and should be
considered as part of the public API for the build tools for any plugin
developer to contribute their plugin's specification.
An ignore parameter has been introduced for the task to allow specific
file to be ignored from the validation. The ignored files in this PR
will soon get issues logged and a link so they can be fixed.
Closes#54314
Prior to this commit Watcher explicitly copied test between two
projects with a copy task. This commit removes the explicit copy in favor
of adding the Watcher tests to the available restResources that may be
copied between projects.
This is how inter-project dependencies should be modeled. However, only
Watcher is included here since it is (currently) the only project with
inter-project test dependencies.
Note - this re-introduces: commit: 4f48e053f9
with some additional fixes.
This PR addresses the unnecessary copying of the rest specs and allows
for better semantics for which specs and tests are copied. By default
the rest specs will get copied if the project applies
`elasticsearch.standalone-rest-test` or `esplugin` and the project
has rest tests or you configure the custom extension `restResources`.
This PR also removes the need for dozens of places where the x-pack
specs were copied by supporting copying of the x-pack rest specs too.
The plugin/task introduced here can also copy the rest tests to the
local project through a similar configuration.
The new plugin/task allows a user to minimize the surface area of
which rest specs are copied. Per project can be configured to include
only a subset of the specs (or tests). Configuring a project to only
copy the specs when actually needed should help with build cache hit
rates since we can better define what is actually in use.
However, project level optimizations for build cache hit rates are
not included with this PR.
Also, with this PR you can no longer use the includePackaged flag on
integTest task.
The following items are included in this PR:
* new plugin: `elasticsearch.rest-resources`
* new tasks: CopyRestApiTask and CopyRestTestsTask - performs the copy
* new extension 'restResources'
```
restResources {
restApi {
includeCore 'foo' , 'bar' //will include the core specs that start with foo and bar
includeXpack 'baz' //will include x-pack specs that start with baz
}
restTests {
includeCore 'foo', 'bar' //will include the core tests that start with foo and bar
includeXpack 'baz' //will include the x-pack tests that start with baz
}
}
```
This commit replaces the existing RandomizedTestingTask and supporting code with Gradle's built-in JUnit support via the Test task type. Additionally, the previous workaround to disable all tasks named "test" and create new unit testing tasks named "unitTest" has been removed such that the "test" task now runs unit tests as per the normal Gradle Java plugin conventions.
This commit replaces the existing RandomizedTestingTask and supporting code with Gradle's built-in JUnit support via the Test task type. Additionally, the previous workaround to disable all tasks named "test" and create new unit testing tasks named "unitTest" has been removed such that the "test" task now runs unit tests as per the normal Gradle Java plugin conventions
We put the rest api spec into a jar for upload to maven, so that we can
use within external rest tests. This change adds making a pom for maven
(as well as producing sources and javadoc jars, even though they will be
empty, because maven central requires them).
Running `gradle install` on the rest-api-spec fails because there is no available install
task. This change applies the nexus plugin so that we can install and should also enable
publishing as part of the uploadArchives task.
Przemyslaw Gomulka
Armin Braun
Nik Everett
Rene Groeschke
Julie Tibshirani
Joe Gallo
David Turner
Jason Tedor
Jake Landis
Mark Vieira
Christoph Büscher
Gordon Brown
Ryan Ernst
Alpar Torok
Ryan Ernst
jaymode