synced flush is going to be replaced by flush. This commit allows to synced_flush api only in v7 compatibility mode.
Worth noting - sync_id is gone and won't be available in v7 responses from indices.stats
relates removal pr #50882
relates #51816
* Adding shard count to _nodes/stats api
Added a shards section to each node returned by the _nodes/stats api. Currently this new section only contains a total count of all shards on the node.
the exception message has changed in #55291. This is not covered by rest
api compatibility, so no need to return the old message for v7 requests.
This commit adds a transformation to allow for the 7.x test to pass with
a new exception message
relates #51816
Adds formal API docs for the following APIs:
* Clear SQL cursor
* SQL search
* SQL translate
Other changes:
* Removes and redirects the "Supported REST parameters section." This is now covered in the SQL search API docs.
* Updates a few related xrefs.
Closes#75085
Date histogram interval parameter was deprecated in 7.2, in favor of the more specific fixed_interval and calendar_interval parameters. The old logic used some poorly understood guessing to decide if it should operate in fixed or calendar mode. The new logic requires a specific choice by the user, which is more explicit. In 7.x REST compatibility mode, we will parse the interval as calendar if possible, and otherwise interpret it as fixed.
Previously the compatibility layer was always returning an _doc in mappings for get
template.
This commit does not return _doc in empty mappings.
Returning just {} empty object (v7 and v8 behaviour)
also moving term lookups tests which are already fixed (relates #74544)
relates #70966
relates main meta issue #51816
relates types removal meta #54160
previously disallowed in #70209. However since now the recommendation is
to not set the field at all - and rely on a default - this would be a
shape change for users prefering to set -1 so far.
With request compatible with v7 they will still be allowed to do this
and will get a warning.
relates #69548
relates #51816
* Add missing repositories API REST specs
A new experimental X-Pack API was added for repositories, but is missing a REST spec. The API was added in #60371
* Move this under the nodes namespace
* Rename max_version_to_clear parameter
* Update docs URL to use current
Fixes a broken link in the `documentation.url` for the field usage stats API and
enroll Kibana API.
These broken links caused the build for the JS client docs to fail.
Warning related transformations missed the possibility to apply per single test only.
Also a warning changed in #67158 for indices.close so this PR also applies the transformation for 7.x test
relates #51816
Since the index may have more than one shard, we can't always predict how many
documents will fall in each slice. This PR simply removes the checks, since we
already test the slicing logic extensively in an integration test. The REST test
is now just a sanity check for the API.
Fixes#75212.
Adds a field usage API that reports shard-level statistics about which Lucene fields have been accessed, and which
parts of the Lucene data structures have been accessed.
Field usage statistics are automatically captured when queries are runnning on a cluster. A shard-level search request
that accesses a given field, even if multiple times during that request, is counted as a single use.
Adds formal API docs and JSON specs for the following APIs:
* Get async SQL search
* Get async SQL search status
* Delete async SQL search
Closes#74845
previously removed in #38514, deprecated in 7.x and defaulted to true.
With rest api compatibility and when value is true it is ignored and warning is emitted
when value is false, an exception is thrown
relates #51816
the adjust_pure_negative value used to be always present in to_xcontent
response, but it was changed in #49543 to only serialise this field when
the value is false
relates #51816
indices upgrade api (/_upgrade or /{index}/_upgrade) was removed and _reindex is suggested to be used instead.
There is no easy way to translate _upgrade request to _reindex requests. The dummy Upgrade action will return an exception to a user with a message indicating that _reindex should be used.
upgrade api removal #64732
relates #51816
This PR adds support for using the `slice` option in point-in-time searches. By
default, the slice query splits documents based on their Lucene ID. This
strategy is more efficient than the one used for scrolls, which is based on the
`_id` field and must iterate through the whole terms dictionary. When slicing a
search, the same point-in-time ID must be used across slices to guarantee the
partitions don't overlap or miss documents.
Closes#65740.
this PR removes tests which are not meant to be fixed (ml/, vectors/) to a separate "not to be fixed list" so that we can see which compatible changes are meant to be implemented.
relates #51816
Changes:
* Adds a tutorial for search templates.
* Adds reference docs for the render search template API.
* Improves parameter documentation for the multi search template API.
* Removes duplicate examples from the search template API, multi search API, and create stored script API docs.
* Splits the source files for the search template API and the multi search template API docs.
This adds support for a `dry_run` parameter for the
`_ilm/migrate_to_data_tiers` API. This defaults to `false`, but when
configured to `true` it will simulate the migration of elasticsearch
entities to data tiers based routing, returning the entites that need to
be updated (indices, ILM policies and the legacy index template that'd
be deleted, if any was configured in the request).
With types removal changes being available under rest api compatibility I have removed the block entries for tests which are already fixed
relates #51816
The filter by filter terms aggregation optimization only kicks in when
its targeting a non-empty shard. An empty shard is fast to collect no
matter what so there isn't really any need to do anything complex.
Anyway, this locks the test to a single sharded index so there isn't a
chance of the debugging data coming back from an empty shard. Which
would cause the test to fail because its expecting the optimization to
run.
Closes#74612
This adds the _ilm/migrate_to_data_tiers API to expose the service for
migrating the elasticsearch abstractions (indices, ILM policies and an
optional legacy template to delete) to data tiers routing allocation
(away from custom node attributes)
Previously removed in #46943
parsing type field in term lookup is now possible with rest
compatible api. The type field is ignored
relates main meta issue #51816
relates type removal meta issue #54160
This PR returns the get snapshots API to the 7.x format (and transport client behavior) and enhances it for requests that ask for multiple repositories.
The changes for requests that target multiple repositories are:
* Add `repository` field to `SnapshotInfo` and REST response
* Add `failures` map alongside `snapshots` list instead of returning just an exception response as done for single repo requests
* Pagination now works across repositories instead of being per repository for multi-repository requests
closes#69108closes#43462
This disables the filter-by-filter aggregation optimization used by
`terms`, `range`, `date_histogram`, and `date_range` aggregations unless
we're *sure* that its faster than the "native" implementation. Mostly this
is when the top level query is empty or we can merge it into the filter
generated by the agg rewrite process.
Now that we have hard and fast rules we can drop the cost estimation
framework without too much fear. So we remove it in this change. It
stomps a bunch of complexity. Sadly, without the cost estimation stuff
we have to add a separate mechanism for blocking the optimization
against runtime fields for which it'd be kind of garbage. For that I
added another rule preventing the filter-by-filter aggregation from
running against the queries made by runtime fields. Its not fool-proof,
but we have control over what queries we pass as a filter so its not
wide open.
I spent a lot of time working on an alternative to this that preserved
that fancy filter-by-filter collection mechanism and was much more kind
to the query cache. It detected cases where going full filter-by-filter
was bad and grouped those filters together to collect in one pass with a
funny ORing collector. It *worked*. And, if we were super concerned with
the performance of the `filters` aggregation it'd be the way to go. But
it was very complex and it was actually slower than using the native
aggregation for things like `terms` and `date_histogram`. It was
glorious. But it was wrong for us. Too complex and optimized the wrong
things.
So here we are. Hopefully this is a fairly simple solution to a sneaky
problem.
previously removed in #55622 use_field_mapping option can be used on doc
value format under rest api compatibility.
The value itself is ignored (replaced with null) as it is a default
behaviour to use field mapping format.
relates https://github.com/elastic/elasticsearch/issues/51816
This is a quality of life improvement for typical users. Almost all anomaly jobs will receive their data through a datafeed.
The datafeed config can now be supplied and is available in the datafeed field in the job config for creation and getting jobs.
If the after key cannot parse back to the same value we generated it from, when the client sends it back we will land on a wrong page. If this page is earlier, it is likely we will (eventually) generate the same wrong after key, resulting in an infinite loop as the client repeatedly ask to retrieve the same page or pages of data. This change fixes that by failing the composite aggregation (with an exception) if the after key is unparsable with the given format. We provide as much information about what failed as possible.
Keith Massey
Przemyslaw Gomulka
Lisa Cawley
Dan Hermann
James Rodewig
Mark Tozzi
Steve Gordon
Julie Tibshirani
Yannick Welsch
Dimitris Athanasiou
Julie Tibshirani
Nhat Nguyen
Andrei Dan
Nik Everett
Armin Braun
Ioannis Kakavas
Benjamin Trent