This commit updates two task names:
```
yamlRestCompatTest -> yamlRestTestV7CompatTest
transformV7RestTests -> yamlRestTestV7CompatTransform
```
`7` is the N-1 version and calculated, such that when `8` is
N-1 version the task names will be `yamlRestTestV8CompatTest` and
`yamlRestTestV8CompatTransform`
The motivation for `yamlRestCompatTest -> yamlRestTestV7CompatTest` is that
many projects have configured `yamlRestCompatTest`
but that configuration is specific to the N-1 version. For example,
if we blacklist tests when running compatibility with v7, we don't also
want to blacklist those tests when running compatibility with v8.
By introducing a version-specific identifier in the name, the task will not
even exist when bumping the version creating the need to (correctly) remove
the version-specific condition.
The motivation for `transformV7RestTests -> yamlRestTestV7CompatTransform`
is to provide more consistent naming.
The idea behind the naming is the main task people
are likely familiar with is :
`yamlRestTest` so we will use that as a base.
`yamlRestTestV7CompatTest` to run the version-specific compat tests
`yamlRestTestV7CompatTransform` to run the version-specific transformations for the compat tests
CI should be un-effected since since we introduced a lifecycle task
name `checkRestCompat` which is what CI should be configured to use.
This new parameter is a boolean parameter that allows
users to put in a compressed model without it having
to be inflated on the master node during the put
request
This is useful for system/module set up and then later
having the model validated and fully parsed when it
is being loaded on a node for usage
This commit allows the compatible REST API tests to execute on Windows.
They were previously excluded from Windows due to a command line limit
when defining a very large exclusion list. That exclusion list is much
smaller now and they will now execute properly on Windows.
Also, an empty exclusion list has been removed from the build config.
This adds the pattern into the error message returned when trying to
fetch fields. So this:
```
POST _search {
"fields": [ { "field": "*", "format": "date_time" } ]
}
```
Will return an error message like
```
error fetching [foo] which matches [*]: Field [foo] of type [keyword] doesn't support formats
```
This introduces a basic public yaml rest test plugin that is supposed to be used by external
elasticsearch plugin authors. This is driven by #76215
- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins
Co-authored-by: Mark Vieira <portugee@gmail.com>
Adds the REST API /_ml/trained_models/{model_id}/deployment/_stats.
The API reports stats for live deployments -those that are started and running
- such as on which nodes the model(s) are deployed, avg inference
time and the inference count
This commit introduces into the node stats API various statistics to
track the time that the elected master spends in various phases of the
cluster state publication process.
Relates #76625
When building empty responses for shards that don't have the term field in question,
significant terms ignored the background filter.
This commit fixes this bug by respecting the background filter count, even
when building empty results.
closes#76729
v7compatibilityNotSupportedTests was introduced to make it easier to
track tests that have been identified as not needing compatible changes
and those that still need to be checked.
We have checked all tests now and the separate list is no longer needed.
relates #51816
relates #73912
This commit extends the `inference_processor` to support allocated
models. In particular, the internal infer action is now checking
if the model is allocated, and if so, it redirects the request
to an instance of the allocated model. If not, then it proceeds
as previously to load the model through the `ModelLoadingService`.
In addition, we now check that a model is not allocated when
the `StopTrainedModelDeploymentAction`.
Note that no new `InferenceConfigUpdate` objects are introduced
as there are no settings currently that can be set on inference time
for allocated models.
* Gracefully nandle very large sizes on terms
Folks often as for the `terms` agg to have *very* large `size`
parameters in an effort to get everything they can. But they never fill
that large size. They can't! There isn't really enough heap to return
two billion buckets.
After #74096 we try to pre-allocate an array of `size + 1` length,
regardless of how many results are returned. When folks ask for
`MAX_INT` buckets this helpfully fails off in lucene land with an error
message that fairly esoterit to someone reading it outside of
Elasticsearch.
This change handles that `MAX_INT` case by building a
`TopBucketsBuilder` designed to handle large max sizes that are rarely
filled. When you ask for 1024 or more buckets you get that one instead
and we don't preallocate the entire array for the reduction. You get the
old preallocated one when you have few buckets.
Closes#76492
* fixup skip
* Moar comment
Adds new field to recovery API to keep track of amount of data
recovered from snapshots.
The normal recovered_bytes field remains and is also increased for
recovery from snapshot but can go backwards in the unlikely case
that recovery from snapshot fails to download a file.
Relates #73496
This change introduces a Service Account for Kibana to use when
authenticating to Elasticsearch. The Service Account with
kibana service name under the elastic namespace,
uses the same RoleDescriptor as the existing kibana_system
built-in user and is its functional equivalent when it comes to
AuthZ.
Previously removed in #42654. The query and the parameter won't work under rest api compatibility and an exception with a message is returned advising that just use of match/multi_match is enough
relates #51816
Type query support was removed in #47207. This query will throw an exception in v7 rest api compatibility indicating that the support was removed + deprecation warnings.
In v8 it will not be available and error about type query being not found will be returned.
relates main meta issue #51816
relates types removal meta #54160
Today we often encounter users that are confused by the behaviour of
calling `GET _cluster/allocation/explain` without a body: it _seems_ to
work, but it explains a random shard, and if this isn't the shard
they're thinking of then it's unclear how to proceed.
With this commit we add a note to the response when a shard was randomly
chosen indicating that it is possible, and possibly useful, to explain a
different shard. We also adjust the exception message in the case when
all shards are assigned to indicate why it's an invalid request and what
to do to make it valid.
synced flush is going to be replaced by flush. This commit allows to synced_flush api only in v7 compatibility mode.
Worth noting - sync_id is gone and won't be available in v7 responses from indices.stats
relates removal pr #50882
relates #51816
* Adding shard count to _nodes/stats api
Added a shards section to each node returned by the _nodes/stats api. Currently this new section only contains a total count of all shards on the node.
the exception message has changed in #55291. This is not covered by rest
api compatibility, so no need to return the old message for v7 requests.
This commit adds a transformation to allow for the 7.x test to pass with
a new exception message
relates #51816
Adds formal API docs for the following APIs:
* Clear SQL cursor
* SQL search
* SQL translate
Other changes:
* Removes and redirects the "Supported REST parameters section." This is now covered in the SQL search API docs.
* Updates a few related xrefs.
Closes#75085
Date histogram interval parameter was deprecated in 7.2, in favor of the more specific fixed_interval and calendar_interval parameters. The old logic used some poorly understood guessing to decide if it should operate in fixed or calendar mode. The new logic requires a specific choice by the user, which is more explicit. In 7.x REST compatibility mode, we will parse the interval as calendar if possible, and otherwise interpret it as fixed.
Previously the compatibility layer was always returning an _doc in mappings for get
template.
This commit does not return _doc in empty mappings.
Returning just {} empty object (v7 and v8 behaviour)
also moving term lookups tests which are already fixed (relates #74544)
relates #70966
relates main meta issue #51816
relates types removal meta #54160
previously disallowed in #70209. However since now the recommendation is
to not set the field at all - and rely on a default - this would be a
shape change for users prefering to set -1 so far.
With request compatible with v7 they will still be allowed to do this
and will get a warning.
relates #69548
relates #51816
* Add missing repositories API REST specs
A new experimental X-Pack API was added for repositories, but is missing a REST spec. The API was added in #60371
* Move this under the nodes namespace
* Rename max_version_to_clear parameter
* Update docs URL to use current
Fixes a broken link in the `documentation.url` for the field usage stats API and
enroll Kibana API.
These broken links caused the build for the JS client docs to fail.
Warning related transformations missed the possibility to apply per single test only.
Also a warning changed in #67158 for indices.close so this PR also applies the transformation for 7.x test
relates #51816
Since the index may have more than one shard, we can't always predict how many
documents will fall in each slice. This PR simply removes the checks, since we
already test the slicing logic extensively in an integration test. The REST test
is now just a sanity check for the API.
Fixes#75212.
Adds a field usage API that reports shard-level statistics about which Lucene fields have been accessed, and which
parts of the Lucene data structures have been accessed.
Field usage statistics are automatically captured when queries are runnning on a cluster. A shard-level search request
that accesses a given field, even if multiple times during that request, is counted as a single use.
Adds formal API docs and JSON specs for the following APIs:
* Get async SQL search
* Get async SQL search status
* Delete async SQL search
Closes#74845
Jake Landis
Nik Everett
Benjamin Trent
Seth Michael Larson
Philip Krauss
Henning Andersen
Rene Groeschke
Przemysław Witek
David Kyle
David Turner
Benjamin Trent
Przemyslaw Gomulka
Dimitris Athanasiou
Yang Wang
Francisco Fernández Castaño
Ioannis Kakavas
Armin Braun
Joe Gallo
James Rodewig
Keith Massey
Lisa Cawley
Dan Hermann
Mark Tozzi
Steve Gordon
Julie Tibshirani
Yannick Welsch