https://github.com/elastic/elasticsearch/pull/74201 documents `null` handling to the arg descriptions of several string functions.
This PR moves pre-existing docs for `null` handling and similar edge case handling for string functions to arg descriptions for consistency.
Relates to #74193
Pagination and snapshots for get snapshots API, build on top of the current implementation to enable work that needs this API for testing. A follow-up will leverage the changes to make things more efficient via pagination.
Relates https://github.com/elastic/elasticsearch/pull/73570 which does part of the under-the-hood changes required to efficiently implement this API on the repository layer.
Both of these APIs don't parse request bodies, the parameters are all taken
from the query string. Also, included the master timeout param include
as it was missing here also.
In #74138 we noted that index settings aren't copied in a clone. In fact
that's not true, we copy everything except explicitly-excluded ones,
`number_of_replicas` and `auto_expand_replicas`. This fixes the mistake.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Today if sending file chunks is CPU-bound (e.g. when using compression)
then we tend to concentrate all that work onto relatively few threads,
even if `indices.recovery.max_concurrent_file_chunks` is increased. With
this commit we fork the transmission of each chunk onto its own thread
so that the CPU-bound work can happen in parallel.
Adds a new API that allows a user to reset
an anomaly detection job.
To use the API do:
```
POST _ml/anomaly_detectors/<job_id>_reset
```
The API removes all data associated to the job.
In particular, it deletes model state, results and stats.
However, job notifications and user annotations are not removed.
Also, the API can be called asynchronously by setting the parameter
`wait_for_completion` to `false` (defaults to `true`). When run
that way the API returns the task id for further monitoring.
In order to prevent the job from opening while it is resetting,
a new job field has been added called `blocked`. It is an object
that contains a `reason` and the `task_id`. `reason` can take
a value from ["delete", "reset", "revert"] as all these
operations should block the job from opening. The `task_id` is also
included in order to allow tracking the task if necessary.
Finally, this commit also sets the `blocked` field when
the revert snapshot API is called as a job should not be opened
while it is reverted to a different model snapshot.
It is useful to know the following information when reading datafeed stats:
- Is the datafeed a "real-time" datafeed, i.e. a datafeed without a configured `end` time
- Has the datafeed processed all past data available at the time of starting.
This object is only available if the datafeed task has been created.
It has the form:
```
"running_state": {
"is_real_time": <boolean>,
"look_back_finished": <boolean>
}
```
Changes:
* Removes a reference to the
`-XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode` JVM option. This
option is no longer supported.
* Combines `Xms/Xmx` recommendations for compressed oops.
Closes#71644.
Co-authored-by: Rick Boyd <boyd.richardj@gmail.com>
Adds a new keep_values gap policy that works like skip, except if the metric
calculated on an empty bucket provides a non-null non-NaN value, this value is
used for the bucket.
Fixes#27377
Co-authored-by: Mark Tozzi <mark.tozzi@gmail.com>
Changes:
* Combines the `Document counts are approximate` and `Calculating document count
error` sections.
* Rewrites the section to include `sum_other_doc_count` and
`doc_count_error_upper_bound` for easier on-page (ctrl+f) searching.
Closes#73200
Improve the error message when inconsistent mappings cause doc value formatting errors. For example, trying to format a binary encoded IP address as a UTF8 string often fails with something unexpected, like `ArrayIndexOutOfBounds`. This change catches that and wraps it with a message suggesting the user check their mappings. Also gets rid of anonymous instances for doc value formatters, which made it hard to see what format was failing to be applied.
In #55805, we added a setting to allow single data node clusters to
respect the high watermark. In #73733 we added the related deprecations.
This commit ensures the only valid value for the setting is true and
adds deprecations if the setting is set. The setting will be removed
in a future release.
Co-authored-by: David Turner <david.turner@elastic.co>
This commit adds a short note to the docs on repository backups
indicating that the repository must not be modified while registered, so
that a restore from a repository backup must complete before
registration.
Relates #73730
This adds a new pipeline aggregation for calculating Kolmogorov–Smirnov test for a given sample and buckets path.
For now, the buckets path resolution needs to be `_count`. But, this may be relaxed in the future.
It accepts a parameter `fractions` that indicates the distribution of documents from some other pre-calculated sample.
This particular version of the K-S test is Two-sample, meaning, it calculates if the `fractions` and the distribution of `_count` values in the buckets_path are taken from the same distribution.
This in combination with the hypothesis alternatives (`less`, `greater`, `two_sided`) and sampling logic (`upper_tail`, `lower_tail`, `uniform`) allow for flexibility and usefulness when comparing two samples and determining the likelihood of them being from the same overall distribution.
Usage:
```
POST correlate_latency/_search?size=0&filter_path=aggregations
{
"aggs": {
"buckets": {
"terms": { <1>
"field": "version",
"size": 2
},
"aggs": {
"latency_ranges": {
"range": { <2>
"field": "latency",
"ranges": [
{ "to": 0.0 },
{ "from": 0, "to": 105 },
{ "from": 105, "to": 225 },
{ "from": 225, "to": 445 },
{ "from": 445, "to": 665 },
{ "from": 665, "to": 885 },
{ "from": 885, "to": 1115 },
{ "from": 1115, "to": 1335 },
{ "from": 1335, "to": 1555 },
{ "from": 1555, "to": 1775 },
{ "from": 1775 }
]
}
},
"ks_test": { <3>
"bucket_count_ks_test": {
"buckets_path": "latency_ranges>_count",
"alternative": ["less", "greater", "two_sided"]
}
}
}
}
}
}
```
This commit adds the ability to specify exclusion patterns in Auto-Follow patterns. This allows excluding indices that match any of the inclusion patterns and also match some of the exclusion patterns giving more fine grained control in scenarios where this is important.
Related #67686
The value of `*.ssl.client_authentication` is `required` for
everything except `xpack.security.http.ssl.client_authentication`, for
which is it `none`.
The doc template for this setting was configured to have a default
value, and allow an override. However, the default was set to `none`
when it should have been `required`.
The override for `http` was correctly set to `none` (but that didn't
really do anything, since that was the same as the default).
This commit changes the default to `required`, which matches the code
(see `XPackSettings.CLIENT_AUTH_DEFAULT`), and leaves the override for
http as `none` (see `XPackSettings.HTTP_CLIENT_AUTH_DEFAULT`).
* Add new thread pool for critical operations
* Split critical thread pool into read and write
* Add POJO to hold thread pool names
* Add tests for critical thread pools
* Add thread pools to data streams
* Update settings for security plugin
* Retrieve ExecutorSelector from SystemIndices where possible
* Use a singleton ExecutorSelector
* [DOCS] Add retriving from flattened fields
* Clarify sub-field syntax
* Moving sub-field retrieval to flattened field docs
* Remove full example and de-emphasize runtime fields
* Remove extraneous sample tag
James Rodewig
Nik Everett
bellengao
Armin Braun
Adam Locke
François-Clément Brossard
Nicole Albee
David Turner
ymao1
Dimitris Athanasiou
Stef Nestor
Benjamin Trent
Igor Motov
István Zoltán Szabó
Mark Tozzi
Daisuke Harada
Henning Andersen
Glen Smith
Francisco Fernández Castaño
Tim Vernum
Lisa Cawley
William Brafford