* [DOCS] New landing page
* Add links for alerting, painless, and plugins
* Change 'Get started with Elastic Cloud' into 'Get started'
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
The docs introduced in #91333 apply to older versions in which the `systemd`
startup timeout was 75s by default, but in #91338 we extended the `systemd`
startup timeout to 900s from 8.7.0 and 7.17.8 onwards. This commit adjusts the
docs to match.
Synthetic _source's array flattening activities can remove some arrays
entirely. Specifically:
```
{
"foo": [
{
"bar": 1
},
{
"baz": 2
}
]
}
```
Turns into:
```
{
"foo": {
"bar": 1,
"baz": 2
}
}
```
See, no more array! It's because the values are flattend to the leaf
fields and didn't have multiple values. This is implied by the docs we
had, but sure wasn't obvious. So now it's documented specifically.
This PR adds the first part of the Prevalidate Node Removal API. This
API allows checking whether attempting to remove some node(s) from the
cluster is likely to succeed or not. This check is useful when a node
needs to be removed from a RED cluster, without risking loosing the last
copy of some RED shards.
In this PR, we only check whether a RED index is a Searchable Snapshot
index or not, in which case the removal of any node is safe as the RED
index is backed by a snapshot.
Relates #87776
This is the continuation of #90176 which leverages #90425 to count query types. This PR adds search usage stats to the existing telemetry by counting sections being used as part of a search request, as well as query types. Each distinct query type is counted once per search request.
The counting is performed while parsing, for the following REST search endpoints:
- _search
- _msearch
- _async_search
- _search/template
- _msearch/template
- _fleet/_fleet_search
- _fleet/_fleet_msearch
All other API using search internally, like reindex, ML transform, rank eval, sql etc. are not counted as part of these search usage stats. Such additional functionalities should have its own dedicated telemetry if needed.
The counting of the search sections is not extensive, only the ones that are interesting to collect counts for are tracked.
The following is the new section added to the cluster stats API response, including some sample stats:
```
"search" : {
"total" : 63,
"sections" : {
"knn" : 42,
"query" : 21,
"aggs" : 46
},
"query" : {
"match" : 58
}
}
```
A big part of the change is actually the plumbing to make a common service class that holds the counters available to all the different callers of the parsing methods, especially plugins. Ideally, there would be a separate component that exposes the search parsing functionality rather than static methods, but changing that would require making the additional component available to the REST layer which is not trivial. I reused the existing UsageService which the RestController already holds, and is already used to count access to the different REST endpoints.
Co-authored-by: Mayya Sharipova mayya.sharipova@elastic.co
If the recovery node bandwidth settings exist,
then the default value for max snapshot speed will
be infinite, and the speed will be rate limited
by the recovery rate limit as well.
Fixes#57023
This change adds support fielddata and subsequently scripting for byte vectors. This is a follow up to
#90774 and completes the initial work for #89784.
The documentation claimed that for the most_fields type, the score is equal to
the sum of all matches divided by the number of matches. This is not correct,
we actually don't divide by the number of matches.
This line in the documentation was added several years ago as part of a large
PR, and was likely just a mistake.
Loading of stored fields is currently handled directly in FetchPhase, with
some fairly complex logic examining various bits of the FetchContext to work
out what fields need to be loaded. This is further complicated by synthetic
source, which may have its own stored field requirements.
This commit tries to separate out these concerns a little by adding a new
StoredFieldsSpec record that holds information about which stored fields
need to be loaded. Each FetchSubPhaseProcessor can now report a
StoredFieldsSpec detailing what its requirements are, and these specs can
be merged together, along with requirements from a SourceLoader, to
determine up-front what fields should be loaded by the StoredFieldLoader.
The stored fields themselves are added into the SearchHit by a new
StoredFieldsPhase, which handles alias resolution and value post-
processing. The logic to determine when source should be loaded and
when not, based on the presence of script fields or stored fields, is
moved into FetchContext, which highlights some inconsistencies that
can be fixed in follow-up commits.
* [DOCS] Replace dependencies list with a link to the generated dependencies list. Closes#84863
* Made conditional based on the release state.
* Fixed missing end statement
Adds the query option to the _semantic_search endpoint for hybrid retrieval.
Scoring is controlled by the boost fields of the knn search and the query.
This renames the explain Health API parameter to verbose.
We decided to rename explain because verbose is a more established
term in the industry for "opt-in to get more information" and allows for more
flexibility to control what exactly that extra information is (explain is already
pushing the limits of what it semantically represents as it's controlling both
the diagnosis insights and the raw details information)
Before it linked to script_score and approximate kNN separately, but now we have
a single page that describes both approaches. This change also removes a link to
the deprecated _knn_search API.
Currently the documentation on network threading suggests that we still
use a model where we have individual workers dedicated to server
sockets. That is no longer true and server sockets are assigned to
normal workers. This commit updates the documentation.
This adds model_alias support for native pytorch models.
Model aliases can be used in `_infer` or within the inference processor. This way the alias can be atomically changed without down time to another deployed model.
Restrictions:
- Model alias changes need to be done between two models of the same kind (e.g. pytorch -> pytorch)
- Model alias change is not allowed between a model that is deployed to a model that is not
- Model alias change is not allowed between a model that deployed AND allocated to a model that is deployed but NOT allocated (not assigned to any nodes).
- A deployment cannot be stopped (without supplying the `force` parameter) when the model has a model alias that is used by a pipeline.
closes: https://github.com/elastic/elasticsearch/issues/90960
This adds a new parameter to the start trained model deployment API,
namely `priority`. The available settings are `normal` and `low`.
For normal priority deployments the allocations get distributed so that
node processors are never oversubscribed.
Low priority deployments allow users to test model functionality even if there
are no node processors available. They are limited to 1 allocation with a single thread.
In addition, the process is executed in low priority which limits the amount of
CPU that can be used when the CPU is under pressure. The intention of this is to
limit the impact of low priority deployments on normal priority deployments.
When we rebalance model assignments we now:
1. compute a plan just for normal priority deployments
2. fix the resources used by normal deployments
3. compute a plan just for low priority deployments
4. merge the two plans
Closes#91024
Abdon Pijpelink
David Turner
David Kyle
Nik Everett
David Kilfoyle
Nikola Grcevski
Iraklis Psaroudakis
Pooya Salehi
István Zoltán Szabó
Artem Prigoda
tmgordeeva
Luca Cavanna
Luigi Dell'Aquila
Lisa Cawley
Jack Conradson
Julie Tibshirani
Alan Woodward
debadair
Keith Massey
Etki
David Roberts
Andrei Dan
Tim Brooks
Benjamin Trent
saryeHaddadi
Liam Thompson
Hendrik Muhs
Dimitris Athanasiou