We sometimes see a `ShardLockObtainFailedException` when a shard failed
to shut down as fast as we expected, often because a node left and
rejoined the cluster. Sometimes this is because it was held open by
ongoing scrolls or PITs, but other times it may be because the shutdown
process itself is too slow. With this commit we add the ability to
capture and log a thread dump at the time of the failure to give us more
information about where the shutdown process might be running slowly.
Relates #93226
Delete-snapshot tasks can sometimes be very long-running, but they're
hard to monitor because the tasks API does not include a description of
the snapshot(s) which are being deleted. This commit adds such a
description.
Some random finds while working with this code. We shouldn't use a Consumer<Long> instead of a LongConsumer
as we never pass `null` to the consumer.
Also, way simplified the locking around the Lucene `Bytebuffer b` to simplify the code and technically make it
a little faster/less-contenting as well.
Plus, made use of modern Java's buffer slicing to simplify the slicing of the Lucene buffer.
We still don't properly understand why this test is failing, and it
doesn't reproduce locally, so this commit adds a little extra logging to
capture extra detail from a failure in CI.
In #90977 we made the list tasks API fully async, but failed to notice
that if we waited for a task to complete then we would respond in the
thread context of the last-completing task. This commit fixes the
problem by restoring the context of the list-tasks task before
responding.
Closes#93428
Similar to #92983, this commit reworks the tests in
`TransportBroadcastByNodeActionTests` to use the `ReachabilityChecker`
to check that things are released on cancellation, and adds a test
showing the cancellation behaviour of the shard-level operations.
These tests try and execute `maxThreads` concurrent tasks to ensure that
the rest of the executor's queue has been processed, but due to #93443
(and the executor's zero timeout) this sometimes doesn't work. This
commit fixes the problem by making every thread a core thread so that
they do not time out.
Closes#92910Closes#92747
Support for synthetic source is also added to `unsigned_long` field as part of this change.
This is required because `unsigned_long` field types can be used in tsdb indices and
this change would prohibit the usage of these field type otherwise.
Closes#92319
Though the remote access is implemented with the transport profile. Its
configuration and behaviour should not be tied to the default transport
profile like other profiles do. Users should be able to enable or
disable SSL separately for the remote access port and configure differnt
values for all SSL settings. These settings can also have different
defaults.
This PR implements the above by:
* Adds a new xpack.security.remote_cluster.ssl.enabled setting to
control whether SSL is enabled separately for the remote access port
* The above enabled setting defaults to true (unlike the default for
tranport SSL)
* Client auth defaults to none for the remote access port
* Separate server SSL configuration validation
The PR also moves the remote access profile to be built first for more
consistent error message.
This adds term query capabilities for rank_features fields. term queries against rank_features are not scored in the typical way as regular fields. This is because the stored feature values take advantage of the term frequency storage mechanism, and thus regular BM25 does not work.
Instead, a term query against a rank_features field is very similar to linear rank_feature query. If more complicated combinations of features and values are required, the rank_feature query should be used.
This adds a new option to the knn search clause called query_vector_builder. This is a pluggable configuration that allows the query_vector created or retrieved.
Since we know which indices were involved in the Bulk
request we can refresh only those instead of all indices,
and expand to hidden indices so that they are also
refreshed.
Relates #93160
Currently we log in internal test cluster cases where there are
in-flight bytes. However, the log message is not clear it is referring
to the number of bytes associated with the request. This commit fixes
this log.
When executing the copy_to mappings on a nested doc, if the location was
underneath another nested mapper then the document parser could end up
adding a new nested path metadata field for the source to the destination
document. This was mostly ignored, but could make calculation of a
NestedIdentity incorrect, leading to exceptions when loading the source of
a nested document during the fetch phase.
This commit moves all of the nested path handling directly into
DocumentParserContext.createNestedContext(), which already has some
logic to detect if we're in a copy_to context.
Fixes#93117
Lookups on sub-objects in flattened fields don't support global ordinals, but only segment ordinals. Currently code documentation claims support for segment and global ordinals. This change fixes the documentation and adds further details how to check for global ordinal support. This also adds testcase for flattened as example for this special case.
The error message should refer to the setting of ssl.key instead of
ssl.keystore.path
PS: Labelling this as non-issue since we don't consider error message as
actual production changes (that need go into the release notes).
Lucene introduced new numeric fields that index both points and doc
values. This has the same semantics as indexing one field for points and
another one for doc values as we did before, but covering both data
structures in a single field yielded a speedup in Lucene's nightly
benchmarks (see annotation
[AH](http://people.apache.org/~mikemccand/lucenebench/sparseResults.html#index_throughput))
which would be interesting to get too.
This commit does not switch to factory methods for queries such as
`LongField#newRangeQuery` for now, we'll need to look into it in a
follow-up.
If a document has `null` specified as the vector value, the vector field will not be parsed or indexed.
This is useful for when deleting vector values from an indexed document.
closes: https://github.com/elastic/elasticsearch/issues/70470
Artem Prigoda
Joe Gallo
Keith Massey
David Turner
Ievgen Degtiarenko
Nikolaj Volgushev
Armin Braun
Francisco Fernández Castaño
Mary Gouseti
Hendrik Muhs
Martijn van Groningen
Yang Wang
Mark Vieira
Benjamin Trent
Jake Landis
Tanguy Leroux
Tim Brooks
Simon Cooper
Nikola Grcevski
David Kyle
Enrico Zimuel
Craig Taverner
Alan Woodward
Iraklis Psaroudakis
Pooya Salehi
Abdon Pijpelink
Adrien Grand