Removes support for using a system property to disable the automatic release of
the write block applied when a node exceeds the flood-stage watermark.
Relates #42559
This commit changes the ForceMergeRequest.validate() method so that it does
not accept the parameters only_expunge_deletes and max_num_segments
to be set at the same time.
The motivation is that InternalEngine.forceMerge() just ignores the max. number
of segments parameter when the only expunge parameter is set to true, leaving
the wrong impression to the user that max. number of segments has been applied.
It also changes InternalEngine.forceMerge() so that it now throws an exception
when both parameters are set, and modifies tests where needed.
Because it changes the behavior of the REST API I marked this as >breaking.
Closes#43102
Today we fail the node at startup if it contains an index that is too old to be
compatible with the current version, unless that index is closed. If the index
is closed then the node will start up and this puts us into a bad state: the
index cannot be opened and must be reindexed using an earlier version, but we
offer no way to get that index into a node running an earlier version so that
it can be reindexed. Downgrading the node in-place is decidedly unsupported and
cannot be expected to work since the node already started up and upgraded the
rest of its metadata. Since #41731 we actively reject downgrades to versions ≥
v7.2.0 too.
This commit prevents the node from starting in the presence of any too-old
indices (closed or not). In particular, it does not write any upgraded metadata
in this situation, increasing the chances an in-place downgrade might be
successful. We still actively reject the downgrade using #41731, because we
wrote the node metadata file before checking the index metadata, but at least
there is a way to override this check.
Relates #21830, #44230
* HLRC: Fix '+' Not Correctly Encoded in GET Req.
* Encode `+` correctly as `%2B` in URL paths
* Keep encoding `+` as space in URL parameters
* Closes#33077
* Provide an Option to Use Path-Style-Access with S3 Repo
* As discussed, added the option to use path style access back again and
deprecated it.
* Defaulted to `false`
* Added warning to docs
* Closes#41816
Now that the transport client has been removed, the client transport
profile filter can be removed from security. This filter prevented node
actions from being executed using a transport client.
This commit removes the nested_path and nested_filter options deprecated in 6x.
This change also checks that the sort field has a [nested] option if it is under a nested
object and throws an exception if it's not the case.
Closes#27098
This commit finalizes the work done to rename size to max_docs in
reindex and update/delete by query. size is no longer supported in URL
or outer level body for the 3 APIs (though size in update/delete-by-query
will and has always been interpreted as scroll_size, it is not to be relied
upon).
Continuation of #41894Closes#24344
#42090 PR added support
for requesting snapshots from multiple repositories. And it has changed
the response format in a non-BwC way.
There is a mentioning of a response format change in the breaking
changes docs, however, there is no example of how new format looks
like. Pointed out by @dakrone.
This commit adds the missing example.
This commit adds multiple repositories support to get snapshots
request.
If some repository throws an exception this method does not fail fast
instead, it returns results for all repositories.
This PR is opened in favour of #41799, because we decided to change
the response format in a non-BwC manner. It makes sense to read a
discussion of the aforementioned PR.
This is the continuation of work done here #15151.
Remove `common` query and `cutoff_frequency` parameter of
`match` and `multi_match` queries. Both have already been
deprecated for the next 7.x version.
Closes: #37096
With the removal of node.max_local_storage_nodes, there is no need anymore to keep the data in
subfolders indexed by a node ordinal. This commit makes it so that ES 8.0 will store data directly in
$DATA_DIR instead of $DATA_DIR/nodes/$nodeOrdinal.
Upon startup, Elasticsearch will check to see if there is data in the old location, and automatically
move it to the new location. This automatic migration only works if $nodeOrdinal is 0, i.e., multiple
node instances have not previously run on the same data path, which required for
node.max_local_storage_nodes to explicitly be configured.
We deprecated these settings awhile ago, in favor of cluster.remote. In
7.x we were gentle and provided automatic upgrade of these settings to
the new settings. Now it is time for them to go. This commit removes the
deprecated search.remote settings.
This setting, which prior to Elasticsearch 5 was enabled by default and caused all kinds of
confusion, has since been disabled by default and is not recommended for production use. The
preferred way going forward is for users to explicitly specify separate data folders for each started
node to ensure that each node is consistently assigned to the same data path.
Relates to #42426
We deprecated `restoreShard` and `snapshotShard` in #42213
This change removes the deprecated methods and their usage and adds
a note in the migration docs.
This commit removes the deprecated migrate tool which was used to
migrate users from the file realm to native realm when the native realm
was first created.
Removed the leniency when encoding remote reindex search requests that
was introduced in 7.x. All index-names are now encoded before being sent
to the remote host.
Follow-up to #40303
This is related to #36652. There are a number of transport settings that
were deprecated and replaced with new versions in 7.x. This commit
removes them from 8.0.
The current java implementation of Fuzziness leaves a lot of room for
initializing it with illegal values that either cause errors later when the
queries reach the shards where they are executed or values that are silently
ignored in favour of defaults. We should instead tighten the java implementation
of the class so that we only accept supported values. Currently those are
numeric values representing the edit distances 0, 1 and 2, optionally also as
float or string, and the "AUTO" fuzziness, which can come in a cusomizable
variant that allows specifying two value that define the positions in a term
where the AUTO option increases the allowed edit distance.
This change removes several redundant ways of object construction and adds input
validation to the remaining ones. Java users should either use one of the
predefined constants or use the static factory methods `fromEdits(int)` or
`fromString(String)` to create instances of the class, while other ctors are
hidden. This allows for instance control, e.g. returning one of the constants
when creating instances from an integer value.
Previously the class would accept any positive integer value and any float
value, while in effect the maximum allowed edit distance was capped at 2 in
practice. These values while throw an error now, as will any other String value
other than "AUTO" that where previously accepted but led to numeric exceptions
when the query was executed.
Removes the deprecated accept_default_password setting.
This setting become redundant when default passwords were removed
from 6.0, but the setting was kept for BWC.
Removes native role store cache settings.
These have been unused since 5.2 but were kept for BWC.
The Migration Assistance API has been functionally replaced by the
Deprecation Info API, and the Migration Upgrade API is not used for the
transition from ES 6.x to 7.x, and does not need to be kept around to
repair indices that were not properly upgraded before upgrading the
cluster, as was the case in 6.
Changed default of compress setting from false to true for blob store
repositories. This aligns the code with existing documentation and also
seems like the better default.
Removes all traces of Zen1 from the code base. Some of these commits will also be backported to
7.0/7.x (#39470) as the cluster.coordination package was making use of some things in
discovery.zen and we want to keep 7.x as close as possible to master.
Blob store compression was all implemented generally, except reading the
setting for it. Moved the setting to BlobStoreRepository to unify this.
Also removed deprecated env setting 'repositories.fs.compress'.
This is a follow up on #39073
The removal of the "nGram" and "edgeNGram" token filter was filed in a new
migrate_8_0/mappings.asciidoc document but it should better be in an
analysis.asciidoc section and file.
In #30209 we deprecated the camel case `nGram` filter name in favour of `ngram` and
did the same for `edgeNGram` and `edge_ngram`. Using these names has been deprecated
since 6.4 and is issuing deprecation warnings since then.
I think we can remove these filters in 8.0. In a backport of this PR I would change what was a
dreprecation warning from 6.4. to an error starting with new indices created in 7.0.
In #38333 and #38350 we moved away from the `discovery.zen` settings namespace
since these settings have an effect even though Zen Discovery itself is being
phased out. This change aligns the documentation and the names of related
classes and methods with the newly-introduced naming conventions.
We have had various reports of problems caused by the maxRetryTimeout
setting in the low-level REST client. Such setting was initially added
in the attempts to not have requests go through retries if the request
already took longer than the provided timeout.
The implementation was problematic though as such timeout would also
expire in the first request attempt (see #31834), would leave the
request executing after expiration causing memory leaks (see #33342),
and would not take into account the http client internal queuing (see #25951).
Given all these issues, it seems that this custom timeout mechanism
gives little benefits while causing a lot of harm. We should rather rely
on connect and socket timeout exposed by the underlying http client
and accept that a request can overall take longer than the configured
timeout, which is the case even with a single retry anyways.
This commit removes the `maxRetryTimeout` setting and all of its usages.
Elasticsearch has long [supported](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-versioning) compare and set (a.k.a optimistic concurrency control) operations using internal document versioning. Sadly that approach is flawed and can sometime do the wrong thing. Here's the relevant excerpt from the resiliency status page:
> When a primary has been partitioned away from the cluster there is a short period of time until it detects this. During that time it will continue indexing writes locally, thereby updating document versions. When it tries to replicate the operation, however, it will discover that it is partitioned away. It won’t acknowledge the write and will wait until the partition is resolved to negotiate with the master on how to proceed. The master will decide to either fail any replicas which failed to index the operations on the primary or tell the primary that it has to step down because a new primary has been chosen in the meantime. Since the old primary has already written documents, clients may already have read from the old primary before it shuts itself down. The version numbers of these reads may not be unique if the new primary has already accepted writes for the same document
We recently [introduced](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/optimistic-concurrency-control.html) a new sequence number based approach that doesn't suffer from this dirty reads problem.
This commit removes support for internal versioning as a concurrency control mechanism in favor of the sequence number approach.
Relates to #1078
With this change we no longer support pluggable discovery implementations. No
known implementations of `DiscoveryPlugin` actually override this method, so in
practice this should have no effect on the wider world. However, we were using
this rather extensively in tests to provide the `test-zen` discovery type. We
no longer need a separate discovery type for tests as we no longer need to
customise its behaviour.
Relates #38410
David Turner
Gordon Brown
Lee Hinman
Jason Tedor
Tanguy Leroux
Lisa Cawley
Armin Braun
Jay Modi
Jim Ferenczi
Henning Andersen
Andrey Ershov
Zachary Tong
Marios Trivyzas
Yannick Welsch
Julie Tibshirani
Simon Willnauer
Ryan Ernst
Colin Goodheart-Smithe
James Rodewig
Tim Brooks
Christoph Büscher
Deb Adair
Tim Vernum
Tamara Braun
Luca Cavanna
Boaz Leskes