We have a few leftover mentions of `zen` discovery, mostly for
historical/BwC reasons, which this commit removes.
Prior to this commit the default value for `discovery.type` was `zen`
but this was not written down anywhere or officially supported: the two
options were to set it to `single-node` or to omit it entirely. This
commit changes the default to `multi-node` and documents this.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Returning 408 for a cluster health timeout was deprecated in #78180 and backported to 7.x in #78940
Now we can do a breaking change in 8.0 respecting the user choice to run ES in 7.x compatible mode via the REST Compatibility layer.
Fixes#70849
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.
Relates to #79309, #31619
Changes:
* Adds a transient settings migration guide to the 7.16 docs.
* Updates the related deprecation docs to link to the guide.
Closes#80055
Relates to #79167.
This change introduces a new CLI tool that can be used to set and
reset the password of all the built-in users and users in the native
realm in Elasticsearch. It depends on the file realm being enabled
(which it is, by default) and can (re)set one built-in user password at a time.
It removes the previously introduced elasticsearch-reset-elastic-password
and elasticsearch-reset-kibana-system-password as their functionality is
covered by this new tool.
This PR adds a framework for migrating system indices as necessary prior
to Elasticsearch upgrades. This framework uses REST APIs added in
another commit:
- GET _migration/system_features
This API, which gets the status of "features" (plugins which own system
indices) with regards to whether they need to be upgraded or not. As of
this PR, this API also reports errors encountered while migrating system
indices alongside the index that was being processed when this occurred.
As an example of this error reporting:
```json
{
"feature_name": "logstash_management",
"minimum_index_version": "8.0.0",
"upgrade_status": "ERROR",
"indices": [
{
"index": ".logstash",
"version": "8.0.0",
"failure_cause": {
"error": {
"root_cause": [
{
"type": "runtime_exception",
"reason": "whoopsie",
"stack_trace": "<omitted for brevity>"
}
],
"type": "runtime_exception",
"reason": "whoopsie",
"stack_trace": "<omitted for brevity>"
}
}
}
]
}
```
- POST _migration/system_features
This API starts the migration process. The API for this has no changes,
but when called, any system indices which need to be migrated will be
migrated, with status information stored in the cluster state for later
use by the GET _migration/system_features API.
Recently we have deprecated a number of settings in monitoring. These settings should be represented in the deprecation info API. This PR will be backported with some minor changes to the 7.x branch so that we can start the deprecation process in that release cycle.
* Add note in breaking changes for nameid_format
We changed the default for `nameid_format` in 8.0 in #44090 but
did not add anything to the breaking changes in the release notes.
This change amends that.
* remove reference to settings
* Fix docs build
* Accepting most of James' suggested changes
Thanks James!
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
* Incorporating changes from Ioannis
* Apply suggestions from code review
Co-authored-by: Tim Vernum <tim@adjective.org>
* Apply suggestions from code review
Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Tim Vernum <tim@adjective.org>
This change adds all the JodaCompatibleZonedDateTime methods that no longer exist to the
migration docs with their ZonedDateTime equivalents.
Fixes: #78739
This commit adds the ability to configure a list of settings that will be ignored by the deprecation info
API. Any deprecation messages for any of the settings given will be suppressed. This can be used to hide
settings that users do not have the ability to change.
Relates #78725
* Adjusted integration tests to use geoip test fixture or to use test databases provided via config dirs (for qa module / docs).
* Kept the geolite2-databases dependency for most of the unit tests only.
* Made fallback_to_default_databases parameter on geoip processor a noop and emit deprecation warning upon using it.
* If no geoip databases are available yet to a node then the geoip processor factory returns a processor implementation that flags documents that databases are unavailable. This allows these documents to be reindex later with a pipeline. These documents will have a tag string array field, which contains a string _geoip_database_unavailable_{database_name} for each missing database in a pipeline.
* Added reload pipeline capabilities is IngestService, so that when databases are available again on a node then pipelines with geoip processor definition can be reloaded.
Relates to #68920
* Implement and test get feature upgrade status API
* Add integration test for feature upgrade endpoint
* Use constant enum for statuses
* Add unit tests for transport class methods
This PR adds a MonitoringIndexTemplateRegistry to the monitoring plugin which automatically
installs all monitoring templates locally when the plugin is initialized. Exporters have been
updated to no longer attempt installation of the monitoring templates, and instead will wait for
the templates to become available before setting themselves as started. Some older
functionality related to templates has been removed as well, such as the expectation that
version 6 monitoring templates are installed, as well as the setting that controls their installation
(xpack.monitoring.exporters.<EXPORTER>.index.template.create_legacy_templates).
This change removes several pieces of deprecated code from stored scripts.
Stored scripts/templates are no longer allowed to be an empty and will throw an exception when used
with PutStoredScript.
ScriptMetadata will now drop any existing stored scripts that are empty with a deprecation warning in
the case they have not been previously removed.
The code field is now only allowed as source as part of a PutStoredScript JSON blob.
* Add stubs for get API
* Add stub for post API
* Register new actions in ActionModule
* HLRC stubs
* Unit tests
* Add rest api spec and tests
* Add new action to non-operator actions list
This change removes JodaCompatibleZonedDateTime and replaces it with ZonedDateTime for use in
scripting.
Breaking changes:
* JodaCompatibleDateTime no longer exists and cannot be cast to in Painless. Use ZonedDateTime
instead.
* The dayOfWeek method on ZonedDateTime returns the DayOfWeek enum instead of an int from
JodaCompatibleDateTime. dayOfWeekEnum still exists on ZonedDateTime as an augmentation to
support the transition to ZonedDateTime, but is now deprecated in favor of dayOfWeek on
ZonedDateTime.
* [DOCS] Always enable file and native realms by default
Adds an 8.0 breaking change for PR #69096.
The copy is based on the 7.13 deprecation notice added with PR #69320.
* reword
* Update docs/reference/migration/migrate_8_0/security.asciidoc
Co-authored-by: Yang Wang <ywangd@gmail.com>
* Update docs/reference/migration/migrate_8_0/security.asciidoc
Co-authored-by: Yang Wang <ywangd@gmail.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>
Monitoring installs a number of ingest pipelines which have been historically used
to upgrade documents when mappings and document structures change between
versions. Since there aren't any changes to the document format, nor will there be
by the time the format is completely retired, we can comfortably remove these
pipelines.
We deprecated support for multiple data paths (MDP) in 7.13. However,
we won't remove support until after 8.0.
Changes:
* Reverts PR #72267, which removed MDP docs
* Removes a related item from the 8.0 breaking changes.
In 8.x, we'll ignore the `ecs` option for the `user_agent` ingest processor.
This adds a related breaking change to the 8.0 migration docs.
Relates to #38828
Several sentences in the 8.0 breaking changes reference setting
system properties in `elasticsearch.yml`, which is not supported. This corrects
those sentences.
It also fixes a sentence that references the `http.content_type.required`
setting as a system property.
* Make the ILM `freeze` action a no-op
This changes the ILM `freeze` action to not actually freeze the index, instead performing no
operation.
Relates to #70192
* zoop -> noop in documentation anchor
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
When running in FIPS mode, (fips_mode.enabled: true), the default
password hasher is now "pbkdf2_stretch"
In non-FIPS mode the default is still "bcrypt"
In 7.x and earlier, the default hasher was always "bcrypt"
In 8.0-alpha1, the default hasher on FIPS was "pbkdf2"
Resolves: #66840
A tag is required to reuse Elasticsearch breaking changes in the Stack
Guide. To display properly, the breaking changes must use external
links rather than xrefs.
This PR correctly places those tags for reuse. It also replaces
several xrefs with external links for reuse.
In the upcoming Lucene 9 release, `indices.query.bool.max_clause_count` is
going to apply to the entire query tree rather than per `bool` query. In order
to avoid breaks, the limit has been bumped from 1024 to 4096.
The semantics will effectively change when we upgrade to Lucene 9, this PR
is only about agreeing on a migration strategy and documenting this change.
To avoid further breaks, I am leaning towards keeping the current setting name
even though it contains `bool`. I believe that it still makes sense given that
`bool` queries are typically the main contributors to high numbers of clauses.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Date histogram interval parameter was deprecated in 7.2, in favor of the more specific fixed_interval and calendar_interval parameters. The old logic used some poorly understood guessing to decide if it should operate in fixed or calendar mode. The new logic requires a specific choice by the user, which is more explicit. In 7.x REST compatibility mode, we will parse the interval as calendar if possible, and otherwise interpret it as fixed.
* Removes docs and references for the following `geo_shape` mapping parameters:
* `tree`
* `tree_levels`
* `strategy`
* `distance_error_pct`
* Updates a related breaking change.
Relates to #70850
This PR returns the get snapshots API to the 7.x format (and transport client behavior) and enhances it for requests that ask for multiple repositories.
The changes for requests that target multiple repositories are:
* Add `repository` field to `SnapshotInfo` and REST response
* Add `failures` map alongside `snapshots` list instead of returning just an exception response as done for single repo requests
* Pagination now works across repositories instead of being per repository for multi-repository requests
closes#69108closes#43462
In #55805, we added a setting to allow single data node clusters to
respect the high watermark. In #73733 we added the related deprecations.
This commit ensures the only valid value for the setting is true and
adds deprecations if the setting is set. The setting will be removed
in a future release.
Co-authored-by: David Turner <david.turner@elastic.co>
These settings were deprecated in 7.13+ in #72835 and are now removed by this commit.
This commit also ensures that the settings are removed from index metadata when the metadata is
loaded. The reason for this is that if we allow the settings to remain (because they are not
technically "invalid"), then the index will not be able to be allocated, because the
FilterAllocationDecider will be looking for nodes with the _tier attribute.
This commit removes the bootstrap.system_call_filter setting, as
starting in Elasticsearch 8.0.0 we are going to require that system call
filters be installed and that this is not user configurable. Note that
while we force bootstrap to attempt to install system call filters, we
only enforce that they are installed via a bootstrap check in production
environments. We can consider changing this behavior, but leave that for
future consideration and thus a potential follow-up change.
We are going to require system call filters. This commit is the first
step in that journey, which is to deprecate the setting that allows
disabling system call filters.
Changes:
* Renames 'full copy searchable snapshot' to 'fully mounted index.'
* Renames 'shared cache searchable snapshot' to 'partially mounted index.'
* Removes some unneeded cache setup instructions for the frozen tier. We added a default cache size with #71844.
We previously allowed but deprecated the ability for the shared cache to
be positively sized on nodes without the frozen role. This is because we
only allocate shared_cache searchable snapshots to nodes with the frozen
role. This commit completes our intention to deprecate/remove this
ability.
This commit removes support for JAVA_HOME. As we previously deprecated
usage of JAVA_HOME to override the path for the JDK, this commit follows
up by removing support for JAVA_HOME. Note that we do not treat
JAVA_HOME being set as a failure, as it is perfectly reasonable for a
user to have JAVA_HOME configured at the system level.
This commit removes the following deprecated settings in v8:
- `gateway.expected_nodes`
- `gateway.expected_master_nodes`
- `gateway.recover_after_nodes`
- `gateway.recover_after_master_nodes`
Co-authored-by: ShawnLi1014 <shawnli1014@gmail.com>
In 7.x the close indices API defaulted to `?wait_for_active_shards=0`
but from 8.0 it defaults to respecting the index settings instead. This
commit introduces the `index-setting` value for this parameter on this
API allowing users to opt-in to the future behaviour today, and emits a
deprecation warning indicating that the default no longer needs to be
used and will be unsupported in future.
In 7.x a follow up PR will introduce support for the same
`index-setting` value for this parameter and will emit deprecation
warnings if users try and use the default instead.
Relates #66419
In 8.x the default for `?wait_for_active_shards` changes from `NONE` to
`DEFAULT` on calls to `POST /index/_close`. This commit adds this change
to the breaking changes docs.
Relates #66419, #66542
Artem Prigoda
David Turner
Jake Landis
James Rodewig
Lisa Cawley
Dan Hermann
Adam Locke
Ioannis Kakavas
bellengao
Przemyslaw Gomulka
Gordon Brown
James Baiera
William Brafford
Jack Conradson
Keith Massey
Martijn van Groningen
debadair
Joe Gallo
Ryan Ernst
István Zoltán Szabó
Lee Hinman
Tim Brooks
Tim Vernum
Adrien Grand
Mark Tozzi
Francisco Fernández Castaño
Ignacio Vera
Armin Braun
Henning Andersen
Jason Tedor
Tianlun Li