There's no need to talk about unwelcome behaviour or power outages here,
and it caused at least one user some concern. This commit clarifies the
message.
Relates #95166
* Add note about ENV for systemd installs
This ES_TMPDIR env variable is not referenced in the /guide/en/elasticsearch/reference/current/setting-system-settings.html#sysconfig section, and when searching for the error mentioned in this page, it might not become too obvious.
* Restructure
---------
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
The systemd unit file is part of the Elasticsearch package and should
not be edited. Instead, we recommend creating a service override file.
This commit tweaks the docs for setting tmp dir with systemd to use the
override file instead of editing the unit file.
relates #93121
Today we suggest that users set `ES_TMPDIR` using `export`, which only
works if you're running things directly from the shell. Yet most users
encountering `ES_TMPDIR` problems seem to on RHEL and trying to run
things via `systemd`, for whom the `export` suggestion doesn't work.
This commit adds to the docs a suggestion of how to adjust the `systemd`
service file to set the appropriate environment variable.
Relates #80651
With this commit we specify `level` in JVM logs. `level` helps to filter
more severe messages from mere debugging messages in logs and with this
change we are able to use it also for JVM logs.
Here are a few example lines:
Before:
```
[2022-12-15T06:19:16.936+0000][15181][gc,start ] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
```
After:
```
[2022-12-15T06:22:27.932+0000][16320][info][gc,start ] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
[2022-12-15T06:22:27.932+0000][16320][debug][gc,age ] GC(0) Desired survivor size 14680064 bytes, new threshold 15 (max threshold 15)
[2022-12-15T06:22:27.935+0000][16320][info ][gc,phases ] GC(0) Pre Evacuate Collection Set: 0.1ms
[2022-12-15T06:22:27.935+0000][16320][trace][gc,age ] GC(0) Age table with threshold 15 (max threshold 15)
```
Notice, that the log level might have trailing spaces.
See also
https://docs.oracle.com/en/java/javase/19/docs/specs/man/java.html#xlog-tags-and-levels
for the available log levels.
The docs for the TCP readiness port have a preview warning, with custom
text. However, this doesn't render correctly in the docs. Rather than
figure out why the custom text doesn't render, this commit simply
removes the custom text in favor of the default preview text.
The current command to start the first Elasticsearch node includes `-p 9200:9200` and `-p 9300:9300`. This PR removes the extra ports so that they're not open unnecessarily.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The docs introduced in #91333 apply to older versions in which the `systemd`
startup timeout was 75s by default, but in #91338 we extended the `systemd`
startup timeout to 900s from 8.7.0 and 7.17.8 onwards. This commit adjusts the
docs to match.
ES_JAVA_OPTS is still the correct way to pass options to
the Elasticsearch process, CLI_JAVA_OPTS affects only the
command line tool. CLI_JAVA_OPTS is the correct way to pass
options for plugin installation or other tools.
* Create restart-cluster.asciidoc
As per https://github.com/elastic/elasticsearch/issues/49972 and https://github.com/elastic/elasticsearch/issues/56578, if a node is above low disk threshold when being restarted (rolling restart, network disruption or crash), the disk threshold decider prevents reusing the shard content on the restarted node.
The consequence of the event is the node may take a long time to start.
* Update docs/reference/setup/restart-cluster.asciidoc
LGTM! Thanks!
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The build_flavor was previously removed since it is no longer relevant;
only the default distribution now exists. However, the removal of build
flavor included removing it from the version information on the info
response for the root path. This API is supposed to be stable, so
removing that key was a compatibility break. This commit adds the
build_flavor back to that API, hardcoded to `default`. Additionally, a
test is added to ensure the key exists going forward, until it can be
properly deprecated.
closes#88318
* [doc] Add information for how to find if compressed ordinary object pointers is in use using the REST APIs.
* Update docs/reference/setup/advanced-configuration.asciidoc
Co-authored-by: Nikola Grcevski <6207777+grcevski@users.noreply.github.com>
Co-authored-by: Nikola Grcevski <6207777+grcevski@users.noreply.github.com>
Ensures that on every page of the docs that mentions
`cluster.initial_master_nodes` also mentions that this setting must be
removed after bootstrapping completes.
The default distribution is the only remaining build flavor, and has been for
quite a while now. This commit removes flavor from the internal Build
class. It keeps rest api compat for nodes info for now by hardcoding
`default`.
Since 8.0.0, we only support Java 17 and up. This change cleans up unsupported Java versions from jvm.options and updates documentation to reference supported Java versions, in case it is confusing to users.
In #84340 we adjusted the docs to show instructions on how to
retrieve TLS keystores passwords from the elasticsearch keystore
but the security-files-reference.asciidoc was not referenced in
all places where the instructions should be used. This commit
changes that and also removes references for beta versions.
The way this specific doc was structured meant that the path for
DEB/RPM packages were wrong (an extra `config` was added to the
dir path). This commit changes the structure to be relevant to the
ES_CONF_PATH env variable that should work uniformly for all of
our installation means.
Fixes an intro sentence for the Docker install instructions.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
(cherry picked from commit 472a7d8e91)
Co-authored-by: Alexander Reelsen <alexander@reelsen.net>
The use of `apt-key` is deprecated and will no longer be available after
Debian 11 and Ubuntu 22.04. This updates the installation instructions
for Debian-based distributions.
Closes#84644
In 8.0.0. we introduce TLS autoconfiguration. We store the key and
certificate materials in password protected PKCS#12 keystores and
we store these passwords in the elasticsearch keystore.
This commit adds instructions on how to get hold of the passwords
for users to inspect or alter the PKCS#12 keystores.
In #80617 we upgraded to a version of JNA which would report the message
`Failed to allocate closure` if the temp dir for `libffi` did not permit
executables (rather than crashing with a segfault). This commit extends
the docs to mention this message.
* Clarify commands shown for "permanently" setting max_map_count
The current text shows a grep, which of course does not set the value at all. I've clarified why it's offered (to CHECK the value).
I've also moved the command for setting the value temporarily to above that discussion, to make it more clear to readers that that's what it does.
* Reorder commands, add headings, and fix typos
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Charlie Arehart <charlie@carehart.org>
* [docs] Add note to Elasticsearch logging that log4j customization is out of support scope
* Update language for default logging recommendation
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Quin Hoxie <qhoxie@gmail.com>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Closes#69533.
The Docker docs mention bind-mounting the `config`, `data` and
`logs` directories when using an arbitrary UID / GID, but they fail
to mention that the `plugins` dir must also be mounted in order to
install plugins.
* Rolling restart instructions
In a rolling restart scenario, indexing do not need to be stopped. This step is optional. I have updated Rolling restart/ step 2.
Note: I am not sure how to check if the doc, once generated, will keep the numbering as section 1 and 3 are references to the full restart, can you check this detail?
* Clarify that shard recovery can be faster
Co-authored-by: Adam Locke <adam.locke@elastic.co>
* [DOCS] Remove extraneous Elasticsearch Docker image information
In the step for starting Elasticsearch with the generated enrollment token, `docker.elastic.co/elasticsearch/elasticsearch:` was included in conjunction with the `{docker-image}` variable. This include led to a duplicate Docker image that displayed as `ocker.elastic.co/elasticsearch/elasticsearch:docker.elastic.co/elasticsearch/elasticsearch:8.0.0-rc1`. This PR removes the duplicate image information.
* Update ifeval statements and add sub-heading for setting JVM heap size
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
We (mostly I) were initially advocating for the auto-generated files to
use unique names (the name containing a timestamp particle), in order to
avoid that subsequent invocations of the config step conflict with
itself. Moreover, I was wishing that these files will not have to be
handled directly by admins (that the enrollment process was to be used).
However, experience proved us otherwise, admins have to manipulate these
files, and unique configuration names are hard to deal with in scripts
and docs, so this PR is all about using a fixed name for all the
generated files. _Labeling as a bug fix because the feedback is that it
very negatively impacts usabilty._ Closes
https://github.com/elastic/elasticsearch/issues/81057
After 7.16.2, we'll no longer produce Windows MSI installer packages for Elasticsearch. These packages were previously released in beta and didn't receive widespread adoption.
### Changes:
* Adds a related 7.17 breaking change.
* Adds a related 7.16 deprecation.
* Removes the MSI installation instructions.
* Removes references to the MSI installer.
I plan to port the applicable changes to 8.1 (main), 8.0, 7.17, and 7.16. In the 7.16 ports, I'll leave in the MSI install docs and add related deprecation notes to them instead.
Removes a section covering configuration management tools from the
installation instructions.
After 7.16.2, Elastic will no longer maintain these tools. Previously,
the tools were only supported on a "best effort" basis.
* [DOCS] Enroll additional nodes on Docker
* Remove -p option for second node
Co-authored-by: Fabio Busatto <52658645+bytebilly@users.noreply.github.com>
* Rename nodes to align with other Docker docs
* Add elastic network to first node docker run command
* Remove hyphen from node names
Co-authored-by: Fabio Busatto <52658645+bytebilly@users.noreply.github.com>
* [DOCS] Add docs for verifying CA fingerprint
* Update openssl command and explanatory text
* Explain copying CA cert if fingerprint validation isn't possible
* Incorporate new section into the main security config page
* Clarify how cert is used
Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
* Split into two, separate sections
* Rename file and update text based on feedback
* Update ref to use new filename
* Remove extra word
Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [DOCS] Update ES quick start for security ON by default
* Remove code.asciidoc, which is part of the overall doc build now
* Update node names for cleanup
* Add note with links to tools
* Add --net elastic network
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
* [DOCS] Update Windows .zip install instructions for security ON by default
* Rework instructions for running as a service on Windows
* Update wording and add variable for back/forward slashes
* Relocating enroll nodes steps and introducing variables
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
#80556 reverted the deprecation of transient cluster settings. This replaces deprecation language in the docs with a warning/recommendation to avoid transient settings.
Closes#80557
# Conflicts:
# docs/reference/migration/migrate_7_16.asciidoc
Makes several changes to consolidate snapshot and backup-related docs.
Highlights:
* Adds info about supported ESS snapshot repository types
* Adds docs for Kibana's Snapshot and Restore feature
* Combines tutorial pages related to taking and managing snapshots
* Consolidates explanations of the snapshot process
* Incorporates SLM into the snapshot tutorial
* Removes duplicate "back up a cluster" pages
Today if `libffi` cannot allocate pages of memory which are both
writeable and executable then it will attempt to write code to a
temporary file. Elasticsearch configures itself a suitable temporary
directory for use by JNA but by default `libffi` won't find this
directory and will try various other places. In certain configurations,
none of the other places that `libffi` tries are suitable. With older
versions of JNA this would result in a `SIGSEGV`; since #80617 the JVM
will exit with an exception.
With this commit we use the `LIBFFI_TMPDIR` environment variable to
configure `libffi` to use the same directory as JNA for its temporary
files if they are needed.
Closes#18272Closes#73309Closes#74545Closes#77014Closes#77053
Relates #77285
Co-authored-by: Rory Hunter <roryhunter2@gmail.com>
Closes#76681. Our approach to using `scratch` for building Docker
images has caused problems at Docker Hub. Fix this situation by
removing the whole process of using scratch and instead bases the
default distribution on `almalinux:8.4-minimal`. Alma Linux is
binary-compatible with RHEL, and therefore very similar to UBI.
We have a few leftover mentions of `zen` discovery, mostly for
historical/BwC reasons, which this commit removes.
Prior to this commit the default value for `discovery.type` was `zen`
but this was not written down anywhere or officially supported: the two
options were to set it to `single-node` or to omit it entirely. This
commit changes the default to `multi-node` and documents this.
Co-authored-by: Adam Locke <adam.locke@elastic.co>
The `elasticsearch.yml` file that ships with our Docker image includes the
`network.host: 0.0.0.0` setting by default. If a user bind-mounts a custom
config file, it should include this setting to ensure Elasticsearch is reachable.
Closes#77937.
* [DOCS] Update archive install docs for security ON by default
* Remove extra attribute references that aren't needed
* Incorporate security info into start page
* Update heading
* Adjust packaged installation docs for security on by default
This commit introduces necessary changes to guide users through
the installation of our DEB/RPM packages, now that security is
enabled and configured by default.
* Update security docs and configure includes
* Update wording in check-running.asciidoc
* Adding hidden GET request
* Update heading
* Updated reconfigure heading
Co-authored-by: Adam Locke <adam.locke@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Changes:
* Adds a transient settings migration guide to the 7.16 docs.
* Updates the related deprecation docs to link to the guide.
Closes#80055
Relates to #79167.
Today we have a short note in one place in the docs saying not to touch
the contents of the data path. This commit expands the warning to
describe more precisely what is forbidden, and to give some more detail
of the consequences, and also duplicates the warning to the other
location that documents the `path.data` setting.
This PR changes uses of transient cluster settings to
persistent cluster settings.
The PR also deprecates the transient settings usage.
Relates to #49540
Introduces a setting cluster.deprecation_indexing.x_opaque_id_used.enabled to disable use of
x-opaque-id in RateLimitingFilter. This will be used for deprecation
logs indexing and will not affect logging to files (it uses different
instance of RateLimitingFilter with this flag enabled by default)
Changes the indices backing a deprecation log data stream to be hidden.
Refactors DeprecationHttpIT to be more reliable
relates #76292closes#77936
We deprecated support for multiple data paths (MDP) in 7.13. However,
we won't remove support until after 8.0.
Changes:
* Reverts PR #72267, which removed MDP docs
* Removes a related item from the 8.0 breaking changes.
* [DOCS] Update remote cluster docs
* Add files, rename files, write new stuff
* Plethora of changes
* Add test and update snippets
* Redirects, moved files, and test updates
* Moved file to x-pack for tests
* Remove older CCS page and add redirects
* Cleanup, link updates, and some rewrites
* Update image
* Incorporating user feedback and rewriting much of the remote clusters page
* More changes from review feedback
* Numerous updates, including request examples for CCS and Kibana
* More changes from review feedback
* Minor clarifications on security for remote clusters
* Incorporate review feedback
Co-authored-by: Yang Wang <ywangd@gmail.com>
* Some review feedback and some editorial changes
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>
Replaces the hard-coded ESS lead-in with the docs attribute.
Previously, this copy omitted Microsoft Azure. This ensures these docs are better maintained.
PR #77155 updated the keystore instructions for Docker. However, it removed an
example that included the `KEYSTORE_PASSWORD` env variable.
This replaces a docker compose example with the original example from PR #51123.
* [DOC] Update Persist Keystore via Docker
From feedback from ES Devs summarized in [^1], I believe this needs to reflect a directory mount rather than file mount to not error. Also adding in the two common mounting errors, but not sure if this is the right place for them.
[^1] https://discuss.elastic.co/t/persist-elasticsearch-kibana-keystores-with-docker/283099
* feedback
* Reorganize
* reword
* fix formatting
* address review feedback
* remove extra whitespace
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Abdon Pijpelink
David Turner
David Kilfoyle
Alessandro Stoltenberg
Rafi Estrada
Ryan Ernst
Nikola Grcevski
Daniel Mitterdorfer
Adam Locke
Flavio
Adam Locke
debadair
Leaf-Lin
Yang Wang
Leaf-Lin
Przemyslaw Gomulka
Nicole Albee
Rune Antonsen
Rick Boyd
Ioannis Kakavas
James Rodewig
er0k
DeDe Morton
Quentin Pradet
Fabio Busatto
Toby Sutor
Rory Hunter
Alex Perrin
Albert Zaharovits
James Rodewig
Kyle.K.Zhao
Stef Nestor