Commit Graph

69 Commits

Author SHA1 Message Date
David Turner 2adeb4a666
Expand and consolidate networking docs (#68051)
Today's network config docs are split into "Network", "HTTP" and
"Transport" pages, with unclear relationships between them. We often
encounter users with weird configs that indicate they don't really
understand how these settings all relate. In fact these pages are all
very interrelated, and the HTTP and Transport pages are almost all only
for advanced users. This commit brings these docs into a single page and
rewords some things to try and guide users away from the advanced
settings unless their configuration needs all the extra complexity.

It also adds a section entitled "Binding and publishing" which clarifies
the meanings of the `bind_host` and `publish_host` parameters. This is
also a common source of confusion amongst users.

It also clarifies that many of these settings accept a list of
addresses, and warns that this may not be what you want. Closes #67956.

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-02-01 13:06:20 +00:00
James Rodewig 3e34247570
[DOCS] Add security privileges to cluster API docs (#67589) 2021-01-19 10:18:59 -05:00
bellengao d14492ca13
[DOCS] Fix some typos in docs (#66672) 2020-12-21 12:45:51 +02:00
James Rodewig 1ea83359bb
[DOCS] Fix case for 'Boolean' (#64299) 2020-10-29 09:04:43 -04:00
Adam Locke 789ee2d73e
[DOCS] Combining important config settings into a single page (#63849)
* Combining important config settings into a single page.

* Updating ids for two pages causing link errors and implementing redirects.
2020-10-19 10:02:22 -04:00
James Rodewig 136275e3e6
[DOCS] Fix typo in nodes stats docs (#61601) (#61716)
Co-authored-by: Henry <henryloh@ucla.edu>
2020-08-31 09:29:40 -04:00
James Rodewig a94e5cb7c4
[DOCS] Replace Wikipedia links with attribute (#61171) 2020-08-17 09:44:24 -04:00
Tim Brooks b1a6271ec8
Add configured indexing memory limit to node stats (#60342)
This commit adds the configured memory limit to the node stats API.
2020-07-29 11:20:59 -06:00
David Turner 940d618186
Log and track open/close of transport connections (#60297)
Transport connections between nodes remain in place until one or other
node shuts down or the connection is disrupted by a flaky network.
Today it is very difficult to demonstrate that transient failures and
cluster instability are caused by the network even though this is often
the case. In particular, transport connections open and close without
logging anything, even at `DEBUG` level, making it very hard to quantify
the scale of the problem or to correlate the networking problems with
external events.

This commit adds the missing `DEBUG`-level logging when transport
connections open and close, and also tracks the total number of
transport connections a node has opened as a measure of the stability of
the underlying network.
2020-07-28 16:58:00 +01:00
Tim Brooks 5c227dac88
Implement human readable indexing pressure stats (#60022)
The indexing pressure stats do not currently have human readable
variants. This commit add human readable variants and updates the
documentation.
2020-07-22 09:54:51 -06:00
Tim Brooks 08506de861
Add indexing pressure documentation (#59456)
This commit adds documentation about the new indexing pressure memory
limit setting and exposure of this metrics in node stats.
2020-07-20 19:35:26 -06:00
David Turner 7bb748da8c
Remove sporadic min/max usage estimates from stats (#59755)
Today `GET _nodes/stats/fs` includes `{least,most}_usage_estimate`
fields for some nodes. These fields have rather strange semantics. They
are only reported on the elected master and on nodes that have been the
elected master since they were last restarted; when a node stops being
the elected master these stats remain in place but we stop updating them
so they may become arbitrarily stale.

This means that these statistics are pretty meaningless and impossible
to use correctly. Even if they were kept up to date they're never
reported for data-only nodes anyway, despite the fact that data nodes
are the ones where we care most about disk usage. The information needed
to compute the path with the least/most available space is already
provided in the rest the stats output, so we can treat the inclusion of
these stats as a bug and fix it by simply removing them in this commit.
Since these stats were always optional and mostly omitted (for opaque
reasons) this is not considered a breaking change.
2020-07-20 14:48:53 +01:00
David Turner 83d6589b2a
Account for remaining recovery in disk allocator (#58029)
Today the disk-based shard allocator accounts for incoming shards by
subtracting the estimated size of the incoming shard from the free space on the
node. This is an overly conservative estimate if the incoming shard has almost
finished its recovery since in that case it is already consuming most of the
disk space it needs.

This change adds to the shard stats a measure of how much larger each store is
expected to grow, computed from the ongoing recovery, and uses this to account
for the disk usage of incoming shards more accurately.
2020-07-01 08:04:45 +01:00
Lisa Cawley 8b9293b3bf
[DOCS] Replace docdir attribute with es-repo-dir (#57489) 2020-06-01 15:55:05 -07:00
James Rodewig 0037334d23
[DOCS] Collapse nested objects in node stats API response (#54755)
Replaces dot notation with collapsed nested object formatting
per the [Elastic API reference template][0].

[0]:https://github.com/elastic/docs/blob/master/shared/api-ref-ex.asciidoc
2020-04-06 15:18:58 -04:00
James Rodewig 52e8f6db99
[DOCS] Document missing data types for node stats API's response parameters (#53475)
Documents missing data types for several response parameters returned
by the node stats API.

Also adds several missing human-readable parameters returned by the API.
2020-03-25 08:25:26 -04:00
James Rodewig 8c50523d9d
[DOCS] Add missing `indices` parms returned by `_nodes/stats` (#52055)
Adds several human-readable `indices` parameters returned by the
`_nodes/stats` API.
2020-02-21 08:07:28 -05:00
James Rodewig b9fafc7dc6
[DOCS] Document node stats response meta (#51263)
Documents several metadata-related parameters returned by the
`GET _nodes/stats` API.
2020-02-03 08:33:02 -05:00
James Rodewig 380cc7c290
[DOCS] Collapse node stats response sections (#51063)
elastic/docs#1687 added support for the `[%collapsible]` Asciidoc
attribute, which creates collapsible sections in the HTML output.

This PR makes two related changes to the nodes stats API documentation:

* Makes the response parameter sections collapsible. This allows users
  to more easily navigate the page without long walls of text.

* Reorders the response parameter sections to match the default order
  returned by the API.

Relates to #47524.
2020-01-16 13:18:36 -05:00
James Rodewig ae2ec858a2
[DOCS] Document `breakers`, `script`, and `discovery` node stats (#50509)
Documents the `breakers`, `script`, and `discovery` parameters returned
by the `_nodes/stats` API.
2020-01-14 16:48:35 -05:00
James Rodewig 11b1ebfd9e
[DOCS] Document `transport` and `http` node stats (#50473)
Documents the `transport` and `http` parameters returned by the
`_nodes/stats` API.
2019-12-26 07:41:23 -05:00
James Rodewig b8a62ce8f7
[DOCS] Document `thread_pool` node stats (#50330) 2019-12-18 16:57:38 -05:00
James Rodewig ff2259bf81
[DOCS] Document JVM node stats (#49500)
* [DOCS] Document JVM node stats

Documents the `jvm` parameters returned by the `_nodes/stats` API.

Co-Authored-By: James Baiera <james.baiera@gmail.com>
2019-12-12 15:41:20 -05:00
James Rodewig f982534df3
[DOCS] Document indices response parameters for node stats API (#47525) 2019-11-12 08:34:51 -05:00
James Rodewig 88b5450ebd
[DOCS] Sort cluster API docs alphabetically (#48198) 2019-10-22 12:27:31 -05:00
James Rodewig cfa76b7b5d
[DOCS] Document ingest pipeline and processor node stats (#47884) 2019-10-17 09:17:58 -04:00
James Rodewig 5772c1c7dd
[DOCS] [2 of 5] Change // CONSOLE comments to [source,console] (#46353) 2019-09-09 13:13:41 -04:00
James Rodewig 565ba9cb88
[DOCS] Reformat index stats API docs (#46322) 2019-09-05 15:03:33 -04:00
James Rodewig 46d7849032
Change `{var}` convention to `<var>` (#45904) 2019-08-23 10:57:20 -04:00
István Zoltán Szabó 5cba5ac01c
[DOCS] Reformats cluster node stats API (#45441)
Co-Authored-By: James Rodewig <james.rodewig@elastic.co>
2019-08-13 12:46:47 +02:00
James Rodewig fab98dfa55
[DOCS] Remove heading offsets for REST APIs (#44568)
Several files in the REST APIs nav section are included using
:leveloffset: tags. This increments headings (h2 -> h3, h3 -> h4, etc.)
in those files and removes the :leveloffset: tags.

Other supporting changes:
* Alphabetizes top-level REST API nav items.
* Change 'indices APIs' heading to 'index APIs.'
* Changes 'Snapshot lifecycle management' heading to sentence case.
2019-07-19 14:35:36 -04:00
Jason Tedor 31a7df8620
Remove reference to fs.data.spins in docs
We long ago removed fs.data.spins from the nodes stats. This commit
removes reference to this in the docs.
2019-05-10 11:47:05 -04:00
Josh Soref edb48321ba [DOCS] Various spelling corrections (#37046) 2019-01-07 14:44:12 +01:00
Jason Tedor 0045111ce2 Deprecate the suggest metrics (#29627)
The suggest stats were folded into the search stats as part of the
indices stats API in 5.0.0. However, the suggest metric remained as a
synonym for the search metric for BWC reasons. This commit deprecates
usage of the suggest metric on the indices stats API.

Similarly, due to the changes to fold the suggest stats into the search
stats, requesting the suggest index metric on the indices metric on the
nodes stats API has produced an empty object as the response since
5.0.0. This commit deprecates this index metric on the indices metric on
the nodes stats API.
2018-04-20 09:47:38 -04:00
Jason Tedor 105dcb544c
Enable selecting adaptive selection stats
The node stats API enables filtlering the top-level stats for only
desired top-level stats. Yet, this was never enabled for adaptive
replica selection stats. This commit enables this. We also add setting
these stats on the request builder, and fix an inconsistent name in a
setter.

Relates #28721
2018-02-19 16:56:36 -05:00
David Roberts a292740b9e Add cgroup memory usage/limit to OS stats on Linux (#26166)
This change adds cgroup memory usage/limit to the OS stats section of
the node stats on Linux.  This information is useful because in Docker
containers the standard node stats report the host memory limit, not
taking account of extra restrictions that may have been applied to the
container.

The original idea was to store these values as Long, truncating any values
outside the range of long.  However, this meant that in the relatively common
case of no limit being applied, users would not see the same value in the OS
stats as they see by querying Linux directly.  So instead the values are stored
as String.  This change places a burden on consumers of the strings to
convert the strings to numbers and decide what to do about extremely large
values, but there will be very few consumers and they would need to have a
policy for dealing with "no limit" in any case.
2017-10-03 12:08:36 +01:00
Peter Dyson 1f9e0fd0dd [Docs] improved description for fs.total.available_in_bytes (#26657) 2017-09-18 16:56:19 +10:00
Clinton Gormley 1b0c93b07c Documented the level parameter to nodes stats
Closes #24999
2017-06-01 12:11:21 +02:00
Jason Tedor 41ffb008ad Fix doc bug for cgroup cpuacct usage metric
This commit fixes a silly doc bug where the field that represents the
total CPU time consumed by all tasks in the same cgroup was mistakenly
reported as "usage" instead of "usage_nanos".

Relates #21029
2016-12-15 23:22:54 -05:00
Clinton Gormley cfabc95f59 Fixed bad asciidoc ID in node stats 2016-11-15 17:39:15 +00:00
Jason Tedor f5ac0e5076 Remove lenient stats parsing
Today when parsing a stats request, Elasticsearch silently ignores
incorrect metrics. This commit removes lenient parsing of stats requests
for the nodes stats and indices stats APIs.

Relates #21417
2016-11-15 12:17:26 -05:00
Jason Tedor aec09a76d6 Clarify requesting all stats in node stats docs
This commit clarifies how to explicitly obtain all stats from the node
stats API.
2016-11-08 13:47:15 -05:00
Jason Tedor 900ee0536e Strengthen handling of unavailable cgroup stats
On some systems, cgroups will be available but not configured. And in
some cases, cgroups will be configured, but not for the subsystems that
we are expecting (e.g., cpu and cpuacct). This commit strengthens the
handling of cgroup stats on such systems.

Relates #21094
2016-10-24 16:36:51 -04:00
Jason Tedor 3d642ab0eb Add basic cgroup CPU metrics
This commit adds basic cgroup CPU metrics to the node stats API.

Relates #21029
2016-10-24 08:26:56 -04:00
Jason Tedor ecce53f0df Add I/O statistics on Linux
This commit adds a variety of real disk metrics for the block devices
that back Elasticsearch data paths. A collection of statistics are read
from /proc/diskstats and are used to report the raw metrics for
operations and read/write bytes.

Relates #15915
2016-05-17 16:16:39 -04:00
Martijn van Groningen 2fa33d5c47 Added ingest statistics to node stats API
The ingest stats include the following statistics:
* `ingest.total.count`- The total number of document ingested during the lifetime of this node
* `ingest.total.time_in_millis` - The total time spent on ingest preprocessing documents during the lifetime of this node
* `ingest.total.current` - The total number of documents currently being ingested.
* `ingest.total.failed` - The total number ingest preprocessing operations failed during the lifetime of this node

Also these stats are returned on a per pipeline basis.
2016-03-10 13:21:43 +01:00
Jason Tedor df598e8129 Modify load average formats
This commit modifies the load_average in the node stats API response
to be an object containing the one-minute, five-minute and
fifteen-minute load averages as fields (if those values are
available). Additionally, this commit modifies the cat nodes API
response to format the one-minute, five-minute and fifteen-minute load
averages as null if any of the respective values are not available.
2016-01-18 11:41:34 -05:00
Jason Tedor 1de2081ed3 Reintroduce five-minute and fifteen-minute load averages on Linux
This commit reintroduces the five-minute and fifteen-minute load stats
on Linux, and changes the format of the load_average field back to an
array.
2016-01-11 23:42:47 -05:00
Jason Tedor 6872d545ac Add system CPU percent to OS stats
This commit adds the system CPU percent reflecting the recent CPU usage
for the whole system.
2015-11-17 13:48:46 -05:00
xuzha 97ecd7bf5a Expose pending cluster state queue size in node stats
Add 3 stats about the queue: total queue size, number of committed cluster
states, and number of pending cluster states.
2015-10-28 10:59:15 -07:00