Commit Graph

8863 Commits

Author SHA1 Message Date
Ioannis Kakavas cb37989bd9
Add a tool for creating enrollment tokens (#74890)
This change introduces a CLI tool that can be used to create
enrollment tokens. It doesn't require credentials, but simply
write access to the local filesystem of a node. It uses an
auto-generated user in the file-realm with superuser role.

For this purpose, this change also introduces a base class for a
CLI tool that can be used by any CLI tool needs to perform actions
against an ES node as a superuser without requiring credentials
from the user. It is worth noting that this doesn't change our
existing thread model, because already an actor with write access
to the fs of an ES node, can become superuser (again, by
adding a superuser to the file realm, albeit manually).

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-07-15 16:55:35 +03:00
Yannick Welsch db814f403b
Track Lucene field usage (#74227)
Adds a field usage API that reports shard-level statistics about which Lucene fields have been accessed, and which
parts of the Lucene data structures have been accessed.

Field usage statistics are automatically captured when queries are runnning on a cluster. A shard-level search request
that accesses a given field, even if multiple times during that request, is counted as a single use.
2021-07-14 13:21:11 +02:00
Lisa Cawley c8c7f0ef52
[DOCS] Anomaly detection: Visualize delayed data (#75098) 2021-07-13 18:06:07 -07:00
Mayya Sharipova 23781bb555
Remove xpack actions for vector field (#75017)
We have already decided not to have xpack usage for field mappers
(see #53076). As mappings stats of all fields is already tracked
in cluster stats.
Moreover xpack usage for vector field is a quite expensive operation
(see #74974).

This removes xpack actions for vector field.
2021-07-13 09:52:50 -04:00
James Rodewig a9ed7dc1eb
[DOCS] Document async SQL APIs (#75078)
Adds formal API docs and JSON specs for the following APIs:

* Get async SQL search
* Get async SQL search status
* Delete async SQL search

Closes #74845
2021-07-12 10:12:02 -04:00
James Rodewig 75c585c0f1
[DOCS] Mute snippet tests for #75069 (#75237) 2021-07-12 09:50:18 -04:00
James Rodewig d9597da0ef
[DOCS] Update security prereqs for delete async EQL API (#75091) 2021-07-12 08:49:55 -04:00
Nhat Nguyen 5fb4c05095
Deprecate SimpleFS and replace it with NIOFS (#75156) (#75196)
SimpleFS is deprecated and will be removed in Lucene 9. This commit
deprecates SimpleFS in 7.x and uses NIOFS for SimpleFS in Elasticsearch
7.15 or later as it offers superior or equivalent performance to
SimpleFS.
2021-07-09 18:22:41 -04:00
Lisa Cawley 3c76bcb3a5
[DOCS] Fixes links to machine learning concepts (#75194) 2021-07-09 13:09:03 -07:00
Rory Hunter d08b851a5b
Accept settings in snake case in Docker image (#74327)
Closes #74036. Since some orchestration platforms forbid periods in
environment variable names, allow Docker users to pass settings to ES
using an alternative name scheme. For example:

    bootstrap.memory_lock

...becomes:

    ES_BOOTSTRAP_MEMORY__LOCK

The setting name is uppercased, prefixed, all underscores are converted
to double underscores, and all periods are converted to underscores.
2021-07-09 19:46:58 +01:00
Francisco Fernández Castaño abc7a4776d
Avoid auto following leader system indices in CCR (#72815)
Relates #67686
2021-07-09 18:08:41 +02:00
André Pessanha bb37e09d92
Rename field_masking_span to span_field_masking (#74718)
`field_masking_span` is the only span query that does not begin with
`span_`.  This commit deprecates the existing name and adds a new
name `span_field_masking` to better fit with the other queries.
2021-07-09 08:56:38 +01:00
Julie Tibshirani cdf67e0fd5
Support search slicing with point-in-time (#74457)
This PR adds support for using the `slice` option in point-in-time searches. By
default, the slice query splits documents based on their Lucene ID. This
strategy is more efficient than the one used for scrolls, which is based on the
`_id` field and must iterate through the whole terms dictionary. When slicing a
search, the same point-in-time ID must be used across slices to guarantee the
partitions don't overlap or miss documents.

Closes #65740.
2021-07-08 15:33:41 -07:00
István Zoltán Szabó 6a4de77e11
[DOCS] Adds classification and regression links back to DFA docs. (#74930) 2021-07-08 16:37:16 +02:00
Felix Barnsteiner 0a8f725647
Support for wildcards and override option for dot_expander processor (#74601) 2021-07-08 14:39:27 +02:00
Felix Barnsteiner 404e9865b6
Json processor: add_to_root_conflict_strategy option (#74967) 2021-07-08 14:36:22 +02:00
James Rodewig 4e005c041c
[DOCS] EQL: Fix delete async EQL search snippet (#75093)
The delete async EQL search API doesn't support the `keep_alive` query parameter.
2021-07-07 16:49:22 -04:00
James Rodewig a961d828d9
[DOCS] Fix allowed `refresh` args for several doc APIs (#74749)
We incorrectly list `wait_for` as a valid `refresh` argument for the
following APIs:

* Delete by query
* Multi get
* Reindex

This fixes that error. It also updates the get API docs for consistency.

Closes #65031
2021-07-07 10:18:21 -04:00
James Rodewig 76938006ab
[DOCS] Note required node roles and data tiers (#74566)
Closes #74528 and #74565
2021-07-07 09:57:32 -04:00
James Rodewig 3971522c65
[DOCS] EQL: Document cross-cluster search support (#74995) (#75045)
Closes #74842.
2021-07-07 09:41:07 -04:00
James Rodewig 0f5a0e711e
[DOCS] SQL: Document async SQL search (#74975) (#75037)
Documents async SQL search functionality.

I plan to add formal API documentation for the async APIs with a later PR.

Relates to #73991 and #74845.
# Conflicts:
#	docs/reference/release-notes/highlights.asciidoc
2021-07-07 09:07:03 -04:00
Marios Trivyzas dd302dcfef
EQL: [Docs] Add documentation for the CircuitBreaker (#74897)
Add documentation for the newly introduced CircuitBreaker, which is
used to restrict the memory usage for an EQL sequence query to avoid
OutOfMemory exceptions.

Follows: #74381
2021-07-07 09:20:25 +02:00
István Zoltán Szabó 841cfb9214
[DOCS] Adds outlier detection links to DFA API docs (#74748) 2021-07-06 15:10:41 +02:00
Felix Barnsteiner 67fbc337ea
Json processor: allow duplicate keys (#74956) 2021-07-06 15:02:32 +02:00
Lisa Cawley b71b7d0866
[DOCS] Fix links to anomaly detection overview (#74943) 2021-07-05 13:19:54 -07:00
Lisa Cawley 4c85852cc7
[DOCS] Update forecasting links in ML APIs (#74942) 2021-07-05 12:34:03 -07:00
Lisa Cawley 5bcd318e29
[DOCS] Move ML functions to appendix (#74802) 2021-07-05 11:53:17 -07:00
István Zoltán Szabó 483d145f78
[DOCS] Fixes an attribute in PUT DFA API docs. (#74931) 2021-07-05 17:08:11 +02:00
István Zoltán Szabó 6c6e6874ff
[DOCS] Removes link to classification and regression. (#74926) 2021-07-05 16:28:14 +02:00
Esduard b3e171b600
changing hard coded 10k page size limit to 65k (#74651)
changes hard code 10k page size limit to 65k(default for `search.max_bucket`)

Closes #57719

Co-authored-by: Hendrik Muhs <hendrik.muhs@elastic.co>
2021-07-05 15:21:36 +02:00
David Turner 963d9aac96
Generalize TCP retxn docs to cover remote clusters (#74732)
Today the docs on setting `tcp_retries2` only talk about intra-cluster
connections, but in fact this setting is equally important to the
resilience of remote cluster connections too. This commit rewords these
docs to cover both cases.

Relates #34405
2021-07-05 13:38:29 +01:00
Yannick Welsch 412ac1a042
Update docs that composite agg no longer uses global ords (#74754)
Follow-up to #74559
2021-07-05 11:26:30 +02:00
James Rodewig 845446aec3 [DOCS] EQL: Remove erroneous CSS reference 2021-07-01 17:05:44 -04:00
James Rodewig 70764de4b6
[DOCS] Move EQL APIs to separate page (#74846)
In preparation for #74845, we need to create formal API reference documentation for our SQL APIs.

Due to the number of SQL APIs, we'll likely need to create a separate nested page for them. For parity, this PR moves
our EQL APIs to a separate page as well. Previously, they were listed under our search APIs.
2021-07-01 13:59:35 -04:00
James Rodewig d522c28533 [DOCS] Reword EQL limitations intro. 2021-07-01 10:24:32 -04:00
Adam Locke 6974f5861a
[DOC] Update tests for collapse search results (#74721)
* [DOC] Update tests for collapse search results

* Add filter for inner hits

* Fixing tests

* Fix typos
2021-07-01 10:21:59 -04:00
Liam MacPherson ba227f99f0
[DOCS] Fix <index> param def for delete index API (#74674)
Clarifies that you cannot specify an alias in the delete index API. You _can_ delete indices with an alias.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-07-01 10:07:14 -04:00
James Rodewig 3d1cb4944d
[DOCS] Overhaul search template docs (#72583)
Changes:

* Adds a tutorial for search templates.
* Adds reference docs for the render search template API.
* Improves parameter documentation for the multi search template API.
* Removes duplicate examples from the search template API, multi search API, and create stored script API docs.
* Splits the source files for the search template API and the multi search template API docs.
2021-06-30 16:03:49 -04:00
Adam Locke b759c2fdd8
[DOCS] Word changes for runtime field incentives (#74769)
Incorporates feedback from #74454
2021-06-30 13:43:26 -04:00
Lisa Cawley 9ab6808206
[DOCS] Clean up xpack.ml.enabled details (#74573) 2021-06-30 09:34:46 -07:00
debadair 29bb8fd67e
[DOCS] Add info about GET/POST to the API conventions. (#74529)
* [DOCS] Add info about GET/POST to the API conventions.

* Fixed anchor

* Added redirect

* Incorporated review suggestions
2021-06-30 09:24:42 -07:00
Nhat Nguyen c23cb99b71
Add index disk usage API (#74051)
This change adds a new API that supports analyzing the disk usage of 
each field of an index.
2021-06-30 11:50:01 -04:00
Stef Nestor fdcb9c3247
[DOCS] Note ILM `searchable_snapshot` action requires data tiers (#74706) 2021-06-30 11:03:11 -04:00
Mayya Sharipova aa76ebbfbe
Set max allowed size for stored async response (#74455)
Add a dynamic transient cluster setting search.max_async_search_response_size
that controls the maximum allowed size for a stored async search
response. The default max size is 10Mb. An attempt to store
an async search response larger than this size will result in error.

Relates to #67594
2021-06-30 10:21:28 -04:00
István Zoltán Szabó a4f9f4fae1
[DOCS] Comments out links to outlier detection. (#74745) 2021-06-30 14:24:34 +02:00
Tim Brooks 293d490ded
Add additional transport compression options (#74587)
This commit is related to #73497. It adds two new settings. The first setting
is transport.compression_scheme. This setting allows the user to
configure LZ4 or DEFLATE as the transport compression. Additionally, it
modifies transport.compress to support the value indexing_data. When
this setting is set to indexing_data only messages which are primarily
composed of raw source data will be compressed. This is bulk, operations
recovery, and shard changes messages.
2021-06-29 12:14:47 -06:00
bellengao cd190dec2f
[DOCS] Fix typos in get follower stats API doc (#74599)
There are some typos in the document of get follower stats API.
2021-06-29 11:29:52 -04:00
Adam Locke b890f9380c
[DOCS] Add performance info for runtime fields (#74454)
* [DOCS] Add performance info for runtime fields

* Add script-based sorting and clarify performance

* Changing title to Incentives and reworking the intro
2021-06-29 10:23:00 -04:00
Andrei Dan 5e9405a513
[DOCS]: allow multiple searchable_snapshot actions in the same policy (#74679)
Since `7.12` we allow multiple searchable_snapshot actions in the same
policy. This updates the docs to reflect this.
2021-06-29 14:20:26 +01:00
James Rodewig d4ed43c5a4
[DOCS] Remove deprecated `geo_shape` parameters (#74519)
* Removes docs and references for the following `geo_shape` mapping parameters:
  * `tree`
  * `tree_levels`
  * `strategy`
  * `distance_error_pct`
* Updates a related breaking change.

Relates to #70850
2021-06-29 08:52:05 -04:00