Commit Graph

8333 Commits

Author SHA1 Message Date
Nik Everett fe457f156d
Docs: Call out that you can't update analyzer (#69889)
You can't update the `analyzer` parameter in the PUT mappings API even if
the index is closed. This adds a TIP to call that out. And adds a TIP
for `search_quote_analyzer` which you *can* update.
2021-03-03 10:28:55 -05:00
Joe Gallo 1e8b5fa7c2
Remove the _ml/find-file-structure docs (#69823) 2021-03-03 09:49:28 -05:00
James Rodewig 67288a1e4d [DOCS] Fix gap policy xref 2021-03-03 09:31:02 -05:00
Mike Barretta d6047a966f
[DOCS] Fix typo (#69838) 2021-03-03 09:16:39 -05:00
James Rodewig 630604bd45
[DOCS] Fix case sensitivity for elision token filter (#69873) 2021-03-03 09:09:05 -05:00
James Rodewig d6492c6392 [DOCS] Reword `terms` rollup config 2021-03-02 16:08:51 -05:00
James Rodewig e21cab640f
[DOCS] Reformat avg bucket agg reference (#69751) 2021-03-02 13:44:43 -05:00
Gordon Brown ce8a0c0cea
Change Get Snapshottable Features endpoint to `_features` (#69755)
The endpoint `_snapshottable_features` is long and implies incorrect
things about this API - it is used not just for snapshots, but also for
the upcoming reset API. Following discussions on the team, this commit
changes the endpoint to `_features` and removes the connection between
this API and snapshots, as snapshots are not the only use for the output
of this API.
2021-03-02 11:30:02 -07:00
Nik Everett ea131e5f5a
Docs: Switch terms agg scripting to runtime fields (#69628)
We expect runtime fields to perform a little better than our "native"
aggregation script so we should point folks to them instead of the
"native" aggregation script.
2021-03-02 11:27:21 -05:00
Andrei Stefan bf1b7a36b5
SQL: Adapt the limitations page to the new "fields" API usage (#69616)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-03-02 17:19:32 +02:00
Lisa Cawley 4c39136837
[DOCS] Fix link in machine learning nightly maintenance setting (#69760) 2021-03-01 17:14:21 -08:00
James Rodewig 3c70b0e3d0
[DOCS] Add xref for runtime fields (#69738) 2021-03-01 16:14:23 -05:00
Lyudmila Fokina ee66d6f11f
Support audit ignore policy by actions (#67477)
* Support audit ignore policy by index privileges

Adding new audit ignore policy - privileges
For example, following policy will filter out all events, which actions
minimal required privilege is either "read" or "delete":

xpack.security.audit.logfile.events.ignore_filters:
  example:
    privileges: ["read", "delete"]

Resolve: #60877
Related: #10836
Related: #37148

* Support audit ignore policy by index privileges

Adding new audit ignore policy - privileges
For example, following policy will filter out all events, which actions
required privilege is either "read" or "delete":

xpack.security.audit.logfile.events.ignore_filters:
  example:
    privileges: ["read", "delete"]

Resolve: #60877
Related: #10836
Related: #37148

* To avoid ambiguity (as cluster and index policies may have the same
name) changing implementation to have to separate policies for
`index_privileges` and `cluster_privileges`.
If both are set for the same policy, throw the IllegalArgumentException.

* To avoid ambiguity (as cluster and index policies may have the same
name) changing implementation to have to separate policies for
`index_privileges` and `cluster_privileges`.
If both are set for the same policy, throw the IllegalArgumentException.

* Fixing Api key related privilege check which expects request and
authentication by introducing overloaded
version of findPrivilegesThatGrant
just checking if privileges which can grant the action regardless of the
 request and authentication context.

* Fixing a test; adding a caching mechanism to avoid calling
findPrivilegesThatGrant each
 time.

* Support audit ignore policy by index privileges

Addressing review feedback

* Support audit ignore policy by index privileges

Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing

* Support audit ignore policy by index privileges

Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing

* Support audit ignore policy by index privileges

Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing

* Support audit ignore policy by index privileges

Addressing review comments + changing approach:
- use permission check instead of simple "checkIfGrants"
- adding more testing

* Revert "Support audit ignore policy by index privileges"

This reverts commit 152821e7

* Revert "Support audit ignore policy by index privileges"

This reverts commit 79649e9a

* Revert "Support audit ignore policy by index privileges"

This reverts commit 96d22a42

* Revert "Support audit ignore policy by index privileges"

This reverts commit 67574b2f

* Revert "Support audit ignore policy by index privileges"

This reverts commit 35573c8b

* Revert "Fixing a test; adding a caching mechanism to avoid calling findPrivilegesThatGrant each  time."

This reverts commit 7faa52f3

* Revert "Fixing Api key related privilege check which expects request and authentication by introducing overloaded version of findPrivilegesThatGrant just checking if privileges which can grant the action regardless of the  request and authentication context."

This reverts commit 72b9aefe

* Revert "To avoid ambiguity (as cluster and index policies may have the same name) changing implementation to have to separate policies for `index_privileges` and `cluster_privileges`. If both are set for the same policy, throw the IllegalArgumentException."

This reverts commit 7dd8fe7d

* Revert "To avoid ambiguity (as cluster and index policies may have the same name) changing implementation to have to separate policies for `index_privileges` and `cluster_privileges`. If both are set for the same policy, throw the IllegalArgumentException."

This reverts commit cb5bc09c

* Revert "Support audit ignore policy by index privileges"

This reverts commit a918da10

* Support audit ignore policy by actions

Getting back to action filtering

* Support audit ignore policy by actions

Cleaning up some tests

* Support audit ignore policy by actions

Cleaning up some tests

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-03-01 18:44:57 +01:00
Benjamin Trent 2279cafb4e
[ML] adding new _preview endpoint for data frame analytics (#69453)
This commit adds a new `_preview` endpoint for data frame analytics. 

This allows users to see the data on which their model will be trained. This is especially useful 
in the arrival of custom feature processors.

The API design is a similar to datafeed `_preview` and data frame analytics `_explain`.
2021-03-01 12:25:50 -05:00
James Rodewig 783769d8d9
[DOCS] Add `fields` parameter to EQL search API (#69634) 2021-03-01 12:00:27 -05:00
Yannick Welsch 529c6227fe
Support include_unloaded_segments in node stats (#69682)
Adds support for the include_unloaded_segments flag in node stats, which helps with understanding resource usage of
shared_cache-style searchable snapshots on a per-node basis.
2021-03-01 17:18:47 +01:00
José Arthur Benetasso Villanova 3b1c03dc49
[DOCS] Fix typo (#69654) 2021-03-01 09:34:56 -05:00
James Rodewig cb25ae06ed
[DOCS] Fix name of `cluster_version` parameter (#69615) 2021-03-01 08:54:47 -05:00
RomainGeffraye fe7afb9d36
[DOCS] Update example for `serial_diff` agg (#69635) 2021-03-01 08:37:29 -05:00
David Turner 86b97ab5c9
Note that forcemerges now run in parallel in docs (#69688)
Relates #69416
2021-03-01 13:15:24 +00:00
István Zoltán Szabó 88bc27592d
[DOCS] Reviews ML decider conceptual docs (#69524)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2021-03-01 09:56:39 +01:00
James Rodewig 593cac391d [DOCS] Make whitespace consistent in JSON snippets 2021-02-25 16:12:47 -05:00
James Rodewig 3ff1a17a79
[DOCS] EQL: Document field existence checks (#69614) 2021-02-25 12:04:22 -05:00
Lisa Cawley efa9b095aa
[DOCS] Adds model alias to inference processor and agg (#69576) 2021-02-24 13:12:39 -08:00
Tal Levy c1c5103756
Generate random rollup index names for RollupILMAction (#69237)
This commit moves away from the static `rollup-{indexName}` rollup index
naming strategy and moves towards a randomized rollup index name scheme.

This will reduce the complications that exist if the RollupStep fails and retries
in any way. A separate cleanup will still be required for failed temporary indices,
but at least there will not be a conflict.

This commit generates the new rollup index name in the LifecycleExecutionState so
that it can be used in RollupStep and UpdateRollupIndexPolicyStep on a per-index
basis.
2021-02-24 12:31:36 -08:00
Adam Locke 1ee4c50217
[DOCS] Remove beta admonition for runtime fields. (#69550)
* [DOCS] Remove beta admonition for runtime fields.

* Remove other beta admonition from Painless guide.
2021-02-24 11:35:11 -05:00
Lisa Cawley 138224b398
[DOCS] Edits trained model alias API (#69491) 2021-02-24 08:17:49 -08:00
James Rodewig 2048eb7eef
[DOCS] Note `index.number_of_routing_shards` affects doc distribution (#69541) 2021-02-24 10:31:01 -05:00
Dimitris Athanasiou bbf81a2603
[ML] Expand usage stats for data frame analytics and trained models (#69477)
This adds additional statistics into the usage API for data frame analytics
and trained models.

For data frame analytics the added stats are:

  - count of jobs by analysis type
  - stats for peak_usage_bytes

For trained models the added stats are:
  - counts of: total, prepackaged, other (not created by data frame analytics)
  - counts by analysis type based on the inference config
  - stats for estimated heap usage
  - stats for estimated number of operations
2021-02-24 15:45:26 +02:00
David Turner e88038575d
Document searchable snapshots supported repos (#69508)
Adds a note listing the repository types that can be used with
searchable snapshots.
2021-02-24 09:02:16 +00:00
Igor Motov 7ad0201b25
Clarify the intended use case for multi_terms aggs (#69397)
This PR clarifies when multi_terms aggs should be used instead of composite
aggs or nested term aggs.

Relates to #65623
2021-02-23 15:11:53 -05:00
Benjamin Trent 1438434b6c
[Transform] add support for geo_line aggregation in pivot function (#69299)
This commit adds support for the Gold+ licensed `geo_line` aggregation.

This aggregation takes a collection of `geo_point` values and constructs a line
according to some sort value. Adding to transforms allows users to create these
potentially expensive lines out of band of visualizations and then do additional aggs/queries
against the pivoted data. 

Examples would be:

"Do these daily user paths ever intersect?"
"Does this path enter and leave this area?"
2021-02-23 14:53:36 -05:00
James Rodewig 35c02c45f7
[DOCS] Note `case_sensitive` param was added in 7.10 (#69405) (#69466)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Bhavya Gupta <46423346+bhavya121999@users.noreply.github.com>
2021-02-23 13:12:28 -05:00
Adam Locke 2362549818
[DOCS] Adding grok support for runtime fields. (#69308)
* [DOCS] Adding grok support for runtime fields.

* Update response.

* Adding testresponse replacements.

* Update runtime field context and add dissect.

* Fixing backslash in the response.

* Fixing testresponse.

* Incorporating review feedback.

* Updates emit and adds cross link from ES runtime fields page.
2021-02-23 12:47:11 -05:00
James Rodewig 5ff8b8c730
[DOCS] Remove outdated default distro refs (#69465) 2021-02-23 12:26:57 -05:00
James Rodewig a32cf65705
[DOCS] Reword node roles docs (#69301) 2021-02-23 11:32:46 -05:00
James Rodewig 9af74ec561
[DOCS] Remove added admons (#69452) 2021-02-23 10:35:21 -05:00
James Rodewig a85f9cade8
[DOCS] Use consistent @timestamp field name (#69435) (#69448)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Koji Kawamura <ijokarumawak@users.noreply.github.com>
2021-02-23 10:20:17 -05:00
István Zoltán Szabó b4057d7c22
[DOCS] Adds new screenshot to Transform tutorial (#69194)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2021-02-23 15:23:51 +01:00
Marios Trivyzas c5cd7e51ef
SQL: [Docs] Use the most common `yyyy` year pattern in examples (#69407)
To avoid confusion for the users replace the `YYYY` and `uuuu` year
patterns in the examples of `DATETIME_FORMAT/PARSE` with the most common
`yyyy` to avoid any confusion for users that might just copy paste those
queries for their own use case.

Relates to #68030
2021-02-23 13:48:03 +01:00
István Zoltán Szabó 77d0f56581
[DOCS] Adds anomaly detection alert documentation (#68923)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2021-02-23 10:29:54 +01:00
Lisa Cawley 50fd9e40a7
[DOCS] Fixes italics and capitalization (#69387) 2021-02-22 13:20:06 -08:00
David Turner bb3ea99850
Skip zone/host awareness with auto-expand replicas (#69334)
Today if an index is set to `auto_expand_replicas: N-all` then we will
try and create a shard copy on every node that matches the applicable
allocation filters. This conflits with shard allocation awareness and
the same-host allocation decider if there is an uneven distribution of
nodes across zones or hosts, since these deciders prevent shard copies
from being allocated unevenly and may therefore leave some unassigned
shards.

The point of these two deciders is to improve resilience given a limited
number of shard copies but there is no need for this behaviour when the
number of shard copies is not limited, so this commit supresses them in
that case.

Closes #54151
Closes #2869
2021-02-22 16:53:58 +00:00
James Rodewig a453a9267d
[DOCS] Add frozen node to cat nodes API (#69228) 2021-02-22 11:24:21 -05:00
James Rodewig b7aaaad20a
[DOCS] Improve docs for `geo_shape` field type's `circle` type (#69285) 2021-02-22 10:24:24 -05:00
James Rodewig e4962994ff
[DOCS] Remove performance warning for script fields (#69309) 2021-02-22 10:05:49 -05:00
Henning Andersen d4a7aa26c1
Autoscaling test scale from empty with node attrs (#68730)
Autoscaling expects data tiers to be used exclusively both for node
roles and in ILM policies. This commit adds a test demonstrating that
as well as documentation for the behavior.
2021-02-22 15:47:15 +01:00
James Rodewig 0cbab23e80
[DOCS] Update ILM tutorial docs for UI changes (#69189) 2021-02-19 12:56:58 -05:00
Dimitris Athanasiou 7fb98c0d3c
[ML] Add runtime mappings to data frame analytics source config (#69183)
Users can now specify runtime mappings as part of the source config
of a data frame analytics job. Those runtime mappings become part of
the mapping of the destination index. This ensures the fields are
accessible in the destination index even if the relevant data frame
analytics job gets deleted.

Closes #65056
2021-02-19 16:29:19 +02:00
Jean-Louis Leysens 867e656df7
[ILM][Docs] Updated existing screenshots (#69173)
* updated existing screenshots

* change 365 days -> 90 days for customize policy tutorial
2021-02-19 10:11:36 +01:00