Commit Graph

8565 Commits

Author SHA1 Message Date
James Rodewig 7cd6952986
[DOCS] Move JSON encoding section (#71508) 2021-04-10 09:07:24 -04:00
James Rodewig 0f3d53b517 [DOCS] Fix create enrich policy API title 2021-04-08 17:53:27 -04:00
James Rodewig a613f039fb
[DOCS] Document REST API uses UTF-8 encoding (#71474) 2021-04-08 15:44:09 -04:00
James Rodewig 3d172da8dd
[DOCS] Fix create enrich policy API title (#71494) 2021-04-08 15:35:53 -04:00
Julie Tibshirani 3da738e5db
Support fetching _tier field value (#71379)
Now that the `fields` option allows fetching metadata fields, we can support
loading the new `_tier` metadata field.

Relates to #63569 and #68135.
2021-04-08 11:41:52 -07:00
Adam Locke 5baabff667
[DOCS] Update mapping API to require index name (#71489) 2021-04-08 13:54:37 -04:00
James Rodewig 44922f6b3d [DOCS] Fix search template label 2021-04-08 13:14:10 -04:00
Nhat Nguyen 5c9969250d
Allow specify dynamic templates in bulk request (#69948)
This change allows users to specify dynamic templates in a bulk request.

```
PUT myindex
{
  "mappings": {
    "dynamic_templates": [{
      "time_histograms": {
        "mapping": {
          "type": "histogram",
          "meta": {
            "unit": "s"
          }
        }
      }
    }]
  }
}
```

```
POST myindex/_bulk
{ "index": { "dynamic_templates": { "response_times": "time_histograms" } } }
{ "@timestamp": "2020-08-12", "response_times": { "values": [1, 10], "counts": [5, 1] }}
```

Closes #61939
2021-04-08 12:44:36 -04:00
James Rodewig de228ee153
[DOCS] Reorder EQL sections. Remove duplicated content. (#71477) 2021-04-08 10:45:33 -04:00
James Rodewig 3ca5a93446 [DOCS] Fix GeoIP capitalization 2021-04-07 14:26:32 -04:00
Adam Locke 343c52c19f
[DOCS] Adding page for indexing runtime fields (#71366)
* [DOCS] Adding page for indexing runtime fields

* Fixing tests.

* Incorporating review feedback to enhance and improve examples.

* Changing note to indicate immutable script when indexing, plus adding on_script_error.
2021-04-07 13:07:39 -04:00
James Rodewig 14e6b073a1
[DOCS] Add redirect for missing geoIP stats API docs (#71402) 2021-04-07 09:15:44 -04:00
Gonzalo Servat 3a5eba485b
Fix typo in "handing" (missing "l") (#71383) 2021-04-07 08:07:50 -04:00
David Turner 6ed2d25458
Include node roles in cluster state JSON response (#71386)
Today the response to `GET _cluster/state` does not include the roles of
the nodes in the cluster. In the past this made sense, roles were
relatively unchanging things that could be determined from elsewhere.
These days we have an increasingly rich collection of roles, with
nontrivial BWC implications, so it is important for debugging to be able
to see the specific roles as viewed by the master. This commit adds the
role names to the cluster state API output.

Relates #71385
2021-04-07 10:44:35 +01:00
James Rodewig b127447548 [DOCS] Fix cat API name 2021-04-06 17:17:05 -04:00
James Rodewig 1f361ad2d1
[DOCS] Update size your shards for `max_primary_shard_size` (#71367) 2021-04-06 17:08:24 -04:00
James Rodewig 40b491b2f1
[DOCS] Use HTML-unescaped Mustache variables in ingest pipelines (#71360) 2021-04-06 15:33:10 -04:00
Nik Everett e158bc10b1
Convert `boolean` field example to runtime fields (#71341)
Runtime fields are much more flexible than `script_fields` because you
can filter and aggregate on them so we hope folks use them! This
converts the example of using a `boolean` field in a script to a runtime
field so folks get used to seeing them and hopefully using them.

While I was editing this I took the opportunity to replace the script
with a real-ish example. Scripts that just load the field value are nice
and short but I hope no one uses them in real life because they just add
overhead when compared to accessing the field directly. So I made the
script do *something*.

Relates to #69291
2021-04-06 14:42:44 -04:00
James Rodewig 9fff70c3b2 [DOCS] Fix xref 2021-04-06 12:10:39 -04:00
James Rodewig 65be40ac7b
[DOCS] Refactor rollover API docs (#70938) 2021-04-06 11:51:02 -04:00
Adam Locke af700f4628
[DOCS] Update runtime fields for script query (#71338)
Fixes typo, moves example out of a NOTE admonition, and puts context before the example.
2021-04-06 10:12:08 -04:00
Alan Woodward 98c9a95e12
Add note that scripted fields will reject documents with a source value in their field (#71340) 2021-04-06 14:28:20 +01:00
James Rodewig 07e9c6aca4 [DOCS] Swap `event.original` for `message` 2021-04-06 06:51:58 -04:00
Tanguy Leroux e26e14441e
Document CCR auto-follow patterns and searchable snapshots indices (#70863)
This commit adds a note in CCR document about auto-follow 
patterns that should not match searchable snapshots indices.

Relates #70580 (comment)
2021-04-06 10:47:00 +02:00
Ryan Ernst f6b833cf27
Add multiple data paths deprecation to docs (#71312)
This commit adds a deprecation note to the multiple data paths doc. It also removes mention of multiple paths support in the setup settings table.

relates #71205
2021-04-05 16:28:19 -07:00
James Rodewig 94d88082ec [DOCS] Swap `message` for `event.original` 2021-04-05 13:57:59 -04:00
Nik Everett 5677c6822e
Point script query docs at runtime fields (#71291)
This adds a "note" on the docs for the script query pointing folks to
runtime fields because they are more flexible. It also translates the
request example into runtime fields.

Relates to #69291

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-04-05 13:11:29 -04:00
Nik Everett 6a1220e7f3
Convert metric aggs docs runtime fields (#71260)
This replaces the `script` docs for bucket aggregations with runtime
fields. We expect runtime fields to be nicer to work with because you
can also fetch them or filter on them. We expect them to be faster
because their don't need this sort of `instanceof` tree:
a92a647b9f/server/src/main/java/org/elasticsearch/search/aggregations/support/values/ScriptDoubleValues.java (L42)

Relates to #69291

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-04-05 13:08:13 -04:00
James Rodewig c23f001151
[DOCS] Add how-to guide for time series data (#71195) 2021-04-05 13:00:06 -04:00
James Rodewig d3c56e6fca [DOCS] Remove unneeded articles for Elasticsearch Service and Elastic Agent 2021-04-02 16:01:59 -04:00
Adam Locke 14aba7bcff
[DOCS] Expand examples for runtime fields in a search query (#71237)
* Add warning admonition for removing runtime fields.

* Add cross-link to runtime fields.

* Expanding examples for runtime fields in a search request.

* Clarifying language and simplifying response tests.
2021-04-02 15:00:54 -04:00
Jason Tedor d340432622
Remove frozen cache setting leniency (#71013)
We previously allowed but deprecated the ability for the shared cache to
be positively sized on nodes without the frozen role. This is because we
only allocate shared_cache searchable snapshots to nodes with the frozen
role. This commit completes our intention to deprecate/remove this
ability.
2021-04-02 14:26:01 -04:00
Nik Everett a9d9ee0d4b
Convert bucket aggs docs to runtime fields (#71202)
This replaces the `script` docs for bucket aggregations with runtime
fields. We expect runtime fields to be nicer to work with because you
can also fetch them or filter on them. We expect them to be faster
because their don't need this sort of `instanceof` tree:
a92a647b9f/server/src/main/java/org/elasticsearch/search/aggregations/support/values/ScriptDoubleValues.java (L42)

Relates to #69291

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-04-02 12:12:06 -04:00
James Rodewig 55f3db74a6
[DOCS] Fix 'Register a snapshot repo' title (#71224) 2021-04-02 09:26:20 -04:00
James Rodewig 0b9f71f3f5
[DOCS] Rename Glossary (#71222)
Changes title from "Glossary of terms" to "Glossary."
"Glossary of terms" is redundant.
2021-04-02 09:12:05 -04:00
James Rodewig 4963118fcc
[DOCS] Update button copy (#71220) 2021-04-02 09:06:50 -04:00
Dan Hermann 579d0367b1
[DOCS] http.client_stats.enabled setting (#71188) 2021-04-02 07:58:05 -05:00
Jason Tedor a5a5278954
Remove legacy role settings (#71163)
This commit removes the previously deprecated legacy role
settings. These settings have been replaced by node.roles.
2021-04-01 19:31:55 -04:00
James Rodewig 3c02ab4190
[DOCS] Add ECS and runtime fields tip to data stream tutorial (#71183) 2021-04-01 15:54:53 -04:00
James Rodewig 2dd034bc65
[DOCS] Rename ES Reference to ES Guide (#71198) 2021-04-01 15:38:41 -04:00
James Rodewig 115d2b2680 [DOCS] Reword data stream indexing section 2021-04-01 10:23:48 -04:00
James Rodewig a4fa22c099 [DOCS] Reword link to Fleet data stream docs 2021-04-01 10:15:56 -04:00
James Rodewig 671dc331cd [DOCS] Remove unused widget files 2021-04-01 08:35:46 -04:00
James Rodewig f41320616c
[DOCS] Refactor data stream setup tutorial (#71074) 2021-03-31 17:28:55 -04:00
Adam Locke f06dc219b2
[DOCS] Fixes deprecation message for Geo-polygon query (#71141)
* [DOCS] Fixes deprecation message for Geo-polygon query

* Change deprecation to full block admonition.
2021-03-31 16:37:29 -04:00
William Brafford 6b349af7d9
Add breaking change notice for action.destructive_requires_name (#71007)
* Add breaking change notice for action.destructive_requires_name
2021-03-31 16:00:29 -04:00
Yash Jipkate 60f4d22722
Change default value of `action.destructive_requires_name` to True. (#66908)
This PR sets the default value of `action.destructive_requires_name`
to `true.` Fixes #61074. Additionally, we set this value explicitly in
test classes that rely on wildcard deletions to clear test state.
2021-03-31 15:59:57 -04:00
Julie Tibshirani d02df3f3fe
Remove the beta flag from 'fields' option. (#71130)
Now that we've addressed the open issues, the 'fields' option can be considered GA.

Relates to #60985.
2021-03-31 10:00:42 -07:00
James Rodewig 9ab1a6caa3
[DOCS] Fix put lifecycle policy API title (#71124) 2021-03-31 11:37:45 -04:00
markharwood 3aee4c1f1f
New queryable "_tier" metadata field (#69288)
New _tier metadata field that supports term, terms, exists and wildcard queries on the first data tier preference stated for an index.

Closes #68135
2021-03-31 15:37:37 +01:00
James Rodewig 693807a6d3
[DOCS] Fix double spaces (#71082) 2021-03-31 09:57:47 -04:00
James Rodewig 27abdd9f2a
[DOCS] Document ingest pipelines for Fleet and Elastic Agent (#70907) 2021-03-31 09:01:18 -04:00
James Rodewig 955df4337b
[DOCS] Fix typos for Elasticsearch Service and Elastic Agent (#71076) 2021-03-31 08:36:21 -04:00
James Rodewig db6da195d0 [DOCS] Fix typo 2021-03-31 08:14:47 -04:00
Alan Woodward 1653f2fe91
Add script parameter to long and double field mappers (#69531)
This commit adds a script parameter to long and double fields that makes
it possible to calculate a value for these fields at index time. It uses the same
script context as the equivalent runtime fields, and allows for multiple index-time
scripted fields to cross-refer while still checking for indirection loops.
2021-03-31 11:14:11 +01:00
Henning Andersen 0f28e97857
Total data set size in stats (#70625)
With shared cache searchable snapshots we have shards that have a size
in S3 that differs from the locally occupied disk space. This commit
introduces `store.total_data_set_size` to node and indices stats, allowing to
differ between the two.

Relates #69820
2021-03-30 15:23:29 +02:00
Benjamin Trent c8415a7924
[ML] adding support for composite aggs in anomaly detection (#69970)
This commit allows for composite aggregations in datafeeds. 

Composite aggs provide a much better solution for having influencers, partitions, etc. on high volume data. Instead of worrying about long scrolls in the datafeed, the calculation is distributed across cluster via the aggregations. 

The restrictions for this support are as follows:

- The composite aggregation must have EXACTLY one `date_histogram` source
- The sub-aggs of the composite aggregation must have a `max` aggregation on the SAME timefield as the aforementioned `date_histogram` source
- The composite agg must be the ONLY top level agg and it cannot have a `composite` or `date_histogram` sub-agg
- If using a `date_histogram` to bucket time, it cannot have a `composite` sub-agg.
- The top-level `composite` agg cannot have a sibling pipeline agg. Pipeline aggregations are supported as a sub-agg (thus a pipeline agg INSIDE the bucket).

Some key user interaction differences:
- Speed + resources used by the cluster should be controlled by the `size` parameter in the `composite` aggregation. Previously, we said if you are using aggs, use a specific `chunking_config`. But, with composite, that is not necessary. 
- Users really shouldn't use nested `terms` aggs anylonger. While this is still a "valid" configuration and MAY be desirable for some users (only wanting the top 10 of certain terms), typically when users want influencers, partition fields, etc. they want the ENTIRE population. Previously, this really wasn't possible with aggs, with `composite` it is.
- I cannot really think of a typical usecase that SHOULD ever use a multi-bucket aggregation that is NOT supported by composite.
2021-03-30 08:25:40 -04:00
James Rodewig 15f863813c [DOCS] Fix wording 2021-03-30 06:50:53 -04:00
James Rodewig 69db7ce171
[DOCS] Remove dupe `wait_for_completion` def (#71012) 2021-03-30 06:46:57 -04:00
James Rodewig ea347302e2
[DOCS] Add reusable tests to ingest docs (#70990) 2021-03-30 06:38:34 -04:00
Jim Ferenczi fa88a46b9c
Expose if a field is a metadata field in the field capabilities response (#69977)
This change exposes for each field in the _field_caps response if the field is a metadata field.
This is needed for consumers of this API that want to filter these fields. Currently ML keeps a static list
and QL checks that the family type starts with `_`. In order to ease the addition of new metadata fields, this
change reworks the strategy in this solution and now only checks for the new flag.
Note that the new flag is also applied at the coordinator level in a best-effort to apply the logic on older nodes
in a mixed-version cluster.
2021-03-30 12:13:03 +02:00
Martijn van Groningen 9089e45173
Update docs that closing a data stream's write index is allowed (#71039)
This was forgotten as part of #70908
2021-03-30 11:49:09 +02:00
markharwood 2f9c7318c2
Search - make wildcard field use constant scoring queries for wildcard queries and caching fix (#70452)
* Make wildcard field use constant scoring queries for wildcard queries. Add a note about ignoring rewrite parameters on wildcard queries.

Also fixes caching issue where case sensitive and case insensitive results were cached as the same

Closes #69604
2021-03-30 10:37:39 +01:00
István Zoltán Szabó 1db2b85e45
[DOCS] Adds source index privileges required for Explain DFA API docs. (#70978) 2021-03-30 10:42:48 +02:00
Shahzad f7efa3eaba
Extract device type from user agent info (#69322) 2021-03-29 16:34:53 -05:00
James Rodewig b2cd89ec09
[DOCS] Split enrich examples from enrich tutorial (#71001) 2021-03-29 12:54:08 -04:00
James Rodewig 20df933e6b
[DOCS] Update data stream rollover references (#70993) 2021-03-29 10:37:14 -04:00
James Rodewig a73631eab8 [DOCS] Resize test pipeline image 2021-03-29 10:34:34 -04:00
James Rodewig fdbea16e15
[DOCS] Move EQL event category section (#70955)
Combines the basic syntax and event category sections for better visibility.
2021-03-29 09:40:34 -04:00
Ignacio Vera a35563aaaf
Fix infinite loop when polygonizing a circle with centre on the pole (#70875)
This PR prevents the algorithm to run on circles that contain a pole.
2021-03-29 07:36:29 +02:00
Benjamin Trent b796632582
[ML] Allow datafeed and job configs for datafeed preview API (#70836)
Previously, a datafeed and job must already exist for the `_preview` API to work.

With this change, users can get an accurate preview of the data that will be sent to the anomaly detection job
without creating either of them. 

closes https://github.com/elastic/elasticsearch/issues/70264
2021-03-26 12:52:23 -04:00
James Rodewig 77cd0b5b74 [DOCS] Sync timestamps 2021-03-26 12:30:44 -04:00
James Rodewig 805cc77aae [DOCS] Remove leading slash for consistency 2021-03-26 11:48:24 -04:00
James Rodewig 1514fa8619 [DOCS] Fix whitespace consistency 2021-03-26 11:45:43 -04:00
James Rodewig 06010b96aa [DOCS] Remove leading slash for consistency 2021-03-26 11:06:58 -04:00
James Rodewig fa93666b6a
[DOCS] Document ingest processor description (#70899) 2021-03-26 09:42:46 -04:00
James Rodewig 493741dd7f
[DOCS] Remove docs for rollup refactor (#70885) 2021-03-26 09:03:00 -04:00
István Zoltán Szabó 9a8c6fb66f
[DOCS] Removes beta labels from DFA related docs. (#70808) 2021-03-26 09:46:41 +01:00
James Rodewig 911293fb4a
[DOCS] Use ECS fields in ingest pipeline tutorial (#70884) 2021-03-25 17:30:45 -04:00
Adam Locke 184cb449cc
[DOCS] Clarify language for supported APIs with CCS and older clusters (#70734)
* [DOCS] Clarify supported features for CCS.

* Clarify text and add subsection with title.

* Moving APIs to supported API section and paring down text.
2021-03-25 13:08:34 -04:00
Adam Locke 8dd563134c
[DOCS] Overhaul TLS security docs (#68946)
* Removing security overview and condensing.

* Adding new security file.

* Minor changes.

* Removing link to pass build.

* Adding minimal security page.

* Adding minimal security page.

* Changes to intro.

* Add basic and basic + http configurations.

* Lots of changes, removed files, and redirects.

* Moving some AD and LDAP sections, plus more redirects.

* Redirects for SAML.

* Updating snippet languages and redirects.

* Adding another SAML redirect.

* Hopefully fixing the ci/2 error.

* Fixing another broken link for SAML.

* Adding what's next sections and some cleanup.

* Removes both security tutorials from the TOC.

* Adding redirect for removed tutorial.

* Add graphic for Elastic Security layers.

* Incorporating reviewer feedback.

* Update x-pack/docs/en/security/securing-communications/security-basic-setup.asciidoc

Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>

* Update x-pack/docs/en/security/securing-communications/security-minimal-setup.asciidoc

Co-authored-by: Yang Wang <ywangd@gmail.com>

* Update x-pack/docs/en/security/securing-communications/security-basic-setup.asciidoc

Co-authored-by: Yang Wang <ywangd@gmail.com>

* Update x-pack/docs/en/security/index.asciidoc

Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>

* Update x-pack/docs/en/security/securing-communications/security-basic-setup-https.asciidoc

Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>

* Additional changes from review feedback.

* Incorporating reviewer feedback.

* Incorporating more reviewer feedback.

* Clarify that TLS is for authenticating nodes

Co-authored-by: Tim Vernum <tim@adjective.org>

* Clarify security between nodes

Co-authored-by: Tim Vernum <tim@adjective.org>

* Clarify that TLS is between nodes

Co-authored-by: Tim Vernum <tim@adjective.org>

* Update title for configuring Kibana with a password

Co-authored-by: Tim Vernum <tim@adjective.org>

* Move section for enabling passwords between Kibana and ES to minimal security.

* Add section for transport description, plus incorporate more reviewer feedback.

* Moving operator privileges lower in the navigation.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Ioannis Kakavas <ikakavas@protonmail.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>
Co-authored-by: Tim Vernum <tim@adjective.org>
2021-03-25 11:54:39 -04:00
James Rodewig 6504b541e9
[DOCS] EQL: Use data streams in docs (#70822) 2021-03-25 09:41:06 -04:00
James Rodewig 48f0ae6509 [DOCS] Add newline to end of file 2021-03-25 09:33:59 -04:00
James Rodewig bd4439be24
[DOCS] Replace hard-coded admons with cloud-only attribute (#70864) 2021-03-25 09:21:35 -04:00
James Rodewig 224bf7943a
[DOCS] Move enrich policy object def to API docs (#70825) 2021-03-25 09:17:41 -04:00
Nik Everett 75342a2a71
Add `teardown` support for doc tests (#70831)
This adds named `teardown` support for doc tests similar to its support
for named `setup` section. This is useful when many doc files want to
share a similar `setup` AND `teardown`. I've introduced an example of
this in the CCR docs just to prove its works. We expect we'll use it for
datastreams as well.

Closes #70830

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-03-24 16:57:50 -04:00
Nik Everett 2b9ed7d36f
Docs: Clean doc for agg parameter (#70675)
This adds a heading for `shard_min_doc_count` and merges the paragraphs
for them. I wanted to link to this section earlier today and it wasn't a
"real" section so I couldn't.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-03-24 16:22:26 -04:00
Nhat Nguyen 5bb440cdca
Move point in time to server (#70704)
This change moves the implementation of point in time to the server package.
2021-03-24 14:29:20 -04:00
James Rodewig 1b06f4cc62
[DOCS] Add info about allowed profile names (#70440) (#70815)
Co-authored-by: Robin Clarke <robin.clarke@elastic.co>
2021-03-24 10:10:21 -04:00
Dan Hermann f3b27541ff
[DOCS] Add MurmurHash3 as a supported hash method for fingerprint processor (#70737) 2021-03-24 07:25:22 -05:00
István Zoltán Szabó 178563c224
[DOCS] Reorganizes Transforms limitations (#70638) 2021-03-24 11:22:59 +01:00
Martijn van Groningen ccb473c56c
Adjust the docs for template-v1 API (#70801)
The delete legacy template API doesn't support comma-separated list of names in any version.

Forward-port of #70649
2021-03-24 11:15:15 +01:00
David Turner dd69ae95d7
Note recovery settings affect searchable snapshots (#70771)
Adds a short note that `max_restore_bytes_per_sec` and
`indices.recovery.max_bytes_per_sec` also affect the recovery of a
searchable snapshot index.
2021-03-24 09:22:44 +00:00
James Rodewig d58ab817a2
[DOCS] Fix sidebar for built-in index patterns (#70788) 2021-03-23 20:18:33 -04:00
James Rodewig 5805ae2eb7
[DOCS] Fix 'How To' title (#70772) 2021-03-23 14:02:48 -04:00
James Rodewig 7bddd7db5c
[DOCS] Fix collapsible properties role (#70756) 2021-03-23 11:14:46 -04:00
James Rodewig e76c229b33
[DOCS] Note you can omit `type` for custom analyzers (#70754) 2021-03-23 11:13:20 -04:00
James Rodewig cfe13a843e
[DOCS] Fix heading for ILM shrink example (#70733) 2021-03-23 11:07:02 -04:00
István Zoltán Szabó c4a33f2413
[DOCS] Adds latest transform example to transform docs (#70562)
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
2021-03-23 15:34:10 +01:00
James Rodewig deb4805779
[DOCS] Fix simulate pipeline API's request parameter docs (#70678) 2021-03-23 09:40:17 -04:00