Commit Graph

10810 Commits

Author SHA1 Message Date
Nik Everett 649ceb74ab
ESQL docs: generate references for functions (#98856)
This generates a "railroad diagram" svg image that can be embedded into
the docs for any function to explain it's syntax. It's basic, but it's
something we can iterate on.

It also generates a table of supported types from the list of types that
we test. It can be included in the docs for reference as well.
2023-08-25 09:07:25 -04:00
Yang Wang f307e6fedd
[Docs] Document IP filtering for RCS 2.0 (#98553)
This PR adds documentation for RCS 2.0 IP filtering.

Relates: #98472
2023-08-24 23:23:40 -04:00
Abdon Pijpelink 1955bd8ad4
[DOCS] New docs for remote clusters using API key authentication (#98330)
* New docs structure for remote clusters

* Fix broken cross-book link errors

* More broken cross-book link errors

* Remove redirects for new pages

* Link to generic remote cluster docs instead

* Drop 'API' from the abbreviated title

* Add 'Establish trust with a remote cluster' section

* Restructure 'Establish trust' section into Prprequisite/local/remote instructions

* Add 'Configure roles and users' section

* Add 'Connect to a remote cluster' section

* Move version compatibility to prerequisites

* Fix test errors

* Incorporate review feedback

* Mention version 8.10 or later in the intro for API keys

* Add license prerequisite
2023-08-24 12:30:03 +02:00
Mary Gouseti b9b818e28e
Allow explain data stream lifecycle to accept a data stream. (#98811)
Currently the `GET target/_lifecycle/explain` API only works for
indices. In this PR we extend this behaviour to allow the target to be a
data stream so we can get the overview lifecycle status for all the
backing indices of a data stream.
2023-08-24 06:29:09 -04:00
Dianna Hohensee a25e176692
Add node "roles" to allocation explain response (#98550)
Report node "roles" in the /_cluster/allocation/explain response.
Nodes with limited sets of roles may affect shard distribution in ways
users did not originally consider, so it is helpful to surface this
information along with node allocation decision explanations.
2023-08-23 08:30:35 -04:00
István Zoltán Szabó 947128e76d
[DOCS] Fixes NOTE display error. (#98783) 2023-08-23 12:18:54 +02:00
Peter Dyson ef7b7ccded
[DOCS] mention closed indices are included by default for snapshot (#98766) 2023-08-23 10:41:35 +02:00
Lee Hinman 815d596daa
Add 'dataset' size to cat indices and cat shards (#98622)
* Add 'dataset' size to cat indices and cat shards

This adds the `dataset` computed size for the `/_cat/indices` and `/_cat/shards` APIs. This new
column is reported by default.

Resolves #95092
2023-08-22 15:36:32 -06:00
Nik Everett 65ea90d3fd
ESQL: LEAST and GREATEST functions (#98630)
Adds `LEAST` and `GREATEST` functions to find the min or max of the
values in many columns.
2023-08-22 14:15:04 -04:00
Bogdan Pintea 372458c9fd
ESQL: date_trunc(): swap order of arguments (#98624)
Swap arguments order so that the range parameter is first and datetime
one second, inline with other languages.
2023-08-22 18:20:05 +02:00
Andrei Dan 01ed7de99f
GA the data stream lifecycle (#98644)
This makes the data stream lifecycle generally available. This will allow
data streams to take advantage of a native simplified and resilient
lifecycle implementation.
2023-08-21 17:28:54 +01:00
Abdon Pijpelink 68aee20d01
[DOCS] Removing 'coming' admonition from 8.9.1 release notes (#98658) 2023-08-21 14:05:42 +02:00
Abdon Pijpelink 2bbc1df9bd
[DOCS] Remove 'coming' notice from 8.8.2 release notes (#98659) 2023-08-21 14:05:15 +02:00
Abdon Pijpelink d04c0f7976
Fix response for IS NULL (#98671)
The response shown for IS NULL here is actually the response for the IS NOT NULL snippet. This PR fixes that.
2023-08-21 13:20:36 +02:00
Abdon Pijpelink 4d7193d76d
[main] Update docs for v8.9.1 release (#98416) (#98655)
* Update docs for v8.9.1 release (#98416)

(cherry picked from commit 7985f8ef0f)

* Resolve merge conflicts

---------

Co-authored-by: Volodymyr Krasnikov <129072588+volodk85@users.noreply.github.com>
2023-08-21 10:53:39 +02:00
Nik Everett 44e61341f2
ESQL: COALESCE function (#98542)
This adds a `COALESCE` function that returns the first non-null value.
2023-08-17 13:51:44 -04:00
Roberto Seldner 79d2879564
Add deprecated note for `balanced` allocator (#98610)
Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2023-08-17 12:50:52 -04:00
Nik Everett a380e8c369
ESQL: LTRIM, RTRIM and fix unicode whitespace (#98590)
Here we add support for the following two ESQL functions:
* LTRIM: remove leading spaces from a string
* RTRIM: remove trailing spaces from a string

We also fix an issue with the handling of unicode white spaces. We
make use of unicode code points to identify unicode whitespace
characters instead of relying on ASCII codes.

Moreover, iterating bytes in a Unicode string needs to consider
that some Unicode characters are encoded using multiple bytes.
2023-08-17 11:30:12 -04:00
Craig Taverner aad16b7d6b
Simple ESQL pow() docs fixes after re-reviewing (#98601) 2023-08-17 17:29:06 +02:00
Kathleen DeRusso 8c12a7b7cd
Query rules docs clarification (#98605)
* Query rules docs clarification

* Update docs/reference/search/search-your-data/search-using-query-rules.asciidoc

* Update docs/reference/search/search-your-data/search-using-query-rules.asciidoc
2023-08-17 11:11:49 -04:00
Carlos Delgado 2b838ae853
Dense vector field types are indexed by default (#98268)
* First version

* Spotless, I liked my version better

* Fix param default values

* Add a supplier for default value to ensure it's calculated correctly

* Can't improve this without breaking tests

* Added checks for not specifying a body in PUT requests

* Fix default provider for enum params

* Added yaml test

* Changed docs and fix TODO

* Removing synonyms changes

* Added separate methods for providing default value as suppliers in enums

* Fixed test

* Add a supplier for default value to ensure it's calculated correctly

* Added checks for not specifying a body in PUT requests

* Remove synonyms changes

* Remove some supplier changes

* Better call enumParam with supplier version

* Fix compiler error on supplier

* Apply validators or requires depending on index version

* Solved BWC tests that involved using validators instead of requiresParameters

* Add tests

* Spotless

* Update docs/changelog/98268.yaml

* Update changelog

* Update docs/changelog/98268.yaml

* PR comments

* PR feedback

* Serialize index only for new index versions

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-08-17 10:53:14 -04:00
Craig Taverner dfe9bdc45f
Simple grammar fix for MVT docs (#98591) 2023-08-17 16:10:26 +02:00
ChrisHegarty 21dcb75836 Merge upstream 2023-08-17 11:40:42 +01:00
Simon Cooper 9538208fb7
Fix shard-stores doc test (#98581)
The replacement wasn't actually doing anything due to different JSON types
2023-08-17 11:38:31 +01:00
Nick Chow 5de0a9013f
Documentation update that fixes a query rules code example (#98540)
* Change example field in rule query guide

* Change fuzzy to contains to get tests to work

---------

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
2023-08-16 15:14:32 -07:00
Andrei Stefan 014bd33f45
ESQL: replace the is_null function with IS NULL and IS NOT NULL predicates (#98412) 2023-08-16 20:19:40 +03:00
ChrisHegarty d7c0f623bf Merge upstream 2023-08-16 15:57:21 +01:00
Michael Peterson 4d3fb8d993 Bump to version 8.11.0 2023-08-16 10:35:00 -04:00
Kostas Krikellas b498ce9ff4
`Sqrt` function for ESQL (#98449)
* Sqrt function for ESQL

Introduces a unary scalar function for square root, which is a thin
wrapper over the Java.Math implementation.

* Fix area for ESQL integration changelog.

* Restore changelog.

* Restore area in changelog.
2023-08-16 16:33:30 +03:00
ChrisHegarty 107174051b Merge upstream 2023-08-16 08:56:40 +01:00
James Baiera 7d990d5a09
Allow custom geo ip database files to be downloaded (#97850)
This PR extends the assumptions we make about database file availability to all database file 
names instead of the default ones we host at Elastic. When creating a geo ip processor with 
a database name that is not recognized we unilaterally convert the processor to one that 
tags documents with a missing database message until the database file requested is 
downloaded or provided via the manual configuration route. This allows a pipeline to be 
created and for the download service to be started, potentially sourcing the needed files.

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-08-16 00:31:51 -04:00
Yang Wang b337f9b6f3
[Docs] Misc doc update for RCS 2.0 (#98472)
This PR adds docs for the following items: * Remote indices privileges *
Remote cluster network settings * Remote cluster security settings * New
privileges * New response field for RemoteInfo API

List of preview pages: * [Remote indices in defining
roles](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/defining-roles.html#roles-remote-indices-priv)
* [Remote indices in PutRole
API](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-api-put-role.html#security-api-put-role-request-body)
* [Remote cluster server SSL
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-settings.html#_remote_cluster_server_api_key_based_model_tlsssl_settings)
* [Remote cluster client SSL
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-settings.html#_remote_cluster_client_api_key_based_model_tlsssl_settings)
* [Remote cluster network
settings](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/modules-network.html#remote-cluster-network-settings)
and
[here](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/modules-network.html#common-network-settings)
* [Remote cluster credentials
setting](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/remote-clusters-settings.html)
* [New
privileges](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/security-privileges.html)
* [New response field for RemoteInfo
API](https://elasticsearch_98472.docs-preview.app.elstc.co/guide/en/elasticsearch/reference/master/cluster-remote-info.html#cluster-remote-info-api-response-body)
2023-08-15 20:11:21 -04:00
Jake Landis 80cc651614
Fix jwt and oidc doc (#97942)
This commit corrects the following issues with JWT and OIDC `jwkset_path` documentation:
* only https is supported for the JWT realm (OIDC support both https and http)
* JWT realm does not use a file watcher to reload the file every 5 seconds
* simplify "path" to  "file name" ..technically it is resolved path, but 99% of the time it will be just 
     a file name in the config directory and "path" is ambiguous
* remove special mention of using the absolute path in cloud. .. this is an unnecessary implementation 
     detail and the only setting (of many) that calls out the cloud config directly by absolute path
* ensure the 2 different JWT documentations are the same
* make mention of when the JWT file will be reloaded (it is not backed by the file watcher, only OIDC is)
2023-08-15 15:50:17 -05:00
amyjtechwriter 7e907f804a
[DOCS]Corrections for index-mgmt page. (#98481)
* Corrections for index-mgmt page.

* Update docs/reference/indices/index-mgmt.asciidoc

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-08-15 16:38:04 +01:00
Abdon Pijpelink e766050edc
[DOCS] Update geohash_grid agg field description (#98494) 2023-08-15 15:59:16 +02:00
Abdon Pijpelink b4ee955bb1
[DOCS] Add tests to behavioral analytics code snippets (#98378) 2023-08-15 10:10:37 +02:00
ChrisHegarty 8a4e42358b Merge upstream 2023-08-15 09:01:43 +01:00
Leaf-Lin db078f78dd
Update post-analytics-collection-event.asciidoc (#97826)
* Update post-analytics-collection-event.asciidoc

Update API endpoint. Instead of `click`, it should be `search_click`. 

Expanded on request body in the API as well.

* Update post-analytics-collection-event.asciidoc

* Update post-analytics-collection-event.asciidoc

* Update post-analytics-collection-event.asciidoc
2023-08-15 09:08:14 +10:00
ChrisHegarty 2e4440ff1c Remove space in dissect feature list 2023-08-14 14:54:37 +01:00
ChrisHegarty 7cb310b9d0 Merge upstream 2023-08-14 09:55:02 +01:00
Yang Wang d0f64941f0
Remove RCS 2.0 feature flag for beta release (#98307)
This PR removes the RCS 2.0 feature flag so that it is ready for beta
release.
2023-08-14 08:33:37 +10:00
Keith Massey b155444b95
Fixing data stream lifecycle documentation links that break clients (#98365) 2023-08-11 10:30:33 -05:00
Jim Ferenczi a5d21ce800
Add the total dense vector count in the indices stats output (#98275)
This change adds the total dense vector count to the output of the indices stats.
This is useful for observability in order to track the number of indexed vectors
in a cluster.

---------

Co-authored-by: Benjamin Trent <ben.w.trent@gmail.com>
2023-08-11 23:17:38 +09:00
Abdon Pijpelink cac9eef000
[DOCS] Set up temporary redirects for new remote clusters docs (#98398) 2023-08-11 15:40:06 +02:00
Felix Barnsteiner c0f10b1cd2
[DOCS] ILM force merge action doesn't make index read-only (#98382) 2023-08-11 08:38:55 -04:00
ChrisHegarty d07492d712 Merge upstream 2023-08-11 12:10:23 +01:00
Mary Gouseti e71ea6e6d7
Add data stream lifecycle by default (#97823)
In this PR we enable all new data streams to be managed by the data
stream lifecycle by default. This is implemented by adding an empty
`lifecycle: {}` upon new data stream creation. 

Opting out is represented by a the `enabled` flag:

```
{
  "lifecycle": {
    "enabled": false
  }
}
```

This change has the following implications on when is an index managed
and by which feature:

| Parent data stream lifecycle| ILM| `prefer_ilm`|Managed by|
|----------------------------|----|----------------|-| | default | yes|
true| ILM| | default | yes| false| data stream lifecycle| |default |
no|true/false|data stream lifecycle| |opt-out or
missing|yes|true/false|ILM| |opt-out or missing|no|true/false|unmanaged|

Data streams that have been created before the data stream lifecycle is
enabled will not have the default lifecycle.

Next steps: - We need to document this when the feature will be GA
(https://github.com/elastic/elasticsearch/issues/97973).
2023-08-11 06:28:37 -04:00
James Rodewig c1eb581da8
[DOCS] Force merge doesn't mark index as read-only (#98357) (#98366)
(cherry picked from commit d4c6208727)

Co-authored-by: Gilad Gal <gilad.gal@elastic.co>
2023-08-10 18:28:52 -04:00
Carlos Delgado c596f121b4
Synonyms Overview Documentation (#98202) 2023-08-10 18:07:12 +02:00
Ioana Tagirta b2ba56a804
Search Applications: Remove inconsistent indices warning (#98348)
* Remove inconsistent indices warning

* Revert docs changes for inconsistent indices warning
2023-08-10 16:14:08 +02:00