Commit Graph

10367 Commits

Author SHA1 Message Date
David Turner 7a517cb4a0
Add note on jstack frequency for troubleshooting (#95764)
Suggest calling `jstack` every 15s to ensure that at least one capture
shows a stuck thread. Also adds a link to this guide to the list on the
troubleshooting overview page.
2023-05-03 10:04:13 +01:00
Mayya Sharipova 808dffe65e
Docs: Add updateable flag to synonym filters (#95746)
In #45135 we introduced `updateable` flag to synonym filters,
but we forgot to add this flag to documentation.
This PR fixes this.

Relates to #45135
2023-05-02 17:38:47 -04:00
Quentin Pradet 4733996ec4
[DOCS] Forward port 8.7.1 release notes 2023-05-02 17:33:15 +04:00
Abdon Pijpelink e48ce6c756
[DOCS] Update default monitoring method on Elastic Cloud (#95662) 2023-05-02 11:31:33 +02:00
Abdon Pijpelink f2d9a3fbca
[DOCS] Fix 'shards_size' typo (#95696)
* [DOCS] Fix 'shards_size' typo

* Second occurrence of 'shards_size'
2023-05-01 15:24:43 +02:00
Mayya Sharipova eab19b56da
Clarify path_hierarchy documentation (#44910)
* Resolve merge conflicts and apply review suggestions

* Fix URLs

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-05-01 11:32:38 +02:00
amyjtechwriter 3d6143b829
Nodes need access to storage.googleapis.com for geoip. (#95554) 2023-04-28 10:40:18 +01:00
Mayya Sharipova 96171591af Fix version in feature-migration test 2023-04-27 14:39:26 -04:00
Mayya Sharipova 07adeb0901
Create .synonyms system index (#95548)
Create .synonyms system index that is exposed
under es.synonyms_api_feature_flag.

This is the first task for creating Synonyms API management,
where synonyms will be stored in the .synonyms system index.

Relates to #38523
2023-04-27 14:11:08 -04:00
Stef Nestor 65b4fe28d4
[+DOC] Troubleshooting / Mapping Explosion (#95397)
* [+DOC] Troubleshooting / Mapping Explosion

---------

Co-authored-by: Steffanie Nestor <steffanie.nestor@elastic.co>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
2023-04-27 11:08:56 -06:00
Michael Peterson 5169011325
Allow multiple field names/patterns for (path_)(un)match (#66364) (#95558)
* Allow multiple field names/patterns for (path_)(un)match (#66364)

Arrays of patterns are now allowed for dynamic_templates in the match,
unmatch, path_match and path_unmatch fields. DynamicTemplate has been modified to
support List<String> for these fields. The patterns can be either simple wildcards
or regex. As with previous functionality, when match_pattern="regex", simple wildcards
will be flagged with an error, but when match_pattern="simple", using regular expressions
in the match will not throw an error.

One new error pathway was added: if a user specifies a list of non-strings for
one of these pattern fields (e.g., "match": [10, false]) a MapperParserException
will be thrown.

A dynamic_template yamlRestTest was added. This is a BWC change, so the REST test
that uses arrays of patterns is limited to v8.9 and above.

Closes #66364.
2023-04-27 12:58:49 -04:00
David Kilfoyle 626db84fac
[Docs] Small fixes for hot spotting page (#95627) 2023-04-27 10:18:21 -04:00
Kathleen DeRusso 63524f8efc
Add template information to search application API docs (#95591)
Co-authored-by: Chris Cressman <chris@chriscressman.com>
Co-authored-by: Carlos Delgado <6339205+carlosdelest@users.noreply.github.com>
2023-04-27 09:32:25 -04:00
Stef Nestor 4c5a3fb4da
[+Doc] Troubleshooting / Hot Spotting (#95429)
* [+Doc] Troubleshooting / Hot Spotting

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-04-26 12:29:47 -06:00
Kathleen DeRusso ba93ddb11d
Add _render_query API call for Search Applications (#95513) 2023-04-26 13:47:41 -04:00
Albert Zaharovits b90060538a
Fix _xpack_usage for enterprise_search (#95565)
Fixes #95564
2023-04-26 19:23:36 +03:00
Abdon Pijpelink 7e34b52c5d
[DOCS] Remove 8.8 release highlights link to fix docs build (#95578) 2023-04-26 15:21:55 +02:00
gmarouli 246c6a9623 Bump to version 8.9.0 2023-04-26 15:20:24 +03:00
Martijn van Groningen 49e8ee4269
Remove remaining tsdb tech preview labels (#95563)
Remove tech preview label from a number of tsdb settings and mapping attributes.
2023-04-26 12:11:03 +02:00
Pablo Alcantar Morales 132290f8a3
fix flaky docs tests `get-lifecycle` (#95529)
The `data-streams/downsampling.asciidoc` test was missing a teardown clean of the ILM policies created. Due to this tests *do not have* the string `ilm` in its name, the automatic teardown process that cleans up the resources (check `ESRestTestCase.java#L815` & `DocsClientYamlTestSuiteIT.java` lines 177 & 195) is not executed for this specific test. In the case this test runs right before the `get-lifecycle` test, the policy won't be automatically deleted hence the test checking the version will fail. Finally, the order of execution of the test is not guaranteed by the suite.
2023-04-26 12:10:49 +02:00
Abdon Pijpelink 2e31832c34
[DOCS] Clarify configuring indices/data streams used by stack monitoring (#95285)
* [DOCS] Clarify indices/data streams using by stack monitoring

* Incorporate review feedback

* Update docs/reference/monitoring/indices.asciidoc

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

---------

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
2023-04-25 17:16:23 +02:00
Sloane Perrault b57286aa0f
Add Search Application telemetry to xpack/usage (#95306)
* search application telemetry via xpack.usage

* impl EnterpriseSearchInfoTransportAction, fix test

* fix docs yaml tests and security operator tests

* set enabled & available with settings & license

* test count increases and decreases

* include xpack instead of xpack.usage specifically

* rename enterprise search usage serializing tests

* call action directly

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>

* handle `enabled = false` for entsearch usage

* return empty stats on error

* use an OriginSettingClient

---------

Co-authored-by: Kathleen DeRusso <kathleen.derusso@elastic.co>
2023-04-25 09:56:09 -04:00
Stef Nestor 1ee528dc3f
[Doc] Troubleshoot Cluster State / Linkable subsections (#95468)
👋🏼 howdy, team! Could we make these sub-sections sub-header link-able?
2023-04-25 10:35:14 +02:00
Jack Conradson 5314e5dd55
Add support for Reciprocal Rank Fusion to the search API (#93396)
This change at a high level adds global ranking on the coordinating node at the end of query reduction 
prior to the fetch phase. Individual rank methods are defined in plugins.

The first rank plugin added as part of this change is reciprocal rank fusion (RRF). RRF uses a relatively 
simple formula for merging 1...n results sets together with sum(1/(k+d)) where k is a ranking constant 
and d is a document's scored position within a result set from a query.
2023-04-24 15:07:34 -07:00
Pablo Alcantar Morales 0ca5dc2eac
fix incorrect TESTRESPONSE definition (#95508)
`TESTRESPONSE` definitions have to be escaped to properly override the value
coming from the response.
2023-04-24 18:16:25 +02:00
Martijn van Groningen 6566bb4075
Add global ordinal info to stats APIs. (#94500)
This change adds:
* Total global ordinal build time for all fields and per field.
* Max shard value count per field. The value count is per shard and of the shard with the highest count. Reporting value on index level or across indices is too expensive to report or keep track of.

This is added to common stats, which
is exposed in several stats APIs.

The following api call:

```
GET /_nodes/stats?filter_path=nodes.*.indices.fielddata&fields=key,key2
```

Returns:

```
{
    "nodes": {
        "pcMNy4GsQ8ef6Rw-bI2EFg": {
            "indices": {
                "fielddata": {
                    "memory_size_in_bytes": 2552,
                    "evictions": 0,
                    "fields": {
                        "key2": {
                            "memory_size_in_bytes": 1320
                        },
                        "key": {
                            "memory_size_in_bytes": 1232
                        }
                    },
                    "global_ordinals": {
                        "build_time_in_millis": 8,
                        "fields": {
                            "key2": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            },
                            "key": {
                                "build_time_in_millis": 4,
                                "shard_max_value_count": 4
                            }
                        }
                    }
                }
            }
        }
    }
}
```
2023-04-24 10:45:27 +02:00
Leaf-Lin 8c06aea3dd
[DOCS] CCR disaster recovery (#91491)
* Add bi-directional disaster recovery

* add ccr bi-directional disaster recovery image

* add link to bi-directional disaster recovery

* add image

* add [source]

* fix language

* Update bi-directional-disaster-recovery.asciidoc

* Update bi-directional-disaster-recovery.asciidoc

* Update bi-directional-disaster-recovery.asciidoc

* Apply suggestions from code review

Remove immutable restrictions and add update/delete by query instructions.

* Apply suggestions from code review

* Apply suggestions from code review

Fixing reference

* Apply suggestions from code review

fix list format

* Apply suggestions from code review

remove space to fix format

* add test

* Update docs/reference/ccr/bi-directional-disaster-recovery.asciidoc

* Add test

* Add uni-directional DR doc

* Add uni-directional image

* add uni-directional doc reference

* Update docs/reference/ccr/uni-directional-disaster-recovery.asciidoc

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>

* Apply suggestions from code review

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>

* Apply suggestions from code review

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>

* Pushing up minor edits to restart build. Previous build failure 'Could not determine the dependencies of task ':x-pack:plugin:ml:explodedBundlePlugin'

* Apply suggestions from code review

* Tip formatting and renaming follwer index to _copy in uni-direction

* Fix failing CI doc check

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Co-authored-by: Amy Jonsson <amy.jonsson@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-04-21 10:02:54 +01:00
Stef Nestor dd968f5433
Update ilm-searchable-snapshot.asciidoc (#95255)
👋🏼 howdy, team! Can we explicate that ILM Searchable Snapshot retention ([doc](https://www.elastic.co/guide/en/elasticsearch/reference/master/ilm-searchable-snapshot.html)) is not effected by SLM policies? TIA!

---------

Co-authored-by: amyjtechwriter <61687663+amyjtechwriter@users.noreply.github.com>
2023-04-20 07:03:31 -06:00
Mayya Sharipova 4d6e451d8b
Add an experimental label for 2048 vector dims (#95395)
Add an experimental lable for increased vector dims.

Relates to PR#95257
2023-04-20 07:48:12 -04:00
Salvatore Campagna ec2bdee31b
Add time_series_dimensions param to flattened docs (#95374) 2023-04-20 10:58:12 +02:00
Ievgen Degtiarenko c2c0ced9b1
Reset desired balance (#94525)
This introduces an endpoint to reset the desired balance.
It could be used if computed balance diverged from the actual one a lot 
to start a new computation from the current state.
2023-04-20 08:03:48 +02:00
Keith Massey efcdbbc952
Adding origination date to DLM (#95113)
This adds support for an index's index.lifecycle.origination_date setting in DLM. If an index has a
value for index.lifecycle.origination_date then it is used instead of the creation date or rollover
date (except in the case of the write index when the write index has not been rolled over yet).
2023-04-19 08:15:24 -05:00
Pablo Alcantar Morales 253fe6325d
Add shards capacity troubleshooting guide (#95208) 2023-04-19 09:24:07 +02:00
Toby Sutor ba9dc81115
[docs] Clarify that index template settings take precedence over comp… (#87374)
* [docs] Clarify that index template settings take precedence over component templates.

[docs] Clarify that index template settings take precedence over component templates.

* Update docs/reference/indices/index-templates.asciidoc

Co-authored-by: Adam Locke <adam.locke@chronosphere.io>

---------

Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
Co-authored-by: Adam Locke <adam.locke@chronosphere.io>
2023-04-19 09:18:33 +02:00
Felix Barnsteiner 11b598a519
Add reroute processor (#76511) 2023-04-18 19:09:25 +02:00
Martijn van Groningen 1f40ced134
Tiny tsdb docs update (#95333)
Update definition of metric type counter to include it resets to zero.

Just like is defined on the tsdb page:
https://www.elastic.co/guide/en/elasticsearch/reference/current/tsds.html#time-series-metric
2023-04-18 11:17:31 -04:00
Cauê Marcondes 91cd61a454
[Fleet] Support for Profiling symbolization (#95241)
* It adds the profiling index pattern profiling-* to the fleet server service privileges.
* And adds profiling-* to kibana system role privileges.

---------

Co-authored-by: Daniel Mitterdorfer <daniel.mitterdorfer@elastic.co>
2023-04-18 07:12:44 +02:00
Kathleen DeRusso 8432a5a366
Update Search Application API docs to reflect Tech Preview status (#95308) 2023-04-17 14:14:30 -04:00
David Kyle 6de8469a51
[ML] Include model definition install status for Pytorch models (#95271)
Adds a new include flag definition_status to the GET trained models API.
When present the trained model configuration returned in the response 
will have the new boolean field fully_defined if the full model definition 
is exists.
2023-04-17 18:12:26 +01:00
Andrei Dan 7b994ba8d0
Document that DS backing indices can have gaps in the name counter (#95237) 2023-04-17 17:11:05 +01:00
David Turner 822dc713d8
Add note on name resolution during startup (#95266)
Notes that the transport publish address is resolved once during
startup, plus advice to ensure that this name resolution doesn't vary by
location.
2023-04-17 14:42:15 +01:00
Przemysław Witek 2b70165ffd
[Transform] Allow specifying destination index aliases in the Transform's dest config (#94943) 2023-04-17 15:08:43 +02:00
Mayya Sharipova 32c17d79c5
Increase max number of vector dims to 2048 (#95257)
Currently Lucene limits the max number of vector dimensions to 1024.
This commit overrides KnnFloatVectorField and KnnByteVectorField
classes to increase the limit to 2048 for indexed vectors in ES.
2023-04-17 09:05:49 -04:00
David Kyle 7d90c519ef
[ML] Add embedding_size to text embedding config (#95176) 2023-04-17 11:49:35 +01:00
Andrei Dan 731e6dd097
GET _lifecycle returns data streams without lifecycle (#95192)
This changes the `GET _data_stream/ds_name/_lifecycle` endpoint to
return the data stream name even if it doesn't have a lifecycle
configured.

e.g.
```
{
  "data_streams": [
    {
      "name": "logs-nginx"
    }
  ]
}
```

* Fix xcontent and tests

* Update docs

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-04-17 10:35:22 +01:00
David Turner b4b9292ce9
Small changes to corruption troubleshooting docs (#95265)
- Mention that third-party software may be to blame too
- Mention `strace` as a last resort
- Minor rewordings
2023-04-17 09:07:27 +01:00
Joe Gallo 9bc09d576a
Fix ignore_missing docs for a couple of Ingest processors (#95244) 2023-04-13 16:34:40 -04:00
Aurélien FOUCRET 9071d114f5
[Ingest Processor] Add `ignore_missing` param to the `uri_parts` ingest processor. (#95068) 2023-04-13 15:11:19 +02:00
David Kilfoyle 2c8e29807e
[DOCS] Add note about /tmp removal (#95166) 2023-04-13 08:39:53 -04:00
Andrei Dan 83402605ea
Remove the lifecycle field in the PUT _lifecycle request (#95201)
This removes a redundant `lifecycle` field in the `PUT _lifecycle`
request.

Before we had
```
PUT _data_stream/logs-nginx/_lifecycle
{
    "lifecycle": {
        "data_retention": "7d"
    }
}
```

This changes the request to
```
PUT _data_stream/logs-nginx/_lifecycle
{
  "data_retention": "7d"
}
```
2023-04-13 09:12:48 +01:00