Commit Graph

1104 Commits

Author SHA1 Message Date
James Rodewig 2f03112b5b
[DOCS] Synced with 8.0 stack upgrade changes (#83489) (#83596)
This moves the bulk of the upgrade information into the consolidated upgrade guide, but leaves the primary upgrade topic in place as a cross reference.

Relates to: https://github.com/elastic/stack-docs/pull/1970

Co-authored-by: gchaps <33642766+gchaps@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
(cherry picked from commit f6473d71f9)

Co-authored-by: debadair <debadair@elastic.co>
2022-02-07 11:01:42 -05:00
James Rodewig 882bac8948 [DOCS] Fix CCS compatibility typo 2022-02-02 13:09:40 -05:00
Eric Beahan 540a40093c
[DOCS] Correct header syntax (#83275)
* correct header syntax

* Update docs/reference/search/search-your-data/retrieve-selected-fields.asciidoc

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2022-01-28 14:55:54 -05:00
Julie Tibshirani e7ba03e0a6
Add notes on indexing to kNN search guide (#83188)
This change adds a new 'indexing considerations' section that explains why index
calls can be slow and how force merge can help search latency.
2022-01-28 10:23:35 -08:00
Christoph Büscher 61e1b080dd
[Docs] Add supported _terms_enum field types (#83244)
The `_terms_enum` API currently supported keyword, constant_keyword and
flattened fields. This should be documented more clearly.
2022-01-28 12:47:12 +01:00
James Rodewig dfb9f6f18d
[DOCS] Document 8.0 BWC support for CCS (#80809)
As of 8.0, the compatibility window for cross-cluster search (CCS) to an earlier release will be one minor release. This updates the CCS docs and adds a related 8.0 breaking change.

Closes https://github.com/elastic/elasticsearch/issues/80782
2022-01-11 10:33:12 -05:00
James Rodewig 7142b47e69
[DOCS] Add prerequisites for CCS (#81782)
* Adds a prerequisites section covering remote cluster config, node roles, and security.
* Moves existing content about remote cluster config to the prereqs.
* Updates the remote cluster docs to include information about eligible gateway nodes and tagging for gateway nodes.

Closes https://github.com/elastic/elasticsearch/issues/72001
2022-01-10 09:17:44 -05:00
Bogdan Pintea 13a0e420a3
SQL: Add CCS SQL documentation (#81545)
This adds the documentation for CCS SQL.

Co-authored-by: James Rodewig <james.rodewig@elastic.co>
2022-01-05 20:01:01 +01:00
Ignacio Vera 8c6ed1efc0
Remove experimental flag from geo field format mvt (#81721)
Small left over from 7.16 where mvt feature become GA
2021-12-14 15:21:05 +01:00
Julie Tibshirani 19eed47159
Improve kNN error message when index is disabled (#81561)
In order to perform a kNN search on a `dense_vector` field, it must have
`index: true` in its mapping. This commit clarifies the error message. Before
the message was confusing, because the user likely didn't touch the `index`
parameter and might not even be aware of it.

It adds a note to the docs clarifying that when coming from 7.x, you must
explicitly update `index: true` and reindex the vectors.

Relates to #78473.
2021-12-08 16:20:35 -08:00
Nhat Nguyen d0d91c690e
Handle partial search result with point in time (#81349)
Today, a search request with PIT would fail immediately if any 
associated indices or nodes are gone, which is inconsistent when
allow_partial_search_results is true.

Relates #81256
2021-12-08 10:04:38 -05:00
James Rodewig 229d2d7a77
[DOCS] Add high-level guide for kNN search (#80857)
Adds a high-level guide for running an approximate or exact kNN search in Elasticsearch.

Relates to https://github.com/elastic/elasticsearch/issues/78473.
2021-11-30 14:17:39 -05:00
happybin92 0aa9767f3d
Support combining _shards preference param with <custom-string> (#80024)
Adds support for combining the _shards search preference parameter with the <custom-string> search preference parameter.

Closes #80021
2021-11-10 14:08:27 +01:00
Julie Tibshirani 8ca693b271
Add docs for kNN search endpoint (#80378)
This commit adds docs for the new `_knn_search` endpoint.

It focuses on being an API reference and is light on details in terms of how
exactly the kNN search works, and how the endpoint contrasts with
`script_score` queries. We plan to add a high-level guide on kNN search that
will explain this in depth.

Relates to #78473.
2021-11-09 09:28:12 -08:00
James Rodewig f56a0f4b66
[DOCS] Remove `testenv` annotations from doc snippet tests (#80023)
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.

Relates to #79309, #31619
2021-11-05 18:38:50 -04:00
Ignacio Vera 508ed02ed2
Document _key tag added on the agg layer features (#80205) 2021-11-03 07:12:46 +01:00
James Rodewig ee1f71d421
[DOCS] Add experimental label to TSDB mapping params and settings (#79647)
Adds an `experimental` annotation to the following:

* `time_series_metric` mapping parameter
* `time_series_dimension` mapping parameter
* `index.mapping.dimension_fields.limit` index setting
*  `time_series_dimension` and `time_series_metric` properties in the field caps API response
2021-10-27 09:09:54 -04:00
Dan Hermann 4a36d5cd79
Remove endpoint for freezing indices (#78918) 2021-10-26 06:37:56 -05:00
James Rodewig 7940e0777c
[DOCS] Re-add several query params to search API docs (#79716)
PR #55884 removed documentation for several query parameters from the search API
docs. During tests, I failed to notice that these are valid parameters but require other parameters to use.

Changes:

* Notes the following search API parameters require the `q` query string parameter:

  * `analyzer`
  * `analyze_wildcard`
  * `default_operator`
  * `df`
  * `lenient`

* Notes the following search API parameters require the `suggest_field` and `suggest_text` query parameters:

  * `suggest_mode`
  * `suggest_size`

* Re-adds the above parameters to the search API docs.

These changes also affect API documentation that reuses the search API parameters:

* Delete by query API
* Update by query API
* Count API
* Explain API
* Validate API

Closes #79674
2021-10-25 11:58:54 -04:00
Gilad Gal 5276ee9b8b Update search-vector-tile-api.asciidoc
Vector tiles are GA in 7.16
2021-10-20 09:21:51 -04:00
Christoph Büscher 8b56362dbf
[Docs] Retrieving metadata using the fields option (#79174)
Adding a small section to the field retrieval article about which metadata
fields now can be retrieved via the `fields` option.
2021-10-15 11:47:17 +02:00
Igor Motov f6034e643a
TSDB: Add time series information to field caps (#78790)
Exposes information about dimensions and metrics via field caps. This
information will be needed for PromQL support.

Relates to #74660
2021-10-13 11:03:38 -10:00
markharwood 0f9848e243
BWC change following backport of PR 78697 to 7.x (#79067)
BWC change following backport of PR 78697 to 7.x
Closes #74121
2021-10-13 15:18:25 +01:00
markharwood 228992bf7e
Search - return ignored field values from fields api. (#78697)
Since Kibana's Discover switched to retrieving values via the fields API rather than source there have been gaps in the display caused by "ignored" fields (those that fall foul of ignore_above and ignore_malformed size and formatting rules).

This PR returns ignored values from source when a user-requested field fails to be parsed for a document. In these cases the corresponding hit adds a new ignored_field_values section in the response.

Closes #74121
2021-10-13 11:05:17 +01:00
Ignacio Vera 920b3b52c2
Add support for metrics aggregations to mvt end point (#78614)
It adds support for several aggregations.
2021-10-05 09:17:25 +02:00
Ignacio Vera e4cde37111
Add centroid grid type in mvt request (#78305)
For this grid type, the features on the aggregation layer are represented by a point that is computed from the 
centroid of the data inside the cell

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-10-01 06:56:13 +02:00
Ignacio Vera 9033faffff
Add cross cluster search test for mvt end point (#78054)
This commit adds a test to check that it is supported and document it.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-09-23 07:59:44 +02:00
Adam Locke 6940673e8a
[DOCS] Update remote cluster docs (#77043)
* [DOCS] Update remote cluster docs

* Add files, rename files, write new stuff

* Plethora of changes

* Add test and update snippets

* Redirects, moved files, and test updates

* Moved file to x-pack for tests

* Remove older CCS page and add redirects

* Cleanup, link updates, and some rewrites

* Update image

* Incorporating user feedback and rewriting much of the remote clusters page

* More changes from review feedback

* Numerous updates, including request examples for CCS and Kibana

* More changes from review feedback

* Minor clarifications on security for remote clusters

* Incorporate review feedback

Co-authored-by: Yang Wang <ywangd@gmail.com>

* Some review feedback and some editorial changes

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Yang Wang <ywangd@gmail.com>
2021-09-22 16:02:33 -04:00
Ignacio Vera 75b7b0db03
Add track_total_hits support in mvt API (#78074)
This allows consumers of the API to be able to know exactly if all the features in a tile has been considered 
when building the hits layer of a vector tile

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-09-22 08:37:50 +02:00
Ignacio Vera 2c98de99f6
Add documentation for _index property in mvt response (#78019)
Document that we are including a _index property for each hit in _mvt response.
2021-09-21 07:13:09 +02:00
edh-oss 62a471aefe
Update JSON parser and snippets (#77983)
Related to issue  #77823

This does the following:

- Updates several asciidoc files that contained code snippets with
  invalid JSON, most involving unnecessary trailing commas.

- Makes the switch from the Groovy JSON parser to the Jackson parser,
  pursuant to the general goal of eliminating Groovy dependence.

- Makes testing of JSON validity at build time more strict.

Note that this update still allows backslash escaping for any
character. Currently that matters because of the file
"docs/reference/ml/anomaly-detection/apis/get-datafeed-stats.asciidoc",
specifically this part:

    "attributes" : {
      "ml.machine_memory" :
        "$body.datafeeds.0.node.attributes.ml\.machine_memory",
      "ml.max_open_jobs" : "512"
    }

It's not clear to me what change, if any, is appropriate there. So,
I've left in the escaped period and configured the parser to ignore
it for the time being.
2021-09-20 11:08:26 +01:00
Nik Everett ed522ba26b
Disable bwc tests before backporting #77633 (#77639)
We make wire changes in #77633 so we need to disable the backwards
compatibility tests in master before merging the wire changes. We'll
re-enable them after the backport is merged.
2021-09-13 14:58:27 -04:00
Nik Everett c2c0165fd2
Profile the fetch phase (#77064)
This adds profiling to the fetch phase so we can tell when fetching is
slower than we'd like and we can tell which portion of the fetch is
slow. The output includes which stored fields were loaded, how long it
took to load stored fields, which fetch sub-phases were run, and how
long those fetch sub-phases took.

Closes #75892

* Skip bwc

* Don't compare fetch profiles

* Use passed one

* no npe

* Do last rename

* Move method down

* serialization tests

* Fix sneaky serialization

* Test for sneaky bug

* license header

* Document

* Fix test

* newline

* Restore assertion

* unit test merging

* Handle inner hits

* Fixup

* Revert unneeded

* Revert inner hits profiling

* Fix names

* Fixup names

* Move results building

* Drop loaded_nested

* Checkstyle

* Fixup more

* Finish writeable cleanup

Add unit tests for merge

* Remove null checking builder

* Fix wire mistake

How did this pass before?!

* Rename

* Remove funny builder

* Remove name munging
2021-09-13 10:00:36 -04:00
James Rodewig cfae69717a
[DOCS] Update anchor and add redirect for aliases (#77349)
PRs #73062 and #73043 repurposed the `alias` anchor for a new guide for index
and data stream aliases. Previously, this anchor was used for our field alias
documentation.

Repurposing the anchor has caused continuity errors for users selecting
different versions of the ES docs. It could also cause confusion for users with
a `/current/` link to the `alias` page.

This updates the anchor for the alias guide and adds a redirect page to
disambiguate the `alias` anchor.

It also fixes a bread crumb issue for redirects following the 'Modifying your
Data' redirect page.

Closes #77034.
2021-09-07 09:42:42 -04:00
Joe Gallo a127154042
[DOCS] Fix the terms enum API docs for search_after (#76991) 2021-08-26 14:07:06 -04:00
James Rodewig bb9587351e
[DOCS] Fix typos (#76875) 2021-08-24 08:47:12 -04:00
James Rodewig 31153e0002
[DOCS] Fix multi-value search `preference` docs (#76822)
You can't combine a `_shards` search `preference` argument with `<custom-string>`.
2021-08-23 08:33:12 -04:00
James Rodewig 7d70414c00
[DOCS] Update vector tile search API title (#76506) 2021-08-13 11:55:11 -04:00
James Rodewig 96c4ee3e5c
[DOCS] Document `_mvt` API (#75384)
* [DOCS] Document `_mvt` API

Documents the `_mvt` API endpoint added with #73872.

Relates to #75242.

* Reword

* Rename API

* Fix doc.url in JSON spec

* Reword

* Reword

* Add content type to JSON spec

* Edits

* Fix typo

* Reword

* Update docs after meeting

* Fix typos

* Fix `size` default

* Updates for #75522

* Fixes

* Clean up JSON spec

* Fix extent tag

* [DOCS] Add `<field>` constraints

* Minor clarification

* Update for #75697

* Reword

* Update for #75621

* Reword default sort

* Update for #75367

* Remove unneeded whitespace

* Add experimental admon and if flags

* [DOCS] Remove ifdefs

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-08-05 15:04:07 -04:00
Elasticsearch addict 7e7b5c9367
[DOCS] Clarify copy for terms enum API's `complete` flag (#76066)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-08-05 08:28:16 -04:00
Julie Tibshirani cdf67e0fd5
Support search slicing with point-in-time (#74457)
This PR adds support for using the `slice` option in point-in-time searches. By
default, the slice query splits documents based on their Lucene ID. This
strategy is more efficient than the one used for scrolls, which is based on the
`_id` field and must iterate through the whole terms dictionary. When slicing a
search, the same point-in-time ID must be used across slices to guarantee the
partitions don't overlap or miss documents.

Closes #65740.
2021-07-08 15:33:41 -07:00
James Rodewig 3971522c65
[DOCS] EQL: Document cross-cluster search support (#74995) (#75045)
Closes #74842.
2021-07-07 09:41:07 -04:00
Adam Locke 6974f5861a
[DOC] Update tests for collapse search results (#74721)
* [DOC] Update tests for collapse search results

* Add filter for inner hits

* Fixing tests

* Fix typos
2021-07-01 10:21:59 -04:00
James Rodewig 3d1cb4944d
[DOCS] Overhaul search template docs (#72583)
Changes:

* Adds a tutorial for search templates.
* Adds reference docs for the render search template API.
* Improves parameter documentation for the multi search template API.
* Removes duplicate examples from the search template API, multi search API, and create stored script API docs.
* Splits the source files for the search template API and the multi search template API docs.
2021-06-30 16:03:49 -04:00
debadair 29bb8fd67e
[DOCS] Add info about GET/POST to the API conventions. (#74529)
* [DOCS] Add info about GET/POST to the API conventions.

* Fixed anchor

* Added redirect

* Incorporated review suggestions
2021-06-30 09:24:42 -07:00
Mayya Sharipova aa76ebbfbe
Set max allowed size for stored async response (#74455)
Add a dynamic transient cluster setting search.max_async_search_response_size
that controls the maximum allowed size for a stored async search
response. The default max size is 10Mb. An attempt to store
an async search response larger than this size will result in error.

Relates to #67594
2021-06-30 10:21:28 -04:00
Adam Locke b890f9380c
[DOCS] Add performance info for runtime fields (#74454)
* [DOCS] Add performance info for runtime fields

* Add script-based sorting and clarify performance

* Changing title to Incentives and reworking the intro
2021-06-29 10:23:00 -04:00
Nik Everett 8904ffe2be
Add extra profiling information to terms agg (#73636)
I was helping some folks debug an issue with the terms agg and noticed
that we didn't always have the `total_buckets` debug information. I also
noticed that we can't tell how many buckets we build, so I added that
too as `built_buckets`.

Finally, I noticed that when we're using segment ords we count segments
without any values as "multi-valued". We can do better there and count
them as no-valued. That will, mostly, just improve the profiling. When
we collect from global ords we have no way to tell how many values are
on the segment so segments without any values will, sadly, in this case
still be miscounted as multi-valued.
2021-06-21 10:10:41 -04:00
James Rodewig c04fd6e301 [DOCS] Remove 'closed data stream' reference 2021-05-27 16:07:09 -04:00
James Rodewig 5729bb8d49
[DOCS] Update alias references (#73427)
Updates several `index aliases` references to `aliases`.
2021-05-27 16:00:57 -04:00