Commit Graph

1008 Commits

Author SHA1 Message Date
James Rodewig a5feaf978d
[DOCS] Add security privileges to search API docs (#68009) 2021-02-03 16:30:51 -05:00
Adam Locke 5de8613c62
[DOCS] Add links to ESS for setting up remote clusters on cloud. (#68401) 2021-02-02 14:32:54 -05:00
Philippus Baalman c011181eec
[DOCS] Fix typo (#68364) 2021-02-02 09:59:57 -05:00
James Rodewig 4625a18540
[DOCS] Fix minor suggester doc errors (#68244) 2021-02-01 09:33:12 -05:00
James Rodewig 5b3b9060bc
[DOCS] Fix title abbrevs for API docs (#68118) 2021-01-28 10:28:41 -05:00
James Rodewig 141fa5fe17
[DOCS] Fix explain API anchors (#68007) 2021-01-26 12:02:56 -05:00
Adam Locke c5c5d27fa7
[DOCS] Add beta admonition for runtime fields. (#67847)
* Add beta admonition for runtime fields.

* Add beta admonition to search your data page.
2021-01-21 17:10:37 -05:00
Adam Locke 744f7c67c8
[DOCS] Updates to the runtime fields docs for 7.11+ (#67484)
* Moving examples to the page for retrieving runtime fields.

* Adding runtime_mappings to request body of search API.

* Updating runtime_mappings properties and adding runtime fields to search your data.

* Updating examples and hopefully fixing build failure.

* Fixing snippet formatting that was causing test failure.

* Adding page in Painless guide for runtime fields.

* Fixing typo.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-01-19 10:31:17 -05:00
Nhat Nguyen 59082c0d3a
Retry point in time on other copy when possible (#66713)
Relates #61062
2021-01-09 09:43:59 -05:00
James Rodewig 58742610c1
[DOCS] Rewrite agg breakdown docs for profile API (#67042) 2021-01-05 16:56:53 -05:00
Przemyslaw Gomulka 5e74f79e22
Support response content-type with versioned media type (#65500)
This commit allows returning a correct requested response content-type - it did not work for versioned media types.
It is done by adding new vendor specific instances to XContent and TextFormat enums. These instances can then "format" the response content type string when provided with parameters. This is similar to what SQL plugin does with its media types.

#51816
2021-01-05 09:23:22 +01:00
Alan Tan ee6cbef7c2
[DOCS] Fix typo (#66721) 2020-12-22 09:02:33 -05:00
Simon b4790edc7d [Docs] Add option to reference stored templates for `_rank_eval` requests
The option to use templates already defined in the cluster is not explicitly stated in the docs.
This PR adds and example to the `rank_eval` documentation.
2020-12-10 15:14:52 +01:00
Adam Locke bce1081c73
[DOCS] Add docs for runtime fields (#62653)
* First steps in docs for runtime fields.

* Adding new page for runtime fields.

* Adding page for runtime fields.

* Adding more to the runtime fields topic.

* Adding parameters and retrieval options for runtime fields.

* Adding TESTSETUP for index creation.

* Incorporating review feedback.

* Incorporating reviewer feedback.

* Adding examples for runtime fields.

* Adding more context and simplifying the example.

* Changing timestamp to @timestamp throughout.

* Removing duplicate @timestamp field.

* Expanding example to hopefully fix CI builds.

* Adding skip test for result.

* Adding missing callout.

* Adding TESTRESPONSEs, which are currently broken.

* Fixing TESTRESPONSEs.

* Incorporating review feedback.

* Several clarifications, better test cases, and other changes.

* Adding missing callout in example.

* Adding substitutions to TESTRESPONSE for shorter results shown.

* Shuffling some information and adding link to script-fields.

* Fixing typo.

* Updates for API redesign -- will break builds.

* Updating examples and including info about overriding fields.

* Updating examples.

* Adding info for using runtime fields in the search request.

* Adding that queries against runtime fields are expensive.

* Incorporating feedback from reviewers.

* Minor changes from reviews.

* Adding alias for test case.

* Adding aliases to PUT example.

* Fixing test cases, for real this time.

* Updating use cases and introducing overlay throughout.

* Edits, adding 'shadowing', and explaining shadowing better.

* Streamlining tests and other changes.

* Fix formatting in example for test.

* Apply suggestions from code review

* Incorporating reviewer feedback 7 Dec

* Shifting structure of mapping page to fix cross links.

* Revisions for shadowing, overview, and other sections.

* Removing dot notation section and incorporating review changes.

* Adding updated example for shadowing.

* Streamlining shadowing example and TESTRESPONSEs.
2020-12-09 17:54:58 -05:00
Christoph Büscher 3c3a43249f
Support unmapped fields in search 'fields' option (#65386)
Currently, the 'fields' option only supports fetching mapped fields. Since
'fields' is meant to be the central place to retrieve document content, it
should allow for loading unmapped values. This change adds implementation and
tests for this feature.

Closes #63690
2020-12-01 21:40:27 +01:00
bellengao 2d9b83e32c
[DOCS] Fix formatting issue in search explain docs (#65303) 2020-11-20 09:01:01 -05:00
James Rodewig 1f8d1e9ee1
[DOCS] Fix indentation (#64941) (#64962)
Co-authored-by: kemalizing <36174398+kemalizing@users.noreply.github.com>
2020-11-12 08:47:20 -05:00
James Rodewig 7ed7a9584e
[DOCS] Fix typo (#64675) (#64799)
Co-authored-by: Ashish Jayan <58534490+chasexd@users.noreply.github.com>
2020-11-09 11:12:46 -05:00
Mayya Sharipova 074f7d2e8a
Async search status (#62947)
Introduce async search status API

GET /_async_search/status/<id>

The API is restricted to the monitoring_user role.

For a running async search, the response is:

```js
{
  "id" : <id>,
  "is_running" : true,
  "is_partial" : true,
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "_shards" : {
      "total" : 562,
      "successful" : 188,
      "skipped" : 0,
      "failed" : 0
  }
}
```

For a completed async search, an additional
`completion_status` fields is added.

```js
{
  "id" : <id>,
  "is_running" : false,
  "is_partial" : false,
  "start_time_in_millis" : 1583945890986,
  "expiration_time_in_millis" : 1584377890986,
  "_shards" : {
      "total" : 562,
      "successful" : 562,
      "skipped" : 0,
      "failed" : 0
  },
 "completion_status" : 200
}
```

Closes #57537
2020-11-03 11:35:28 -05:00
James Rodewig 37b6adaf91
[DOCS] Rewrite aggs overview (#64318)
- Replaces more abstract docs about object structure and values source with task-based examples.
- Relocates several sections from the current `misc.asciidoc` file.
- Alphabetically sorts agg categories in the nav.
- Removes the matrix agg family. Moves the stats matrix agg under the metric agg family

Co-authored-by: debadair <debadair@elastic.co>
2020-10-30 08:39:38 -04:00
James Rodewig 1ea83359bb
[DOCS] Fix case for 'Boolean' (#64299) 2020-10-29 09:04:43 -04:00
Nik Everett 3af540b50d
Remove aggregation's postCollect phase (#64016)
After #63811 it became clear to me that `postCollect` is kind of
dangerous and not all that useful. So this removes it.

The trouble with `postCollect` is that it all happened right after we
finished calling `collect` on the `LeafBucketCollectors` but before we
built the aggregation results. But in #63811 we found out that we can't
call `postCollect` on the children of `parent` or `child` aggregators
until we know which *which* aggregation results we're building.

So this removes `postCollect` and moves all of the things we did at
post-collect phase into `buildAggregations` or into hooks called in
those methods.
2020-10-28 17:33:27 -04:00
Julie Tibshirani a98fd8b6e1
Mark the search 'fields' option as beta. (#63699)
We've identified two important enhancements that may affect the API. We expect
any API changes from these enhancements to be minor, but want to leave open the
possibility for small breaks. For example, we may end up returning unmapped
fields by default, or omitting nested fields from the root hit. The impact to
users should be quite small.

We're tracking the issues we need to resolve before removing the 'beta' label
here: #60985.
2020-10-16 17:13:29 -07:00
James Rodewig 2fd11f58a0
[DOCS] Fix typo in search profile docs (#63522) (#63557)
Co-authored-by: ondrejbardon <70531797+ondrejbardon@users.noreply.github.com>
2020-10-12 10:12:45 -04:00
Gordon Brown 91f4b58bf7
Deprecate REST access to System Indices (#60945)
This PR adds deprecation warnings when accessing System Indices via the REST layer. At this time, these warnings are only enabled for Snapshot builds by default, to allow projects external to Elasticsearch additional time to adjust their access patterns.

Deprecation warnings will be triggered by all REST requests which access registered System Indices, except for purpose-specific APIs which access System Indices as an implementation detail a few specific APIs which will continue to allow access to system indices by default:

- `GET _cluster/health`
- `GET {index}/_recovery`
- `GET _cluster/allocation/explain`
- `GET _cluster/state`
- `POST _cluster/reroute`
- `GET {index}/_stats`
- `GET {index}/_segments`
- `GET {index}/_shard_stores`
- `GET _cat/[indices,aliases,health,recovery,shards,segments]`

Deprecation warnings for accessing system indices take the form:
```
this request accesses system indices: [.some_system_index], but in a future major version, direct access to system indices will be prevented by default
```
2020-10-06 11:13:48 -06:00
James Rodewig 382c454c8b
[DOCS] Corrected track_total_hits def (#62830) (#63180)
Co-authored-by: John Berryman <jnbrymn@github.com>
2020-10-02 09:46:09 -04:00
Nik Everett 2c93cd38e4
Docs: Support """ in console-response (#62876)
We support `"""` in `console` snippets to emulate kibana's CONSOLE.
CONSOLE also spits out `"""` when a json field contains a new line or a
double quote. This adds support for those sorts of responses to the
handling of `console-response` snippets.
2020-09-28 10:49:12 -04:00
James Rodewig a0149576ef
[DOCS] Correct the documented behaviour of `track_total_hits` (#62837) (#62866)
If `track_total_hits=true` is used, the exact value of the number of hits is returned - i.e. the value is effectively limitless, and not the default value of 10,000

Co-authored-by: AndyHunt66 <andrew.hunt@elastic.co>
2020-09-24 09:16:56 -04:00
Nik Everett 049bca0959
Add more debugging information for cardinality agg (#62317)
This adds two extra bits of info to the profiler:
1. Count of the number of different types of collectors. This lets us figure
   out if we're using the optimization for segment ordinals. It adds a few
   more similar counters just for good measure.
2. Profiles the `getLeafCollector` and `postCollection` methods. These are
   non-trivial for some aggregations, like cardinality.
2020-09-15 08:49:13 -04:00
Julie Tibshirani a19503e9a6
Add the fields option to the search API docs. (#62260) 2020-09-14 13:43:30 -07:00
Julie Tibshirani f29c743a47
Support the 'fields' option in inner_hits and top_hits. (#62259)
This PR adds support for the 'fields' option in the following places:
* Anytime `inner_hits` is used, for both fetching nested/ child docs and field collapsing
* The `top_hits` aggregation

Addresses #61949.
2020-09-14 10:08:58 -07:00
James Rodewig 2fa5891559 [DOCS] Note PIT searches should omit target 2020-09-09 15:34:07 -04:00
James Rodewig 3ec8f8a451
[DOCS] Fix field caps API docs (#62110) 2020-09-08 11:56:29 -04:00
James Rodewig 04dff9e792
[DOCS] Remove indices from PIT searches (#62105) 2020-09-08 10:58:16 -04:00
James Rodewig 1b51acbbab
[DOCS] Add PIT to search after docs (#61593)
Co-authored-by: Jim Ferenczi <jim.ferenczi@elastic.co>
2020-09-08 09:53:21 -04:00
James Rodewig ea50c24eb7 [DOCS] Update shard allocation awareness xref 2020-09-02 11:32:35 -04:00
James Rodewig f881a695e1
[DOCS] Add redirects for wildcard and constant keyword (#61815) 2020-09-01 15:32:35 -04:00
James Rodewig 49350ddae8
[DOCS] Reorg field data types page (#61117) 2020-08-26 14:01:34 -04:00
James Rodewig 590b7f1cb8 [DOCS] Fix typo in search your data docs 2020-08-25 17:00:45 -04:00
James Rodewig 915b353f36
[DOCS] Display point in time API docs (#61527) 2020-08-25 11:03:41 -04:00
Nhat Nguyen 879279c9b4
Introduce point in time APIs in x-pack basic (#61062)
This commit introduces a new API that manages point-in-times in x-pack 
basic. Elasticsearch pit (point in time) is a lightweight view into the
state of the data as it existed when initiated. A search request by
default executes against the most recent point in time. In some cases,
it is preferred to perform multiple search requests using the same point
in time. For example, if refreshes happen between search_after requests,
then the results of those requests might not be consistent as changes
happening between searches are only visible to the more recent point in
time.

A point in time must be opened before being used in search requests. The 
`keep_alive` parameter tells Elasticsearch how long it should keep a
point in time around.

```
POST /my_index/_pit?keep_alive=1m
```

The response from the above request includes a `id`, which should be 
passed to the `id` of the `pit` parameter of search requests.

```
POST /_search
{
    "query": {
        "match" : {
            "title" : "elasticsearch"
        }
    },
    "pit": {
            "id":  "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWICBXV1aWQyAAAFdXVpZDEAAQltYXRjaF9hbGw_gAAAAA==",
            "keep_alive": "1m"
    }
}
```

Point-in-times are automatically closed when the `keep_alive` is 
elapsed. However, keeping point-in-times has a cost; hence,
point-in-times should be closed as soon as they are no longer used in
search requests.

```
DELETE /_pit
{
    "id" : "46ToAwMDaWR4BXV1aWQxAgZub2RlXzEAAAAAAAAAAAEBYQNpZHkFdXVpZDIrBm5vZGVfMwAAAAAAAAAAKgFjA2lkeQV1dWlkMioGbm9kZV8yAAAAAAAAAAAMAWIBBXV1aWQyAAA="
}
```

#### Notable works in this change:

- Move the search state to the coordinating node: #52741
- Allow searches with a specific reader context: #53989
- Add the ability to acquire readers in IndexShard: #54966

Relates #46523
Relates #26472

Co-authored-by: Jim Ferenczi <jimczi@apache.org>
2020-08-24 20:24:35 -04:00
James Rodewig d03012fbd1
[DOCS] Fix typo in profile API docs (#61445) (#61502)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: shashikumarec088 <shashikumarec088@gmail.com>
2020-08-24 15:30:25 -04:00
James Rodewig fdc4e83050
[DOCS] Combine `Search your data` files (#61477)
No-op changes to:

* Move `Search your data` source files into the same directory
* Rename `Search your data` source files based on page ID
* Remove unneeded includes
* Remove the `Request` dir
2020-08-24 11:22:56 -04:00
James Rodewig c688cb6bfd
[DOCS] Fix hyphenation for "time series" (#61472) 2020-08-24 10:34:41 -04:00
James Rodewig d46931840b
[DOCS] Prune `Search your data` content (#61303)
Changes:
* Removes narrative around URI searches. These aren't commonly used in production. The `q` param is already covered in the search API docs: https://www.elastic.co/guide/en/elasticsearch/reference/master/search-search.html#search-api-query-params-q
* Adds a common options section that highlights narrative docs for query DSL, aggregations, multi-index search, search fields, pagination, sorting, and async search.
* Adds a `Search shard routing` page. Moves narrative docs for adaptive replica selection, preference, routing , and shard limits to that section.
* Moves search timeout and cancellation content to the `Search your data` page.
* Creates a `Search multiple data streams and indices` page. Moves related narrative docs for multi-target syntax searches and `indices_boost` to that page.
* Removes narrative examples for the `search_type` parameters. Moves documentation for this parameter to the search API docs.
2020-08-24 08:38:20 -04:00
Nhat Nguyen 2a3a8dd296 Fix anchor doc for msearch cancellation paragraph
Relates #61418
2020-08-21 12:11:00 -04:00
James Rodewig c21930e4ce
[DOCS] Remove URI search examples from API reference (#61423) 2020-08-21 10:57:35 -04:00
Nhat Nguyen 35ccd06918
Add cancellation doc for multi search (#61418)
Relates #61337
2020-08-21 10:10:05 -04:00
James Rodewig a94e5cb7c4
[DOCS] Replace Wikipedia links with attribute (#61171) 2020-08-17 09:44:24 -04:00
James Rodewig 39f92f2a02
[DOCS] Fix typo in suggester docs (#61077) (#61204)
Co-authored-by: Arash Layeghi <arashlayeghi57@gmail.com>
2020-08-17 09:14:37 -04:00