Commit Graph

718 Commits

Author SHA1 Message Date
Mayya Sharipova bf3208b028
Add scripted_metric agg context to unsigned_long (#64422)
Also enhance documentation to provide more examples
how unsigned_long field should be used in scripts

Closes #64347
2022-02-02 15:27:42 -05:00
Yannick Welsch 7f0595abe6
Implement all queries on doc-values only keyword fields (#83404)
Adds doc-values-only search support for wilcard/regexp/prefix/fuzzy etc. queries on keyword fields.

Relates #81210 and #52728
2022-02-02 14:06:27 +01:00
Yannick Welsch eec26826d6
Allow doc-values only search on geo_point fields (#83395)
Similar to #82409, but for geo_point fields.

Allows searching on geo_point fields when those fields are not indexed (index: false) but just doc values are enabled.

Also adds distance feature query support for date fields (bringing date field to feature parity with runtime fields)

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-02-02 11:56:19 +01:00
Adam Locke 620fe44c6b
[DOCS] Update dynamic mapping docs to clarify supported match_mapping_type (#83274)
* [DOCS] Update dynamic mapping docs to clarify supported match_mapping_type

* Add ES data type column header

* Remove sentence about always choosing the larger data type

* Clarify that JSON doesn't distinguish types

* Add frame to table
2022-02-01 10:37:27 -05:00
Julie Tibshirani e7ba03e0a6
Add notes on indexing to kNN search guide (#83188)
This change adds a new 'indexing considerations' section that explains why index
calls can be slow and how force merge can help search latency.
2022-01-28 10:23:35 -08:00
Mitar b65fb17a48 Fixed documentation for built in date formats. (#83036)
We had a lot of `ZZ` on the end of formats. But it's just `Z`.
2022-01-26 14:22:02 -05:00
James Rodewig d3fb014914
[DOCS] Reuse multi-level `join` warning (#82976)
Updates and reuses a warning against creating multi-level `join` fields to make it more prominent.

The current warning is low on the page, where some users may not seeing until they've already begun mapping fields.

Closes https://github.com/elastic/elasticsearch/issues/82818.
2022-01-25 13:51:42 -05:00
Yannick Welsch d9f77fa3a6
Allow doc-values only search on ip fields (#82929)
Allows searching on ip fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-25 09:24:12 +01:00
Yannick Welsch 0592c4cd7e
Allow doc-values only search on boolean fields (#82925)
Allows searching on boolean fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-24 14:27:06 +01:00
Yannick Welsch fd7f69cea6
Allow doc-values only search on keyword fields (#82846)
Allows searching on keyword fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-24 08:57:11 +01:00
James Rodewig d8229ddd5b
[DOCS] Clarify that `null` values don't create dynamic field mappings (#82769)
Closes #82641.
2022-01-19 09:08:36 -05:00
Yannick Welsch 928c09a373
Allow doc-values only search on date types (#82602)
Similar to #82409, but for date fields.

Allows searching on date field types (date, date_nanos) when those fields are not indexed (index: false) but just doc
values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-17 11:57:31 +01:00
Yannick Welsch e421477ac8
Allow docvalues-only search on number types (#82409)
Allows searching on number field types (long, short, int, float, double, byte, half_float) when those fields are not
indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Note to reviewers:

I have split isSearchable into two separate methods isIndexed and isSearchable on MappedFieldType. The former one is
about whether actual indexing data structures have been used (postings or points), and the latter one on whether you
can run queries on the given field (e.g. used by field caps). For number field types, queries are now allowed whenever
points are available or when doc values are available (i.e. searchability is expanded).

Relates #81210 and #52728
2022-01-13 16:23:01 +01:00
Julie Tibshirani 6c442920ba
Reject zero-length vectors when using cosine similarity (#82241)
Cosine similarity is not defined when one of the vectors has zero magnitude.
Before, the kNN search endpoint threw a confusing exception related to top docs
collection. Now we reject vectors early with a clear error message, failing
indexing if the vector has zero magnitude.
2022-01-11 09:34:04 -08:00
eltomello 38a74a4545
[DOCS] Fix field name to match description (#81621) 2021-12-13 15:51:42 -05:00
James Rodewig 229d2d7a77
[DOCS] Add high-level guide for kNN search (#80857)
Adds a high-level guide for running an approximate or exact kNN search in Elasticsearch.

Relates to https://github.com/elastic/elasticsearch/issues/78473.
2021-11-30 14:17:39 -05:00
Colin Ng dd2424b79c
Fix typo (#80925) 2021-11-23 16:28:53 -05:00
James Rodewig cbcd901096
[DOCS] Relocate `index.mapping.dimension_fields.limit` setting docs (#80964)
Moves `index.mapping.dimension_fields.limit` so that its co-located with
other mapping limit settings.
2021-11-23 14:51:28 -05:00
Dan Hermann 0d21b032b6
[DOCS] Custom routing for data streams 2021-11-10 07:11:50 -06:00
Julie Tibshirani 8ca693b271
Add docs for kNN search endpoint (#80378)
This commit adds docs for the new `_knn_search` endpoint.

It focuses on being an API reference and is light on details in terms of how
exactly the kNN search works, and how the endpoint contrasts with
`script_score` queries. We plan to add a high-level guide on kNN search that
will explain this in depth.

Relates to #78473.
2021-11-09 09:28:12 -08:00
Julie Tibshirani 44198c6f34
Check nested fields earlier in kNN search (#80516)
Currently, we don't support kNN search against fields in a `nested` mapping.
Before, we were checking this at search-time. This commit moves it earlier, so
you aren't even allowed to set `index: true` if the vector is in a nested
mapping. That way, users are aware of the limitation before they start to index
documents.

Relates to #78473.
2021-11-09 09:06:53 -08:00
Yannick Welsch 6eef523674
Revert 74559 (Avoid global ordinals in composite) (#78846) (#80498)
This reverts the change to use segment ordinals in composite terms aggregations due to a performance degradation when the field is high cardinality.

Co-authored-by: Mark Tozzi <mark.tozzi@elastic.co>
2021-11-08 17:11:46 +01:00
James Rodewig f56a0f4b66
[DOCS] Remove `testenv` annotations from doc snippet tests (#80023)
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.

Relates to #79309, #31619
2021-11-05 18:38:50 -04:00
Julie Tibshirani 36ebac38bf Remove a stray backtick in the dense vector docs 2021-11-05 10:21:44 -07:00
Julie Tibshirani 075d08eb64
Update `dense_vector` docs with kNN indexing options (#80306)
This commit updates the `dense_vector` docs to include information on the new
`index`, `similarity`, and `index_options` parameters. It also tries to clarify
the difference between `similarity` and `index_options` with the existing
parameters that have the same name.

Relates to #78473.
2021-11-04 11:44:13 -07:00
James Rodewig 3734dada85
[DOCS] Add collapsible section to TSDB mapping parameters + index setting (#80230) (#80278) 2021-11-03 10:13:48 -04:00
Tobias Frey 9cddd78674
[DOCS] Fix typo (#79609) 2021-10-27 11:05:09 -04:00
James Rodewig ee1f71d421
[DOCS] Add experimental label to TSDB mapping params and settings (#79647)
Adds an `experimental` annotation to the following:

* `time_series_metric` mapping parameter
* `time_series_dimension` mapping parameter
* `index.mapping.dimension_fields.limit` index setting
*  `time_series_dimension` and `time_series_metric` properties in the field caps API response
2021-10-27 09:09:54 -04:00
Dan Hermann 4a36d5cd79
Remove endpoint for freezing indices (#78918) 2021-10-26 06:37:56 -05:00
Christoph Büscher f522de6b56
[Docs] Clarify ignore_above behaviour (#79705)
Clarify that `keyword` fields that exceed the optional `ignore_above` setting
are inlcuded in the `_ignored` fields since 7.14.

Closes #79605
2021-10-25 20:27:02 +02:00
James Rodewig dbb8a015ad [DOCS] Fix typos in flattened field type docs 2021-10-05 14:15:07 -04:00
James Rodewig ce4b95e5b0
[DOCS] Document `time_series_metric` mapping parameter (#78013)
Changes:
* Documents the `time_series_metric` mapping parameter for PR #76766.
* Renames the `dimension` parameter to `time_series_dimension` for PR #78012.
* Adds support for `unsigned_long` to `time_series_dimension` for PR #78204.
2021-09-23 08:54:19 -04:00
Adam Locke 7d61b0261c
[DOCS] Add composite runtime fields (#78050)
* [DOCS] Add composite runtime fields

* Update snippets and tests

* Add note that composite runtime fields cannot be indexed yet
2021-09-22 07:56:50 -04:00
James Rodewig e729c3f543
[DOCS] Clarify geoshape orientation docs (#75888)
Adds additional information about how Elasticsearch uses polygon orientation. Elasticsearch only uses a polygon's orientation to determine if it crosses the international dateline. If so, Elasticsearch splits the polygon at the dateline.

Closes #74891
2021-09-08 11:10:03 -04:00
Adam Locke 32e364d394
[DOCS] Clarify indexing a runtime field (#77117)
* [DOCS] Clarify indexing a runtime field

* Clarify wording based on reviewer feedback
2021-09-01 11:59:11 -04:00
James Rodewig 1acc7e5d5e
[DOCS] Remove unneeded sidebar from array docs (#76664) 2021-08-18 14:00:30 -04:00
Julie Tibshirani 2ddbd62291
Mention match_only_text in disk usage docs (#76416)
* Mention match_only_text in disk usage docs

Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.

* Fixing typo and minor grammar changes

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-08-13 09:31:09 -04:00
James Rodewig 1fa6e79a1c
[DOCS] Clarify multi-field relationship to parent field (#76244)
Closes #71659
2021-08-09 11:43:06 -04:00
James Rodewig 32a516807a
[DOCS] Update routing formulas (#76203)
The `_routing` metadata field docs currently include formulas for how
Elasticsearch routes documents to shards. However, these formulas were not
updated for #18699.  This updates the routing formulas and adds xrefs for
related settings.

Closes #76072
2021-08-09 11:42:33 -04:00
Adam Locke c9901429c2
[DOCS] Add retrieving runtime fields to introduction (#76084) 2021-08-04 11:17:28 -04:00
James Rodewig fc0ac1923d
[DOCS] Correct spelling for geo terms (#76028)
Changes:
* Use "geopoint" when not referring to the literal field type
* Use "geoshape" when not referring to the literal field type or query type
* Use "GeoJSON" consistently
2021-08-03 09:55:48 -04:00
a-k-g d671e3f7a8
[Docs] Include `index` param in `geo_point` docs (#75798) 2021-08-03 08:57:16 -04:00
James Rodewig 1eaf1beffd [DOCS] Reword internal use copy for `dimension` mapping parameter 2021-07-30 09:01:46 -04:00
Adrien Grand feb6620d14
`indices.query.bool.max_clause_count` now limits all query clauses (#75297)
In the upcoming Lucene 9 release, `indices.query.bool.max_clause_count` is
going to apply to the entire query tree rather than per `bool` query. In order
to avoid breaks, the limit has been bumped from 1024 to 4096.

The semantics will effectively change when we upgrade to Lucene 9, this PR
is only about agreeing on a migration strategy and documenting this change.

To avoid further breaks, I am leaning towards keeping the current setting name
even though it contains `bool`. I believe that it still makes sense given that
`bool` queries are typically the main contributors to high numbers of clauses.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-07-21 12:16:30 +02:00
James Rodewig 1f04319826
[DOCS] Document time series dimension mapping parameters (#75414)
Changes:
* Documents the `dimension` mapping parameter for `ip`, `keyword`, and `numeric`
  fields.

* Documents the `index.mapping.dimension_fields.limit` index setting.
2021-07-19 11:24:30 -04:00
Yannick Welsch 412ac1a042
Update docs that composite agg no longer uses global ords (#74754)
Follow-up to #74559
2021-07-05 11:26:30 +02:00
Adam Locke b759c2fdd8
[DOCS] Word changes for runtime field incentives (#74769)
Incorporates feedback from #74454
2021-06-30 13:43:26 -04:00
Adam Locke b890f9380c
[DOCS] Add performance info for runtime fields (#74454)
* [DOCS] Add performance info for runtime fields

* Add script-based sorting and clarify performance

* Changing title to Incentives and reworking the intro
2021-06-29 10:23:00 -04:00
James Rodewig d4ed43c5a4
[DOCS] Remove deprecated `geo_shape` parameters (#74519)
* Removes docs and references for the following `geo_shape` mapping parameters:
  * `tree`
  * `tree_levels`
  * `strategy`
  * `distance_error_pct`
* Updates a related breaking change.

Relates to #70850
2021-06-29 08:52:05 -04:00
Benjamin Trent 07b336f1b0
Add support for range aggregations on histogram mapped fields (#74146)
This adds support for the range aggregation over `histogram` mapped fields.

Decisions made for implementation:

 - Sub-aggregations are not allowed. This is to simplify implementation and follows the prior art set by the `histogram` aggregation
 - Nothing fancy is done with the ranges. No filter translations as we cannot easily do a `range` filter query against histogram fields. This may be an optimization in the future.
 - Ranges check the histogram value ONLY. No interpolation of values is done. If we have better statistics around the histogram this MAY be possible.
2021-06-29 07:24:54 -04:00