Commit Graph

761 Commits

Author SHA1 Message Date
Yannick Welsch d9f77fa3a6
Allow doc-values only search on ip fields (#82929)
Allows searching on ip fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-25 09:24:12 +01:00
Yannick Welsch 0592c4cd7e
Allow doc-values only search on boolean fields (#82925)
Allows searching on boolean fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-24 14:27:06 +01:00
Yannick Welsch fd7f69cea6
Allow doc-values only search on keyword fields (#82846)
Allows searching on keyword fields when those fields are not indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-24 08:57:11 +01:00
James Rodewig d8229ddd5b
[DOCS] Clarify that `null` values don't create dynamic field mappings (#82769)
Closes #82641.
2022-01-19 09:08:36 -05:00
Yannick Welsch 928c09a373
Allow doc-values only search on date types (#82602)
Similar to #82409, but for date fields.

Allows searching on date field types (date, date_nanos) when those fields are not indexed (index: false) but just doc
values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Relates #81210 and #52728
2022-01-17 11:57:31 +01:00
Yannick Welsch e421477ac8
Allow docvalues-only search on number types (#82409)
Allows searching on number field types (long, short, int, float, double, byte, half_float) when those fields are not
indexed (index: false) but just doc values are enabled.

This enables searches on archive data, which has access to doc values but not index structures. When combined with
searchable snapshots, it allows downloading only data for a given (doc value) field to quickly filter down to a select set
of documents.

Note to reviewers:

I have split isSearchable into two separate methods isIndexed and isSearchable on MappedFieldType. The former one is
about whether actual indexing data structures have been used (postings or points), and the latter one on whether you
can run queries on the given field (e.g. used by field caps). For number field types, queries are now allowed whenever
points are available or when doc values are available (i.e. searchability is expanded).

Relates #81210 and #52728
2022-01-13 16:23:01 +01:00
Julie Tibshirani 6c442920ba
Reject zero-length vectors when using cosine similarity (#82241)
Cosine similarity is not defined when one of the vectors has zero magnitude.
Before, the kNN search endpoint threw a confusing exception related to top docs
collection. Now we reject vectors early with a clear error message, failing
indexing if the vector has zero magnitude.
2022-01-11 09:34:04 -08:00
eltomello 38a74a4545
[DOCS] Fix field name to match description (#81621) 2021-12-13 15:51:42 -05:00
James Rodewig 229d2d7a77
[DOCS] Add high-level guide for kNN search (#80857)
Adds a high-level guide for running an approximate or exact kNN search in Elasticsearch.

Relates to https://github.com/elastic/elasticsearch/issues/78473.
2021-11-30 14:17:39 -05:00
Colin Ng dd2424b79c
Fix typo (#80925) 2021-11-23 16:28:53 -05:00
James Rodewig cbcd901096
[DOCS] Relocate `index.mapping.dimension_fields.limit` setting docs (#80964)
Moves `index.mapping.dimension_fields.limit` so that its co-located with
other mapping limit settings.
2021-11-23 14:51:28 -05:00
Dan Hermann 0d21b032b6
[DOCS] Custom routing for data streams 2021-11-10 07:11:50 -06:00
Julie Tibshirani 8ca693b271
Add docs for kNN search endpoint (#80378)
This commit adds docs for the new `_knn_search` endpoint.

It focuses on being an API reference and is light on details in terms of how
exactly the kNN search works, and how the endpoint contrasts with
`script_score` queries. We plan to add a high-level guide on kNN search that
will explain this in depth.

Relates to #78473.
2021-11-09 09:28:12 -08:00
Julie Tibshirani 44198c6f34
Check nested fields earlier in kNN search (#80516)
Currently, we don't support kNN search against fields in a `nested` mapping.
Before, we were checking this at search-time. This commit moves it earlier, so
you aren't even allowed to set `index: true` if the vector is in a nested
mapping. That way, users are aware of the limitation before they start to index
documents.

Relates to #78473.
2021-11-09 09:06:53 -08:00
Yannick Welsch 6eef523674
Revert 74559 (Avoid global ordinals in composite) (#78846) (#80498)
This reverts the change to use segment ordinals in composite terms aggregations due to a performance degradation when the field is high cardinality.

Co-authored-by: Mark Tozzi <mark.tozzi@elastic.co>
2021-11-08 17:11:46 +01:00
James Rodewig f56a0f4b66
[DOCS] Remove `testenv` annotations from doc snippet tests (#80023)
Removes `testenv` annotations and related code. These annotations originally let you skip x-pack snippet tests in the docs. However, that's no longer possible.

Relates to #79309, #31619
2021-11-05 18:38:50 -04:00
Julie Tibshirani 36ebac38bf Remove a stray backtick in the dense vector docs 2021-11-05 10:21:44 -07:00
Julie Tibshirani 075d08eb64
Update `dense_vector` docs with kNN indexing options (#80306)
This commit updates the `dense_vector` docs to include information on the new
`index`, `similarity`, and `index_options` parameters. It also tries to clarify
the difference between `similarity` and `index_options` with the existing
parameters that have the same name.

Relates to #78473.
2021-11-04 11:44:13 -07:00
James Rodewig 3734dada85
[DOCS] Add collapsible section to TSDB mapping parameters + index setting (#80230) (#80278) 2021-11-03 10:13:48 -04:00
Tobias Frey 9cddd78674
[DOCS] Fix typo (#79609) 2021-10-27 11:05:09 -04:00
James Rodewig ee1f71d421
[DOCS] Add experimental label to TSDB mapping params and settings (#79647)
Adds an `experimental` annotation to the following:

* `time_series_metric` mapping parameter
* `time_series_dimension` mapping parameter
* `index.mapping.dimension_fields.limit` index setting
*  `time_series_dimension` and `time_series_metric` properties in the field caps API response
2021-10-27 09:09:54 -04:00
Dan Hermann 4a36d5cd79
Remove endpoint for freezing indices (#78918) 2021-10-26 06:37:56 -05:00
Christoph Büscher f522de6b56
[Docs] Clarify ignore_above behaviour (#79705)
Clarify that `keyword` fields that exceed the optional `ignore_above` setting
are inlcuded in the `_ignored` fields since 7.14.

Closes #79605
2021-10-25 20:27:02 +02:00
James Rodewig dbb8a015ad [DOCS] Fix typos in flattened field type docs 2021-10-05 14:15:07 -04:00
James Rodewig ce4b95e5b0
[DOCS] Document `time_series_metric` mapping parameter (#78013)
Changes:
* Documents the `time_series_metric` mapping parameter for PR #76766.
* Renames the `dimension` parameter to `time_series_dimension` for PR #78012.
* Adds support for `unsigned_long` to `time_series_dimension` for PR #78204.
2021-09-23 08:54:19 -04:00
Adam Locke 7d61b0261c
[DOCS] Add composite runtime fields (#78050)
* [DOCS] Add composite runtime fields

* Update snippets and tests

* Add note that composite runtime fields cannot be indexed yet
2021-09-22 07:56:50 -04:00
James Rodewig e729c3f543
[DOCS] Clarify geoshape orientation docs (#75888)
Adds additional information about how Elasticsearch uses polygon orientation. Elasticsearch only uses a polygon's orientation to determine if it crosses the international dateline. If so, Elasticsearch splits the polygon at the dateline.

Closes #74891
2021-09-08 11:10:03 -04:00
Adam Locke 32e364d394
[DOCS] Clarify indexing a runtime field (#77117)
* [DOCS] Clarify indexing a runtime field

* Clarify wording based on reviewer feedback
2021-09-01 11:59:11 -04:00
James Rodewig 1acc7e5d5e
[DOCS] Remove unneeded sidebar from array docs (#76664) 2021-08-18 14:00:30 -04:00
Julie Tibshirani 2ddbd62291
Mention match_only_text in disk usage docs (#76416)
* Mention match_only_text in disk usage docs

Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.

* Fixing typo and minor grammar changes

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-08-13 09:31:09 -04:00
James Rodewig 1fa6e79a1c
[DOCS] Clarify multi-field relationship to parent field (#76244)
Closes #71659
2021-08-09 11:43:06 -04:00
James Rodewig 32a516807a
[DOCS] Update routing formulas (#76203)
The `_routing` metadata field docs currently include formulas for how
Elasticsearch routes documents to shards. However, these formulas were not
updated for #18699.  This updates the routing formulas and adds xrefs for
related settings.

Closes #76072
2021-08-09 11:42:33 -04:00
Adam Locke c9901429c2
[DOCS] Add retrieving runtime fields to introduction (#76084) 2021-08-04 11:17:28 -04:00
James Rodewig fc0ac1923d
[DOCS] Correct spelling for geo terms (#76028)
Changes:
* Use "geopoint" when not referring to the literal field type
* Use "geoshape" when not referring to the literal field type or query type
* Use "GeoJSON" consistently
2021-08-03 09:55:48 -04:00
a-k-g d671e3f7a8
[Docs] Include `index` param in `geo_point` docs (#75798) 2021-08-03 08:57:16 -04:00
James Rodewig 1eaf1beffd [DOCS] Reword internal use copy for `dimension` mapping parameter 2021-07-30 09:01:46 -04:00
Adrien Grand feb6620d14
`indices.query.bool.max_clause_count` now limits all query clauses (#75297)
In the upcoming Lucene 9 release, `indices.query.bool.max_clause_count` is
going to apply to the entire query tree rather than per `bool` query. In order
to avoid breaks, the limit has been bumped from 1024 to 4096.

The semantics will effectively change when we upgrade to Lucene 9, this PR
is only about agreeing on a migration strategy and documenting this change.

To avoid further breaks, I am leaning towards keeping the current setting name
even though it contains `bool`. I believe that it still makes sense given that
`bool` queries are typically the main contributors to high numbers of clauses.

Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
2021-07-21 12:16:30 +02:00
James Rodewig 1f04319826
[DOCS] Document time series dimension mapping parameters (#75414)
Changes:
* Documents the `dimension` mapping parameter for `ip`, `keyword`, and `numeric`
  fields.

* Documents the `index.mapping.dimension_fields.limit` index setting.
2021-07-19 11:24:30 -04:00
Yannick Welsch 412ac1a042
Update docs that composite agg no longer uses global ords (#74754)
Follow-up to #74559
2021-07-05 11:26:30 +02:00
Adam Locke b759c2fdd8
[DOCS] Word changes for runtime field incentives (#74769)
Incorporates feedback from #74454
2021-06-30 13:43:26 -04:00
Adam Locke b890f9380c
[DOCS] Add performance info for runtime fields (#74454)
* [DOCS] Add performance info for runtime fields

* Add script-based sorting and clarify performance

* Changing title to Incentives and reworking the intro
2021-06-29 10:23:00 -04:00
James Rodewig d4ed43c5a4
[DOCS] Remove deprecated `geo_shape` parameters (#74519)
* Removes docs and references for the following `geo_shape` mapping parameters:
  * `tree`
  * `tree_levels`
  * `strategy`
  * `distance_error_pct`
* Updates a related breaking change.

Relates to #70850
2021-06-29 08:52:05 -04:00
Benjamin Trent 07b336f1b0
Add support for range aggregations on histogram mapped fields (#74146)
This adds support for the range aggregation over `histogram` mapped fields.

Decisions made for implementation:

 - Sub-aggregations are not allowed. This is to simplify implementation and follows the prior art set by the `histogram` aggregation
 - Nothing fancy is done with the ranges. No filter translations as we cannot easily do a `range` filter query against histogram fields. This may be an optimization in the future.
 - Ranges check the histogram value ONLY. No interpolation of values is done. If we have better statistics around the histogram this MAY be possible.
2021-06-29 07:24:54 -04:00
Christos Soulios df941367df
Add dimension mapping parameter (#74450)
Added the dimension parameter to the following field types:

    keyword
    ip
    Numeric field types (integer, long, byte, short)

The dimension parameter is of type boolean (default: false) and is used 
to mark that a field is a time series dimension field.

Relates to #74014
2021-06-24 20:16:27 +03:00
Luca Cavanna 5bfdcd2ec7
[DOCS] add missing dynamic runtime option (#74294) 2021-06-21 09:13:21 -04:00
Luca Cavanna 1d88fe639b
Dynamic runtime to not dynamically create objects (#74234)
When we introduced dynamic:runtime (#65489) we decided to have it create objects dynamically under properties, as the runtime section did not (and still does not) support object fields. That proved to be a poor choice, because the runtime section is flat, supports dots in field names, and does not really need objects. Also, these end up causing unnecessary mapping conflicts.

With this commit we adapt dynamic:runtime to not dynamically create objects.

Closes #70268
2021-06-18 14:12:43 +02:00
James Rodewig 8a899419bc [DOCS] Change `multi field` to `multi-field` 2021-06-15 11:40:03 -04:00
Daisuke Harada fa61bf814e
Update runtime.asciidoc (#73802)
it looks typo in a few numbers there.
2021-06-07 09:36:05 -04:00
Adam Locke e2c470abed
[DOCS] Retrieve values from flattened fields w/ runtime fields (#73630)
* [DOCS] Add retriving from flattened fields

* Clarify sub-field syntax

* Moving sub-field retrieval to flattened field docs

* Remove full example and de-emphasize runtime fields

* Remove extraneous sample tag
2021-06-03 11:52:53 -04:00
Adam Locke 0aa0171ce1
[DOCS] Create a new page for grok content in scripting docs (#73118)
* [DOCS] Moving grok to its own scripting page

* Adding examples

* Updating cross link for grok page

* Adds same runtime field in a search request for #73262

* Clarify titles and shift navigation

* Incorporating review feedback

* Updating cross-link to Painless
2021-05-27 15:18:34 -04:00
Adam Locke 89ed0c8e29
[DOCS] Expand information on using a runtime field without a script (#73219)
* [DOCS] Expand information on when to use a runtime field without a script

* Reworking information based on review feedback

* Clarify case where doc_values are disabled

* A few minor changes from review feedback
2021-05-25 15:09:31 -04:00
James Rodewig 8ec893a425
[DOCS] Change field alias anchor (#73043) 2021-05-13 09:32:36 -04:00
James Rodewig 8dddca77aa
[DOCS] Remove and redirect frozen index overview content (#72990)
Changes:

* Removes and adds redirects for the frozen indices [overview][0], [best
  practices][1], [search][2], and [monitoring][3] pages.
* Removes glossary terms related to frozen indices.
* Updates several xrefs to point to the freeze index API docs.

Relates to elastic/elasticsearch#72946 and elastic/elasticsearch#70192.

[0]: https://www.elastic.co/guide/en/elasticsearch/reference/7.12/frozen-indices.html
[1]: https://www.elastic.co/guide/en/elasticsearch/reference/master/best_practices.html
[2]: https://www.elastic.co/guide/en/elasticsearch/reference/master/searching_a_frozen_index.html
[3]: https://www.elastic.co/guide/en/elasticsearch/reference/master/monitoring_frozen_indices.html
2021-05-12 12:54:20 -04:00
Dominic Page dc43d05816
docs amendment match-only-text has limited support for aggs (#72985) 2021-05-12 16:48:57 +02:00
Christoph Büscher f34c9a8a40
Enhance error message for copy-to (#72820)
We currently don't support `copy_to` for fields that take the form of objects
(e.g. `date_range` or certain kinds of `geo_point` variants). The current
problem with objects is that when DocumentParser parses anything other than
single values, it potentially advances the underlying parser past the value that
we would need to stay on for parsing the value again. While we might want to
support this in the future, for now this PR enhances the otherwise confusing
MapperParsingException with something more helpful and adds a short note in the
documentation about this restriction.

Closes #49344
2021-05-11 13:27:45 +02:00
James Rodewig bbfa090a19
[DOCS] Fix bulk API xref (#72685) 2021-05-04 11:07:19 -04:00
Ignacio Vera 4fff3788f3
Disallow creating geo_shape mappings with deprecated parameters (#70850)
With the introduction of BKD-based geo shape indexing in #32039, the prefix tree indexing method has 
been deprecated. From 8.0.0, it will not be allowed to create new mappings using deprecated parameters.
2021-04-30 11:08:58 +02:00
Adrien Grand 83113ec8d3
Add `match_only_text`, a space-efficient variant of `text`. (#66172)
This adds a new `match_only_text` field, which indexes the same data as a `text`
field that has `index_options: docs` and `norms: false` and uses the `_source`
for positional queries like `match_phrase`. Unlike `text`, this field doesn't
support scoring.
2021-04-22 08:41:47 +02:00
Mayya Sharipova f8215e752c
Add doc on rank_feature(s) negative score impact (#71795)
Add a warning about consequences of negative score impact
for documents that don't have values for rank_feature(s)
fields.

Related to #69994
2021-04-20 06:56:05 -04:00
Alan Woodward ee3510b766
Add index-time scripts to geo_point field mapper (#71861)
This commit adds the ability to define an index-time geo_point field
with a script parameter, allowing you to calculate points from other
values within the indexed document.
2021-04-20 10:24:25 +01:00
Luca Cavanna d8057bfe71
Rename on_script_error options to fail or continue (#71841)
As we started thinking about applying on_script_error to runtime fields, to handle script errors at search time, we would like to use the same parameter that was recently introduced for indexed fields. We decided that continue or fail gives a better indication of the behaviour compared to the current ignore or reject which is too specific to indexing documents.

This commit applies such rename.
2021-04-20 09:59:42 +02:00
Mayya Sharipova 853e68dfdf
Add access to dense_vector values (#71313)
Allow direct access to a dense_vector' values in script
through the following functions:

- getVectorValue – returns a vector's value as an array of floats
- getMagnitude – returns a vector's magnitude

Closes #51964
2021-04-19 08:02:05 -04:00
Christoph Büscher 948d02e4d6
Support fetching flattened subfields (#70916)
Currently the `fields` API fetches the root flattened field and returns it in a
structured way in the response. In addition this change makes it possible to
directly query subfields. However, requesting flattened subfields via wildcard
patterns is not possible.

Closes #70605
2021-04-15 12:28:58 +02:00
Alan Woodward 05551dd77b
Add index-time scripts to date field mapper (#71633)
This commit allows you to set 'script' and 'on_script_error' parameters
on date field mappers, meaning that runtime date fields can be made indexed
simply by moving their definitions from the runtime section of the mappings
to the properties section.
2021-04-14 09:18:05 +01:00
Nik Everett 6607a48435
Advise against dates with decimal points (#71578)
We accept dates with a decimal point like `2113413.13241324` and parse
them *somehow*. But there are cases where we'll lose precision on those
dates, see #70085. This advises folks not to use that format. We'll
continue to accept those dates for backwards compatibility but you
should avoid using them.

Co-authored-by: Adrien Grand <jpountz@gmail.com>
2021-04-13 15:11:05 -04:00
Nik Everett b2caf4d230
Convert parent-join example script to runtime field (#71423)
Runtime fields are much more flexible than script_fields because you
can filter and aggregate on them so we hope folks use them! This
converts the example of using a `parent_join` field in a script to a
runtime field so folks get used to seeing them and hopefully using them.

While I was editing this I took the opportunity to replace the script
with a real-ish example. Scripts that just load the field value are nice
and short but I hope no one uses them in real life because they just add
overhead when compared to accessing the field directly. So I made the
script do something.

Relates to #69291
2021-04-13 09:00:18 -04:00
Alan Woodward 67db2538f8
Add index-time scripts to IP field mapper (#71617)
This commit allows you to set 'script' and 'on_script_error' parameters
on IP field mappers, meaning that runtime IP fields can be made indexed
simply by moving their definitions from the runtime section of the mappings
to the properties section.
2021-04-13 13:40:10 +01:00
Nik Everett e4451bda05
Convert date_nanos example script to runtime field (#71351)
Runtime fields are much more flexible than script_fields because you
can filter and aggregate on them so we hope folks use them! This
converts the example of using a `date_nanos` field in a script to a
runtime field so folks get used to seeing them and hopefully using them.

While I was editing this I took the opportunity to replace the script
with a real-ish example. Scripts that just load the field value are nice
and short but I hope no one uses them in real life because they just add
overhead when compared to accessing the field directly. So I made the
script do something.

Relates to #69291

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-04-12 17:22:02 -04:00
Alan Woodward 5e11709693
Add scripts to keyword field mapper (#71555)
This commit adds script and on_script_error parameters to
keyword field mappers, allowing you to define index-time scripts
for keyword fields.
2021-04-12 16:46:02 +01:00
Luca Cavanna 1469e18c98
Add support for script parameter to boolean field mapper (#71454)
Relates to #68984
2021-04-12 10:04:12 +02:00
Julie Tibshirani 3da738e5db
Support fetching _tier field value (#71379)
Now that the `fields` option allows fetching metadata fields, we can support
loading the new `_tier` metadata field.

Relates to #63569 and #68135.
2021-04-08 11:41:52 -07:00
Nhat Nguyen 5c9969250d
Allow specify dynamic templates in bulk request (#69948)
This change allows users to specify dynamic templates in a bulk request.

```
PUT myindex
{
  "mappings": {
    "dynamic_templates": [{
      "time_histograms": {
        "mapping": {
          "type": "histogram",
          "meta": {
            "unit": "s"
          }
        }
      }
    }]
  }
}
```

```
POST myindex/_bulk
{ "index": { "dynamic_templates": { "response_times": "time_histograms" } } }
{ "@timestamp": "2020-08-12", "response_times": { "values": [1, 10], "counts": [5, 1] }}
```

Closes #61939
2021-04-08 12:44:36 -04:00
Adam Locke 343c52c19f
[DOCS] Adding page for indexing runtime fields (#71366)
* [DOCS] Adding page for indexing runtime fields

* Fixing tests.

* Incorporating review feedback to enhance and improve examples.

* Changing note to indicate immutable script when indexing, plus adding on_script_error.
2021-04-07 13:07:39 -04:00
Nik Everett e158bc10b1
Convert `boolean` field example to runtime fields (#71341)
Runtime fields are much more flexible than `script_fields` because you
can filter and aggregate on them so we hope folks use them! This
converts the example of using a `boolean` field in a script to a runtime
field so folks get used to seeing them and hopefully using them.

While I was editing this I took the opportunity to replace the script
with a real-ish example. Scripts that just load the field value are nice
and short but I hope no one uses them in real life because they just add
overhead when compared to accessing the field directly. So I made the
script do *something*.

Relates to #69291
2021-04-06 14:42:44 -04:00
Alan Woodward 98c9a95e12
Add note that scripted fields will reject documents with a source value in their field (#71340) 2021-04-06 14:28:20 +01:00
Adam Locke 14aba7bcff
[DOCS] Expand examples for runtime fields in a search query (#71237)
* Add warning admonition for removing runtime fields.

* Add cross-link to runtime fields.

* Expanding examples for runtime fields in a search request.

* Clarifying language and simplifying response tests.
2021-04-02 15:00:54 -04:00
markharwood 3aee4c1f1f
New queryable "_tier" metadata field (#69288)
New _tier metadata field that supports term, terms, exists and wildcard queries on the first data tier preference stated for an index.

Closes #68135
2021-03-31 15:37:37 +01:00
James Rodewig 693807a6d3
[DOCS] Fix double spaces (#71082) 2021-03-31 09:57:47 -04:00
Alan Woodward 1653f2fe91
Add script parameter to long and double field mappers (#69531)
This commit adds a script parameter to long and double fields that makes
it possible to calculate a value for these fields at index time. It uses the same
script context as the equivalent runtime fields, and allows for multiple index-time
scripted fields to cross-refer while still checking for indirection loops.
2021-03-31 11:14:11 +01:00
markharwood 2f9c7318c2
Search - make wildcard field use constant scoring queries for wildcard queries and caching fix (#70452)
* Make wildcard field use constant scoring queries for wildcard queries. Add a note about ignoring rewrite parameters on wildcard queries.

Also fixes caching issue where case sensitive and case insensitive results were cached as the same

Closes #69604
2021-03-30 10:37:39 +01:00
James Rodewig d8a78b9d26
[DOCs] Add tip for `index_options` parameter (#70450) (#70498)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: yudidi <972656027@qq.com>
2021-03-17 10:43:41 -04:00
Jim Ferenczi 701abc6bea
Change default format for date_nanos field (#70463)
This commit updates the default format of date_nanos field
on existing and new indices to use `strict_date_optional_time_nanos` instead of
`strict_date_optional_time`.
Using `strict_date_optional_time` as the default format for date_nanos doesn't
make sense because it accepts and parses dates with nanosecond precision,
but when it formats it drops the nanoseconds.
The change should be transparent for users, these formats accept the same input.

Relates #69192
Closes #67063
2021-03-17 11:40:32 +01:00
James Rodewig 5c75d004fa
[DOCS] Replace `put` with `create or update` in API names (#70330)
Co-authored-by: debadair <debadair@elastic.co>
Co-authored-by: Lisa Cawley <lcawley@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2021-03-15 14:49:44 -04:00
Mayya Sharipova 1de0b616eb
Add positive_score_impact to rank_features type (#69994)
rank_features field type misses positive_score_impact parameter
that rank_feature type has. This adds this parameter.

Closes #68619
2021-03-10 14:55:54 -05:00
Julie Tibshirani 796284a190
Move flattened field to core. (#68780)
This field mapper only lived in its own module so it could be licensed as x-pack
basic. Now it can be moved to core, which matches its status as a core type.
2021-03-08 16:56:16 -08:00
James Rodewig f1e911d13d
[DOCS] Add guidance for mapping unstructured content (#69079) 2021-03-08 12:31:42 -05:00
Mayya Sharipova aab3f3021a
Remove size of dense_vector (#70024)
Remove not completely correct statement about the size of dense_vectors

We do store a dense_vector as binary doc value  with size `4*dims+4`.
But this is size before compression. As compressed size depends on
data itself, it is better to remove completely any statement
about the size.
2021-03-08 07:49:06 -05:00
Christoph Büscher 6011d99b14
[DOCS] Improve tip about updating search_analyzer (#69621)
The tip about updating a `search_analyzer` currently does not mention that most
of the time (when the current analyzer is not "default"), user need to repeat
the currently set "analyzer" parameter in the field definition. Adding this as a
short note.
2021-03-03 16:31:29 +01:00
Nik Everett fe457f156d
Docs: Call out that you can't update analyzer (#69889)
You can't update the `analyzer` parameter in the PUT mappings API even if
the index is closed. This adds a TIP to call that out. And adds a TIP
for `search_quote_analyzer` which you *can* update.
2021-03-03 10:28:55 -05:00
Adam Locke 1ee4c50217
[DOCS] Remove beta admonition for runtime fields. (#69550)
* [DOCS] Remove beta admonition for runtime fields.

* Remove other beta admonition from Painless guide.
2021-02-24 11:35:11 -05:00
Adam Locke 2362549818
[DOCS] Adding grok support for runtime fields. (#69308)
* [DOCS] Adding grok support for runtime fields.

* Update response.

* Adding testresponse replacements.

* Update runtime field context and add dissect.

* Fixing backslash in the response.

* Fixing testresponse.

* Incorporating review feedback.

* Updates emit and adds cross link from ES runtime fields page.
2021-02-23 12:47:11 -05:00
James Rodewig 9af74ec561
[DOCS] Remove added admons (#69452) 2021-02-23 10:35:21 -05:00
James Rodewig a85f9cade8
[DOCS] Use consistent @timestamp field name (#69435) (#69448)
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>

Co-authored-by: Koji Kawamura <ijokarumawak@users.noreply.github.com>
2021-02-23 10:20:17 -05:00
James Rodewig b7aaaad20a
[DOCS] Improve docs for `geo_shape` field type's `circle` type (#69285) 2021-02-22 10:24:24 -05:00
S. J. Lee 42bca5bfc2
[DOCS] Relocate scriptless runtime document (#68916)
* Reallocate runtime document

Reallocate document `runtime-fields-scriptless` from `runtime-search-request` to `runtime-mapping-fields`

* Move runtime without script section

Move runtime without script section to under the dynamic runtime mapping section

* Fix snippet formatting and remove discrete heading.

* Update test snippet.

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-02-17 12:38:06 -05:00
James Rodewig 9b88ae92e6
[DOCS] Fix typos for duplicate words (#69125) 2021-02-17 10:34:20 -05:00
Luca Cavanna bd3467a305
Add the ability to remove a runtime field (#68992)
Currently, existing runtime fields can be updated, but they cannot be removed. That allows to correct potential mistakes, but once a runtime field is added to the index mappings, it is not possible to remove it.

With this commit we introduce the ability to remove an existing runtime field by providing a null value for it through the put mapping API. If a field with such name does not exist, such specific instruction will have no effect on other existing runtime fields.

Note that the removal of runtime fields makes the recently introduced assertRefreshItNotNeeded assertion trip, because when each local node merges mappings back in, the runtime fields that were previously removed by the master node, get added back again locally. This is only a problem for the assertion that verifies that the removed refresh operation is never needed. We worked around this by tweaking the assertion to ignore runtime fields completely, for simplicity, by assertion on the serialized merged mappings and incoming mappings without the corresponding runtime section.

Co-authored-by: Adam Locke <adam.locke@elastic.co>
2021-02-16 21:00:16 +01:00
James Rodewig bc0d37dffa
[DOCS] Document what makes a geo-point malformed (#59045) (#68556)
Co-authored-by: Ben Moskovitz <ben@mosk.nz>
2021-02-04 17:01:59 -05:00
Adam Locke c5c5d27fa7
[DOCS] Add beta admonition for runtime fields. (#67847)
* Add beta admonition for runtime fields.

* Add beta admonition to search your data page.
2021-01-21 17:10:37 -05:00
Adam Locke 1fb257545f
[DOCS] Add runtime fields to Painless Guide (#67781)
* Adding runtime fields page for Painless context.

* Adds beta admonition to runtime fields and Painless docs.

* Fixing test errors and improving content sections.

* Adding refresh to fix test cases.

* Simplifying the ingest request to include refresh.

* Removing beta (will add in another PR) and updating examples.
2021-01-21 15:52:01 -05:00