elasticsearch/docs/reference/mapping.asciidoc

93 lines
3.4 KiB
Plaintext
Raw Normal View History

[[mapping]]
= Mapping
[partintro]
--
Mapping is the process of defining how a document, and the fields it contains,
are stored and indexed.
Each document is a collection of fields, which each have their own
<<mapping-types,data type>>. When mapping your data, you create a mapping
definition, which contains a list of fields that are pertinent to the document.
A mapping definition also includes <<mapping-fields,metadata fields>>, like the
`_source` field, which customize how a document's associated metadata is
handled.
Use _dynamic mapping_ and _explicit mapping_ to define your data. Each method
provides different benefits based on where you are in your data journey. For
example, explicitly map fields where you don't want to use the defaults, or to
gain greater control over which fields are created. You can then allow {es} to
add other fields dynamically.
NOTE: Before 7.0.0, the mapping definition included a type name.
{es} 7.0.0 and later no longer accept a _default_ mapping. See <<removal-of-types>>.
.Experiment with mapping options
****
<<runtime-search-request,Define runtime fields in a search request>> to
experiment with different mapping options, and also fix mistakes in your index
mapping values by overriding values in the mapping during the search request.
****
[discrete]
[[mapping-dynamic]]
== Dynamic mapping
<<dynamic-field-mapping,Dynamic mapping>> allows you to experiment with
and explore data when youre just getting started. {es} adds new fields
automatically, just by indexing a document. You can add fields to the top-level
mapping, and to inner <<object,`object`>> and <<nested,`nested`>> fields.
Use <<dynamic-templates,dynamic templates>> to define custom mappings that are
applied to dynamically added fields based on the matching condition.
[discrete]
[[mapping-explicit]]
== Explicit mapping
<<explicit-mapping,Explicit mapping>> allows you to precisely choose how to
define the mapping definition, such as:
* Which string fields should be treated as full text fields.
* Which fields contain numbers, dates, or geolocations.
* The <<mapping-date-format,format>> of date values.
* Custom rules to control the mapping for
<<dynamic-mapping,dynamically added fields>>.
Use <<runtime-mapping-fields,runtime fields>> to make schema changes without
reindexing. You can use runtime fields in conjunction with indexed fields to
balance resource usage and performance. Your index will be smaller, but with
slower search performance.
[discrete]
[DOCS] Add docs for runtime fields (#62653) * First steps in docs for runtime fields. * Adding new page for runtime fields. * Adding page for runtime fields. * Adding more to the runtime fields topic. * Adding parameters and retrieval options for runtime fields. * Adding TESTSETUP for index creation. * Incorporating review feedback. * Incorporating reviewer feedback. * Adding examples for runtime fields. * Adding more context and simplifying the example. * Changing timestamp to @timestamp throughout. * Removing duplicate @timestamp field. * Expanding example to hopefully fix CI builds. * Adding skip test for result. * Adding missing callout. * Adding TESTRESPONSEs, which are currently broken. * Fixing TESTRESPONSEs. * Incorporating review feedback. * Several clarifications, better test cases, and other changes. * Adding missing callout in example. * Adding substitutions to TESTRESPONSE for shorter results shown. * Shuffling some information and adding link to script-fields. * Fixing typo. * Updates for API redesign -- will break builds. * Updating examples and including info about overriding fields. * Updating examples. * Adding info for using runtime fields in the search request. * Adding that queries against runtime fields are expensive. * Incorporating feedback from reviewers. * Minor changes from reviews. * Adding alias for test case. * Adding aliases to PUT example. * Fixing test cases, for real this time. * Updating use cases and introducing overlay throughout. * Edits, adding 'shadowing', and explaining shadowing better. * Streamlining tests and other changes. * Fix formatting in example for test. * Apply suggestions from code review * Incorporating reviewer feedback 7 Dec * Shifting structure of mapping page to fix cross links. * Revisions for shadowing, overview, and other sections. * Removing dot notation section and incorporating review changes. * Adding updated example for shadowing. * Streamlining shadowing example and TESTRESPONSEs.
2020-12-10 06:54:58 +08:00
[[mapping-limit-settings]]
== Settings to prevent mapping explosion
Defining too many fields in an index can lead to a mapping explosion, which can
cause out of memory errors and difficult situations to recover from.
Consider a situation where every new document inserted
introduces new fields, such as with <<dynamic-mapping,dynamic mapping>>.
Each new field is added to the index mapping, which can become a
problem as the mapping grows.
[DOCS] Add docs for runtime fields (#62653) * First steps in docs for runtime fields. * Adding new page for runtime fields. * Adding page for runtime fields. * Adding more to the runtime fields topic. * Adding parameters and retrieval options for runtime fields. * Adding TESTSETUP for index creation. * Incorporating review feedback. * Incorporating reviewer feedback. * Adding examples for runtime fields. * Adding more context and simplifying the example. * Changing timestamp to @timestamp throughout. * Removing duplicate @timestamp field. * Expanding example to hopefully fix CI builds. * Adding skip test for result. * Adding missing callout. * Adding TESTRESPONSEs, which are currently broken. * Fixing TESTRESPONSEs. * Incorporating review feedback. * Several clarifications, better test cases, and other changes. * Adding missing callout in example. * Adding substitutions to TESTRESPONSE for shorter results shown. * Shuffling some information and adding link to script-fields. * Fixing typo. * Updates for API redesign -- will break builds. * Updating examples and including info about overriding fields. * Updating examples. * Adding info for using runtime fields in the search request. * Adding that queries against runtime fields are expensive. * Incorporating feedback from reviewers. * Minor changes from reviews. * Adding alias for test case. * Adding aliases to PUT example. * Fixing test cases, for real this time. * Updating use cases and introducing overlay throughout. * Edits, adding 'shadowing', and explaining shadowing better. * Streamlining tests and other changes. * Fix formatting in example for test. * Apply suggestions from code review * Incorporating reviewer feedback 7 Dec * Shifting structure of mapping page to fix cross links. * Revisions for shadowing, overview, and other sections. * Removing dot notation section and incorporating review changes. * Adding updated example for shadowing. * Streamlining shadowing example and TESTRESPONSEs.
2020-12-10 06:54:58 +08:00
Use the <<mapping-settings-limit,mapping limit settings>> to limit the number
of field mappings (created manually or dynamically) and prevent documents from
causing a mapping explosion.
--
include::mapping/dynamic-mapping.asciidoc[]
[DOCS] Add docs for runtime fields (#62653) * First steps in docs for runtime fields. * Adding new page for runtime fields. * Adding page for runtime fields. * Adding more to the runtime fields topic. * Adding parameters and retrieval options for runtime fields. * Adding TESTSETUP for index creation. * Incorporating review feedback. * Incorporating reviewer feedback. * Adding examples for runtime fields. * Adding more context and simplifying the example. * Changing timestamp to @timestamp throughout. * Removing duplicate @timestamp field. * Expanding example to hopefully fix CI builds. * Adding skip test for result. * Adding missing callout. * Adding TESTRESPONSEs, which are currently broken. * Fixing TESTRESPONSEs. * Incorporating review feedback. * Several clarifications, better test cases, and other changes. * Adding missing callout in example. * Adding substitutions to TESTRESPONSE for shorter results shown. * Shuffling some information and adding link to script-fields. * Fixing typo. * Updates for API redesign -- will break builds. * Updating examples and including info about overriding fields. * Updating examples. * Adding info for using runtime fields in the search request. * Adding that queries against runtime fields are expensive. * Incorporating feedback from reviewers. * Minor changes from reviews. * Adding alias for test case. * Adding aliases to PUT example. * Fixing test cases, for real this time. * Updating use cases and introducing overlay throughout. * Edits, adding 'shadowing', and explaining shadowing better. * Streamlining tests and other changes. * Fix formatting in example for test. * Apply suggestions from code review * Incorporating reviewer feedback 7 Dec * Shifting structure of mapping page to fix cross links. * Revisions for shadowing, overview, and other sections. * Removing dot notation section and incorporating review changes. * Adding updated example for shadowing. * Streamlining shadowing example and TESTRESPONSEs.
2020-12-10 06:54:58 +08:00
include::mapping/explicit-mapping.asciidoc[]
[DOCS] Add docs for runtime fields (#62653) * First steps in docs for runtime fields. * Adding new page for runtime fields. * Adding page for runtime fields. * Adding more to the runtime fields topic. * Adding parameters and retrieval options for runtime fields. * Adding TESTSETUP for index creation. * Incorporating review feedback. * Incorporating reviewer feedback. * Adding examples for runtime fields. * Adding more context and simplifying the example. * Changing timestamp to @timestamp throughout. * Removing duplicate @timestamp field. * Expanding example to hopefully fix CI builds. * Adding skip test for result. * Adding missing callout. * Adding TESTRESPONSEs, which are currently broken. * Fixing TESTRESPONSEs. * Incorporating review feedback. * Several clarifications, better test cases, and other changes. * Adding missing callout in example. * Adding substitutions to TESTRESPONSE for shorter results shown. * Shuffling some information and adding link to script-fields. * Fixing typo. * Updates for API redesign -- will break builds. * Updating examples and including info about overriding fields. * Updating examples. * Adding info for using runtime fields in the search request. * Adding that queries against runtime fields are expensive. * Incorporating feedback from reviewers. * Minor changes from reviews. * Adding alias for test case. * Adding aliases to PUT example. * Fixing test cases, for real this time. * Updating use cases and introducing overlay throughout. * Edits, adding 'shadowing', and explaining shadowing better. * Streamlining tests and other changes. * Fix formatting in example for test. * Apply suggestions from code review * Incorporating reviewer feedback 7 Dec * Shifting structure of mapping page to fix cross links. * Revisions for shadowing, overview, and other sections. * Removing dot notation section and incorporating review changes. * Adding updated example for shadowing. * Streamlining shadowing example and TESTRESPONSEs.
2020-12-10 06:54:58 +08:00
include::mapping/runtime.asciidoc[]
include::mapping/types.asciidoc[]
include::mapping/fields.asciidoc[]
include::mapping/params.asciidoc[]
include::mapping/mapping-settings-limit.asciidoc[]
include::mapping/removal_of_types.asciidoc[]