elasticsearch/docs/reference/mapping
Benjamin Trent 5add44d7d1
Adds new `bit` element_type for dense_vectors (#110059)
This commit adds `bit` vector support by adding `element_type: bit` for
vectors. This new element type works for indexed and non-indexed
vectors. Additionally, it works with `hnsw` and `flat` index types. No
quantization based codec works with this element type, this is
consistent with `byte` vectors.

`bit` vectors accept up to `32768` dimensions in size and expect vectors
that are being indexed to be encoded either as a hexidecimal string or a
`byte[]` array where each element of the `byte` array represents `8`
bits of the vector.

`bit` vectors support script usage and regular query usage. When
indexed, all comparisons done are `xor` and `popcount` summations (aka,
hamming distance), and the scores are transformed and normalized given
the vector dimensions. Note, indexed bit vectors require `l2_norm` to be
the similarity.

For scripts, `l1norm` is the same as `hamming` distance and `l2norm` is
`sqrt(l1norm)`. `dotProduct` and `cosineSimilarity` are not supported.

Note, the dimensions expected by this element_type are always to be
divisible by `8`, and the `byte[]` vectors provided for index must be
have size `dim/8` size, where each byte element represents `8` bits of
the vectors.

closes: https://github.com/elastic/elasticsearch/issues/48322
2024-06-27 04:48:41 +10:00
..
dynamic
fields Enable fallback synthetic source for token_count (#109044) 2024-05-27 10:22:59 -07:00
params Add note about optional times and epochs (#105786) 2024-03-05 08:44:03 -05:00
types Adds new `bit` element_type for dense_vectors (#110059) 2024-06-27 04:48:41 +10:00
dynamic-mapping.asciidoc
explicit-mapping.asciidoc [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
fields.asciidoc
mapping-settings-limit.asciidoc [DOCS] Refactor book-scoped variables in `docs/reference/index.asciidoc` (#107413) 2024-04-17 14:37:07 +02:00
params.asciidoc
removal_of_types.asciidoc
runtime.asciidoc
types.asciidoc [DOCS] Adds docs to semantic text (#108311) 2024-05-31 16:56:07 +02:00