This adds the pattern into the error message returned when trying to
fetch fields. So this:
```
POST _search {
"fields": [ { "field": "*", "format": "date_time" } ]
}
```
Will return an error message like
```
error fetching [foo] which matches [*]: Field [foo] of type [keyword] doesn't support formats
```
This introduces a basic public yaml rest test plugin that is supposed to be used by external
elasticsearch plugin authors. This is driven by #76215
- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins
Co-authored-by: Mark Vieira <portugee@gmail.com>
XContentMapValues.extractRawValues() contains a hack whereby we
ignore extra 'dummy' dotted path values appended to a source path - ie,
loading data from foo.bar.dummy will also load from foo.bar. This was
used to implement source loading for multifields, which do not have a name
that corresponds to an existing path in source data. This was safe in general
because you can't have a field that is a dot prefix of another field in index
mappings.
With runtime fields, however, this changes. You can define scriptless runtime
fields for both foo.bar and foo.bar.dummy, and it is important that the
latter does not pick up values from the former path.
Source loading for multifield highlighting has already moved away from using
this hack by loading everything through value fetchers. This commit removes
the hack entirely, and updates term vector loading to use a value fetcher
as well
Change the formatter config to sort / order imports, and reformat the
codebase. We already had a config file for Eclipse users, so Spotless now
uses that.
The "Eclipse Code Formatter" plugin ought to be able to use this file as
well for import ordering, but in my experiments the results were poor.
Instead, use IntelliJ's `.editorconfig` support to configure import
ordering.
I've also added a config file for the formatter plugin.
Other changes:
* I've quietly enabled the `toggleOnOff` option for Spotless. It was
already possible to disable formatting for sections using the markers
for docs snippets, so enabling this option just accepts this reality
and makes it possible via `formatter:off` and `formatter:on` without
the restrictions around line length. It should still only be used as
a very last resort and with good reason.
* I've removed mention of the `paddedCell` option from the contributing
guide, since I haven't had to use that option for a very long time. I
moved the docs to the spotless config.
When libs/core was created, several classes were moved from server's
o.e.common package, but they were not moved to a new package. Split
packages need to go away long term, so that Elasticsearch can even think
about modularization. This commit moves all the classes under o.e.common
in core to o.e.core.
relates #73784
Sometimes our fancy "run this agg as a Query" optimizations end up
slower than running the aggregation in the old way. We know that and use
heuristics to dissable the optimization in that case. But it turns out
that the process of running the heuristics itself can be slow, depending
on the query. Worse, changing the heuristics requires an upgrade, which
means waiting. If the heurisics make a terrible choice folks need a
quick way out. This adds such a way: a cluster level setting that
contains a list of queries that are considered "too expensive" to try
and optimize. If the top level query contains any of those queries we'll
disable the "run as Query" optimization.
The default for this settings is wildcard and term-in-set queries, which
is fairly conservative. There are certainly wildcard and term-in-set
queries that the optimization works well with, but there are other queries
of that type that it works very badly with. So we're being careful.
Better, you can modify this setting in a running cluster to disable the
optimization if we find a new type of query that doesn't work well.
Closes#73426
I broke composite early termination when reworking how aggregations'
contact for `getLeafCollector` around early termination in #70320. We
didn't see it in our tests because we weren't properly emulating the
aggregation collection stage. This fixes early termination by adhering
to the new contract and adds more tests.
Closes#72078
Co-authored-by: Benjamin Trent <4357155+benwtrent@users.noreply.github.com>
Revamps the integration tests for the `filter` agg to be more clear and
builds integration tests for the `fitlers` agg. Both of these
integration tests are fairly basic but they do assert that the aggs
work.
* Warn users if security is implicitly disabled
Elasticsearch has security features implicitly disabled by default for
Basic and Trial licenses, unless explicitly set in the configuration
file.
This may be good for onboarding, but it also lead to unintended insecure
clusters.
This change introduces clear warnings when security features are
implicitly disabled.
- a warning header in each REST response if security is implicitly
disabled;
- a log message during cluster boot.
This allows many of the optimizations added in #63643 and #68871 to run
on aggregations with sub-aggregations. This should:
* Speed up `terms` aggregations on fields with less than 1000 values that
also have sub-aggregations. Locally I see 2 second searches run in 1.2
seconds.
* Applies that same speedup to `range` and `date_histogram` aggregations but
it feels less impressive because the point range queries are a little
slower to get up and go.
* Massively speed up `filters` aggregations with sub-aggregations that
don't have a `parent` aggregation or collect "other" buckets. Also
save a ton of memory while collecting them.
As a follow-up of moving runtime fields to server, we would like to remove the runtime field xpack plugin portions that are left.
This commit moves the runtime-fields/qa module to x-pack/qa/runtime-fields. Runtime fields are no longer part of x-pack, but their qa tests involve testing with security, async search, which require x-pack, hence their qa tests are left under x-pack.
Nik Everett
Rene Groeschke
Alan Woodward
Rory Hunter
Ryan Ernst
Przemyslaw Gomulka
Lyudmila Fokina
Mark Vieira
Luca Cavanna