Commit Graph

15 Commits

Author SHA1 Message Date
Nik Everett dde7d418a8
Add matching pattern to error in fields (#76903)
This adds the pattern into the error message returned when trying to
fetch fields. So this:
```
POST _search {
  "fields": [ { "field": "*", "format": "date_time" } ]
}
```

Will return an error message like
```
error fetching [foo] which matches [*]: Field [foo] of type [keyword] doesn't support formats
```
2021-09-01 12:15:54 -04:00
Rene Groeschke 35ec6f348c
Introduce simple public yaml-rest-test plugin (#76554)
This introduces a basic public yaml rest test plugin that is supposed to be used by external 
elasticsearch plugin authors. This is driven by #76215

- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins

Co-authored-by: Mark Vieira <portugee@gmail.com>
2021-08-31 08:45:52 +02:00
Rene Groeschke b323726ebd
Resolve system properties in build scripts via provider factory (#76199)
This allows tracking system properties used in the build configuration and brings us
one step closer to be gradle configuration cache compliant.
2021-08-09 09:39:30 +02:00
Alan Woodward 8806c1a6d9
Be stricter about which source paths we load values from (#74348)
XContentMapValues.extractRawValues() contains a hack whereby we
ignore extra 'dummy' dotted path values appended to a source path - ie,
loading data from foo.bar.dummy will also load from foo.bar. This was
used to implement source loading for multifields, which do not have a name
that corresponds to an existing path in source data. This was safe in general
because you can't have a field that is a dot prefix of another field in index
mappings.

With runtime fields, however, this changes. You can define scriptless runtime
fields for both foo.bar and foo.bar.dummy, and it is important that the
latter does not pick up values from the former path.

Source loading for multifield highlighting has already moved away from using
this hack by loading everything through value fetchers. This commit removes
the hack entirely, and updates term vector loading to use a value fetcher
as well
2021-06-23 10:48:42 +01:00
Nik Everett ce245705b6 Fix test assertion for runtime fields
Runtime fields don't have an index so they can't enable the optimization
that we're testing for.
2021-06-22 09:07:40 -04:00
Rory Hunter a5d2251064
Order imports when reformatting (#74059)
Change the formatter config to sort / order imports, and reformat the
codebase. We already had a config file for Eclipse users, so Spotless now
uses that.

The "Eclipse Code Formatter" plugin ought to be able to use this file as
well for import ordering, but in my experiments the results were poor.
Instead, use IntelliJ's `.editorconfig` support to configure import
ordering.

I've also added a config file for the formatter plugin.

Other changes:
   * I've quietly enabled the `toggleOnOff` option for Spotless. It was
     already possible to disable formatting for sections using the markers
     for docs snippets, so enabling this option just accepts this reality
     and makes it possible via `formatter:off` and `formatter:on` without
     the restrictions around line length. It should still only be used as
     a very last resort and with good reason.
   * I've removed mention of the `paddedCell` option from the contributing
     guide, since I haven't had to use that option for a very long time. I
     moved the docs to the spotless config.
2021-06-16 09:22:22 +01:00
Ryan Ernst 68817d7ca2
Rename o.e.common in libs/core to o.e.core (#73909)
When libs/core was created, several classes were moved from server's
o.e.common package, but they were not moved to a new package. Split
packages need to go away long term, so that Elasticsearch can even think
about modularization. This commit moves all the classes under o.e.common
in core to o.e.core.

relates #73784
2021-06-08 09:53:28 -07:00
Przemyslaw Gomulka 4bdd00d452
[Rest Api Compatibility] Typed endpoint for bulk api (#73571)
retrofits typed endpoint and type in request parsing
the original types removal commit
#46983

relates #51816
2021-06-07 19:36:31 +02:00
Nik Everett 4b5aebe8b0
Add setting to disable aggs optimization (#73620)
Sometimes our fancy "run this agg as a Query" optimizations end up
slower than running the aggregation in the old way. We know that and use
heuristics to dissable the optimization in that case. But it turns out
that the process of running the heuristics itself can be slow, depending
on the query. Worse, changing the heuristics requires an upgrade, which
means waiting. If the heurisics make a terrible choice folks need a
quick way out. This adds such a way: a cluster level setting that
contains a list of queries that are considered "too expensive" to try
and optimize. If the top level query contains any of those queries we'll
disable the "run as Query" optimization.

The default for this settings is wildcard and term-in-set queries, which
is fairly conservative. There are certainly wildcard and term-in-set
queries that the optimization works well with, but there are other queries
of that type that it works very badly with. So we're being careful.

Better, you can modify this setting in a running cluster to disable the
optimization if we find a new type of query that doesn't work well.

Closes #73426
2021-06-02 09:12:54 -04:00
Nik Everett 39fee5e908
Fix composite early termination on sorted (#72101)
I broke composite early termination when reworking how aggregations'
contact for `getLeafCollector` around early termination in #70320. We
didn't see it in our tests because we weren't properly emulating the
aggregation collection stage. This fixes early termination by adhering
to the new contract and adds more tests.

Closes #72078

Co-authored-by: Benjamin Trent <4357155+benwtrent@users.noreply.github.com>
2021-04-22 14:32:26 -04:00
Nik Everett 2d6f8d1e0c
Add integration tests for filters (#69439)
Revamps the integration tests for the `filter` agg to be more clear and
builds integration tests for the `fitlers` agg. Both of these
integration tests are fairly basic but they do assert that the aggs
work.
2021-04-14 16:54:23 -04:00
Lyudmila Fokina 3b0b7941ae
Warn users if security is implicitly disabled (#70114)
* Warn users if security is implicitly disabled

Elasticsearch has security features implicitly disabled by default for
Basic and Trial licenses, unless explicitly set in the configuration
file.
This may be good for onboarding, but it also lead to unintended insecure
 clusters.
 This change introduces clear warnings when security features are
 implicitly disabled.
 - a warning header in each REST response if security is implicitly
 disabled;
 - a log message during cluster boot.
2021-04-13 18:33:41 +02:00
Mark Vieira 6339691fe3
Consolidate REST API specifications and publish under Apache 2.0 license (#70036) 2021-03-26 16:20:14 -07:00
Nik Everett 10e2f90560
Speed up aggs with sub-aggregations (#69806)
This allows many of the optimizations added in #63643 and #68871 to run
on aggregations with sub-aggregations. This should:
* Speed up `terms` aggregations on fields with less than 1000 values that
  also have sub-aggregations. Locally I see 2 second searches run in 1.2
  seconds.
* Applies that same speedup to `range` and `date_histogram` aggregations but
  it feels less impressive because the point range queries are a little
  slower to get up and go.
* Massively speed up `filters` aggregations with sub-aggregations that
  don't have a `parent` aggregation or collect "other" buckets. Also
  save a ton of memory while collecting them.
2021-03-03 18:04:47 -05:00
Luca Cavanna 84f7cbd91e
Move runtime fields qa tests to x-pack/qa (#69680)
As a follow-up of moving runtime fields to server, we would like to remove the runtime field xpack plugin portions that are left.

This commit moves the runtime-fields/qa module to x-pack/qa/runtime-fields. Runtime fields are no longer part of x-pack, but their qa tests involve testing with security, async search, which require x-pack, hence their qa tests are left under x-pack.
2021-03-01 20:19:12 +01:00