Commit Graph

3152 Commits

Author SHA1 Message Date
Armin Braun 3945ee75d0
Use assertAcked more (#101201)
Just found that we have a lot of inconsistency and needless verbosity
here in tests. We can just use `assertAcked` in a couple spots
to save `.get`, `.actionGet` etc., especially with the signature
change I added here.
2023-10-23 19:48:10 +02:00
David Turner 1eda6ac74b
Extract ESIntegTestCase#prepareSearch (#101175)
Relates #101172
2023-10-20 06:18:58 -04:00
Ryan Ernst 8a1db8c6c3
Move index version constants to IndexVersions (#101094)
Similar to the TransportVersions holder class, IndexVersions is the new
place to contain all constants for IndexVersion. This commit moves all
existing constants to the new class. It is purely mechanical.
2023-10-19 20:44:51 -04:00
Rene Groeschke 1773d4b5d1
Fix Gradle deprecation warning in examples project (#100988) 2023-10-17 12:00:47 -04:00
Rene Groeschke 8a6df32de6
Update gradle wrapper to 8.4 (#99856)
* Remove deprecated forConfigurationTime usage
2023-10-11 17:02:27 +02:00
Armin Braun b7eafce32c
Make some practically static methods static (#97565)
Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
2023-10-06 23:37:07 +02:00
Panagiotis Bailis d32dc6dd8b
Refactoring & adding tests to rescorer in the example plugin, to account for non strictly increasing doc ids in the results. (#100189)
* Updating ExampleRescoreBuilder.ExampleRescorer#rescore to use the ordered sortedByDocId array when iterating through the leaves to fetch the NumericFieldData used or reranking. 
* Adding test to account for the above behavior, ensuring that the docs will be properly rescored when the original results are in descending _id order.

Closes #39665
2023-10-04 12:28:18 +03:00
Yang Wang 5628392fa5
Differentiate stats for the same blobstore operation with purposes (#99615)
Today blobstore stats are collected against each HTTP operation, e.g.
Get, List. This is not granular enough because the same HTTP operration
can be performed for different purposes, e.g. cluster state, indices or
translog. This PR adds a new Purpose enum to provide further breakdown
for the same HTTP operation. 

Relates: ES-6800
2023-10-02 06:37:08 -04:00
Rene Groeschke b23e000c30
Update gradle wrapper to 8.3 (#97838)
Gradle now fully supports compiling, testing and running on Java 20. 
Among other general performance improvements this release introduces --test-dry-run command line option that allows checking if tests are filtered or not by gradle.
Required updating nebula ospackage plugin as setuid was broken in gradle 8.3.
2023-09-27 08:48:21 +02:00
Tim Vernum d411acecbc
Suppress this-escape warning for JDK21 (#99848)
Adds @SuppressWarnings("this-escape") to all necessary places to that
Elasticsearch can compile with -Werror on JDK21

No investigation has been done to determine whether any of the cases
are a potential source of errors - we have simply suppressed all
existing occurrences.

Resolves: #99845
2023-09-25 18:30:21 +10:00
Armin Braun 6fc05a6bb1
Remove redundant fields on TextFieldMapper and KeywordFieldMapper (#99666)
These fields were also on the mapped field types, no need to waste bytes
on them.
2023-09-19 10:43:42 -04:00
Alan Woodward 4e1fb3fca5
Automatically disable `ignore_malformed` on datastream `@timestamp` fields (#99346)
Data-stream mappings require a @timestamp field to be present and configured
as a date with a specific set of parameters. The index-wide setting of
ignore_malformed can cause problems here if it is set to true, because it needs
to be false for the @timestamp field.

This commit detects if a set of mappings is configured for a datastream by checking
for the presence of a DataStreamTimestampFieldMapper metadata field, and passes
that information on during Mapper construction as part of the MapperBuilderContext.
DateFieldMapper.Builder now checks to see if it is specifically for a data stream timestamp
field, and if it is, sets ignore_malformed to false.

Relates to #96051
2023-09-13 15:02:22 +01:00
Armin Braun 574fb05946
Deduplicate org.apache.lucene.document.FieldType instances across mappers (#99361)
We mostly have a handful of `FieldType` values here across all mappers and none of them contain
attributes. There's only so many combinations here, lets deduplicate these to save some heap and set up
subsequent mapper heap savings.
2023-09-08 22:18:35 +02:00
Armin Braun f1a376c317
Remove CopyTo.Builder (#99368)
The copyTo builder is really hard to reason about when it comes to
mapper merging, because the `reset` method would actually mutate an
existing mapper. That seems dangerous and the whole thing is quite
inefficient as well. -> this PR just removes it and uses a copy
constructor for copy on write, avoiding instance creation on mapper
merges here and there and leaving no doubt about these things being
immutable.
2023-09-08 13:24:31 -04:00
Ryan Ernst fe31e92020
Fix example referencing TransportVersion constant (#99375)
With the recent move of constants to the TransportVersions class, the
example plugins were missed. This commit fixes the example plugins which
referenced a constant.
2023-09-08 10:23:15 -07:00
Mark Vieira 60e178eb3a
Update Gradle Enterprise plugin to 3.14.1 (#98551) 2023-09-06 08:13:46 -07:00
Francisco Fernández Castaño f6a2b5c9ef
Add bulk delete method to BlobStore interface and implementations (#98948) 2023-08-29 12:25:03 +02:00
Simon Cooper b67a9e1ec3
Move text references to index created version to IndexVersion (#98727) 2023-08-23 10:51:56 +01:00
Simon Cooper 041e94d2b0
More migrations of Version for index created version to IndexVersion (#98495) 2023-08-21 16:47:58 +01:00
Simon Cooper a830787b07
Bulk migration of Version.CURRENT for index created to IndexVersion.current() (#98490) 2023-08-15 13:47:27 +01:00
Jim Ferenczi 28a504d7a1
Use the Weight#matches mode for highlighting by default (#96068)
This PR adapts the unified highlighter to use the Weight#matches mode by default when possible.
This is the default mode in Lucene for some time now. For cases where the matches mode won't work (nested and parent-child queries),
 the matches mode is disabled automatically.
I didn't expose an  option to explicitly disable this mode because that should be seen as an internal implementation detail.
With this change, matches that span multiple terms are highlighted together (something that users asked for years) and the clauses that don't match the document are ignored.
2023-08-09 10:44:38 +09:00
Piotr Stankowski a462deaccf
HDFS plugin: add replication_factor param (#94132)
Allows users of the HDFS repository plugin to configure the replication factor for files created by the repository implementation at repository creation time.

---------

Co-authored-by: James Baiera <james.baiera@gmail.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-08-03 14:13:45 -04:00
Simon Cooper 03e92b30df
Bulk migrate uses of settings(Version) to settings(IndexVersion) (#97528)
Mechanically change uses of Version.CURRENT to IndexVersion.current()
2023-07-25 12:25:56 +01:00
Rene Groeschke b8627079b4
Update Gradle Wrapper to 8.2 (#96686)
- Convention usage has been deprecated and was fixed in our build files
- Fix test dependencies and deprecation
2023-07-04 15:35:15 +02:00
Volodymyr Krasnikov 6e7bc27622
Update Guava dependency version to 32.0.1-jre (#97033)
* Update Guava dependency version

* Update gradle verification metadata

* fix :repository-gcs:thirdPartyAudit

* fix :discovery-gce:thirdPartyAudit

* Use single sha256 hash in allowed list
2023-06-27 09:14:10 -07:00
Simon Cooper f554a6ab2d
Remove the DiscoveryNode constructor that takes a Version (#97136) 2023-06-27 16:41:45 +01:00
Simon Cooper a873e26cf7
Convert IndexVersion.CURRENT to a method with a pluggable interface (#97132) 2023-06-27 14:47:32 +01:00
twosom c9f781f196
Cleanups in NoriTokenizerFactory and KuromojiTokenizerFactory (#92574)
Some code restructuring to improve readability.
2023-06-23 13:56:54 +02:00
Armin Braun dd7d381922
Dry up getting cluster admin client in tests (#96952)
Drying this up further and adding the same short-cut for single node
tests. Dealing with most of the spots that I could grab via automatic
refactorings.
2023-06-22 14:27:23 +02:00
Armin Braun 3f8ee82ef8
Use indices admin client shortcut in most integration tests (#96946)
Replacing the remaining usages that I could automatically replace
and a couple that I did by hand in this PR.
Also, added the same shortcut to the single node tests to save some
duplication there.
2023-06-20 13:32:59 +02:00
Simon Cooper 71c12262fb
Migrate index created version to IndexVersion (#96066) 2023-06-14 09:43:31 +01:00
Ryan Ernst 164e97e2ca
Encapsulate TransportVersion.CURRENT (#96681)
This commit changes access to the latest TransportVersion constant to
use a static method instead of a public static field. By encapsulating
the field we will be able to (in a followup) lazily determine what the
latest is, outside of clinit.
2023-06-13 18:44:15 -04:00
Benjamin Trent cba3e274bd
Increase size mapper test coverage (#96746)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2023-06-13 11:18:25 -04:00
Benjamin Trent 8b52d85a37
Add more testing for murmur3 mapper (#96745)
The key here is that doc value fields for the murmur3 mapper are the
hash, where as the `fields` API pulls from `_source` which contains the
unmodified string.

closes: https://github.com/elastic/elasticsearch/issues/96742
2023-06-13 09:21:37 -04:00
David Turner 350beea181
Arbitrary bytes in blob store register (#96019)
Today the blob store register supports recording only a `long`,
represented as an 8-byte blob. We need to store a little more data in
the register, so this commit generalises things to work with a
`BytesReference` directly.
2023-05-16 06:16:21 -04:00
Armin Braun af5c11702b
Use string list setting throughout codebase (#95901)
We can dry things up a little here and also making things a little faster
(in case we missed a corner case where a list setting is hot) with the optimized
string list setting constructor.
2023-05-09 11:22:28 +02:00
Mark Vieira 047a85f30a
Update Gradle Enteprise plugin to 3.13.1 (#95896)
Upgrade Gradle Enterprise plugin to latest available.
2023-05-08 14:01:15 -04:00
Przemyslaw Gomulka 257d633f2d
Add slf4j-nop in order to prevent startup warnings (#95459)
currently when Elasticsearch starts up there are warnings on the console complaining about SLF4J providers not found.
this is emitted twice
1 - when repository-azure module is loaded
2 - when xpack-security (depending on transport-netty) module is loaded 

This commits adds a slf4j-nop to fix that warning

related to #93714
and #93878
2023-05-05 09:38:34 +02:00
Armin Braun c41bda9e3a
Dry up remaining verbose index setting building in tests (#95652)
Lasts spots I could easily find via regex.
Follow-up to #95569
2023-04-28 11:18:07 +02:00
Rene Groeschke 486dfc8ea2
Update Gradle wrapper from 8.1 to 8.1.1 (#95492) 2023-04-24 06:30:45 -04:00
Artem Prigoda a100fefe7f
Remove optional commons-io dependency from azure-discovery (#95378)
`commons-io` is declared as a transitive dependency for `azure-svc-mgmt-compute`
https://mvnrepository.com/artifact/com.microsoft.azure/azure-svc-mgmt-compute/0.9.5, but it seems that it's used only for internal testing and is not required for the consumers of the API. We should be able to safely remove it.
2023-04-21 20:04:46 +02:00
Rene Groeschke 44cc172219
Update Gradle wrapper to 8.1 (#94663)
- Udpate docker compose plugin to use 8.1 compliant version
- Fix deprecations of test task configurations
2023-04-13 16:11:51 +02:00
Rory Hunter fe1083f6c5
Upgrade spotless plugin to 6.17.0 (#94994)
Fixes #82794. Upgrade the spotless plugin, which addresses the issue
around formatting `instanceof` expressions. Formatting of statements
including lambdas seems to have improved too.
2023-04-04 10:03:32 +01:00
Rene Groeschke e735455431
Provide hints to Gradle Task.onlyIf declarations (#94953)
* Update gradle enterprise plugin to show skip reasons in build scans
* Unify OS specific handling in build logic
2023-03-31 23:57:55 +02:00
Alan Woodward 093e36c875
Introduce DocumentParsingException (#92646)
Document parsing methods currently throw MapperParsingException. This
isn't very helpful, as it doesn't contain any information about where the parse
error happened - it is designed for parsing mappings, which are realised into
java maps before being examined. This commit introduces a new exception
specifically for document parsing that extends XContentException, so that
it reports the current position of the parser as part of its error message.

Fixes #85083
2023-03-31 12:14:19 +01:00
Alan Woodward 131da70321
ValueFetchers now return a StoredFieldsSpec (#94820)
This allows us to be more conservative about what needs to be loaded
when using the fields API, and opens up the possibility of avoiding
using stored fields or source altogether if we can use doc values to
fetch values.

This commit also uses this new information from ValueFetchers to 
more efficiently preload stored fields for the `fields` API, while
still allowing the lazy loading of individual fields if they are asked
for by scripts or runtime fields which cannot be introspected.
2023-03-30 10:46:43 +01:00
Simon Cooper 56d53da381
Migrate LuceneDocument.getFields(String) to a List (#94830) 2023-03-29 11:08:36 +01:00
Alan Woodward 35da97214c
Make IndexAnalyzers an interface (#94819)
IndexAnalyzers is currently always a concrete class wrapping several
Maps of NamedAnalyzers. This means that whenever it is used it needs
to instantiate all of its component analyzers, making testing much heavier
than it needs to be. It also means that things like overriding analysis for
legacy indexes is pushed into mapper parameters, rather than being
handled in a single place.

This commit makes IndexAnalyzers into an interface, with an anonymous
concrete implementation that handles reloading and closing for index
shards.
2023-03-28 16:07:08 +01:00
Benjamin Trent c40fc1ffc7
Fix metadata _size when it comes to stored fields extraction (#94483)
Corrects weird regression introduced in:
https://github.com/elastic/elasticsearch/pull/91269

Previous to that change, getting `_size` worked when using
`storedFields`. However, that change now detects if a field type
declares itself as `stored`. Even though `_size` is stored, it didn't
declare itself as such. 

This change fixes that.

closes https://github.com/elastic/elasticsearch/issues/94468
2023-03-14 11:16:26 -04:00
Rene Groeschke 9ce5baed0c
Update Gradle wrapper to 8.0.2 (#92751)
- Remove custom checksum build logic in wrapper task
- Adjust jdk home handling adjusting the change in behaviour in gradle. Requires providing canonical paths for provisioned jdk homes.
- Fix test by add workaround to bug in configuration cache
2023-03-09 13:00:12 +01:00