Another round of automated fixes to this, marking things that can be
made static as static. Saves some JIT cycles but also turns some lambdas
from capturing to non-capturing and makes the "utilityness" of some
classes visible.
Lots of spots where we did weird things around streams like redundant stream creation, redundant collecting
before adding all the collected elements to another collection or so, redundant streams for joining strings
and using less efficient `Collectors.toList` and in a few cases also incorrectly relying on the result being mutable.
The sysprops and envVars members of Command provide cli implementations
with information about the jvm process that is running. This is
convenient for runtime, but difficult for tests to mock because they
must subclass the cli class.
This commit adds a ProcessInfo record, and plumbs it through the
main and execute methods. The new record includes system properties,
environment variables and the working directory. By having this be a
single new parameter, additional information can be added in the future
without again needing to modify the method signatures.
relates #85758
Serveral mechanisms exist for intializing logging in cli tools. Some
base Command classes exist which initialize logging. But they do this
late, when they are constructed, which may be after static init has
occured for classes grabbing a Logger. Other CLIs like node tool
explicitly initialize logging to avoid that problem.
This commit removes all the of the LoggingAware classes, and
unifies logging configuration to occur at the very beginning of the cli
launcher.
relates #85758
CLI scripts have a common infrastructure in that they call to the shared
elasticsearch-cli shell script which launches them with the appropriate
java command line. However, each underlying Java class must implement
its own main method.
This commit introduces a single main method to be shared by CLIs. The
new CliToolLauncher takes in system properties to determine which tool
is being run, and a new CliToolProvider SPI allows defining and finding
the named tools.
relates #85758
Co-authored-by: William Brafford <william.brafford@elastic.co>
The main method of Command is not normally called by cli tests. Instead
they call the execute helper which calls mainWithoutErrorHandling.
Sometimes, though, it is desirable to test the top level behavior. This
commit cleans up the base class tests to not call main directly. This
will help with future refactorings to change the signature of main, so
that less uses need to be changed.
This commit adds a jar separate from the test framework to provide
utilities for testing x-content related code. The first thing moved
there is the base schema validation test case, which also pulls along
the com.networknt dependency and jackson. For now these are direct
dependencies, though we could consider shading them in the future so as
not to expose downstream projects to them, which may have version
conflicts.
Lucene issues that resulted in elasticsearch changes:
LUCENE-9820 Separate logic for reading the BKD index from logic to intersecting it.
LUCENE-10377: Replace 'sortPos' with 'enableSkipping' in SortField.getComparator()
LUCENE-10301: make the test-framework a proper module by moving all test
classes to org.apache.lucene.tests
LUCENE-10300: rewrite how resources are read in ukrainian morfologik analyzer:
LUCENE-10054 Make HnswGraph hierarchical
Fix the split package org.elasticsearch.common.xcontent, between server and the x-content lib. Move the x-content lib exported package from org.elasticsearch.common.xcontent to org.elasticsearch.xcontent ( following the naming convention of similar libraries ). Removing split packages is a prerequisite to modularization.
Change the formatter config to sort / order imports, and reformat the
codebase. We already had a config file for Eclipse users, so Spotless now
uses that.
The "Eclipse Code Formatter" plugin ought to be able to use this file as
well for import ordering, but in my experiments the results were poor.
Instead, use IntelliJ's `.editorconfig` support to configure import
ordering.
I've also added a config file for the formatter plugin.
Other changes:
* I've quietly enabled the `toggleOnOff` option for Spotless. It was
already possible to disable formatting for sections using the markers
for docs snippets, so enabling this option just accepts this reality
and makes it possible via `formatter:off` and `formatter:on` without
the restrictions around line length. It should still only be used as
a very last resort and with good reason.
* I've removed mention of the `paddedCell` option from the contributing
guide, since I haven't had to use that option for a very long time. I
moved the docs to the spotless config.
When libs/core was created, several classes were moved from server's
o.e.common package, but they were not moved to a new package. Split
packages need to go away long term, so that Elasticsearch can even think
about modularization. This commit moves all the classes under o.e.common
in core to o.e.core.
relates #73784
This change fixes number of problems in GeoIPv2 code:
- closes streams from Files.list in GeoIpCli, which should fix tests on Windows
- makes sure that total download time in GeoIP stats is non-negative (we serialize it as vInt which can cause problems with negative numbers and it can happen when clock was changed during operation)
- fixes handling of failed/simultaneous downloads, #69951 was meant as a way to prevent 2 persistent tasks to index chunks but it would prevent any update if single download failed mid indexing, this change uses timestamp (lastUpdate) as sort of UUID. This should still prevent 2 tasks to step on each other toes (overwriting chunks) but in the end still only single task should be able to update task state (this is handled by persistent tasks framework)
Closes#71145
Instead of creating directories manually under temp dir we should simply use createTempDir() twice. This should make Windows be able to delete directories correctly.
This PR also removes unused method.
Fixes#71145
Air-gapped environments can't simply use GeoIp database service provided by Infra, so they have to either use proxy or recreate similar service themselves.
This PR adds tool to make this process easier. Basic workflow is:
download databases from MaxMind site to single directory (either .mmdb files or gzipped tarballs with .tgz suffix)
run the tool with $ES_PATH/bin/elasticsearch-geoip -s directory/to/use [-t target/directory]
serve static files from that directory (for example with docker run -v directory/to/use:/usr/share/nginx/html:ro nginx
use server above as endpoint for GeoIpDownloader (geoip.downloader.endpoint setting)
to update new databases simply put new files in directory and run the tool again
This change also adds support for relative paths in overview json because the cli tool doesn't know about the address it would be served under.
Relates to #68920
Armin Braun
Jake Landis
Rene Groeschke
Ryan Ernst
Lee Hinman
Nik Everett
Mayya Sharipova
Chris Hegarty
Rory Hunter
Ryan Ernst
Przemko Robakowski
Mark Vieira