Commit Graph

200 Commits

Author SHA1 Message Date
Rene Groeschke 35ec6f348c
Introduce simple public yaml-rest-test plugin (#76554)
This introduces a basic public yaml rest test plugin that is supposed to be used by external 
elasticsearch plugin authors. This is driven by #76215

- Rename yaml-rest-test to intern-yaml-rest-test
- Use public yaml plugin in example plugins

Co-authored-by: Mark Vieira <portugee@gmail.com>
2021-08-31 08:45:52 +02:00
Nhat Nguyen 174f65ef82
Upgrade to Lucene 8.9.0 (#74729) 2021-06-30 08:49:53 -04:00
Ryan Ernst ab1a2e4a84
Add precommit task for detecting split packages (#73784)
Modularization of the JDK has been ongoing for several years. Recently
in Java 16 the JDK began enforcing module boundaries by default. While
Elasticsearch does not yet use the module system directly, there are
some side effects even for those projects not modularized (eg #73517).
Before we can even begin to think about how to modularize, we must
Prepare The Way by enforcing packages only exist in a single jar file,
since the module system does not allow packages to coexist in multiple
modules.

This commit adds a precommit check to the build which detects split
packages. The expectation is that we will add the existing split
packages to the ignore list so that any new classes will not exacerbate
the problem, and the work to cleanup these split packages can be
parallelized.

relates #73525
2021-06-08 15:04:23 -07:00
Nhat Nguyen 5efb6eaba6
Update Lucene to 8.9.0-snapshot-ddc238e5df8 (#73568)
Just include LUCENE-9980, which fixes #39591.

Closes #39591
2021-05-31 13:49:29 -04:00
Nhat Nguyen 1764e8ba15
Upgrade to Lucene-8.9.0-SNAPSHOT-efdc43fee18 (#73130)
Upgrades to Lucene-8.9 snapshot which includes:

- LUCENE-9507: Custom order for leaves (/cc @mayya-sharipova)
- LUCENE-9935: Enable bulk merge for stored fields with index sort
2021-05-17 09:37:20 -04:00
Nhat Nguyen a461597c75
Upgrade to Lucene 8.8.2 on 8.0 (#71587) 2021-04-14 08:52:23 -04:00
Jake Landis b1ef1fd800
Introduce yamlRestCompatTests for :plugins projects (#71440) 2021-04-08 16:11:50 -05:00
Mark Vieira 6339691fe3
Consolidate REST API specifications and publish under Apache 2.0 license (#70036) 2021-03-26 16:20:14 -07:00
Mark Vieira a92a647b9f Update sources with new SSPL+Elastic-2.0 license headers
As per the new licensing change for Elasticsearch and Kibana this commit
moves existing Apache 2.0 licensed source code to the new dual license
SSPL+Elastic license 2.0. In addition, existing x-pack code now uses
the new version 2.0 of the Elastic license. Full changes include:

 - Updating LICENSE and NOTICE files throughout the code base, as well
   as those packaged in our published artifacts
 - Update IDE integration to now use the new license header on newly
   created source files
 - Remove references to the "OSS" distribution from our documentation
 - Update build time verification checks to no longer allow Apache 2.0
   license header in Elasticsearch source code
 - Replace all existing Apache 2.0 license headers for non-xpack code
   with updated header (vendored code with Apache 2.0 headers obviously
   remains the same).
 - Replace all Elastic license 1.0 headers with new 2.0 header in xpack.
2021-02-02 16:10:53 -08:00
Ignacio Vera 747773d5af
Upgrade to Lucene 8.8.0 (#68272) 2021-02-01 13:36:03 +01:00
Jim Ferenczi e77c523bd9
Upgrade to a new lucene 8.8.0 snapshot (#67691)
This change upgrades to the latest Lucene 8.8.0 snapshot.
It also restores the compression on binary doc values that was lost in the last snapshot upgrade.
The compression is now configurable on binary doc values but we don't expose this functionality yet so this commit ensures that we pick the same compression mode as previous releases (BEST_COMPRESSION).
2021-01-19 13:33:19 +01:00
Ignacio Vera 604ee06a3b
Upgrade to lucene-8.8-snapshot-f73f6b1 (#67228) 2021-01-12 08:03:00 +01:00
Ignacio Vera 4add5cb65d
Upgrade to Lucene-8.7.0 (#64532) 2020-11-03 16:04:26 +01:00
Ignacio Vera 2a1d8c66af
Upgrade to lucene-8.7.0-snapshot-72d8528c3a6 (#63912) 2020-10-20 13:33:32 +02:00
Mayya Sharipova e6aa10ad89
Upgrade 8.7.0-snapshot-5c4168d (#63444)
This disables sort optim on _doc, which may still be unstable.
2020-10-08 06:47:37 -04:00
Mayya Sharipova fc6e3dc41e
Upgrade to lucene-8.7.0-snapshot-e914862 (#63395)
This fixes a problem with some special cases of sorting on .doc
2020-10-07 08:48:14 -04:00
Mayya Sharipova 5f9ba13750
Upgrade to lucene- 8.7.0-snapshot-66c49a35402 (#63349)
This includes fixing doc iteration during sort optimization
2020-10-06 20:31:51 -04:00
Nhat Nguyen ff43c08443
Upgrade to Lucene-8.7.0-snapshot-77396dbf339 (#63222)
Includes LUCENE-9554, which exposes the pendingNumDocs from IndexWriter.
2020-10-05 09:36:02 -04:00
Mayya Sharipova 1648cf2974
Upgrade lucene to lucene-8.7.0-snapshot-3b59906 (#62970) 2020-09-28 13:57:49 -04:00
Adrien Grand b33dc24a79
Upgrade to lucene-8.7.0-snapshot-830bd186a8d. (#62576) 2020-09-18 07:53:38 +02:00
Adrien Grand 0418e9c09b
Upgrade to lucene-8.7.0-snapshot-9cd3af50f80. (#62450)
This new snapshot contains the following JIRAs that we're interested in:
 - [LUCENE-9525](https://issues.apache.org/jira/browse/LUCENE-9525)
Better handling of small documents. This should improve retrieval times
when documents are less than ~1kB.
 - [LUCENE-9510](https://issues.apache.org/jira/browse/LUCENE-9510)
Faster flushes when index sorting is enabled by not compressing the
temporary files that store stored fields and term vectors.
2020-09-16 17:13:39 +02:00
Adrien Grand 39bde05040
Upgrade to lucene-8.7.0-snapshot-cdfdc1e0851. (#62334)
Upgrade to a new Lucene snapshot that (at least partially) addresses the
indexing rate regression when index sorting is enabled.
2020-09-15 14:19:42 +02:00
Ignacio Vera 8312591353
upgrade to lucene-8.7.0-snapshot-b313618cc1d (#62213) 2020-09-10 15:22:33 +02:00
Ignacio Vera e236054e09
upgrade to Lucene-8.7.0-snapshot-61ea26a (#61957) 2020-09-04 12:08:35 +02:00
Jason Tedor db7c29bc42
Upgrade to Lucene 8.6.2 (#61688)
This commit upgrades the Lucene dependencies to 8.6.2.
2020-08-31 09:52:09 -04:00
Jake Landis 10be5d4c79
Convert most OSS plugins from integTest to [yaml | java]RestTest or internalClusterTest (#59444)
For all OSS plugins (except repository-* and discovery-*) integTest
task is now a no-op and all of the tests are now executed via a test,
yamlRestTest, javaRestTest, or internalClusterTest.

This commit does NOT convert the discovery-* and repository-* since they
are bit more complex then the rest of tests and this PR is large enough.
Those plugins will be addressed in a future PR(s).

This commit also fixes a minor issue that did not copy the rest api
for projects that only had YAML TEST tests.

related: #56841
2020-07-28 16:43:17 -05:00
Ignacio Vera 15bc79da19
upgrade to lucene-8.6.0 release (#59596) 2020-07-15 11:58:28 +02:00
Ignacio Vera 155c9d15ea
upgrade to lucene-8.6.0-snapshot-6a715e2ecc3 (#59091) 2020-07-07 10:50:53 +02:00
Jake Landis 333a5d8cdf
Create plugin for yamlTest task (#56841)
This commit creates a new Gradle plugin to provide a separate task name
and source set for running YAML based REST tests. The only project
converted to use the new plugin in this PR is distribution/archives/integ-test-zip.
For which the testing has been moved to :rest-api-spec since it makes the most
sense and it avoids a small but awkward change to the distribution plugin.

The remaining cases in modules, plugins, and x-pack will be handled in followups.

This plugin is distinctly different from the plugin introduced in #55896 since
the YAML REST tests are intended to be black box tests over HTTP. As such they
should not (by default) have access to the classpath for that which they are testing.

The YAML based REST tests will be moved to separate source sets (yamlRestTest).
The which source is the target for the test resources is dependent on if this
new plugin is applied. If it is not applied, it will default to the test source
set.

Further, this introduces a breaking change for plugin developers that
use the YAML testing framework. They will now need to either use the new source set
and matching task, or configure the rest resources to use the old "test" source set that
matches the old integTest task. (The former should be preferred).

As part of this change (which is also breaking for plugin developers) the
rest resources plugin has been removed from the build plugin and now requires
either explicit application or application via the new YAML REST test plugin.

Plugin developers should be able to fix the breaking changes to the YAML tests
by adding apply plugin: 'elasticsearch.yaml-rest-test' and moving the YAML tests
under a yamlRestTest folder (instead of test)
2020-07-06 12:13:01 -05:00
Rene Groeschke 9526c7a4b3
Replace compile configuration usage with api (#58451)
- Use java-library instead of plugin to allow api configuration usage
- Remove explicit references to runtime configurations in dependency declarations
- Make test runtime classpath input for testing convention
  - required as java library will by default not have build jar file
  - jar file is now explicit input of the task and gradle will ensure its properly build
2020-06-30 09:37:09 +02:00
Rene Groeschke 5f9d1f1d7c
Unify dependency licenses task configuration (#58116)
- Remove duplicate dependency configuration
- Use task avoidance api accross the build
- Remove redundant licensesCheck config
2020-06-17 18:27:16 +02:00
Alan Woodward e19a82d762
Update to lucene snapshot e7c625430ed (#57981)
Includes LUCENE-9148 and LUCENE-9398, which splits the BKD metadata, index and data into separate files and keeps the index off-heap.
2020-06-11 14:36:31 +01:00
markharwood df93987a75
Update Lucene snapshot to 8.6.0-snapshot-9d6c738ffce (#56988)
Update of Lucene snapshot and dealing with API changes
2020-05-21 09:18:35 +01:00
Ignacio Vera e62fb090fa
upgrade to Lucene 8.6.0 snapshot (#56175) 2020-05-13 08:37:35 +02:00
Ignacio Vera a18fca65d6
Upgrade to lucene 8.5.1 release (#55229)
Upgrade to lucene 8.5.1 release that contains a bug fix for a bug that might introduce index corruption when deleting data from an index that was previously shrunk.
2020-04-15 15:14:38 +02:00
Alan Woodward a371113d20
Upgrade to lucene 8.5.0 release (#54077)
Upgrades our lucene dependency to the released 8.5.0 version.
2020-03-24 13:44:28 +00:00
Jake Landis afc2383b72
Optimize which Rest resources are used by the Rest tests. (#53299)
This should help with Gradle's incremental compile such that projects
only depend upon the resources they use.

related #52114
2020-03-18 09:09:29 -05:00
Alan Woodward 5d716bc16c
Upgrade to final lucene 8.5.0 snapshot (#53293)
Lucene 8.5.0 release candidates are imminent. This commit upgrades master to use
the latest snapshot to check that there are no last-minute bugs or regressions.
2020-03-10 09:30:37 +00:00
Nhat Nguyen 8f17b21de8 Revert "upgrade to lucene-snapshot-fa75139efea (#53150)"
This reverts commit 59b841751d.
2020-03-05 17:30:06 -05:00
Ignacio Vera 59b841751d
upgrade to lucene-snapshot-fa75139efea (#53150) 2020-03-05 08:47:10 +01:00
Nhat Nguyen 699b1dccac
Upgrade to Lucene 8.5.0-snapshot-c4475920b08 (#52950)
To give LUCENE-9228 more CI cycles
2020-02-28 21:25:34 -05:00
markharwood cbd224d070
Upgrade Lucene 8.5 to latest snapshot (#52520)
Upgrade Lucene 8.5 to latest snapshot
2020-02-20 10:34:41 +00:00
Ignacio Vera 8b6e6a1639 Upgrade to lucene-8.5.0-snapshot-d62f6307658 (#52039) 2020-02-10 09:08:15 +01:00
Mayya Sharipova 3d53c48509
Upgrade to lucene-8.5.0-snapshot-3333ce7da6d (#51327)
Remove lucene-spatial module
Keep previous behaviour of FieldHighlighter::highlightOffsetsEnums

LUCENE-9093 modified how FieldHighlighter breaks texts into passages,
which doesn't work well with elasticsearch custome BoundedBreakIteratorScanner.
This commit for now keeps previous behaviour of FieldHighlighter.
2020-01-30 10:08:48 -05:00
Adrien Grand de2c0c660e
Upgrade to Lucene 8.4.0. (#50518) 2020-01-08 17:10:38 +01:00
Ignacio Vera 8304977a53
upgrade to lucene 8.4.0-snapshot-08b8d116f8f (#50129) 2019-12-12 12:17:44 +01:00
Adrien Grand 1329acc094
Upgrade to lucene 8.4.0-snapshot-662c455. (#50016)
Lucene 8.4 is about to be released so we should check it doesn't cause problems
with Elasticsearch.
2019-12-10 17:09:36 +01:00
Mayya Sharipova 3514485991
Upgrade lucene to 8.4.0-snapshot-e648d601efb (#48607) 2019-10-29 05:44:29 -04:00
Ignacio Vera cef9883324
upgrade to Lucene-snapshot-25968e3b75e (#48203) 2019-10-18 07:48:52 +02:00
Ignacio Vera 4e91caee59
upgrade to Lucene-8.3.0-snapshot-47aece66b48 (#47770) 2019-10-09 12:21:59 +02:00