_id and _routing now no longer support the 'path' setting on indexes
created with 2.0. Indexes created before 2.0 still support this
setting for backcompat.
closes#6730
The `analyzer` setting is now the base setting, and `search_analyzer`
is simply an override of the search time analyzer. When setting
`search_analyzer`, `analyzer` must be set.
closes#9371
Restrict use of java.io.File to 5 methods (excluded), but otherwise ban.
This is a prerequisite to do any mocking here.
I don't try to do any heavy cleanup on these tests, I am not familiar with them.
So this is mostly a rote straightforward conversion.
Closes#8836
This commit cuts over all of core (not quite all tests) to java.nio.Path
It also adds the file class to the core forbidden APIs to prevent its usage.
This commit also resolves#8254 since we now consistently useing the NIO Path
API. The Changes in this commit allow for more information if IO operations fail
since the NIO API throws exceptions instead of boolean return values. The build-in
methods used in this commit are also more resillient to encodeing errors like
unmappable characters and throw exceptions if those chars are present in a file.
Closes#8254Closes#8666
Interrupting a thread while blocking on an NIO Read / Write Operation
can cause a file to be closed due to the interrupts. This can have unpredictable
effects when files are open by index readers etc. we should prevent interruptions
across the board if possible.
Closes#8494
This prevents too-difficult regular expressions from consuming
excessive RAM/CPU; the default max_determinized_states is 10,000 (same
as Lucene) but query_string and regepx query/filter can override
per-request.
The also upgrades to a new Lucene 5.0.0 snapshot.
Closes#8386Closes#8357
Today we use the File API for file deletion as well as recursive
directory deletions. This API returns a boolean if operations
are successful while hiding the actual reason why they failed.
The Path API throws and actual exception that might provide better
insights and debug information.
Closes#8366
This has a lot of improvements in lucene, particularly around memory usage, merging, safety, compressed bitsets, etc.
On the elasticsearch side, summary of the larger changes:
API changes: postings API became a "pull" rather than "push", collector API became per-segment, etc.
packaging changes: add lucene-backwards-codecs.jar as a dependency.
improvements to boolean filtering: especially ensuring it will not be slow for SparseBitSet.
use generic BitSet api in plumbing so that concrete bitset type is an implementation detail.
use generic BitDocIdSetFilter api for dedicated bitset cache, so there is type safety.
changes to support atomic commits
implement Accountable.getChildResources (detailed memory usage API) for fielddata, etc
change handling of IndexFormatTooOld/New, since they no longer extends CorruptIndexException
Closes#8347.
Squashed commit of the following:
commit d90d53f5f2
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Nov 5 21:35:28 2014 +0100
Make default codec/postings/docvalues format constants
commit cb66c22c71
Merge: d4e2f6dad4ff43
Author: Robert Muir <rmuir@apache.org>
Date: Wed Nov 5 11:41:13 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit d4e2f6dfe7
Merge: 4e5445c4111d93
Author: Robert Muir <rmuir@apache.org>
Date: Wed Nov 5 06:26:32 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit 4e5445c775
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 16:19:19 2014 -0500
FixedBitSet -> BitSet
commit 9887ea73e8
Merge: 1bf8894fc84666
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 15:26:25 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit 1bf8894430
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 15:22:51 2014 -0500
remove nocommit
commit a9c2a2259f
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 13:48:43 2014 -0500
turn jenkins red again
commit 067baaaa4d
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 13:18:21 2014 -0500
unzip from stream
commit 82b6fba33d
Merge: b2214bb 6523cd9
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 13:10:59 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit b2214bb093
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 13:09:53 2014 -0500
go back to my URL until we can figure out what is up with jenkins
commit e7d6141722
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 10:52:54 2014 -0500
try this jenkins
commit 337a3c7704
Author: Simon Willnauer <simonw@apache.org>
Date: Tue Nov 4 16:17:49 2014 +0100
Rename temp-files under lock to prevent metadata reads while renaming
commit 77d5ba80d0
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 10:07:11 2014 -0500
continue to treat too-old/too-new as corruption for now
commit 98d0fd2f48
Author: Robert Muir <rmuir@apache.org>
Date: Tue Nov 4 09:24:21 2014 -0500
fix last nocommit
commit 643fceed66
Author: Simon Willnauer <simonw@apache.org>
Date: Tue Nov 4 14:46:17 2014 +0100
remove NoSuchDirectoryException
commit 2e43c4feba
Merge: 93826e48163107
Author: Simon Willnauer <simonw@apache.org>
Date: Tue Nov 4 14:38:00 2014 +0100
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit 93826e4d56
Merge: 7f1012944e24d3
Author: Simon Willnauer <simonw@apache.org>
Date: Tue Nov 4 12:54:27 2014 +0100
Merge branch 'master' into enhancement/lucene_5_0_upgrade
Conflicts:
src/main/java/org/elasticsearch/index/store/DistributorDirectory.java
src/main/java/org/elasticsearch/index/store/Store.java
src/main/java/org/elasticsearch/indices/recovery/RecoveryStatus.java
src/test/java/org/elasticsearch/index/store/DistributorDirectoryTest.java
src/test/java/org/elasticsearch/index/store/StoreTest.java
src/test/java/org/elasticsearch/indices/recovery/RecoveryStatusTests.java
commit 7f10129364
Author: Adrien Grand <jpountz@gmail.com>
Date: Tue Nov 4 11:32:24 2014 +0100
Fix TopHitsAggregator to not ignore the top-level/leaf collector split.
commit 042fadc860
Author: Adrien Grand <jpountz@gmail.com>
Date: Tue Nov 4 11:31:20 2014 +0100
Remove MatchDocIdSet in favor of DocValuesDocIdSet.
commit 7d877581ff
Author: Adrien Grand <jpountz@gmail.com>
Date: Tue Nov 4 11:10:08 2014 +0100
Make the and filter use the cost API.
Lucene 5 ensured that cost() can safely be used, and this will have the benefit
that the order in which filters are specified is not important anymore (only
for slow random-access filters in practice).
commit 78f1718aa2
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 23:55:17 2014 -0500
fix previous eclipse import braindamage
commit 186c40e925
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 22:32:34 2014 -0500
allow child queries to exhaust iterators again
commit b0b1271305
Author: Ryan Ernst <ryan@iernst.net>
Date: Mon Nov 3 14:50:44 2014 -0800
Fix nocommit for mapping output. index_options will not be printed if
the field is not indexed.
commit ba223eb85e
Author: Ryan Ernst <ryan@iernst.net>
Date: Mon Nov 3 14:07:26 2014 -0800
Remove no commit for chinese analyzer provider. We should have a
separate issue to address not using this provider on new indexes.
commit ca554b03c4
Author: Ryan Ernst <ryan@iernst.net>
Date: Mon Nov 3 13:41:59 2014 -0800
Fix stop tests
commit de67c4653e
Author: Ryan Ernst <ryan@iernst.net>
Date: Mon Nov 3 12:51:17 2014 -0800
Remove analysis nocommits, switching over to Lucene43*Filters for
backcompat
commit 50cae9bec7
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 15:32:25 2014 -0500
add ram accounting and TODO lazy-loading (its no worse than master, can be a followup improvement) for suggesters
commit 7a7f0122f1
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 15:11:26 2014 -0500
bump lucene version
commit cd0cae5c35
Merge: 446bc093c72073
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 14:49:05 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit 446bc09b4e
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 14:46:30 2014 -0500
remove hack
commit a19d85a968
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 12:53:11 2014 -0500
dont create exceptions with circular references on corruption (will open a PR for this)
commit 0beefb9e82
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 11:47:14 2014 -0500
temporarily add craptastic detector for this horrible bug
commit e9f2d298bf
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 10:56:01 2014 -0500
add nocommit
commit e97f1d50a9
Merge: c57a3c8f1f50ac
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 10:12:12 2014 -0500
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit c57a3c8341
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 10:11:46 2014 -0500
fix nocommit
commit dd0e77e4ec
Author: Robert Muir <rmuir@apache.org>
Date: Mon Nov 3 09:54:09 2014 -0500
nocommit -> TODO, this is in much more places in the codebase, bigger issue
commit 3cc3bf56d7
Author: Ryan Ernst <ryan@iernst.net>
Date: Sat Nov 1 23:59:17 2014 -0700
Remove nocommit and awaitsfix for edge ngram filter test.
commit 89f1152451
Author: Ryan Ernst <ryan@iernst.net>
Date: Sat Nov 1 23:57:44 2014 -0700
Fix EdgeNGramTokenFilter logic for version <= 4.3, and fixed instanceof
checks in corresponding tests to correctly check for reverse filter when
applicable.
commit 112df869cd
Author: Robert Muir <rmuir@apache.org>
Date: Sun Nov 2 00:08:30 2014 -0400
execute geo disjoint query/filter as intersects
commit e5061273cc
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 22:58:59 2014 -0400
remove chinese analyzer from docs
commit ea1af11b89
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 22:29:00 2014 -0400
fix ram accounting bug
commit 53c0a42c6a
Merge: e3bcd3c6011a18
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 22:16:29 2014 -0400
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit e3bcd3cc07
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 22:15:01 2014 -0400
fix url-email back compat (thanks ryan)
commit 91d6b096a9
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 22:11:26 2014 -0400
bump lucene version
commit d2bb9568df
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 20:33:07 2014 -0400
remove nocommit
commit 1d049c471e
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 20:28:58 2014 -0400
fix eclipse to group org/com imports together: without this, its madness
commit 09d8c1585e
Author: Robert Muir <rmuir@apache.org>
Date: Sat Nov 1 14:27:41 2014 -0400
remove nocommit, if you dont liek it, print assembly and tell me how it can be better
commit 8a6a294313
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 20:01:55 2014 +0100
Remove deprecated usage of DocIdSets.newDocIDSet.
commit 601bee6054
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 14:13:18 2014 -0400
maybe one of these zillions of annotations will stop thread leaks
commit 9d3f69abc7
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 14:05:39 2014 -0400
fix some analysis nocommits
commit 312e3a29c7
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 18:28:45 2014 +0100
Remove XConstantScoreQuery/XFilteredQuery/ApplyAcceptedDocsFilter.
commit 5a0cb9f8e1
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 17:06:45 2014 +0100
Fix misleading documentation of DocIdSets.toCacheable.
commit 8b4ef2b5b4
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 17:05:59 2014 +0100
Fix CustomRandomAccessFilterStrategy to override the right method.
commit d7a9a407a6
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 16:21:35 2014 +0100
Better handle the special case when there is a single SHOULD clause.
commit 648ad389f0
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 15:53:38 2014 +0100
Cut over XBooleanFilter to BitDocIdSet.Builder.
The idea is similar to what happened to Lucene's BooleanFilter.
Yet XBooleanFilter is a bit more sophisticated and I had to slightly
change the way it is implemented in order to make it work. The main difference
with before is that slow filters are now applied lazily, so eg. if you have 3
MUST clauses, two with a fast iterator and the third with a slow iterator, the
previous implementation used to apply the fast iterators first and then only
check the slow filter for bits which were set in the bit set. Now we are
computing a bit set based on the fast must clauses and then basically returning
a BitsFilteredDocIdSet.wrap(bitset, slowClause).
Other than that, BooleanFilter still uses the bitset optimizations when or-ing
and and-ind filters.
Another improvement is that BooleanFilter is now aware of the cost API.
commit b2dad312b4
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 10:18:53 2014 -0400
clear nocommit
commit 4851d2091e
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 15:15:16 2014 +0100
cut over to RoaringDocIdSet
commit ca6aec24a9
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:57:30 2014 +0100
make nocommit more explicit
commit d0742ee2cb
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 09:55:24 2014 -0400
fix standardtokenizer nocommit
commit 7d6faccaff
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:54:08 2014 +0100
fix compilation
commit a038a405c1
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:53:43 2014 +0100
fix compilation
commit 30c9e307b1
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:52:35 2014 +0100
fix compilation
commit e5139bc5a0
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 09:52:16 2014 -0400
clear nocommit here
commit 85dd2cedf7
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:46:17 2014 +0100
fix CompletionPostingsFormatTest
commit c0f3781f61
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 09:38:00 2014 -0400
add tests for these analyzers
commit 51f9999b4a
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 14:10:26 2014 +0100
remove nocommit - this is not an issue
commit fd1388fa03
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Fri Oct 31 14:07:01 2014 +0100
Remove redundant null check
commit 3d6dd51b09
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Fri Oct 31 14:01:37 2014 +0100
Removed the work around to prevent p/c error when invoking #iterator() twice, because the custom query filter wrapper now doesn't transform the result to a cache doc id set any more.
I think the transforming to a cachable doc id set in CustomQueryWrappingFilter isn't needed at all, because we use the DocIdSet only once and because of that is just slowed things down.
commit 821832a537
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 13:54:33 2014 +0100
one more nocommit
commit 77eb9ea4c4
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Fri Oct 31 13:52:29 2014 +0100
Remove cast
commit a400573c03
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 13:49:24 2014 +0100
fix stop filter
commit 51746087cf
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 13:21:36 2014 +0100
fix changed semantics of FBS.nextSetBit to check for NO_MORE_DOCS
commit 8d0a4e2511
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 08:13:44 2014 -0400
do the bogus cast differently
commit 46a5cc5732
Author: Simon Willnauer <simonw@apache.org>
Date: Fri Oct 31 13:00:16 2014 +0100
I hate it but P/C now passes
commit 580c0c2f82
Merge: a9d3c001645434
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 31 06:54:31 2014 -0400
fix nocommit/classcast
commit a9d3c004d6
Author: Adrien Grand <jpountz@gmail.com>
Date: Fri Oct 31 08:49:31 2014 +0100
Update TODO.
commit aa75af0b40
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 19:18:25 2014 -0400
clear obselete nocommits from lucene bump
commit d438534cf4
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 18:53:20 2014 -0400
throw classcastexception when ES abuses regular filtercache for nested docs
commit 2c751f3a8f
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 18:31:34 2014 -0400
bump lucene revision, fix tests
commit d6ef7f6304
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 22:37:58 2014 +0100
fix merge problems
commit de9d361f88
Merge: 41f6aabf6b37a3
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 22:28:59 2014 +0100
Merge branch 'master' into enhancement/lucene_5_0_upgrade
Conflicts:
pom.xml
src/main/java/org/elasticsearch/Version.java
src/main/java/org/elasticsearch/gateway/local/state/meta/MetaDataStateFormat.java
commit 41f6aab388
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 17:48:46 2014 +0100
fix potiential NPE
commit c4428b12e1
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 17:38:46 2014 +0100
don't advance iterator in a match(doc) method
commit 28ab948e99
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 17:34:58 2014 +0100
don't advance iterator in a match(doc) method
commit eb0f33f663
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 16:55:54 2014 +0100
fix GeoUtilsTest
commit 7f711fe3ea
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 16:43:16 2014 +0100
Use a dedicated default index option if field type is not indexed by default
commit 78e3f37ab7
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 10:56:14 2014 -0400
disable this test with AwaitsFix to reduce noise
commit 9a590f563c
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 09:38:49 2014 +0100
fix lucene version
commit abe3ca1d8b
Author: Simon Willnauer <simonw@apache.org>
Date: Thu Oct 30 09:35:05 2014 +0100
fix AnalyzingCompletionLookupProvider to wrok with new codec API
commit 464293b245
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 00:26:00 2014 -0400
don't try to write stuff to tests class directory
commit 031cc6c19f
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 00:12:36 2014 -0400
AwaitsFix these known issues to reduce noise
commit 4600d51891
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 30 00:06:53 2014 -0400
openbitset lives on
commit 8492bae056
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 23:42:54 2014 -0400
fixes for filter tests
commit 31f24ce4ef
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 23:12:38 2014 -0400
don't use fieldcache
commit 8480789942
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 23:04:29 2014 -0400
ancient index no longer supported
commit 02e78dc7eb
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 23:37:02 2014 +0100
fix more tests
commit ff746c6df2
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 23:08:19 2014 +0100
fix all mapper
commit e4fb84b517
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 22:55:54 2014 +0100
fix distributor tests and cut over to FileStore API
commit 20c850e2cf
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 22:42:18 2014 +0100
use DOCS_ONLY if index=true and current options == null
commit 44169c1084
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 22:33:36 2014 +0100
Fix index=yes|no settings in mappers
commit a3c5f77987
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 21:51:41 2014 +0100
fix several field mappers conversion from setIndexed to indexOptions
commit df84d73690
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 21:33:35 2014 +0100
fix SourceFieldMapper to be not indexed
commit b2bf01d12a
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 21:23:08 2014 +0100
Cut over to .liv files in store and corruption tests
commit 619004df43
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 17:05:52 2014 +0100
fix more tests
commit b7ed653a8b
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 16:19:08 2014 +0100
[STORE] Add dedicated method to write temporary files
Recovery writes temporary files which might not end up in the
right distributor directories today. This commit adds a dedicated
API that allows specifying the target file name in order to create the
tempoary file in the correct directory.
commit 7d574659f6
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 10:28:49 2014 -0400
add some leniency to temporary bogus method
commit f97022ea7c
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 10:24:17 2014 -0400
fix MultiCollector bug
commit b760533128
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:56:08 2014 +0100
CheckIndex is now closeable we need to close it
commit 9dae9fb6d6
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:45:11 2014 +0100
s/Lucene51/Lucene50
commit 7aea9b8685
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:42:30 2014 +0100
fix BloomFilterPostingsFormat
commit 16fea6fe84
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:41:16 2014 +0100
fix some codec format issues
commit 3d77aa97dd
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:30:43 2014 +0100
fix CodecTests
commit 6ef823b1fd
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:26:47 2014 +0100
make it compile
commit 9991eee1fe
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 09:12:43 2014 -0400
add an ugly hack for TopHitsAggregator for now
commit 03e768a01f
Author: Simon Willnauer <simonw@apache.org>
Date: Wed Oct 29 14:01:02 2014 +0100
cut over ES090PostingsFormat
commit 463d281faa
Merge: 0f8740a8eac79c
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 08:30:36 2014 -0400
Merge branch 'master' into enhancement/lucene_5_0_upgrade
commit 0f8740a782
Author: Robert Muir <rmuir@apache.org>
Date: Wed Oct 29 01:00:15 2014 -0400
fix/hack remaining filter and analysis issues
commit df53448856
Author: Robert Muir <rmuir@apache.org>
Date: Tue Oct 28 23:11:47 2014 -0400
fix ngrams / openbitset usage
commit 11f5dc3b98
Author: Robert Muir <rmuir@apache.org>
Date: Tue Oct 28 22:42:44 2014 -0400
hack over sort comparators
commit 4ebdc75435
Author: Robert Muir <rmuir@apache.org>
Date: Tue Oct 28 21:27:07 2014 -0400
compiler errors < 100
commit 2d60c9e29d
Author: Robert Muir <rmuir@apache.org>
Date: Tue Oct 28 03:13:08 2014 -0400
clear some nocommits around ram usage
commit aaf47fe6c0
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 12:27:34 2014 -0400
migrate fieldinfo handling
commit ef6ed6d15d
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 12:07:13 2014 -0400
more simple fixes
commit f475e1048a
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 11:58:21 2014 -0400
more fielddata ram accounting fixes
commit 16b4239eaa
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 16:47:32 2014 +0100
add missing file
commit 5b542fa2a6
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 16:43:29 2014 +0100
cut over completion posting formats - still some nocommits
commit ecdea49404
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 11:21:09 2014 -0400
fielddata accountable fixes
commit d43da26571
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 16:19:53 2014 +0100
cut over BloomFilterPostings to new API
commit 29b192ba62
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 10:22:51 2014 -0400
fix more analyzers
commit 74b4a0c528
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 09:54:25 2014 -0400
fix tests
commit 554084ccb4
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 14:51:48 2014 +0100
maintain supressed exceptions on CorruptIndexException
commit cf882d9112
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 14:47:17 2014 +0100
commitOnClose=false
commit ebb2a9189a
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 14:46:06 2014 +0100
cut over indexwriter closeing in InternalEngine
commit cd21b3d470
Author: Simon Willnauer <simonw@apache.org>
Date: Mon Oct 27 14:38:10 2014 +0100
fix constant
commit f93f900c4a
Author: Robert Muir <rmuir@apache.org>
Date: Mon Oct 27 09:50:49 2014 -0400
fix test
commit a9a752940b
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Mon Oct 27 09:26:18 2014 +0100
Be explicit about the index options
commit d9ee815bab
Author: Simon Willnauer <simonw@apache.org>
Date: Sun Oct 26 20:03:44 2014 +0100
cut over store and directory
commit b3f5c8e390
Author: Robert Muir <rmuir@apache.org>
Date: Sun Oct 26 13:08:39 2014 -0400
more test fixes
commit 8842f2684e
Author: Robert Muir <rmuir@apache.org>
Date: Sun Oct 26 12:14:52 2014 -0400
tests manual labor
commit c43de5aec3
Author: Robert Muir <rmuir@apache.org>
Date: Sun Oct 26 11:04:13 2014 -0400
BytesRef -> BytesRefBuilder
commit 020c0d087a
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Sun Oct 26 15:53:37 2014 +0100
Moved over to BitSetFilter
commit 48dd1b909e
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Sun Oct 26 15:53:11 2014 +0100
Left over Collector api change in ScanContext
commit 6ec248ef63
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Sun Oct 26 15:47:40 2014 +0100
Moved indexed() over to indexOptions != null or indexOptions == null
commit 9937aebfd8
Author: Martijn van Groningen <martijn.v.groningen@gmail.com>
Date: Sun Oct 26 13:26:31 2014 +0100
Fixed many compile errors. Mainly around the breaking Collector api change in 5.0.
commit fec32c4abc
Author: Robert Muir <rmuir@apache.org>
Date: Sat Oct 25 11:22:17 2014 -0400
more easy fixes
commit dab22531d8
Author: Robert Muir <rmuir@apache.org>
Date: Sat Oct 25 09:33:41 2014 -0400
more progress
commit 414767e9a9
Author: Robert Muir <rmuir@apache.org>
Date: Sat Oct 25 06:33:17 2014 -0400
more progress
commit ad9d969fdd
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 24 14:28:01 2014 -0400
current state of fun
commit 464475eecb
Author: Robert Muir <rmuir@apache.org>
Date: Fri Oct 24 11:42:41 2014 -0400
bump to 5.0 snapshot
Don't insist on log file removal until after usage is printed.
Some simple Python code improvements (x.find(y) != -1 --> y in x)
Make sure the git area is "clean" (has no unpushed changes, has pulled
all changes, has no untracked files)
Add label color detail when creating next github version label.
Closes#7913
Lucene will soon release official 4.10.1, but by upgrading sooner we can 1) sidestep the false failures due to the 1.8.0_20 JVM hotspot bug (has caused a number of false failures in recent Jenkins tests), 2) make sure none of the Lucene changes in 4.10.1 are problematic.
Closes#7844
The upgrade tests required to specify the lower and the upper version
that it should run against. This commit adds support for random picks
if either the lower or the upper or both are not specified.
This commit adds a basic full cluster restart upgrade script that tests
upgrades of elasticsearch across major version. In contrast to the BWC
tests we are running using the java test framework this test uses a python
script as well as the REST APIs to ensure upgrades across major version work and
indices are compatible.
The upgrade test starts 2 or more nodes of an old elasticserach version, indexes
a random number of documents into the running nodes and executes a full cluster restart.
After the nodes are recovered a small set of basic checks are executed to ensure all
documents are still searchable and field data can be loaded etc.
Closes#7074
This commit adds the ability to run bwc tests during the release
process to ensure the current release is backwards compatible with
the latest installed previous version. Installed means available
in the configured bwc test path.
Closes#6953
Sandboxes the groovy scripting language with multiple configurable
whitelists:
`script.groovy.sandbox.receiver_whitelist`: comma-separated list of string
classes for objects that may have methods invoked.
`script.groovy.sandbox.package_whitelist`: comma-separated list of
packages under which new objects may be constructed.
`script.groovy.sandbox.class_whitelist` comma-separated list of classes
that are allowed to be constructed.
As well as a method blacklist:
`script.groovy.sandbox.method_blacklist`: comma-separated list of
methods that are never allowed to be invoked, regardless of target
object.
The sandbox can be entirely disabled by setting:
`script.groovy.sandbox.enabled: false`
This commit adds checks for nocommit and tabs in the source code.
The task is executed during the validate phase and can be disabled via
`-Dvalidate.skip`
In our REST tests we already have support for features and skip sections that allow to skip tests if a feature is not supported.
We can then add a skip section based on the benchmark feature to the benchmark tests and execute them only when they are supported, knowing that they need at least a node with node.bench settings within the cluster. We can check that this requirement is met by calling the nodes info api.
This way we can dynamically decide whether to execute those tests or not and we don't need to have a node.bench around all the time. In fact, given that the REST tests use the GLOBAL cluster, we want to be able to randomize settings as much as possible and run tests against default settings as well. Also, this mechanism can be easily supported by the external cluster implementation that is used during the release process.
Introduced ability to disable benchmark nodes which is needed by BenchmarkNegativeTest.
ElasticsearchRestTests extends now ElasticsearchIntegrationTest and makes use of our ordinary test infrastructure, in particular all randomized aspects now come for free instead of having to maintain a separate (custom) tests runner
We previously parsed only the tests that needed to be run given the version of the cluster the tests are running against. This doesn't happen anymore as it didn't buy much and it would be harder to support as the tests get now parsed before the test cluster gets started. Thus all the tests are now parsed regardless of their skip sections, afterwards the ones that don't need to be run will be skipped through assume directives.
Fixed REST tests that rely on a specific number of shards as this change introduces also random number of shards and replicas (through randomIndexTemplate)
Closes#5654
Elasticsearch is release from release-branches but the modifications
to the documentation must be cherry-picked into the current development
branch. To make this easier this commit splits the commits of the
Version and the documenation into seperate commits.
Just call ./es_release_notes.pl <issue-tag> to get all release notes.
By default html output is returned, but you can switch to markdown by calling
./es_release_notes.pl <issue-tag> markdown
Local mode modification done previously faulty. env[‘WORKSPACE’ is not
the sufficient discriminator to see if script is running under Jenkins.
This fails on the Jenkins parent jobs since those type of jobs don’t
have WORKSPACE set.
We currently run always with SecurityManager installed. To make sure we
work also without we should randomly swap it out ie. run without the
security manager.
this commit allows to run the release tool for smoke
testing without being on the actually released branch.
This commit also added a list of plugins that will be installed
for smoke testing to see if the plugin startup mechanism works
correctly.
the build_release.py tool now also downloads and verfyfies the
released packages from S3. It checks integrity based on the sha1
checksums and runs the smoketest against the specs in the current branch.
Currently when importing projects into eclipse you need to run 'mvn
eclipse:eclipse' on the command line to generate the poject files. This
means that when the pom changes you need to re-run the command on the
command line to reflect those changes in the project in eclipse. This
commit allows the developer to import the project as an existing maven
project (can be shared using git after import) and then allows the
application to be run inside eclipse using the .launch file in
/dev-tools enabling easy debugging of the application within eclipse
without requiring a maven build.
1) add build_desc property variable
2) use build_id plus build_number as an unique file name
3) tools subdirectory should be under current PWD when the script is
executing
If RPM tools are not installed the release tool now fails with an
appropriate message. The tool now also fails if any of the required
artifacts is not present.
We use 'coming[x.y.z]' in our ref docs which needs to be updated
when doing a release to 'added[x.y.z]' This commit adds support
for replacing the references where applicable during preparing the release.
This tool builds a release and runs several checks to make sure the
release is in a reasonable shape (smoke test). From a top level
perspective it runs the following steps:
* clean the build environment `mvn clean`
* check if a Java 6 JDK is available
* run the tests with network and local
* generates the checksums for the binary packages
* uploads the binary packages to Amazon S3
* runs a 'mvn deploy' to publish the maven artifacts
The script will create an intermediate branch from a given 'release
branch' updates all versions based on the version we are currently
releasing. Updates the 'pom.xml' file as well as the 'Version.java'
class. Once this is done it commits the changes and rebase with the
branch we want to release from, merges the changes from the intermediate
branch and pushes to the given remote repository including the release
tag.
Alexander Reelsen
Simon Willnauer
Ryan Ernst
Igor Motov
Robert Muir
Michael McCandless
Veres Lajos
Lee Hinman
Clinton Gormley
Bill Hwang
Andrew Raines
javanna
David Pilato
b.hwang
colings86