Commit Graph

72833 Commits

Author SHA1 Message Date
Przemysław Witek db80251fb9
[Transform] Make GetCheckpointAction and GetCheckpointNodeAction time out (#101055) 2023-10-25 13:15:01 +02:00
Lorenzo Dematté 77dac65761
Fix NodeInfo version parsing in bwc tests (#100838)
* Mixed cluster tests with string NodeInfo version

- Move version based feature comparison to a common, deprecated method (to be replaced with real features)
- Use string comparison against old cluster version to partition new/old cluster nodes
2023-10-25 12:59:10 +02:00
Andrei Dan 74ea04fb2d
[DOCS] document tail merging and create tutorial for migrating to DSL (#101117)
This documents tail merging, the enabled flag, and
adds a tutorial to migrate a data stream from ILM to DSL.
2023-10-25 11:12:36 +01:00
Felix Barnsteiner 75d9bd7790
Rename component templates and pipelines according to the new naming conventions (#99975)
- Creates a new StackTemplateRegistry that uses the new names
- The new registry only respects stack.templates.enabled for index templates
- Renames the old registry to LegacyStackTemplateRegistry
- Component templates are not duplicated but registered under two different names
- Documents the new naming convention
- Index templates are not renamed, at least for now, as there are some challenges with it
  See 7fd0423 for more details.
2023-10-25 11:56:28 +02:00
Bogdan Pintea 0c76b4d135
QL: Add error logging for *QL (#101057)
This adds logging for the *QL query failures.
Exceptions resulting in a 5xx-class response are logged on `WARN` level, otherwise `DEBUG`.
2023-10-25 11:52:10 +02:00
Simon Cooper 2f162a48ac
Unmute DeprecationHttpIT.testCompatibleMessagesCanBeIndexed (#101278)
This has been fixed by #101273
2023-10-25 10:50:35 +01:00
Lorenzo Dematté f878a8c308
Fix NodeInfo version parsing in integration tests (#100770)
* Compatible version parsing in YAML tests
* Compatible version parsing in various IT tests
2023-10-25 10:55:15 +02:00
David Turner 5bbfe66b99
Improve string reps for snapshot debugging (#101295)
Various things related to snapshots appear in debug logs, but have no
useful string representation which makes it hard to follow the process.
This commit adds some missing string representations.
2023-10-25 02:52:23 -04:00
Ignacio Vera 126412c8bf
Don't generate bounding box touching tiles in GeoTileGridAggregatorTests (#101137) 2023-10-25 08:35:02 +02:00
Martijn van Groningen 311185311f
Remove index.codec setting from setting up tsdb docs. (#101276)
This is not needed for tsdb, because of synthetic source and slows down indexing / refreshes.
2023-10-25 08:21:18 +02:00
Volodymyr Krasnikov 66f7298e83
Update doc as per SDH finding (#101285) 2023-10-24 15:17:34 -07:00
Mark Vieira 5446d88e3b
Include branch information in build scans for buildkite jobs (#101284) 2023-10-24 14:08:21 -07:00
Nik Everett 46f95a67b4
ESQL: More MV_* tests (#100564)
This adds more tests for some of the `MV_` functions and updates their
docs now that the railroad diagram and table generated by the tests
covers all of the types.
2023-10-24 16:55:17 -04:00
Simon Cooper 9f0a9bc949
Add an extra check to DeprecationHttpIT that the index is actually deleted (#101273)
The tests in DeprecationHttpIT are affecting each other - this adds a check the index is actually deleted between each test. This should stop the regular CI failures we see in DeprecationHttpIT.
2023-10-24 17:01:54 +01:00
Ignacio Vera 8a9f4fed55
Remove explicit SearchResponse references from LegacyGeo, Aggregations and parent-join modules (#101250) 2023-10-24 17:46:25 +02:00
Stuart Tettemer ecd13e3f11
Metrics test framework (#101168)
Adds a test framework that validates instruments are registered before they are called and are not double registered.

Also records all invocations of Instruments and allows test authors to add validation to instruments.
2023-10-24 09:05:16 -05:00
Daniel Mitterdorfer 9b29fa60d4
Align look-back with client-side cache (#101264)
We have recently introduced a 6 hour cache for executables (previously
lifetime in the cache was unbounded) in the host agent. With this commit
we align the look-back so it matched the client-side cache lifetime.
2023-10-24 16:03:27 +02:00
Andrei Stefan 4679b095a0
ESQL: mv_expand pushes down a limit copy and keeps the limit after it untouched (#100782)
- allow mv_expand to push down limit and project past it
- accept a limit after mv_expand when there is also a second limit before the mv_expand
- adds a default TopN for cases when there is only a sort at Lucene level
- adds OrderBy node type to the exceptions for duplicating the limit after mv_expand
2023-10-24 16:18:51 +03:00
Aurélien FOUCRET 29dbeb4693
Rewrite flaky tests (#101164) 2023-10-24 08:22:41 -04:00
David Turner 3031595d6a
Clean up action registration for repo analysis (#101211)
- Removes the registration of the inner actions via `getActions()`.
- Replace the outer action's `ActionType` subclass using `localOnly()`.
- Collapses each outer `Action` class with the inner `TransportAction`.
- Tightens up some unnecessary `public` visibility.

Closes #101198
2023-10-24 06:46:03 -04:00
David Turner b714549689
Improve cancellation in repo analysis (#101213)
Today we rely on an `isRunning` check to check for task cancellation,
but since #82685 we can actively record the failure arising from the
cancellation using a `CancellationListener`.

Closes #101197
2023-10-24 10:59:06 +01:00
Lorenzo Dematté 482cf3b676
Reapply "Making yaml tests version selector parser compatible with versions returned by Build" (#100953)
* Compatible version parsing in YAML tests
* Propagate exception in case of non-semantic version where one is expected
* Removed remove of SNAPSHOT (no longer needed)
2023-10-24 11:24:32 +02:00
Luca Cavanna b07feb507d
Percolator to support parsing script score query with params (#101051)
While dot expansion is disabled when parsing percolator queries at index
time, as that would interfere with query parsing,  we still use a wrapper parser
that is conservative about what methods it supports, assuming that
document parsing needs nextToken and not much more. Turns out that when
parsing queries instead, we need to support all the XContentParser
methods including map, list etc.

This commit adds a test for script score query parsing through document
parsing via percolator field mapper, and removes the limitations in the
wrapper parser when dots expansion is disabled.
2023-10-24 11:03:28 +02:00
Yang Wang aa30dad01f
S3 CAS operation should respect abortMutipartUpload failure (#101253)
We inadvertently made s3 CAS operation to ignore abortMutipartUpload
failures in #98664. This PR fixes it.
2023-10-24 04:37:05 -04:00
Daniel Mitterdorfer c03ebd6314
Provide stable resampling (#101255)
We resample data randomly if required. So far we have initialized the
random number generator based on the hash code of the request with the
intent of providing a random resampling that is still stable if the same
request is issued multiple times. However, the hash code was not stable
in a cluster because a query may use Lucene's `ByteRef` class to store
values (such as the upper and lower bound of a date range). That class
uses a murmur hash for its hash code. The murmur hash is initialized
from `org.apache.lucene.util.StringHelper#GOOD_FAST_HASH_SEED` which
intentionally varies across JVM instances. Consequently, the hash code
of `ByteRef` (and ultimately the request's hash code) varies depending
on which node in the cluster handles a request.

With this commit we instead rely on the string representation of a
query, which is stable across instances and node restarts to initialize
the random number generator. This provides randomness across requests
but also a consistent result for identical requests. Converting the
query builder to its string representation adds around 1ms of overhead.
Given that typical response times are in the range of single digit
seconds, we deem this overhead acceptable.
2023-10-24 10:30:05 +02:00
David Turner 2757e30010
Make S3 anti-contention delay configurable (#101245)
The anti-contention delay in the S3 repository's compare-and-exchange
operation is hard-coded at 1 second today, but sometimes we encounter a
repository that needs much longer to perform a compare-and-exchange
operation when under contention. With this commit we make the
anti-contention delay configurable.
2023-10-24 08:13:55 +01:00
Lorenzo Dematté b094268961
Remove not needed node selector from some aggregation tests (#100947) 2023-10-24 08:58:29 +02:00
Ievgen Degtiarenko 215d3e4299
Set longer settings update task timeout (#101208)
It appears that task cancelation is executed before the settings update is
event starting in testClusterSettingsUpdateNotAcknowledged. This change uses
longer timeout to improve the probability of blocking.
2023-10-24 08:28:03 +02:00
Ryan Ernst b0fb121640
Make node client type setting a noop (#101214)
The node client type is a remnant of the transport client. This commit
cleans up some test reads and an unnecessary override of the setting. It
was already not read anywhere in production. Now it is only registered
in order to provide validation. In the future it should be deprecated
and removed.
2023-10-23 20:11:58 -04:00
Brian Seeders 6e6e85d0b4
[ci] Disable periodic java-matrix, java-fips-matrix, and bwc jobs in Jenkins (#101234) 2023-10-23 16:20:14 -04:00
Brian Seeders 24ef517355
[buildkite] Do collapsing annotations for Terrazzo pipelines as well 2023-10-23 16:19:09 -04:00
Brian Seeders 4d10ea1849
[buildkite] Increase release-tests timeout 2023-10-23 16:05:10 -04:00
AlexB 48f1e4cf89
ESQL small doc improvement (#101226)
Move command list to the top

Co-authored-by: Alexandros Batsakis <abatsakis@splunk.com>
2023-10-23 12:23:58 -07:00
Stuart Tettemer d8b2c52c82
Metrics refactor - split registry and service (#101154)
This splits out the registry and the service, which makes testing easier and removes much of the delegation from the old `APMMeter` to `Instruments` (now renamed `APMMeterRegistry`).

APMMeterService takes care of the lifecycle and APMMeterRegistry holds the instruments.
2023-10-23 13:28:46 -05:00
Armin Braun 3945ee75d0
Use assertAcked more (#101201)
Just found that we have a lot of inconsistency and needless verbosity
here in tests. We can just use `assertAcked` in a couple spots
to save `.get`, `.actionGet` etc., especially with the signature
change I added here.
2023-10-23 19:48:10 +02:00
Rene Groeschke 84afa5cee3
Use gradle toolchain to resolve JDKs by BWC builds (#101224)
Fixes a problem when a versioned java home has been defined but not properly propagated
to the BWC build
2023-10-23 18:54:43 +02:00
AlexB 931dcae41d
Add improvements to the ES|QL docs (#101195)
Content and structural improvements to the ES|QL docs

---------

Co-authored-by: Alexandros Batsakis <abatsakis@splunk.com>
Co-authored-by: Abdon Pijpelink <abdon.pijpelink@elastic.co>
2023-10-23 07:45:42 -07:00
Martijn van Groningen af08d28d38
Fix painless execute api and tsdb issue. (#101212)
Today using painless execute api with tsdb index can fail with a `_id must be unset or set to [cn4exTOUtxytuLkQAAABeRnR_mY] but was [_id] because [test_index] is in time_series mode` error.
This change addresses this.

The painless execute api shouldn't set use a static _id, but
let the TsidExtractingIdFieldMapper generate it.
Otherwise validation TsidExtractingIdFieldMapper fails.

Closes #101072
2023-10-23 16:40:21 +02:00
Daniel Mitterdorfer a579504e11
Remove auto_configure privilege for profiling (#101026)
With this commit we remove the `auto_configure` privilege for the Fleet
service account that targets profiling-related indices. This privilege
was needed to automatically create indices and data streams in the past
but as this managed by the Elasticsearch plugin, there is no need to
grant this privilege to Fleet-managed components.
2023-10-23 13:14:14 +02:00
Daniel Mitterdorfer 2ef6bb3a90
Increase K/V look-back time interval (#101205)
With this commit we increase the look-back time interval from 3 hours to
4 hours by default. This look-back time interval is applied to determine
the correct K/V indices to query around a rollover. As the new index may
not have all data immediately after a rollover, we also need to query
the old index. Clients may cache data for up to 3 hours but to avoid
unlucky timing we add a bit of slack and increase the time interval to 4
hours.
2023-10-23 13:04:52 +02:00
David Turner 4bbf760cda
Repo analysis of uncontended register behaviour (#101185)
Today repository analysis verifies that a register behaves correctly
under contention, retrying until successful, but it turns out that some
repository implementations cannot even perform uncontended register
writes correctly which may cause endless retries in the contended case.
This commit adds another repository analyser which verifies that
uncontended register writes work correctly on the first attempt.
2023-10-23 11:46:30 +01:00
Rene Groeschke f7ba5efcb0
Fix generation of xcontent provider Manifest (#101200)
Fixes #101191
2023-10-23 06:35:02 -04:00
Artem Prigoda 13bf22367d
Add a latch to sync when all max upload tasks have been submitted (#101203)
Before we check the amount of active tasks on the prewarming executor,
we need to verify that all the tasks have been actually submitted.

Otherwise, we have a race in and amount of active tasks can be lower
then the amount of submitted tasks.

Fixes #99124

---------

Co-authored-by: David Turner <david.turner@elastic.co>
2023-10-23 12:05:38 +02:00
Ignacio Vera ea430ec97a
Remove explicit SearchResponse references from spatial module test code (#101196) 2023-10-23 11:40:08 +02:00
David Turner a1c1883a18
Rename RegisterAnalyzeAction to ContendedR... (#101192)
Relates #101185
2023-10-23 04:44:08 -04:00
Rene Groeschke f0ef872c2c
Update bundled JDK to 21.0.1 (#101133)
* Update docs/changelog/101133.yaml
2023-10-23 10:10:18 +02:00
David Turner cfb0780b7a
More robust timeout for repo analysis (#101184)
Replaces the transport-level timeout with an overall timeout on the
whole repository analysis task to ensure that all child tasks terminate
promptly.

Relates #66992 Closes #101182
2023-10-23 03:17:40 -04:00
Ignacio Vera 6b8790571a
Increase GeoHexGridTiler#FACTOR (#101142)
we added a more aggressive way to handle geotile boundaries which affects the correction factor for GeoHexGridTiler.
2023-10-23 07:47:12 +02:00
David Turner 9794c6e205
Use ESIntegTestCase#prepareSearch more (#101179)
The refactoring in #101175 only covered all the one-arg call sites. This
PR does the rest.
2023-10-20 18:33:00 +01:00
Armin Braun 1521484d11
Remove more explicit references to SearchResponse in tests (#101092) (#101172)
Follow-up to #100966.

Add more assertion overloads that consume a requestBuilder as in the
other PRs and start using `assertHitCount` in more places that were
duplicating what it does. Also add a shortcut for
`client().prepareSearch()` to integ tests and bulk-replace some usages
of this pattern to avoid these changes from blowing up test code line
count further.
2023-10-20 07:48:37 -04:00