* Fix ExceptionSerializationTests to use getCodeSource instead of getResource.
Using getResource makes this sensitive to unrelated classpath entries,
such as the entitlement bridge library, that get prepended to the classpath.
* FIx logging tests to use org.elasticsearch.index instead of root logger.
Using the root logger makes this sensitive to unrelated
logging, such as from the entitlement library.
* Fix entitlement error message by stashing the module name in ModuleEntitlements.
Taking the actual module name from the class doesn't work in tests,
where those classes are loaded from the classpath and so their module
info is misleading.
* Ignore server locations whose representative class isn't loaded
* Partial initial implementation
* System properties: testOnlyClasspath and enableForTests
* Trivially allow some packages
* DEBUG: use TreeMap in TestScopeResolver for readability
* Special case bouncycastle for security plugin
* Add CONFIG to TestPathLookup
* Add the classpath to the source path list for every plugin
* Add @WithoutEntitlements to tests that run ES nodes
* Set es.entitlement.enableForTests for all libs
* Use @WithoutEntitlements on ingest plugin tests
* Substitute ALL-UNNAMED for module name in non-modular plugins
* Add missing entitlements found by unit tests
* Comment in TestScopeResolver
* Properly compute bridge jar location for patch-module
* Call out nonServerLibs
* Don't build two TestPathLookups
* More comments for meta-tests
* Remove redundant dependencies for bridgeJarConfig.
These are alread set in ElasticsearchJavaBasePlugin.
* Add bridge+agent dependencies only if those exist.
For serverless, those project dependencies don't exist, and we'll need
to add the dependencies differently, using Maven coordinates.
* [CI] Auto commit changes from spotless
* Pass testOnlyPath in environment instead of command line.
It's typically a very very long string, which made Windows angry.
* [CI] Auto commit changes from spotless
* Split testOnlyPathString at File.pathSeparator
* Use doFirst to delay setting testOnlyPath env var
* Trivially allow jimfs (??)
* Don't enforce entitlements on internalClusterTest for now
* Replace forbidden APIs
* Match testOnlyClasspath using URI instead of String.
We already get the "needle" in the form of a URI, so this skips
a step, and has the benefit of also working on Windows.
* [CI] Auto commit changes from spotless
* More forbidden APIs
* Disable configuration cache for LegacyYamlRestTestPluginFuncTest
* Strip carriage-return characters in expected output for ReleaseNotesGeneratorTest.
The template generator also strips these, so we need to do so to make this pass
on Windows.
Note that we use replace("\r", "") where the template generator uses
replace("\\r", ""). The latter didn't work for me when I tried it on Windows,
for reasons I'm not aware of.
* Move configureEntitlements to ElasticsearchTestBasePlugin as-is
* Use matching instead of if
* Remove requireNonNull
* Remove default configuration
* Set inputs instead of dependencies
* Use test.systemProperty
* Respond to PR comments
* Disable entitlement enforcement for ScopedSettingsTests.
This test works by altering the logging on the root logger.
With entitlements enabled, that will cause additional log statements to appear,
which interferes with the test.
* Address PR comments
* Moritz's configureJavaBaseModuleOptions
* Allow for entitlements not yet enforced in serverless
* fix entitlementBridge config after rename
* drop empty file collections
* Remove workaround in LegacyYamlRestTestPluginFuncTest
---------
Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Lorenzo Dematté <lorenzo.dematte@elastic.co>
Co-authored-by: Moritz Mack <mmack@apache.org>
When return an error from Elasticsearch exceptions may contain values
written as http response headers. ElasticsearchException contains a map
of headers that are added to the response. But these values are also
written to a special "header" section of the response body.
This commit renames the existing "headers" in ElasticsearchException to
"body headers", which are both http headers and written to the response
body. A new "http headers" is added for headers that should only be
written as response headers.
This brings in the fixes from #130020, with minor fixes to address review
nits from that PR.
Co-authored-by: Martijn van Groningen <martijn.v.groningen@gmail.com>
Initial version of patterned_text mapper. Behaves similarly to match_only_text. This version uses a single SortedSetDocValues for a template and another for arguments. It splits the message by delimiters, the classifies a token as an argument if it contains a digit. All arguments are concatenated and inserted as a single doc value. A single inverted index is used, without positions. Phrase queries are still possible, using the SourceConfirmedTextQuery, but are not fast.
- loads the downloaded GeoIP databases from system index to ingest node file system for each project
- each project's databases are loaded to directory `tmp/geoip-databases/{nodeId}/{projectId}`
The vast majority of ingest pipelines are light CPU
operations. We don't want these to be put behind IO work on the write
executor. Instead, execute these on the coordination pool.
Occasional shard allocation issues were causing the YAML tests to fail
because the shard that had the document with the max timestamp in it
would be unavailable.
Fixes#118217
This change makes the GeoIp persistent task executor/downloader multi-project aware.
- the database downloader persistent task will be at the project level, meaning there will be a downloader instance per project
- persistent task id is prefixed with project id, namely `<project-id>/geoip-downloader` for cluster in MP mode
* Enable And Disable Endpoint
* Status Endpoint
* Integration Tests
* REST Spec
* REST Spec tests
* Some documentation
* Update docs/changelog/129474.yaml
* Fix failing security test
* PR Fixes
* PR Fixes - Add missing feature flag name to YAML spec
* PR Fixes - Fix support for timeout and master_timeout parameters
* PR Fixes - Make the REST handler validation happy with the new params
* Delete docs/changelog/129474.yaml
* PR Fixes - Switch to local metadata action type and improve request handling
* PR Fixes - Make enable / disable endpoint cancellable
* PR Fixes - Switch timeout param name for status endpoint
* PR Fixes - Switch timeout param name for status endpoint in spec
* PR Fixes - Enforce local only use for status action
* PR Fixes - Refactor StreamsMetadata into server
* PR Fixes - Add streams module to multi project YAML test suite
* PR Fixes - Add streams cluster module to multi project YAML test suite
Our hypothesis is that the path is not round-tripping on Windows. This
fixes the test by always using the canonical string for the path, when
inserting and when purging.
Fixes#129635
The stateful side tests generally do not exercise multi-project.
Therefore we need to use the default project-id instead of random one.
Relates: #127631
These usages had the potential of causing test failures when a data
stream was created before midnight and the backing index name generation
ran the next day - which would be millisecconds apart. To avoid these
failures, we update the tests to be robust to these time differences.
Resolves#123376
We added a new `projectClient` method on `Client` in #129174. We now
update the usages of the old method (on `ProjectResolver`) to use the
new one and we delete the old method.
This PR updates some of the existing checks for cluster blocks to also consider project global blocks. It adds a few project-aware flavour of existing methods in `ClusterBlocks`. `globalBlockedException(ClusterBlockLevel)` is the mostly used one. I've updated only some of the obvious ones here.
Follow up to https://github.com/elastic/elasticsearch/pull/127978
Relates ES-11209
We originally defined the `projectClient` method on `ProjectResolver` as
a convenience method to execute API calls for specific projects. That
method requires a reference to both a `ProjectResolver` and a `Client`.
We now introduce the same method directly on the `Client` interface and
inject a `ProjectResolver` there, removing the need for a
`ProjectResolver` reference in places that just want to execute API
requests on a specific project.
To reduce the number of changes, this change solely focuses on
introducing the new method. Future changes will migrate the uses of the
original method to the new one and remove the original altogether.
Don't store text and match_only_text field by default when source mode is synthetic and a field is a multi field or when there is a suitable multi field.
Without this change, ES would store field otherwise twice in a multi-field configuration.
For example:
```
...
"os": {
"properties": {
"name": {
"ignore_above": 1024,
"type": "keyword",
"fields": {
"text": {
"type": "match_only_text"
}
}
}
...
```
In this case, two stored fields were added, one in case for the `name` field and one for `name.text` multi-field.
This change prevents this, and would never store a stored field when text or match_only_text field is a multi-field.
Keith Massey
Yang Wang
Quentin Pradet
Jordan Powers
Patrick Doyle
Ryan Ernst
Martijn van Groningen
Sam Xiao
Tim Brooks
Joe Gallo
Parker Timmins
Brendan Cully
David Turner
Niels Bauman
Luke Whiting
Mikhail Berezovskiy
Pete Gillin
Pooya Salehi
Moritz Mack
Simon Cooper
Ignacio Vera
Luigi Dell'Aquila