* Don't remove warning headers on all failure
We shouldn't remove warning when request is failing not because of
security reasons (syntax error for ex.).
Note, that security related failure could happen not only during
authentication (therefore we will check for the rest status), also all
failures happened during authentication will be considered security
related and warnings will be removed from the response.
Resolves: #75739
Adds new field to recovery API to keep track of amount of data
recovered from snapshots.
The normal recovered_bytes field remains and is also increased for
recovery from snapshot but can go backwards in the unlikely case
that recovery from snapshot fails to download a file.
Relates #73496
This removes "wildcard" as an available function in EQL. This has
already been replace with "like" and "regex" embedded syntax (and
respective case insensitive variants).
Licensed feature tracking utilizes the existing license level checks to
track when a feature is used. However, some features check the license
level at the start of an operation or when enabling a feature, but then
the tracking only captures the beginning time.
This commit reworks the licensed feature framework to use a new
LicensedFeature class which will eventually replace
XPackLicenseState.Feature values. There are two LicensedFeature
implementations, one for "momentary" features that are tracked just at
the moment they are used, and "persistent" features that are considered
"on" until the feature is untracked. The usage map of tracked features
is cleaned up every hour, and those features that have not been used in
the last 24 hours are removed from tracking.
Not all features are converted to LicensedFeature yet. Instead, a few
features have been converted to demonstrate how it can be done, so that
the rest can be done in parallel at a future time.
Co-authored-by: Tim Vernum <tim@adjective.org>
Add system data streams to the "snapshot feature state" code block, so that if
we're snapshotting a feature by name we grab that feature's system data streams
too. Handle these data streams on the restore side as well.
* Add system data streams to feature state snapshots
* Don't pass system data streams through index name resolution
* Don't add no-op features to snapshots
* Hook in system data streams for snapshot restoration
This fixes the error message in Painless for when an instance-style method isn't found. It now prints the
number of arguments instead of the memory location of the nodes data structure.
Removes the deprecated kibana_dashboard_only_user from the set of reserved roles in Elasticsearch as this legacy functionality is being removed from Kibana.
Relates: elastic/kibana#54755.
This commit extends the SSL diagnostics message to include descriptions of the
- The KeyUsage and ExtendedKeyUsage of the peer's certificate
- The CipherSuite & Protocol (TLS/SSL version) of the current session
These can be helpful in diagnosing SSL errors.
Co-authored-by: Tim Vernum <tim@adjective.org>
Randomly add to use a snapshot for recovery to searchable snapshot and
snapshot tests to verify that recover from snapshot does not break other
features (those should not care about the flag).
Relates #76237
Outbound responses would not get the expected `decRef`, resulting in
memory and/or circuit breaker leaks. In particular, the
`GetCcrRestoreFileChunkResponse` expects this, causing a leak when
a follower bootstraps.
Relates #65921
This PR adds support for sort and pagination similar to those used with
regular search API. Similar to the query field, the sort field also
supports only a subset of what is available for regular search.
A recent commit added an assertion to make sure that LZ4 is not
configured on an incompatible node. This assertion is currently broken
in tests. This commit removes it.
This is related to #73497. Currently, we only use the configured
transport.compression_scheme setting when compressing a request or a
response. Additionally, the cluster.remote.*.compression_scheme
setting is ignored. This commit fixes this behavior by respecting the
per-cluster setting. Additionally, it resolves confusion around inbound
and outbound connections by always responding with the same scheme that
was received. This allows remote connections to have different schemes
than local connections.
Painless extensions now compile against it's `spi` subproject. Gradle
and IntelliJ rig the dependencies correctly but Eclipse doesn't seem to
manage it. This temporarily declares the `spi` as a `compileOnly`
dependency when in Eclipse land so Eclipse can see the classes.
This change introduces a Service Account for Kibana to use when
authenticating to Elasticsearch. The Service Account with
kibana service name under the elastic namespace,
uses the same RoleDescriptor as the existing kibana_system
built-in user and is its functional equivalent when it comes to
AuthZ.
* Mention match_only_text in disk usage docs
Previously we explained how to manually disable norms, freqs, and positions. We
now have a ready-made solution in the new `match_only_text` field type.
* Fixing typo and minor grammar changes
Co-authored-by: Adam Locke <adam.locke@elastic.co>
* Update redline library to 1.2.10
The redline team just released version 1.2.10 of the redline
library which contains our fix of the rpm signatures / headers.
Also a PR to update that dependency in the ospackage plugin has been
raised at https://github.com/nebula-plugins/gradle-ospackage-plugin/pull/402
* Update common about enforcing redline 1.2.10
On the "Size your shards" page, the shard size recommendation assumes a time
series use case. Similarly, users shouldn't count nearly empty and rarely used
Enterprise Search system indices against the recommended shard count limit.
Closes#76328.
This commit adds peer recoveries from snapshots. It allows establishing a replica by downloading file data from a snapshot rather than transferring the data from the primary.
Enabling this feature is done on the repository definition. Repositories having the setting `use_for_peer_recovery=true` will be consulted to find a good snapshot when recovering a shard.
Relates #73496
This change increases the logging level for LDAP failures in 3 cases:
1. When a search fails, the failure details are now logged at DEBUG
rather than TRACE. The errors are useful for diagnostic purposes and
it should not be necessary to turn on TRACE to see them
2. When a failure occurs when attempting to bind using the configured
"bind_dn" and password, this is now a WARN rather than DEBUG. Failures
for user supplied credentials (e.g. basic auth) are still logged as
DEBUG because these are typically not under the control of the cluster
administrator.
3. When a failure occurs while attempting to retrieve a connection
from an LDAP connection pool, this will be logged at a WARN level.
This is almost always an error that the cluster administrator should
be aware of and seek to resolve.
In some cases this may cause 2 sets of log messages (one from LDAP and
one from the authentication service) however it is not always the case
that both are logged, and even when they are they may have different
levels of detail (for example, the authentication service message does
not always include the DN of the bind user).
When running in FIPS mode, (fips_mode.enabled: true), the default
password hasher is now "pbkdf2_stretch"
In non-FIPS mode the default is still "bcrypt"
In 7.x and earlier, the default hasher was always "bcrypt"
In 8.0-alpha1, the default hasher on FIPS was "pbkdf2"
Resolves: #66840
In 7.15, we intend for the indexing_data compression level and the
compression scheme lz4 to no longer be experimental. This commit
updates the documentation to reflect this. Additionally, it adds
missing docs for the cluster.remote.*.transport.compression_scheme
setting.
Relates to #73497.
Fix docCountError calculation in case of multiple reduces. It fixes 2 mistakes
in #43874. The first error was introduced in the original PR, where unknown doc
count errors were initialized equal to 0, the second was introduced during in
order to fix the first one by ignoring these 0s, which essentially disabled the
original fix.
Fixes#75667
With recent fixes it is never correct to simply remove a snapshot from the cluster state without
updating other snapshot entries if an entry contains any successful shards due to possible dependencies.
This change reproduces two issues resulting from simply removing snapshot without regard for other queued
operations and fixes them by having all removal of snapshot from the cluster state go through the same
code path.
Also, this change moves the tracking of a snapshot as "ending" up a few lines to fix an assertion about finishing
snapshots that forces them to be in this collection.
* Name `BulkItemResponse` ctors
`BulkItemResponse` can contain either a success or failure. This
replaces the two constructors used to build either case with named
static methods. So instead of
```
return new BulkItemResponse(0, OpType.CREATE, createResponse);
return new BulkItemResponse(0, OpType.CREATE, failure);
```
you now use
```
return BulkItemResponse.success(0, OpType.CREATE, createResponse);
return BulkItemResponse.failure(0, OpType.CREATE, failure);
```
This makes it marginally easier to read code building these things - you
don't have to know the type of the parameter to know if its a failure
or success.
* Consistent
* Mock response
This change creates a standard for loading additional allow listed elements for the core contexts
within the ScriptPlugin. For each core context we look for org.elasticsearch.script.<context-name>.txt
and if it exists we load it automatically for that context.
There are two possible race conditions that were not
previously handled in this test.
- Since the syncconfig was null, it may be that the
transform actually gets set to stopping/stopped
and its unable to kick off another indexing pass
- It may also be that the indexer thread is still
finishing up work when the second execution is
requested, so it returns false.
Adding a sync config and assertBusy handles these
cases. Ran 1k+ times locally with this change
and it never failed. Without, it failed ~10 runs.
closes https://github.com/elastic/elasticsearch/issues/76397
Igor Motov
Przemysław Witek
Lyudmila Fokina
Ryan Ernst
Francisco Fernández Castaño
Bogdan Pintea
Seth Michael Larson
William Brafford
Jack Conradson
bellengao
Devon Thomson
hanbj
sindhusp
Henning Andersen
Armin Braun
Yang Wang
Tim Brooks
James Rodewig
Nik Everett
Ioannis Kakavas
Julie Tibshirani
Rene Groeschke
Adam Locke
István Zoltán Szabó
Tim Vernum
Costin Leau
Benjamin Trent