the per type indexing stats is simplified and when _types is requested
it will return total stats for the index repeated under types/_doc/
the removal #47203
relates main meta issue #51816
relates types removal issue #54160
When a dynamic template is applied to a 7.x index, we do some validation checks
against the generated mappings and issue a deprecation warning if the validation
fails. We had some tests that were checking this at a low level, but nothing that
exercised the full logic.
When dynamic runtimes were added, we expected this behaviour to carry over;
however, a bug in building the ParserContext to pass to the runtime Builder meant
that we would instead always throw an error. In fact, erroring here is a good result,
as the only reason we issue a deprecation warning for non-runtime fields is to
preserve backward compatibility; given that runtime fields are new, it has never
been possible to add a bad dynamic runtime template.
This commit adds specific tests for both situations. It ensures that the parser
context passed to a runtime builder will know that it is in a dynamic context,
but it also removes the version check and deprecation warning, as they were
never actually triggered and we can be stricter here.
- Fix new introduced deprecated usages
- Update to newer ospackage snapshot to include provided PR for fixing deprecated usage
This gradle release comes with improvements on incremental compilation which we should benefit from
Pagination and snapshots for get snapshots API, build on top of the current implementation to enable work that needs this API for testing. A follow-up will leverage the changes to make things more efficient via pagination.
Relates https://github.com/elastic/elasticsearch/pull/73570 which does part of the under-the-hood changes required to efficiently implement this API on the repository layer.
Both of these APIs don't parse request bodies, the parameters are all taken
from the query string. Also, included the master timeout param include
as it was missing here also.
Alpine Linux 3.14.0 is incompatible with older versions of Docker, so pin the
version that we use to 3.13. At some point in the future, it will
be possible to upgrade Alpine.
Also when compiling curl, if the configure step fails and a config.log
file exists, then dump it out before exiting to assist diagnosis.
In #74138 we noted that index settings aren't copied in a clone. In fact
that's not true, we copy everything except explicitly-excluded ones,
`number_of_replicas` and `auto_expand_replicas`. This fixes the mistake.
Co-authored-by: James Rodewig <40268737+jrodewig@users.noreply.github.com>
This commit disables the query cache for the `FunctionScoreQuery` and the
`ScriptScoreQuery`. These queries are not meant to be cached. If the score
is not needed, we'll now cache the sub-query and filters independently since
we don't want to keep an unused script in the cache.
Closes#73925
Today if sending file chunks is CPU-bound (e.g. when using compression)
then we tend to concentrate all that work onto relatively few threads,
even if `indices.recovery.max_concurrent_file_chunks` is increased. With
this commit we fork the transmission of each chunk onto its own thread
so that the CPU-bound work can happen in parallel.
RuntimeField.Builder currently extends FieldMapper.Builder so that it can
share some parsing code and re-use the Parameter infrastructure. However,
this also means that we have to have a number of no-op method implementations,
and in addition this will make it complicated to add a fields parameter within
multi-keyed object field types. This commit removes the class relationship
between these two classes.
The metadata index is small and important and only read at startup.
Today we rely on Lucene to spot if any of its components is corrupt, but
Lucene does not necesssarily verify all checksums in order to catch a
corruption. With this commit we run `CheckIndex` on the metadata index
first, and fail on startup if a corruption is detected.
Closes#29358
The version field on all lucene Analyzers is unused, and is being removed
in lucene 9. This commit deprecates setting a version on an analyzer in
index settings and removes the related calls to Analyzer.setVersion()
Relates to #74057
Change the formatter config to sort / order imports, and reformat the
codebase. We already had a config file for Eclipse users, so Spotless now
uses that.
The "Eclipse Code Formatter" plugin ought to be able to use this file as
well for import ordering, but in my experiments the results were poor.
Instead, use IntelliJ's `.editorconfig` support to configure import
ordering.
I've also added a config file for the formatter plugin.
Other changes:
* I've quietly enabled the `toggleOnOff` option for Spotless. It was
already possible to disable formatting for sections using the markers
for docs snippets, so enabling this option just accepts this reality
and makes it possible via `formatter:off` and `formatter:on` without
the restrictions around line length. It should still only be used as
a very last resort and with good reason.
* I've removed mention of the `paddedCell` option from the contributing
guide, since I haven't had to use that option for a very long time. I
moved the docs to the spotless config.
This reduces the number of imports and all deprecation code is now in one place.
Also removed DeprecationChecksTests test case from x-pack core, because the same test already exists in the deprecation module.
Today `StoreFileMetadata#writtenBy` is a `Version`, but sent over the
wire as a `String`. It's included in every file chunk sent during
recovery, but parsing it uses `String#replaceFirst` a few times, which
is nontrivial, representing >50% of the time it takes to deserialize a
`RecoveryFileChunkRequest`. Moreover the value isn't actually used for
anything so the parsing is wasted work.
With this commit we treat this field as an opaque string, avoiding the
unnecessary parsing effort. We don't remove it because it appears in
e.g. a snapshot repository, where it may be useful for debugging.
The API key hashing result is now cached on the creation time of an API key,
i.e. pre-warm the cache. Previously it is cached when the API key is
authenticated for the first time. Since it is reasonable to assume that an API
key will be used shortly after its creation, this change has following
advantages:
* It removes the need for expensive pbkdf2 hashing computation on
authentication time and therefore reduces overall server load
* It makes the first authentication faster
We expect all keys to be used, that is, caching on creation time does not
change the total number of keys need to be cached. Hence this PR does not
introduce any extra logic to fine tune whether a key should be cached (for
example, only cache if the load factor is lower than certain threshold etc.).
This test failure happens fairly rarely, roughly once per month,
including on PRs and 6.8 branch. It almost requires no change. Hence
this PR only bumps the timeout for just another 100 ms. It does not
increases the timeout by too much becausd that would otherwise hide
other real problems, e.g. 2nd auth taking too long is likely an issue
for the caching mechanism.
Date based aggregations accept a timezone, which gets applied to both the bucketing logic and the formatter. This is usually what you want, but in the case of date formats where a timezone doesn't make any sense, it can create problems. In particular, our formatting logic and our parsing logic were doing different things for epoch_second and epoch_millis formats with time zones. This led to a problem on composite where we'd return an after key for the last bucket that would parse to a time before the last bucket, so instead of correctly returning an empty response to indicate the end of the aggregation, we'd keep returning the same last page of data.
* Put all service accounts information on one page
* De-emphasize connection with built-in accounts + edits
* Iterate on the docs: tweak, correction and more details.
* fix test
* Edits and minor text changes
Co-authored-by: Yang Wang <yang.wang@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Deprecation warning is now issued if any realm is configured with a name
prefixed with an underscore. This applies to all realms regardless
whether they are enabled or not.
Relates: #73250
API key authCache is set to expire after write (by default 24 hours).
ExpireAfterWrite is generally preferred over expireAfterAccess because it
guarantees stale entries get evicted eventually in edge cases, e.g. when the
cache misses a notification from the cluster.
However, things are a bit different for the authCache. There is an additional
roundtrip to the security index for fetching the API key document. If the
document does not exists (removed due to expiration) or is invalidated, the
authentication fails earlier on without even consulting the authCache. This
means the stale entries won't cause any security issues when they exist.
Therefore, this PR changes the authCache to be expire after access, which helps
preventing potential cyclic surge of expensive hash computations especially
when a large number of API keys are in use.
To further help the cache efficiency, this PR also actively invalidates the
authCache if the document is either not found or invalidated so it does not
have to wait for 24 hour to happen. Note that these are all edge cases and we
don't expect them to happen often (if at all).
James Rodewig
Armin Braun
Dan Hermann
Przemyslaw Gomulka
Yang Wang
Alan Woodward
Rene Groeschke
Adam Locke
François-Clément Brossard
Nicole Albee
Rory Hunter
David Turner
Benjamin Trent
Jim Ferenczi
Przemysław Witek
Martijn van Groningen
Mark Vieira
Mark Tozzi
David Roberts
Luca Cavanna
Dimitris Athanasiou