Removing the custom dependency checksum functionality in favor of Gradle build-in dependency verification support.
- Use sha256 in favor of sha1 as sha1 is not considered safe these days.
Closes https://github.com/elastic/elasticsearch/issues/69736
Part of #84369. Split out from #87696. Introduce tracing interfaces in
advance of adding APM support to Elasticsearch. The only implementation
at this point is a no-op class.
If we run into a seed that causes many fake exceptions and thus retries,
a 100ms retry interval will add up to minutes of test time for tests like
`testLargeBlobCountDeletion` that trigger thousands of requests.
There's no reason not to speed this up by 10x via more aggressive retry
timings as far as I can see so I reduced the timings to avoid randomly
blocked tests.
When the SAS token is prefixed with ? the SDK parses incorrectly
all the query params and removes the first query param, this
produces an invalid signature preventing the repository to
authenticate against the storage service. This commit reverts
to the previous behaviour where the SAS token was appended to
the URL as it is provided by the user.
Closes#88140
Many shards benchmarks have recently seen a bit of an increase in the time spent on
settings from new settings getting added. This makes some defaults quite a bit
cheaper by saving re-serialization of the same default value which helps with
some cluster state application cases for large index counts.
This is a result of structural search/replace in intellij. This only affects log methods with a signature
logger.info(ParametrizedMessage)
logger.info(ParametrizedMessage, Throwable)
relates https://github.com/elastic/elasticsearch/issues/86549
This is a result of structural search/replace in intellij. This only affects log methods with a signature
logger.info(Supplier<?>) where level could be info/debug etc and supplier argument is in a form of
()-> new ParameterizedMessage
This commit also introduced a Strings utility class to avoid passing Locale.ROOT to every
String.format(Locale.ROOT, pattern, args)
relates #86549
This commit converts all ParameterizedMessages which take no arguments
to direct logging messages. There is no need for lazy evaluation since
these strings are static.
relates #86549
Staying up to date. This upgrade comes with two non-trivial
change. The thread local cache buffer cache has been disabled
for non-transport threads and the default pooled buffer chunk
size has been reduced to 4M from 16M.
The later change I think is without any effect for us since we
allocate buffers in 16k chunks on our end when doing outbound
writes and if anything should save us memory.
The disabling of thread-local buffer cache could have some effect
but is hardly measurable in practice. That combined with the fact
that we can switch back to the old behavior via the Netty system
property made me go with the new behavior here which does save us
some heap memory at hardly measurable CPU cost.
In this version jackson was upgraded to the version 2.13.2.1 that we were using
before. See https://github.com/Azure/azure-sdk-for-java/releases/tag/azure-core_1.27.0
Without the upgrade, the third party SAS tests were failing, once upgraded the tests pass again,
I think that we were hitting some incompatibility with the jackson version that we were
providing.
Relates #83884
This change isolates the Jackson implementation of x-content parsers and generators to a separate classloader. The code is loaded dynamically upon accessing any x-content functionality.
The x-content implementation is embedded inside the x-content jar, as a hidden set of resource files. These are loaded through a special classloader created to initialize the XContentProvider through service loader. One caveat to this approach is that IDEs will no longer trigger building the x-content implementation when it changes. However, running any test from the command line, or running a full Build in IntelliJ will trigger the directory to be built.
Co-authored-by: ChrisHegarty <christopher.hegarty@elastic.co>
Follow-up from #77144 (comment) with converting id/_id to always be strings instead of integers. This makes the type value in the Elasticsearch specification be only string instead of string | number.
this change was generated using following command on ubuntu
find . -type f -name "*.yml" -print0 | xargs -0 sed -i -r 's/([^a-zA-Z0-9_\.]id|[^a-zA-Z0-9_]_id):(\s*)([0-9]+)/\1:\2"\3"/g'
repository azure plugin is using azure-core which depends on sl4j for
logging. Slf4j requires a bridge to log into log4j2 otherwise it emitts
a warning to STDOUT and all further logs also to STDOUT.
This commit addes the missing log4j-sl4j-impl which is a bridge between
slf4j API and log4j implementation
closes#83652
Today scaling thread pools never reject tasks but always add them to the
queue of task the execute, even in the case the thread pool executor is
shutting down or terminated. This behaviour does not work great when a
task is blocked waiting for another task from another scaling thread
pool to complete an I/O operation which will never be executed if the
task was enqueued just before the scaling thread pool was shutting down.
This situation is more likely to happen with searchable snapshots in
which multiple threads can be blocked waiting for parts of Lucene files
to be fetched and made available in cache. We saw tests failures in CI
where Lucene 9 uses concurrent threads (to asynchronously checks
indices) that were blocked waiting for cache files to be available and
failing because of leaking files handles (see #77017, #77178).
This pull request changes the `ForceQueuePolicy` used by scaling thread
pools so that it now accepts a `rejectAfterShutdown` flag which can be
set on a per thread pool basis to indicate when tasks should just be
rejected once the thread pool is shut down. Because we rely on many
scaling thread pools to be black holes and never reject tasks, this flag
is set to `false` on most of them to keep the current behavior. In some
cases where the rejection logic was already implemented correctly this
flag has been set to `true`.
This pull request also reimplements the interface
`XRejectedExecutionHandler` into an abstract class
`EsRejectedExecutionHandler` that allows to share some logic for
rejections.
Closes#81652.
Convert the `repository-azure`, `repository-gcs` and `repository-s3`
plugins into modules, so that they are always included in the
Elasticsearch distribution. Also change plugin installation, removal
and syncing so that attempting to add or remove these plugins still
succeeds but is now a no-op.
Rene Groeschke
Rory Hunter
Armin Braun
Chris Hegarty
Francisco Fernández Castaño
Przemyslaw Gomulka
Ryan Ernst
Tanguy Leroux
Mary Gouseti