Prior to this commit, the `ConcurrentLruCache` implementation would not
perform well under certain conditions. As long as the cache capacity was
not reached, the cache would avoid maintaining an eviction queue
(reordering entries depending with least/most recently read). When the
cache capacity was reached, the LRU queue was updated for each
read/write operation. This decreased performance significantly under
contention when the capacity was reached.
This commit completely rewrites the internals of `ConcurrentLruCache`.
`ConcurrentLruCache` is now a specialized version of the
`ConcurrentLinkedHashMap` [1]. This change focuses on buferring read and
write operations, only processing them at certain times to avoid
contention.
When a cached entry is read, a read operation is queued and buffered
operations are drained if the buffer reached a fixed limit. When a new
cache entry is added or removed, a write operation is queued and
triggers a drain attempt. When the capacity is outgrown, the cache polls
items from the eviction queue, which maintains elements with the
least recently used ones first. Entries are removed until the capacity
is under control.
The behavior described here and the buffer sizes are optimized with the
number of available processors in mind. Work is localized as much as
possible on a per-thread basis to avoid contention on the eviction queue.
The new implementation has been tested with the JMH benchmark provided
here, comparing the former `COncurrentLruCache`, the new implementation
as well as the `ConcurrentLinkedHashMap` [1].
When testing with a cache reaching capacity, under contention, with a
10% cache miss, we're seeing a 40x improvement compared to the previous
implementation and performance on par with the reference.
See [2] for how to replicate the benchmark.
[1] https://github.com/ben-manes/concurrentlinkedhashmap
[2] https://github.com/spring-projects/spring-framework/wiki/Micro-Benchmarks
Closes gh-26320
c470262c8e