root
/
redis
mirror of https://github.com/redis/redis.git
1
0
Fork 0
Code Issues Actions Packages Projects Releases Wiki Activity
redis/src
History
Greg Femec cde69883a1 Fix random element selection for large hash tables. (#8133) 
When a database on a 64 bit build grows past 2^31 keys, the underlying hash table expands to 2^32 buckets. After this point, the algorithms for selecting random elements only return elements from half of the available buckets because they use random() which has a range of 0 to 2^31 - 1. This causes problems for eviction policies which use dictGetSomeKeys or dictGetRandomKey. Over time they cause the hash table to become unbalanced because, while new keys are spread out evenly across all buckets, evictions come from only half of the available buckets. Eventually this half of the table starts to run out of keys and it takes longer and longer to find candidates for eviction. This continues until no more evictions can happen.

This solution addresses this by using a 64 bit PRNG instead of libc random().

Co-authored-by: Greg Femec <gfemec@google.com>
(cherry picked from commit 266949c7fc)
 		2021-02-23 01:35:37 +02:00
..
modules Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
.gitignore
…
Makefile Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
acl.c White space tweaks and skip categories already applied 2021-01-12 16:25:37 +02:00
adlist.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
adlist.h Speedup: unblock clients on keys in O(1). 2020-04-15 16:03:16 +02:00
ae.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
ae.h Added a refcount on timer events to prevent deletion of recursive timer calls 2020-05-15 22:23:24 +02:00
ae_epoll.c
…
ae_evport.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
ae_kqueue.c
…
ae_select.c
…
anet.c
…
anet.h
…
aof.c Disable rehash when redis has child process (#8007) 2021-01-12 16:25:37 +02:00
asciilogo.h string literal should be const char* 2020-03-25 15:54:34 +01:00
atomicvar.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
bio.c Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) 2021-01-12 16:25:37 +02:00
bio.h
…
bitops.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
blocked.c Remove race condition and consistency issues with client pause and threaded IO (#8520) 2021-02-22 23:05:00 +02:00
childinfo.c RM_GetContextFlags provides indication that we're in a fork child (#7783) 2020-10-27 09:12:01 +02:00
cluster.c Fix cluster access to unaligned memory (SIGBUS on old ARM) (#7958) 2021-01-12 16:25:37 +02:00
cluster.h Fix cluster access to unaligned memory (SIGBUS on old ARM) (#7958) 2021-01-12 16:25:37 +02:00
config.c CONFIG REWRITE should honor umask settings. (#8371) 2021-02-23 01:35:37 +02:00
config.h cpu affinity: DragonFlyBSD support (#7956) 2020-10-27 09:12:01 +02:00
connection.c Fix connGetSocketError usage (#7811) 2020-10-27 09:12:01 +02:00
connection.h Module API: Add RM_GetClientCertificate(). (#7866) 2020-10-27 09:12:01 +02:00
connhelpers.h Conns: Fix connClose() / connAccept() behavior. 2020-03-25 15:55:24 +01:00
crc16.c
…
crc16_slottable.h
…
crc64.c Move CRC64 initialization in main(). 2020-05-08 10:37:35 +02:00
crc64.h Implemented CRC64 based on slice by 4 2020-04-28 11:20:15 +02:00
crcspeed.c Fix rdb checksum / crc64 on bigendian (#8270) 2021-01-12 16:25:37 +02:00
crcspeed.h Added crcspeed library 2020-04-28 11:20:15 +02:00
db.c Swapdb should make transaction fail if there is any client watching keys (#8239) 2021-01-12 16:25:37 +02:00
debug.c raspberry build fix. (#8095) 2021-01-12 16:25:37 +02:00
debugmacro.h
…
defrag.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
dict.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
dict.h Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
endianconv.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
endianconv.h
…
evict.c prevent client tracking from causing feedback loop in performEvictions (#8100) 2021-01-12 16:25:37 +02:00
expire.c WATCH no longer ignores keys which have expired for MULTI/EXEC. (#7920) 2020-10-27 09:12:01 +02:00
fmacros.h
…
geo.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
geo.h
…
geohash.c FIX: truncate max/min longitude,latitude related geo_point (ex: {180, 85.05112878} ) 2020-04-15 16:03:16 +02:00
geohash.h
…
geohash_helper.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
geohash_helper.h
…
gopher.c
…
help.h Redis 6.0.9. 2020-10-27 09:12:01 +02:00
hyperloglog.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
intset.c
…
intset.h
…
latency.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
latency.h
…
lazyfree.c Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) 2021-01-12 16:25:37 +02:00
listpack.c fix 32bit build warnings (#7926) 2020-10-27 09:12:01 +02:00
listpack.h
…
listpack_malloc.h
…
localtime.c
…
lolwut.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
lolwut.h add include guard for lolwut.h 2020-05-08 10:37:36 +02:00
lolwut5.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
lolwut6.c
…
lzf.h
…
lzfP.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
lzf_c.c
…
lzf_d.c
…
memtest.c Add support for Haiku OS (#7435) 2020-10-27 09:12:01 +02:00
mkreleasehdr.sh
…
module.c Modules: Fix an integer sign bug in moduleTimerHandler (#8131) 2021-01-12 16:25:37 +02:00
mt19937-64.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
mt19937-64.h Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
multi.c Swapdb should make transaction fail if there is any client watching keys (#8239) 2021-01-12 16:25:37 +02:00
networking.c Remove race condition and consistency issues with client pause and threaded IO (#8520) 2021-02-22 23:05:00 +02:00
notify.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
object.c EXISTS should not alter LRU, OBJECT should not reveal expired keys on replica (#8016) 2021-01-12 16:25:37 +02:00
pqsort.c
…
pqsort.h
…
pubsub.c Boost up performance for redis PUB-SUB patterns matching 2020-03-31 12:47:14 +02:00
quicklist.c fix integer overflow 2020-04-07 16:52:04 +02:00
quicklist.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
rand.c
…
rand.h
…
rax.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
rax.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
rax_malloc.h
…
rdb.c Fix overflow of rdbWriteRaw return value (#8306) 2021-01-12 16:25:37 +02:00
rdb.h Fix redis-check-rdb support for modules aux data (#7826) 2020-10-27 09:12:01 +02:00
redis-benchmark.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
redis-check-aof.c
…
redis-check-rdb.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
redis-cli.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
redis-trib.rb
…
redisassert.h
…
redismodule.h Moved RMAPI_FUNC_SUPPORTED location such that it will be visible to modules (#8037) 2021-01-12 16:25:37 +02:00
release.c
…
replication.c Backup keys to slots map and restore when fail to sync if diskless-load type is swapdb in cluster mode (#8108) 2021-01-12 16:25:37 +02:00
rio.c
…
rio.h
…
scripting.c Fix wrong order of key/value in Lua map response (#8266) 2021-01-12 16:25:37 +02:00
sds.c Allow '\0' inside of result of sdscatvprintf, and efficiency improvements (#6260) 2021-01-12 16:25:37 +02:00
sds.h Mark extern definition of SDS_NOINIT in sds.h 2020-02-12 14:17:54 +01:00
sdsalloc.h add missing file marco 2020-03-05 12:51:15 +01:00
sentinel.c Sentinel: add missing calls for sentinelflushconfig when config master at runtime (#8229) 2021-01-12 16:25:37 +02:00
server.c Fix random element selection for large hash tables. (#8133) 2021-02-23 01:35:37 +02:00
server.h CONFIG REWRITE should honor umask settings. (#8371) 2021-02-23 01:35:37 +02:00
setcpuaffinity.c cpu affinity: DragonFlyBSD support (#7956) 2020-10-27 09:12:01 +02:00
setproctitle.c Fix setproctitle related crashes. (#8150) 2021-01-12 16:25:37 +02:00
sha1.c
…
sha1.h
…
sha256.c
…
sha256.h
…
siphash.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
slowlog.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
slowlog.h
…
solarisfixes.h
…
sort.c sort Command lookupKeyRead and lookupKeyWrite are used on the opposite (#8283) 2021-01-12 16:25:37 +02:00
sparkline.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
sparkline.h
…
stream.h Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
syncio.c
…
t_hash.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
t_list.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
t_set.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
t_stream.c XINFO should use lookupKeyReadOrReply (#8436) 2021-02-23 01:35:37 +02:00
t_string.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
t_zset.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
testhelp.h
…
timeout.c Add missing calls to raxStop (#7532) 2020-09-01 09:27:58 +03:00
tls.c Module API: Add RM_GetClientCertificate(). (#7866) 2020-10-27 09:12:01 +02:00
tracking.c Client tracking tracking-redir-broken push len is 2 not 3 (#8456) 2021-02-23 01:35:37 +02:00
util.c stringmatchlen() should not expect null terminated strings. 2020-05-08 10:37:36 +02:00
util.h
…
valgrind.sup Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
version.h Redis 6.0.10 2021-01-12 16:25:37 +02:00
ziplist.c Fix redundancy use of semicolon in do-while macros in ziplist.c. (#7832) 2020-10-27 09:12:01 +02:00
ziplist.h
…
zipmap.c Squash merging 125 typo/grammar/comment/doc PRs (#7773) 2020-10-27 09:12:01 +02:00
zipmap.h
…
zmalloc.c Fix wrong zmalloc_size() assumption. (#7963) 2020-10-27 09:12:01 +02:00
zmalloc.h
…