openssl/crypto
Neil Horman fbd34c03e3 rcu: Ensure that updates to the ID field of a qp don't lose refs
ppc64le occasionally still fails the threadstest on __rcu_torture

From several days of debugging, I think I've landed on the problem.

Occasionally, under high load I observe the following pattern

CPU0                                     CPU1
update_qp                                get_hold_current_qp
  atomic_and_fetch(qp->users, ID_MASK, RELEASE)
                                         atomic_add_fetch(qp->users, 1, RELEASE
  atomic_or_fetch(qp->users, ID_VAL++, RELEASE)

When this pattern occurs, the atomic or operation fails to see the published
value of CPU1 and when the or-ed value is written back to ram, the incremented
value in get_hold_current_qp is overwritten, meaning the hold that the reader
placed on the rcu lock is lost, allowing the writer to complete early, freeing
memory before a reader is done reading any held memory.

Why this is only observed on ppc64le I'm not sure, but it seems like a pretty
clear problem.

fix it by implementing ATOMIC_COMPARE_EXCHANGE_N, so that, on the write side in
update_qp, we can ensure that updates are only done if the read side hasn't
changed anything.  If it has, retry the operation.

With this fix, I'm able to run the threads test overnight (4000 iterations and
counting) without failure.

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/26478)
2025-01-22 20:16:11 +01:00
..
aes Restore correct registers in aarch64 AES-CTR code 2025-01-21 10:55:56 +01:00
aria
asn1 Improve ASN1_TIME_print documentation and output 2025-01-22 11:28:31 +01:00
async one more empty line for code style consistency 2024-12-06 15:27:51 +01:00
bf
bio Fix potential memory leak in BIO_get_accept_socket() 2025-01-02 14:06:51 +01:00
bn Fix timing side-channel in ECDSA signature computation 2025-01-20 09:30:48 +01:00
buffer
camellia Copyright year updates 2024-09-05 09:35:49 +02:00
cast
chacha Make it able to run asm code on OpenBSD (arm64) 2025-01-14 12:14:27 +01:00
cmac Add FIPS indicator to CMAC. 2024-07-31 09:04:17 +10:00
cmp Fix doc and use of_X509v3_add_extensions() in case sk_X509_EXTENSION_num(exts) <= 0 2024-11-13 17:19:58 +01:00
cms Fix CMS encryption with key agreement when originator set 2025-01-06 11:45:02 +01:00
comp
conf Check returns of various sk_*_push functions 2025-01-08 11:11:00 +01:00
crmf Copyright year updates 2024-09-05 09:35:49 +02:00
ct
des Copyright year updates 2024-09-05 09:35:49 +02:00
dh Fix potential use-after-free in REF_PRINT_COUNT 2024-12-10 14:58:08 +01:00
dsa Fix potential use-after-free in REF_PRINT_COUNT 2024-12-10 14:58:08 +01:00
dso Fix potential use-after-free in REF_PRINT_COUNT 2024-12-10 14:58:08 +01:00
ec Fix timing side-channel in ECDSA signature computation 2025-01-20 09:30:48 +01:00
encode_decode Move some encode helpers to crypt/encode_decode/ 2025-01-21 20:09:57 +11:00
engine Copyright year updates 2024-09-05 09:35:49 +02:00
err fix X509_PURPOSE_add() to take |sname| as primary key and handle |id| in a backwd compat way for new purpose 2025-01-20 08:41:19 +01:00
ess Copyright year updates 2024-09-05 09:35:49 +02:00
evp Pass functions with correct signatures to the evp_generic_fetch_xxx methods 2025-01-07 21:24:13 +01:00
ffc ffc: change FIPS zeroization to use the OPENSSL_PEDANTIC_ZEROIZATION define 2024-11-28 15:13:35 +01:00
hashtable ossl_ht_insert(): Allow for 4 iterations of grow_hashtable() 2024-08-22 14:52:43 +02:00
hmac s390x: Disable HMAC hardware acceleration when an engine is used for the digest 2024-09-03 21:15:00 +02:00
hpke Fix potential memory leak in OSSL_HPKE_CTX_new() 2024-10-30 11:58:16 +00:00
http Fix the use of OPENSSL_HTTP_PROXY / OPENSSL_HTTPS_PROXY 2025-01-09 12:03:52 +01:00
idea Copyright year updates 2024-09-05 09:35:49 +02:00
kdf
lhash Copyright year updates 2024-04-09 13:43:26 +02:00
md2
md4
md5 Optimize x86/aarch64 MD5 implementation 2025-01-06 11:43:36 +01:00
mdc2
modes Make it able to run asm code on OpenBSD (arm64) 2025-01-14 12:14:27 +01:00
objects Tweak case of [fs] variant letter in SLH-DSA algorithms 2025-01-11 14:39:03 +11:00
ocsp Check returns of various sk_*_push functions 2025-01-08 11:11:00 +01:00
pem Make the provider context available to encoders 2025-01-21 17:19:07 +11:00
perlasm Work around to get llvm-mingw working on aarch64 2025-01-17 17:43:39 +01:00
pkcs7 Avoid NULL dereference with PKCS7_OP_SET_DETACHED_SIGNATURE 2024-12-02 09:44:58 +01:00
pkcs12 Fix potential memory leak in PKCS12_add_key_ex() 2025-01-06 21:32:50 +01:00
poly1305 Make it able to run asm code on OpenBSD (arm64) 2025-01-14 12:14:27 +01:00
property Avoid leaking memory when realloc fails 2025-01-03 15:58:26 +01:00
rand Return NULL from ossl_lib_ctx_get_concrete() when it is uninitialized 2025-01-08 11:18:13 +01:00
rc2
rc4
rc5
ripemd
rsa Fix potential use-after-free in REF_PRINT_COUNT 2024-12-10 14:58:08 +01:00
seed
sha Make it able to run asm code on OpenBSD (arm64) 2025-01-14 12:14:27 +01:00
siphash
sm2 sm2_sig_verify(): Do not call BN_CTX_end() without BN_CTX_start() 2024-11-21 11:13:56 +01:00
sm3 Copyright year updates 2024-09-05 09:35:49 +02:00
sm4 Revert "Move rodata to .rodata section for armv8" 2025-01-21 18:21:19 +01:00
srp Fix potential double free through SRP_user_pwd_set1_ids() 2024-10-11 14:22:36 +02:00
stack Copyright year updates 2024-04-09 13:43:26 +02:00
store Copyright year updates 2024-09-05 09:35:49 +02:00
thread Fix no-thread-pool build on Windows 2024-09-10 16:36:39 +02:00
ts Copyright year updates 2024-09-05 09:35:49 +02:00
txt_db
ui Fix Edge Cases in Password Callback Handling 2024-09-09 08:58:03 +02:00
whrlpool Copyright year updates 2024-09-05 09:35:49 +02:00
x509 fix X509_PURPOSE_add() to take |sname| as primary key and handle |id| in a backwd compat way for new purpose 2025-01-20 08:41:19 +01:00
LPdir_nyi.c
LPdir_unix.c
LPdir_vms.c
LPdir_win.c
LPdir_win32.c
LPdir_wince.c
README-sparse_array.md
alphacpuid.pl
arm64cpuid.pl
arm_arch.h Copyright year updates 2024-04-09 13:43:26 +02:00
armcap.c Enable AES and SHA3 optimisations on Apple Silicon M4-based macOS systems 2024-11-22 14:56:04 +01:00
armv4cpuid.pl Mark OPENSSL_armcap_P .hidden in arm asm 2024-11-22 11:22:00 +01:00
asn1_dsa.c
bsearch.c
build.info Add FIPS indicator callback. 2024-07-11 08:29:43 +10:00
c64xpluscpuid.pl
comp_methods.c Move stack of compression methods from libssl to OSSL_LIB_CTX 2024-05-28 08:56:13 +02:00
context.c Return NULL from ossl_lib_ctx_get_concrete() when it is uninitialized 2025-01-08 11:18:13 +01:00
core_algorithm.c
core_fetch.c Take into account no_store when pushing algorithm 2024-12-20 18:20:17 +01:00
core_namemap.c core_namemap.c: Use OPENSSL_STRING instead of defining STRING type 2024-12-31 15:03:13 +01:00
cpt_err.c Use the new hashtable for core_namemap 2024-08-21 15:21:26 +02:00
cpuid.c Extension of OPENSSL_ia32cap to accommodate additional CPUID bits 2024-12-13 14:51:22 +01:00
cryptlib.c
ctype.c
cversion.c Revert API change of OPENSSL_version() 2024-10-17 13:36:58 +02:00
defaults.c Fix typos found by codespell 2024-08-07 19:09:43 +02:00
der_writer.c
deterministic_nonce.c Copyright year updates 2024-09-05 09:35:49 +02:00
dllmain.c
ebcdic.c
ex_data.c
getenv.c
ia64cpuid.S
indicator_core.c Add FIPS indicator callback. 2024-07-11 08:29:43 +10:00
info.c Extension of OPENSSL_ia32cap to accommodate additional CPUID bits 2024-12-13 14:51:22 +01:00
init.c Copyright year updates 2024-04-09 13:43:26 +02:00
initthread.c Recycle the TLS key that holds thread_event_handler 2024-09-05 17:19:53 +02:00
loongarch64cpuid.pl LoongArch64 assembly pack: Really implement OPENSSL_rdtsc 2023-12-19 18:34:34 +01:00
loongarch_arch.h
loongarchcap.c
mem.c Use static array (length 256) for copy of OPENSSL_MALLOC_FAILURES 2024-11-28 17:01:28 +01:00
mem_clr.c
mem_sec.c Copyright year updates 2024-09-05 09:35:49 +02:00
mips_arch.h
o_dir.c
o_fopen.c Copyright year updates 2024-09-05 09:35:49 +02:00
o_init.c
o_str.c crypto: factorize to hex chars conversion code. 2024-08-07 19:25:10 +02:00
o_time.c
packet.c
param_build.c params: drop INT_MAX checks 2023-12-29 10:21:10 +01:00
param_build_set.c ossl_param_build_set_multi_key_bn(): Do not set NULL BIGNUMs 2023-10-18 18:07:13 +02:00
params.c Don't promise a non-zero return size in error cases. 2025-01-21 17:21:52 +11:00
params_dup.c
params_from_text.c Make ossl_trace_param_values an official api function 2024-11-19 08:38:25 -05:00
params_idx.c.in
pariscid.pl
passphrase.c
ppccap.c Copyright year updates 2024-09-05 09:35:49 +02:00
ppccpuid.pl
provider.c
provider_child.c
provider_conf.c Support CLI and API setting of provider configuration parameters 2025-01-18 03:46:37 +11:00
provider_core.c Support boolean queries against provider config 2025-01-22 14:52:26 +11:00
provider_local.h
provider_predefined.c
punycode.c
quic_vlint.c
rcu_internal.h Copyright year updates 2024-04-09 13:43:26 +02:00
riscv32cpuid.pl Copyright year updates 2024-09-05 09:35:49 +02:00
riscv64cpuid.pl riscv: Add basic vector extension support 2023-10-26 15:55:49 +01:00
riscvcap.c Avoid undefined behaviour with the <ctype.h> functions. 2024-10-10 20:47:48 +02:00
s390x_arch.h s390x: Don't probe crypto cards for ME/CRT offloading during initialization 2024-10-23 15:07:01 +02:00
s390xcap.c s390x: Don't probe crypto cards for ME/CRT offloading during initialization 2024-10-23 15:07:01 +02:00
s390xcpuid.pl Copyright year updates 2024-09-05 09:35:49 +02:00
self_test_core.c Copyright year updates 2024-09-05 09:35:49 +02:00
sleep.c For Unix, refactor OSSL_sleep() to use nanosleep() instead of usleep() 2024-05-22 09:59:32 +02:00
sparccpuid.S
sparcv9cap.c
sparse_array.c Copyright year updates 2024-09-05 09:35:49 +02:00
threads_lib.c
threads_none.c threads: follow formatting rules 2024-09-05 17:09:50 +02:00
threads_pthread.c rcu: Ensure that updates to the ID field of a qp don't lose refs 2025-01-22 20:16:11 +01:00
threads_win.c Fix premature reuse of qp's in rcu locks 2025-01-13 17:13:48 -05:00
time.c crypto: fix missing <winsock.h> indirection inclusion. 2024-09-05 17:02:51 +02:00
trace.c Add a QUERY trace category 2024-11-19 08:36:19 -05:00
uid.c
vms_rms.h
x86_64cpuid.pl Extension of OPENSSL_ia32cap to accommodate additional CPUID bits 2024-12-13 14:51:22 +01:00
x86cpuid.pl Extension of OPENSSL_ia32cap to accommodate additional CPUID bits 2024-12-13 14:51:22 +01:00