The BN_GF2m_poly2arr() function converts characteristic-2 field
(GF_{2^m}) Galois polynomials from a representation as a BIGNUM bitmask,
to a compact array with just the exponents of the non-zero terms.
These polynomials are then used in BN_GF2m_mod_arr() to perform modular
reduction. A precondition of calling BN_GF2m_mod_arr() is that the
polynomial must have a non-zero constant term (i.e. the array has `0` as
its final element).
Internally, callers of BN_GF2m_poly2arr() did not verify that
precondition, and binary EC curve parameters with an invalid polynomial
could lead to out of bounds memory reads and writes in BN_GF2m_mod_arr().
The precondition is always true for polynomials that arise from the
standard form of EC parameters for characteristic-two fields (X9.62).
See the "Finite Field Identification" section of:
https://www.itu.int/ITU-T/formal-language/itu-t/x/x894/2018-cor1/ANSI-X9-62.html
The OpenSSL GF(2^m) code supports only the trinomial and pentanomial
basis X9.62 forms.
This commit updates BN_GF2m_poly2arr() to return `0` (failure) when
the constant term is zero (i.e. the input bitmask BIGNUM is not odd).
Additionally, the return value is made unambiguous when there is not
enough space to also pad the array with a final `-1` sentinel value.
The return value is now always the number of elements (including the
final `-1`) that would be filled when the output array is sufficiently
large. Previously the same count was returned both when the array has
just enough room for the final `-1` and when it had only enough space
for non-sentinel values.
Finally, BN_GF2m_poly2arr() is updated to reject polynomials whose
degree exceeds `OPENSSL_ECC_MAX_FIELD_BITS`, this guards against
CPU exhausition attacks via excessively large inputs.
The above issues do not arise in processing X.509 certificates. These
generally have EC keys from "named curves", and RFC5840 (Section 2.1.1)
disallows explicit EC parameters. The TLS code in OpenSSL enforces this
constraint only after the certificate is decoded, but, even if explicit
parameters are specified, they are in X9.62 form, which cannot represent
problem values as noted above.
Initially reported as oss-fuzz issue 71623.
A closely related issue was earlier reported in
<https://github.com/openssl/openssl/issues/19826>.
Severity: Low, CVE-2024-9143
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/25639)
(cherry picked from commit 8e008cb8b2)
c0d3e4d32d