Commit Graph

178 Commits

Author SHA1 Message Date
Michael Niedermayer 8fa71d7b59 avcodec/jpeg2000dec: Check nonzerobits more completely
Fixes: runtime error: shift exponent 36 is too large for 32-bit type 'int'
Fixes: 2239/clusterfuzz-testcase-minimized-5639766592716800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dfb61ea263)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:18 +02:00
Michael Niedermayer 78bd801c74 avcodec/jpeg2000dec: Use ff_set_dimensions()
Fixes: OOM
Fixes: 1890/clusterfuzz-testcase-minimized-6329019509243904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f3da6fbff8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer 1e7874b043 avcodec/jpeg2000dec: Check tile offsets more completely
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9c1812491f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer 6d825e9d5f avcodec/jpeg2000dec: Fix copy and paste error
Found-by: jamrial
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5782e0ba8c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer d09e4a9e8b avcodec/jpeg2000dec: Check tile offsets
Fixes: runtime error: signed integer overflow: 4096 - -2147483648 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89325417e7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer a944744f19 avcodec/jpeg2000dec: More completely check cdef
Fixes out of array access
Fixes: j2k-poc.bin

Found-by: Lucas Leong <wmliang.tw@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0aada30510)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-01 02:12:22 +01:00
Michael Niedermayer 311de799a8 avcodec/jpeg2000dec: Check bpno in decode_cblk()
Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a85b02dcf7)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 12:40:51 +01:00
Michael Niedermayer f1058efc81 avcodec/jpeg2000dec: Fix potential integer overflow with tile dimensions
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 65d3359fb3)

Conflicts:

	libavcodec/jpeg2000dec.c
2015-12-06 12:40:50 +01:00
Michael Niedermayer 50870dd3de avcodec/jpeg2000dec: Check SIZ dimensions to be within the supported range
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6ef819c40b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 12:40:50 +01:00
Michael Niedermayer ac302efb91 avcodec/jpeg2000dec: Check for duplicate SIZ marker
Fixes: 0231a17345734228011c6f35a64e4594/asan_heap-oob_1d92a72_3218_1213809a9e3affec77e4c191fdfdc0a9.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44a7f17d0b)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 12:40:49 +01:00
Michael Niedermayer d73a8ae70f avcodec/jpeg2000dec: Clip all tile coordinates
Fixes out of array access
Fixes: b877a6b788a25c70e8b1d014f8628549/asan_heap-oob_1da2c3f_2324_5a1b329b0b3c4bb6b1d775660ac56717.r3d

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 43492ff3ab)

Conflicts:

	libavcodec/jpeg2000dec.c
2015-12-06 12:40:49 +01:00
Michael Niedermayer 84cf741861 avcodec/jpeg2000dec: Check that coords match before applying ICT
This avoid potential out of array accesses

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 12ba1b2b4d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-08-20 14:29:11 +02:00
Michael Niedermayer 5766f99f71 avcodec/jpeg2000dec: fix boolean operator
Fixes CID1271791 #7-6

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f8f155a18a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer f9cd317223 Merge commit 'fe4d5fe9361162f9033ff1bd84bfc1b2091ba785'
* commit 'fe4d5fe9361162f9033ff1bd84bfc1b2091ba785':
  jpeg2000: Mark static data init functions as av_cold

Conflicts:
	libavcodec/jpeg2000.c
	libavcodec/jpeg2000dec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-17 20:42:10 +02:00
Diego Biurrun fe4d5fe936 jpeg2000: Mark static data init functions as av_cold 2014-06-17 08:44:07 -07:00
Carl Eugen Hoyos ced0d6c14d Use correct msvc type specifiers for ptrdiff_t and size_t.
The Windows runtime aborts if it finds %t or %z.
Fixes ticket #3472.

Reviewed-by: Ronald Bultje
2014-04-24 18:01:30 +02:00
Michael Niedermayer 8d024c5107 Merge commit 'cc8163e1a3601a56f722a4720516e860bf1c6198'
* commit 'cc8163e1a3601a56f722a4720516e860bf1c6198':
  avcodec: more correct printf specifiers

Conflicts:
	libavcodec/4xm.c
	libavcodec/alsdec.c
	libavcodec/dfa.c
	libavcodec/h264_ps.c
	libavcodec/jpeg2000dec.c
	libavcodec/lagarith.c
	libavcodec/mpeg12dec.c
	libavcodec/rv10.c
	libavcodec/svq3.c
	libavcodec/wmaprodec.c
	libavcodec/xwddec.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-22 18:43:40 +01:00
Diego Biurrun cc8163e1a3 avcodec: more correct printf specifiers 2014-03-22 14:08:20 +01:00
Lukasz Marek aeb2905fb7 lavc/jpeg2000dec: silent warning discards qualifiers
Signed-off-by: Lukasz Marek <lukasz.m.luki@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-08 02:54:37 +01:00
Michael Niedermayer 8001e9f7d1 avcodec/jpeg2000dec: fix error detection in pix_fmt_match()
Fixes out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-20 18:46:38 +01:00
Michael Niedermayer 723e7b86ea avcodec/jpeg2000dec: calculate planar and pixelsize from pixel format descriptor
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-20 18:46:37 +01:00
Michael Niedermayer 55fa898969 avcodec/jpeg2000dec: zero Jpeg2000QuantStyle structure before use in get_qcd()
Fixes: msan_uninit-mem_7f50b84aac30_6823_mjp2.mov
Fixes use of uninitialized memory
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-12-28 22:08:49 +01:00
Michael Niedermayer 3d5a5e86be avcodec/jpeg2000dec: Check precno before using it in JPEG2000_PGOD_CPRL
Fixes out of array reads
Fixes: asan_heap-oob_f0de57_6823_mjp2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-11-24 04:02:24 +01:00
Michael Niedermayer 780669ef7c avcodec/jpeg2000dec: non zero image offsets are not supported
Fixes out of array accesses
Fixes Ticket3080
Found-by: ami_stuff
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-26 02:19:13 +02:00
Michael Niedermayer a1b9004b76 avcodec/jpeg2000dec: fix context consistency with too large lowres
Fixes out of array accesses
Fixes Ticket2898

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-15 14:30:17 +02:00
Michael Niedermayer fe448cd28d avcodec/jpeg2000dec: prevent out of array accesses in pixel addressing
Fixes Ticket2921

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-13 21:48:36 +02:00
Michael Niedermayer ac3b01a9c0 avcodec/jpeg2000dec: check transform equality in MCT
Fixes null pointer dereference
Fixes Ticket2843

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-13 19:23:24 +02:00
Michael Niedermayer e96c80154f Merge commit '1018a92219a38a812cf97761c6b3a5e66a400f4b'
* commit '1018a92219a38a812cf97761c6b3a5e66a400f4b':
  jpeg2000: Check block length

See: 914ab4cd1c
See: 582f53349e
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-10 10:32:04 +02:00
Luca Barbato 1018a92219 jpeg2000: Check block length
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2013-10-09 22:25:19 +02:00
Michael Niedermayer fcd19d6c8d avcodec/jpeg2000dec: Zero code and quant style
This prevents future frames from using half initialized/inconsistent
structures
Fixes assertion failure
Fixes Ticket2897

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-10-03 19:03:56 +02:00
Michael Niedermayer 2dd38a1d48 avcodec/jpeg2000dec: move "missing SIZ" check up
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-18 10:04:43 +02:00
Michael Niedermayer 8ae14bf5ce Merge commit '18a245a2aea31b92bc435f2962a71e060d83b5f7'
* commit '18a245a2aea31b92bc435f2962a71e060d83b5f7':
  jpeg2000dec: Add more checks when parsing headers

The check cannot be true in FFmpeg, but does no harm.
I will move it in a subsequent commit so it can detect missing SIZ

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-18 10:03:35 +02:00
Martin Storsjö 18a245a2ae jpeg2000dec: Add more checks when parsing headers
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
2013-09-17 15:57:27 +03:00
Clément Bœsch 04427182bc avcodec: typo fix sepera* → separa* 2013-09-15 21:32:42 +02:00
Michael Niedermayer e428632c1a avcodec/jpeg2000dec: print invalid cdx/y values
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-15 15:37:16 +02:00
Carl Eugen Hoyos f99f7f4025 Set bits_per_raw_sample when reading j2k frames. 2013-09-12 09:23:32 +02:00
Michael Niedermayer 4ced30908f avcodec/jpeg2000dec: make SOC finding code more robust
Fixes file from Ticket860

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-09-05 01:39:20 +02:00
Michael Niedermayer 9a271a9368 jpeg2000: check log2_cblk dimensions
Fixes out of array access
Fixes Ticket2895

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-24 03:42:54 +02:00
Michael Niedermayer 8bb11c3ca7 avcodec/jpeg2000dec: Check cdx/y values more carefully
Some invalid values where not handled correctly in the later pixel
format matching code.
Fixes out of array accesses
Fixes Ticket2848

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-23 01:02:03 +02:00
Michael Niedermayer 63c0e9077e avcodec/jpeg2000dec: fix near null pointer dereference
Fixes Ticket2840

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-18 18:44:12 +02:00
Michael Niedermayer 5cd57e8758 avcodec/jpeg2000dec: check sample sepration for validity
Fixes division by 0
Fixes Ticket2841

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-07 15:09:35 +02:00
Michael Niedermayer bff371e34c jpeg2000dec: simplify jpeg2000_read_bitstream_packets()
23 lines less
less error prone as theres no more need to update *_size on every read

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-08-02 03:55:07 +02:00
Michael Niedermayer 1434df3b93 jpeg2000dec: Support non subsampled 9-16bit planar pixel formats
This applies changes similar to fc6de70c44
to the >8bit codepath

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-21 04:16:53 +02:00
Michael Niedermayer db33010483 jpeg2000dec: silence unused variable warning
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-19 17:54:57 +02:00
Michael Niedermayer fc6de70c44 jpeg2000dec: Support non subsampled 8bit planar pixel formats
Fixes file2.jp2

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-18 21:50:32 +02:00
Michael Niedermayer 99de97cabf jpeg2000dec: parse CDEF
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

Conflicts:

	libavcodec/jpeg2000dec.c

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-18 21:43:42 +02:00
Carl Eugen Hoyos b39a6bbe7f Fix pix_fmt detection in the native jpeg2000 decoder.
Based on b7a928b by Michael Bradshaw.
Fixes ticket #2683.

Reviewed-by: Nicolas Bertrand
2013-07-18 13:51:24 +02:00
Nicolas Bertrand f56fe04de3 jpeg2000: Initialize only once mqc arrays
Improves decoding speed.

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2013-07-13 07:54:12 +02:00
Michael Niedermayer f57119b8e5 jpeg2000: fix overflow in dequantization
Fixes decoding of file generated with:
ffmpeg -f lavfi -i smptehdbars=hd720 -pix_fmt rgb48 /tmp/o.jp2

Reviewed-by: Nicolas BERTRAND <nicoinattendu@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-13 12:44:37 +02:00
Nicolas Bertrand dd1382ac95 jpeg2000: Initialize only once mqc arrays
Increases encoding and decoding speed

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2013-07-12 23:43:09 +02:00