Commit Graph

84 Commits

Author SHA1 Message Date
Michael Niedermayer 6da07b7b6a avcodec/hevc_ps: Check delta_pocs in ff_hevc_decode_short_term_rps()
Fixes: integer overflow
Fixes: 2893/clusterfuzz-testcase-minimized-5809330567774208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2b44dcbc44)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-24 12:57:00 +02:00
Michael Niedermayer dbf29313ca avcodec/hevc_ps: fix integer overflow in log2_parallel_merge_level_minus2
Fixes: runtime error: signed integer overflow: -2147483647 - 2 cannot be represented in type 'int'
Fixes: 2702/clusterfuzz-testcase-minimized-4511932591636480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 74c1c22d7f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:18 +02:00
Michael Niedermayer c299d7060e avcodec/hevc_ps: Fix runtime error: index 32 out of bounds for type 'uint8_t [32]'
Fixes: 2010/clusterfuzz-testcase-minimized-6209288450080768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 29808fff33)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer 81b798e24d avcodec/hevc_ps: Fix runtime error: signed integer overflow: 2147483628 + 256 cannot be represented in type 'int'
Fixes: 1909/clusterfuzz-testcase-minimized-6732072662073344

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6726328f79)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-08-23 13:15:17 +02:00
Michael Niedermayer bd28de1b4d avcodec/hevc_ps: Only discard overread VPS if a previous is available
Fixes Ticket4621

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57078e4d25)
2015-06-19 11:25:23 +02:00
Michael Niedermayer 6d7a0c37b1 avcodec/hevc_ps: Explicitly check num_tile_* for negative values
This fixes nothing but maybe helps coverity which does not see that this is failing later

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 65e5032955)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer 73ea11d721 avcodec/hevc_ps: Check vps_num_hrd_parameters
Fix CID1239052 part2

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b195aa5d52)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer 65b47dddcf avcodec/hevc_ps: More completely check vps_num_layer_sets
Fixes CID1239052  part1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 16c95b1073)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-06-01 23:25:20 +02:00
Michael Niedermayer 2049d95f2f avcodec/hevc_ps: Check cropping parameters more correctly
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06c70d4537)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-29 03:34:23 +02:00
Michael Niedermayer e8a44b8387 avcodec/hevc_ps: Sanity checks for some log2_* values
log2 values which imply numeric overflow are not supported

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 205b2ba3d6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-29 03:34:22 +02:00
Michael Niedermayer 7bce99216f avcodec/hevc_ps: More complete window reset
Fixes out of array read
Fixes: signal_sigsegv_35bcf26_471_cov_2806540268_CAINIT_A_SHARP_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 57e5812198)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-03-29 03:16:27 +02:00
Michael Niedermayer b6dc16bd95 avcodec/hevc_ps: Check diff_cu_qp_delta_depth
Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3281fa8925)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-18 02:32:11 +01:00
Michael Niedermayer 4b8f3c5bf3 avcodec/hevc_ps: Check num_long_term_ref_pics_sps
Fixes out of array access
Fixes: signal_sigsegv_35bd0f0_1182_cov_791726764_STRUCT_B_Samsung_4.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ea38e5a6b7)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:37 +01:00
Michael Niedermayer 2ba17ac96c avcodec/hevc_ps: Check return code from pps_range_extensions()
Fixes out of array read
Fixes: asan_heap-oob_177e222_885_cov_1532528832_MERGE_D_TI_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9f9440bd81)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-30 21:40:36 +01:00
Michael Niedermayer 045670a6f7 avcodec/hevc_ps: Check default display window bitstream and skip if invalid
Fixes Ticket4035

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 852aaead1f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-01 13:28:14 +01:00
Michael Niedermayer cbb277988a avcodec/hevc_ps: Always initialize backup in decode_vui()
Fixes CID1237283

Reviewed-by: Christophe Gisquet <christophe.gisquet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-06 21:36:56 +02:00
Michael Niedermayer aaaf7261b7 avcodec/hevc_ps: fix 1 vs. 0 typo
Found-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-22 23:00:41 +02:00
Christophe Gisquet 0625a38066 hevc_ps: check overflow and test alternate syntax
Some streams were found to have what appeared to be truncated SPS.
Their syntax seem to be valid at least until the end of the VUI, so
try that syntax if the parsing would overflow the SPS in the
conforming syntax.

Fixes ticket #3872.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-21 22:04:43 +02:00
Michael Niedermayer 7caacc50ae avcodec/hevc_ps: do cleanup in case of unsupported bit depth
Fixes memleak
Fixes CID1231989

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-20 15:14:53 +02:00
Christophe Gisquet 5ec85c9750 hevc: do generic validation of bitstream
After finishing parsing VPS/SPS/PPS/slice header, check remaining bits,
and if an overconsumption occurred, report invalid data.

Liked-by: BBB
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-11 02:47:05 +02:00
Christophe Gisquet 0d0d24af01 hevc_ps: verify P/T/L information
This makes the SPS parsing a little, but barely, safer.

Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-10 18:05:33 +02:00
Mickaël Raulet 37822a566f hevc/rext: remove warning for cross component prediction
cherry picked from commit fb595102593c94179502bb1967f978a1d99cc7cb

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-22 13:59:00 +02:00
Mickaël Raulet 627c044f50 hevc/rext: fixing rdpcm and scaling list for Range Extension
-new rext bitstreams:
PERSIST_RPARAM_A_RExt_Sony_1.bit           ok    =
QMATRIX_A_RExt_Sony_1.bit                  ok    =
SAO_A_RExt_MediaTek_1.bit                  ok    =

(cherry picked from commit cdea029d452c521f8e5bcbe589f44b13a4011604)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-22 13:07:54 +02:00
Michael Niedermayer d13a731fc1 avcodec/hevc_ps: Check abs_delta_rps
Fixes integer overflow

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-19 17:30:44 +02:00
Michael Niedermayer ba80b8d29b avcodec/hevc: check nb_cpb
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-19 16:51:05 +02:00
Mickaël Raulet fdb20db642 hevc/rext: put a warning log message instead of an error log message
cherry picked from commit 243cb99cff727d6a14c32cdff2748f6c255dbaf4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-18 23:47:06 +02:00
Michael Niedermayer 0fc2045d5f avcodec/hevc_ps: prevent stale pointer in malloc failure case
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 21:49:56 +02:00
Mickaël Raulet 5a41999d81 hevc/rext: basic infrastructure for supporting range extension
- support for 4:2:2 and 4:4:4 up to 12 bits
- add a new profile for range extension
(cherry picked from commit d3c067fa65bbc871758d28aa07f54123430ca346)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-15 13:47:35 +02:00
Michael Niedermayer ccd6911c18 avcodec/hevc_ps: do not loose all reference to pointers still in use
Fixes leaving a pointer to unreferenced memory
Fixes Ticket 3115

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-12 21:03:09 +02:00
Michael Niedermayer 66558fe715 avcodec/hevc_ps: Check layer_id_included_flags count
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-12 07:46:08 +02:00
Michael Niedermayer 6935ae22ed avcodec/hevc_ps: more complete check for vps_max_dec_pic_buffering
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-07-12 07:46:08 +02:00
Michael Niedermayer acc7cf70dc Merge commit '9b60d9197970658e91daf4b586397f450de9af69'
* commit '9b60d9197970658e91daf4b586397f450de9af69':
  hevc: Allow out of bound values for num_reorder_pics

Conflicts:
	libavcodec/hevc_ps.c

See: bc21260e64
See: ab296c7a9f
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-26 19:58:25 +02:00
Kieran Kunhya 9b60d91979 hevc: Allow out of bound values for num_reorder_pics
This fixes decoding for a sample that cannot be shared

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-06-26 07:14:57 +02:00
gcocherel ba70563d55 hevc/pps: optimized size of min_tb_addr_zs
reduce computation too
(cherry picked from commit 39c4d45c7788081c45c7fae51b7c5d0bcbaece9d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-24 13:11:40 +02:00
gcocherel f7f1f4c7ce avcodec/hevc_ps: remove min_cb_addr_zs
(cherry picked from commit ff7926d5092f9d4158108963e977e8c992322ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-24 13:06:42 +02:00
Kieran bc21260e64 hevc: Fix 4K sample video
Reviewed-by: smarter
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-15 23:24:11 +02:00
Michael Niedermayer eaaa5801ef Merge commit '4d33873c2990b8d6096f60fef384f0efc4482b55'
* commit '4d33873c2990b8d6096f60fef384f0efc4482b55':
  hevc: make pps/sps ids unsigned where necessary

Conflicts:
	libavcodec/hevc.h
	libavcodec/hevc_ps.c

See: d1e6602665
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-09 21:44:40 +01:00
Vittorio Giovara 4d33873c29 hevc: make pps/sps ids unsigned where necessary
Fixes integer overflow and out of array accesses.
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
2014-03-09 17:23:08 +01:00
Michael Niedermayer 920c01adce hevc: Use get_bits_long() in decode_vui()
Fix assertion failure.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-03-09 08:25:39 +01:00
Michael Niedermayer ef08d554e0 Merge commit '175e5063320f585118a5461f15dbacf2ce17e97d'
* commit '175e5063320f585118a5461f15dbacf2ce17e97d':
  hevc: Mention the missing SPS in the error message

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-19 22:10:27 +01:00
Luca Barbato 175e506332 hevc: Mention the missing SPS in the error message 2014-02-19 16:59:40 +01:00
Michael Niedermayer b818637b84 avcodec/hevc_ps: Use get_bits_long() in decode_vui()
Fix assertion failure
Fixes: a225222ef88a0f5b1e93e1d0432debc3-asan_static-oob_124a17d_1448_cov_77608227_DBLK_E_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-02-15 22:30:40 +01:00
Michael Niedermayer 5b93b2722d Merge commit '838740e6420538ad45982da6b1d3aa3ae91307f5'
* commit '838740e6420538ad45982da6b1d3aa3ae91307f5':
  hevc: Prevent some integer overflows

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:31:04 +01:00
Michael Niedermayer 29ba1cff2b Merge commit 'b37e796082b2d787aff3cd5631bb89c4fd374708'
* commit 'b37e796082b2d787aff3cd5631bb89c4fd374708':
  hevc: Use uint64 to check for tile dimensions

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:19:39 +01:00
Michael Niedermayer 42a29015e1 Merge commit 'a7a07cc98ac548297b5b0628cb81280e11952e3f'
* commit 'a7a07cc98ac548297b5b0628cb81280e11952e3f':
  hevc: check that VPS referenced from SPS exists

Conflicts:
	libavcodec/hevc_ps.c

See: d66bab0a69
Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-21 15:04:24 +01:00
Luca Barbato 838740e642 hevc: Prevent some integer overflows
get_ue_golomb_long() returns an unsigned.

Sample-Id: 00001541-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:59:18 +01:00
Luca Barbato b37e796082 hevc: Use uint64 to check for tile dimensions
And use unsigned datatypes.

Otherwise it would overflow.

Sample-Id: 00001315-google
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
2014-01-21 11:57:51 +01:00
Michael Niedermayer a7a07cc98a hevc: check that VPS referenced from SPS exists
This matches how its done for SPS/PPS.
Fixes null pointer dereference.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-01-21 11:57:36 +01:00
Michael Niedermayer ab296c7a9f avcodec/hevc_ps: Override max_dec_pic_buffering when its inconsistent with num_reorder_pics
Fixes Ticket3304

Alternatively max_dec_pic_buffering could be removed completely as its not used.

Based on a patch by Jose Santiago <santiago@haivision.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-15 04:45:42 +01:00
Michael Niedermayer d66bab0a69 avcodec/hevc_ps: check that VPS referenced from SPS exists
This matches how its done for SPS/PPS.
An alternative to this is to check it when its used.

Fixes null pointer dereference
Fixes: signal_sigsegv_e30a43_1437_CIP_A_Panasonic_3.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-06 04:30:04 +01:00