e25a19fc7c
swscale/aarch64/output.S: refactor ff_yuv2plane1_8_neon
...
The benchmarks (before vs after) were gathered using
./tests/checkasm/checkasm --test=sw_scale --bench --runs=6 | grep yuv2yuv1
A78 before:
yuv2yuv1_0_512_accurate_c: 2039.5 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 385.5 ( 5.29x)
yuv2yuv1_0_512_approximate_c: 2110.5 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 385.5 ( 5.47x)
yuv2yuv1_3_512_accurate_c: 2061.2 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 381.2 ( 5.41x)
yuv2yuv1_3_512_approximate_c: 2099.2 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 381.2 ( 5.51x)
yuv2yuv1_8_512_accurate_c: 2054.2 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 385.5 ( 5.33x)
yuv2yuv1_8_512_approximate_c: 2112.2 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 385.5 ( 5.48x)
yuv2yuv1_11_512_accurate_c: 2036.0 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 381.2 ( 5.34x)
yuv2yuv1_11_512_approximate_c: 2115.0 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 381.2 ( 5.55x)
yuv2yuv1_16_512_accurate_c: 2066.5 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 385.5 ( 5.36x)
yuv2yuv1_16_512_approximate_c: 2100.8 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 385.5 ( 5.45x)
yuv2yuv1_19_512_accurate_c: 2059.8 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 381.2 ( 5.40x)
yuv2yuv1_19_512_approximate_c: 2102.8 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 381.2 ( 5.52x)
After:
yuv2yuv1_0_512_accurate_c: 2206.0 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 139.2 (15.84x)
yuv2yuv1_0_512_approximate_c: 2050.0 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 139.2 (14.72x)
yuv2yuv1_3_512_accurate_c: 2205.2 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 138.0 (15.98x)
yuv2yuv1_3_512_approximate_c: 2052.5 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 138.0 (14.87x)
yuv2yuv1_8_512_accurate_c: 2171.0 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 139.2 (15.59x)
yuv2yuv1_8_512_approximate_c: 2064.2 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 139.2 (14.82x)
yuv2yuv1_11_512_accurate_c: 2164.8 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 138.0 (15.69x)
yuv2yuv1_11_512_approximate_c: 2048.8 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 138.0 (14.85x)
yuv2yuv1_16_512_accurate_c: 2154.5 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 139.2 (15.47x)
yuv2yuv1_16_512_approximate_c: 2047.2 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 139.2 (14.70x)
yuv2yuv1_19_512_accurate_c: 2144.5 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 138.0 (15.54x)
yuv2yuv1_19_512_approximate_c: 2046.0 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 138.0 (14.83x)
A72 before:
yuv2yuv1_0_512_accurate_c: 3779.8 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 527.8 ( 7.16x)
yuv2yuv1_0_512_approximate_c: 4128.2 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 528.2 ( 7.81x)
yuv2yuv1_3_512_accurate_c: 3836.2 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 527.0 ( 7.28x)
yuv2yuv1_3_512_approximate_c: 3991.0 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 526.8 ( 7.58x)
yuv2yuv1_8_512_accurate_c: 3732.8 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 525.5 ( 7.10x)
yuv2yuv1_8_512_approximate_c: 4060.0 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 527.0 ( 7.70x)
yuv2yuv1_11_512_accurate_c: 3836.2 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 530.0 ( 7.24x)
yuv2yuv1_11_512_approximate_c: 4014.0 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 530.0 ( 7.57x)
yuv2yuv1_16_512_accurate_c: 3726.2 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 525.5 ( 7.09x)
yuv2yuv1_16_512_approximate_c: 4114.2 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 526.2 ( 7.82x)
yuv2yuv1_19_512_accurate_c: 3812.2 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 530.0 ( 7.19x)
yuv2yuv1_19_512_approximate_c: 4012.2 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 530.0 ( 7.57x)
After:
yuv2yuv1_0_512_accurate_c: 3716.8 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 215.1 (17.28x)
yuv2yuv1_0_512_approximate_c: 3877.8 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 222.8 (17.40x)
yuv2yuv1_3_512_accurate_c: 3717.1 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 217.8 (17.06x)
yuv2yuv1_3_512_approximate_c: 3801.6 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 220.3 (17.25x)
yuv2yuv1_8_512_accurate_c: 3716.6 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 213.8 (17.38x)
yuv2yuv1_8_512_approximate_c: 3831.8 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 218.1 (17.57x)
yuv2yuv1_11_512_accurate_c: 3717.1 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 219.1 (16.97x)
yuv2yuv1_11_512_approximate_c: 3801.6 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 216.1 (17.59x)
yuv2yuv1_16_512_accurate_c: 3716.6 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 213.6 (17.40x)
yuv2yuv1_16_512_approximate_c: 3831.6 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 215.1 (17.82x)
yuv2yuv1_19_512_accurate_c: 3717.1 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 223.8 (16.61x)
yuv2yuv1_19_512_approximate_c: 3801.6 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 219.1 (17.35x)
x13s before:
yuv2yuv1_0_512_accurate_c: 1435.1 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 221.1 ( 6.49x)
yuv2yuv1_0_512_approximate_c: 1405.4 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 219.1 ( 6.41x)
yuv2yuv1_3_512_accurate_c: 1418.6 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 215.9 ( 6.57x)
yuv2yuv1_3_512_approximate_c: 1405.9 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 224.1 ( 6.27x)
yuv2yuv1_8_512_accurate_c: 1433.9 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 218.6 ( 6.56x)
yuv2yuv1_8_512_approximate_c: 1412.9 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 218.9 ( 6.46x)
yuv2yuv1_11_512_accurate_c: 1449.1 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 217.6 ( 6.66x)
yuv2yuv1_11_512_approximate_c: 1410.9 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 221.1 ( 6.38x)
yuv2yuv1_16_512_accurate_c: 1402.1 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 214.6 ( 6.53x)
yuv2yuv1_16_512_approximate_c: 1422.4 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 222.9 ( 6.38x)
yuv2yuv1_19_512_accurate_c: 1421.6 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 217.4 ( 6.54x)
yuv2yuv1_19_512_approximate_c: 1421.6 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 221.4 ( 6.42x)
After:
yuv2yuv1_0_512_accurate_c: 1413.6 ( 1.00x)
yuv2yuv1_0_512_accurate_neon: 80.6 (17.53x)
yuv2yuv1_0_512_approximate_c: 1455.6 ( 1.00x)
yuv2yuv1_0_512_approximate_neon: 80.6 (18.05x)
yuv2yuv1_3_512_accurate_c: 1429.1 ( 1.00x)
yuv2yuv1_3_512_accurate_neon: 77.4 (18.47x)
yuv2yuv1_3_512_approximate_c: 1462.6 ( 1.00x)
yuv2yuv1_3_512_approximate_neon: 80.6 (18.14x)
yuv2yuv1_8_512_accurate_c: 1425.4 ( 1.00x)
yuv2yuv1_8_512_accurate_neon: 77.9 (18.30x)
yuv2yuv1_8_512_approximate_c: 1436.6 ( 1.00x)
yuv2yuv1_8_512_approximate_neon: 80.9 (17.76x)
yuv2yuv1_11_512_accurate_c: 1429.4 ( 1.00x)
yuv2yuv1_11_512_accurate_neon: 76.1 (18.78x)
yuv2yuv1_11_512_approximate_c: 1447.1 ( 1.00x)
yuv2yuv1_11_512_approximate_neon: 78.4 (18.46x)
yuv2yuv1_16_512_accurate_c: 1439.9 ( 1.00x)
yuv2yuv1_16_512_accurate_neon: 77.6 (18.55x)
yuv2yuv1_16_512_approximate_c: 1422.1 ( 1.00x)
yuv2yuv1_16_512_approximate_neon: 78.1 (18.20x)
yuv2yuv1_19_512_accurate_c: 1447.1 ( 1.00x)
yuv2yuv1_19_512_accurate_neon: 78.1 (18.52x)
yuv2yuv1_19_512_approximate_c: 1474.4 ( 1.00x)
yuv2yuv1_19_512_approximate_neon: 78.1 (18.87x)
Signed-off-by: Martin Storsjö <martin@martin.st>
2025-02-07 12:05:06 +02:00
a76b409dd0
aarch64: Reindent all assembly to 8/24 column indentation
...
libavcodec/aarch64/vc1dsp_neon.S is skipped here, as it intentionally
uses a layered indentation style to visually show how different
unrolled/interleaved phases fit together.
Signed-off-by: Martin Storsjö <martin@martin.st>
2023-10-21 23:25:54 +03:00
184103b310
aarch64: Consistently use lowercase for vector element specifiers
...
Signed-off-by: Martin Storsjö <martin@martin.st>
2023-10-21 23:25:18 +03:00
0d7caa5b09
swscale/aarch64: add vscale specializations
...
This commit adds new code paths for vscale when filterSize is 2, 4, or
8. By using specialized code with unrolling to match the filterSize we
can improve performance.
On AWS c7g (Graviton 3, Neoverse V1) instances:
before after
yuv2yuvX_2_0_512_accurate_neon: 558.8 268.9
yuv2yuvX_4_0_512_accurate_neon: 637.5 434.9
yuv2yuvX_8_0_512_accurate_neon: 1144.8 806.2
yuv2yuvX_16_0_512_accurate_neon: 2080.5 1853.7
Signed-off-by: Jonathan Swinney <jswinney@amazon.com>
Signed-off-by: Martin Storsjö <martin@martin.st>
2022-08-16 13:40:42 +03:00
3e708722a2
swscale/aarch64: vscale optimization
...
Use scalar times vector multiply accumlate instructions instead of
vector times vector to remove the need for replicating load instructions
which are slightly slower.
On AWS c7g (Graviton 3, Neoverse V1) instances:
yuv2yuvX_8_0_512_accurate_neon: 1144.8 987.4
yuv2yuvX_16_0_512_accurate_neon: 2080.5 1869.4
Signed-off-by: Jonathan Swinney <jswinney@amazon.com>
Signed-off-by: Martin Storsjö <martin@martin.st>
2022-08-16 13:40:42 +03:00
9025d5c5ce
swscale: aarch64: Don't clobber callee-saved registers v8-v15
...
Signed-off-by: Martin Storsjö <martin@martin.st>
2020-04-21 23:41:13 +03:00
c3a17ffff6
swscale/aarch64: use multiply accumulate and shift-right narrow
...
This patch rewrites the innermost loop of ff_yuv2planeX_8_neon to avoid zips and
horizontal adds by using fused multiply adds. The patch also uses ld1r to load
one element and replicate it across all lanes of the vector. The patch also
improves the clipping code by removing the shift right instructions and
performing the shift with the shift-right narrow instructions.
I see 8% difference on an m6g instance with neoverse-n1 CPUs:
$ ffmpeg -nostats -f lavfi -i testsrc2=4k:d=2 -vf bench=start,scale=1024x1024,bench=stop -f null -
before: t:0.014015 avg:0.014096 max:0.015018 min:0.013971
after: t:0.012985 avg:0.013013 max:0.013996 min:0.012818
Tested with `make check` on aarch64-linux.
Signed-off-by: Sebastian Pop <spop@amazon.com>
Reviewed-by: Clément Bœsch <u@pkh.me>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-01-04 20:59:31 +01:00
c921f4f687
sws/aarch64: add ff_yuv2planeX_8_neon
2016-04-11 16:27:19 +02:00
Krzysztof Pyrkosz
Martin Storsjö
Swinney, Jonathan
Sebastian Pop
Clément Bœsch