Commit Graph

6 Commits

Author SHA1 Message Date
xinhao.zheng e376109335 Integrate Asym int4 ukernels to MNN
Integrate two kernels matmul_clamp_f16_qsi8d32p_qai4c32p
and matmul_clamp_f32_qsi8d32p_qai4c32p. Now KleidiAI support
asymmetric & block wise quantified & f32/f16activation.

Remove .tar.gz package from source code. Will download from
https://gitlab.arm.com/kleidi/kleidiai/-/releases when cmake.
2025-04-24 08:32:00 +08:00
xinhao.zheng 9e57159bce Refine some static into init 2025-02-11 15:33:06 +08:00
xinhao.zheng 332912cb6b Integrate KleidiAI sme int4 kernel
Add logic to select micro kernel functions when SME2 is enable.
Thread number will be forced to 1 when run matmul, for better
energy efficiency ratio.
2025-02-11 14:23:54 +08:00
xinhao.zheng 73344cd4d0 Refine some functions. 2025-01-09 09:58:26 +08:00
xinhao.zheng 82efb15c3f Refactor MNN KleidiAI interface
Refactor MNN_KleidiAI interface to support more model types,
and facilitate subsequent KleidiAI ukernels' integration.

Re-abstract information stored in class KleidiAI:
1) static info: not related to loaded model, initialized when
interface is constructed and never changed.
2) status: will change while pipeline is running.

Let interface and loaded model decouple for more complex mix
of multiple types of models. Add mAccelType in MNN data structure,
kleidiAI interface will rely on this type to decide which branch
to go.

Move some pack functions to mnn_kleidiai_util.cpp.

Add CPU feature detection in source/backend/cpu/CPURuntime.hpp.
Subsequent ukernels need SME information.
2025-01-07 10:18:20 +08:00
yiyang.fan 630db0edbf Update the KleidiAI version to r0.5.0. 2024-12-10 18:24:00 +08:00