mirror of https://github.com/alibaba/MNN.git
70 lines
2.2 KiB
Markdown
70 lines
2.2 KiB
Markdown
# 脚本工具
|
||
一些功能性脚本，提供各种功能。
|
||
|
||
## apply_gptq.py
|
||
将GPTQ的权重写入到量化的MNN权重中。
|
||
|
||
### 用法
|
||
```
|
||
usage: apply_gptq.py [-h] --mnn_graph MNN_GRAPH --mnn_weight MNN_WEIGHT --gptq_tensor GPTQ_TENSOR
|
||
|
||
apply_gptq
|
||
|
||
options:
|
||
-h, --help show this help message and exit
|
||
--mnn_graph MNN_GRAPH
|
||
mnn graph json path.
|
||
--mnn_weight MNN_WEIGHT
|
||
mnn weight file path.
|
||
--gptq_tensor GPTQ_TENSOR
|
||
gptq tensor path.
|
||
```
|
||
|
||
### 参数
|
||
- MNN_GRAPH: 模型计算图的json文件，获取方法：`./MNNDump2Json model.mnn model.json`
|
||
- MNN_WEIGHT: 模型的权重文件，如：`gptq.mnn.weight`
|
||
- GPTQ_TENSOR: GPTQ量化后的权重文件，`model.safetensor`
|
||
|
||
### 示例
|
||
使用该脚本生成gptq量化的权重`gptq.mnn.weight`
|
||
```sh
|
||
cd build
|
||
./MNNDump2Json model.mnn model.json
|
||
cp model.mnn.weight gptq.mnn.weight
|
||
python ../tools/script/apply_gptq.py --mnn_graph model.json --mnn_weight gptq.mnn.weight --gptq_tensor model.safetensor
|
||
```
|
||
|
||
## apply_lora.py
|
||
|
||
合并base模型的计算图和lora模型的权重文件，生成新的计算图。
|
||
|
||
### 用法
|
||
```sh
|
||
usage: apply_lora.py [-h] --base BASE --lora LORA [--scale SCALE] [--fuse FUSE] [--out OUT]
|
||
|
||
apply_lora
|
||
|
||
options:
|
||
-h, --help show this help message and exit
|
||
--base BASE base model json path.
|
||
--lora LORA lora dir path or *.safetensors path.
|
||
--scale SCALE lora scale: `alpha/r`.
|
||
--fuse FUSE fuse A and B.
|
||
--out OUT out file name.
|
||
```
|
||
|
||
### 参数
|
||
- BASE: base.json, base模型计算图的json文件，获取方法：`./MNNDump2Json base.mnn base.json`
|
||
- LORA: lora权重文件夹或者lora权重的safetensors
|
||
- SCALE: lora权重的scale, `lora_alpha / lora_r`, 一般为4.0
|
||
- FUSE: 是否将lora_A与lora_B合并成为一个lora权重，合并后模型较大
|
||
- OUT: 生成新的计算图文件名，默认为`lora.json`，转换为模型：`./MNNRevert2Buffer lora.json lora.mnn`
|
||
|
||
### 示例
|
||
使用该脚本生成lora对应的模型`lora.mnn`, 用法: [LoRA](../transformers/llm.html#lora)
|
||
```sh
|
||
cd build
|
||
./MNNDump2Json base.mnn base.json
|
||
python ../tools/script/apply_lora.py --base base.json --lora lora_dir
|
||
./MNNRevert2Buffer lora.json lora.mnn
|
||
``` |