commit message
This commit is contained in:
Wang_JQ
parent
0ce59cc7b9
commit
884d50e16e
16
README.md
16
README.md
|
|
@ -12,6 +12,9 @@ HuggingFace🤗: [neukg/TechGPT-7B](https://huggingface.co/neukg)
|
|||
随着大模型时代的到来，大模型与知识图谱融合的工作日益成为当前研究的热点。为了对这项工作提供研究基础，东北大学知识图谱研究组于2023年6月发布了 [TechGPT-1.0](https://github.com/neukg/TechGPT) 大模型。
|
||||
|
||||
在 TechGPT-1.0 基础上，经过几个月的努力，东北大学知识图谱研究组发布功能更全、性能更好的 **TechGPT-2.0** 大模型。TechGPT-2.0 延续了 TechGPT-1.0 以“**知识图谱构建**”与“**智能问答**”为核心的构建目标，在继承 TechGPT-1.0 全部功能的基础上，**对其多项功能进行了增强**，并**扩充了一些新功能**。
|
||||
## 更新！！！
|
||||
* 2024/8/22 TechGPT-2.0更新TechGPT-2.0-Qwen1.5-7b 大模型,初步测试较Alpaca与Atom模型表现良好,且ALpaca与Atom模型参数已更新。
|
||||
* 2024/8/22 TechGPT-2.0新加Alpaca与Atom模型的WebGUI界面，用户可通过调整Web界面中的参数测试模型问答能力。
|
||||
## 内容导引
|
||||
| 章节 | 描述 |
|
||||
|----------------------|-----------------------------|
|
||||
|
|
@ -40,12 +43,13 @@ TechGPT-2.0 在继承了 TechGPT-1.0 的能力上进行了重要的改进，具
|
|||
|
||||
## 模型下载与体验
|
||||
### 下载地址
|
||||
| 模型名称 | 训练方式 | 大小 | HuggingFace 下载 | wisemodel 下载 | ModelScope 下载 |
|
||||
|:------------------------|:-------:|:-------:|:------------------------------------------------------------:|:-------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------:|
|
||||
| TechGPT-1.0 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-7B) | [[wisemodel社区]](https://www.wisemodel.cn/models/undefined/TechGPT-1.0) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-1.0) |
|
||||
| TechGPT-2.0-Alpaca 🆕 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-alpaca-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-Alpaca) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-Alpaca) |
|
||||
| TechGPT-2.0-Atom 🆕 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-atom-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-Atom) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-Atom) |
|
||||
| TechGPT-2.0-QLora 🆕 | Lora微调 | 3 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-QLora-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-QLora) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-QLora) |
|
||||
| 模型名称 | 训练方式 | 大小 | HuggingFace 下载 | wisemodel 下载 | ModelScope 下载 |
|
||||
|:-------------------------|:------:|:-----:|:------------------------------------------------------------:|:-------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------:|
|
||||
| TechGPT-1.0 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-7B) | [[wisemodel社区]](https://www.wisemodel.cn/models/undefined/TechGPT-1.0) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-1.0) |
|
||||
| TechGPT-2.0-Alpaca 🆕 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-alpaca-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-Alpaca) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-Alpaca) |
|
||||
| TechGPT-2.0-Atom 🆕 | 全量微调 | 13 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-atom-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-Atom) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-Atom) |
|
||||
| TechGPT-2.0-QLora 🆕 | Lora微调 | 3 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-QLora-hf) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-QLora) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-QLora) |
|
||||
| TechGPT-2.0-Qwen1.5 🆕 | 全量微调 | 14 GB | [[🤗HF]](https://huggingface.co/neukg/TechGPT-2.0-Qwen1.5-7b) | [[wisemodel社区]](https://www.wisemodel.cn/models/neukg/TechGPT-2.0-Qwen1.5) | [[<img src="https://g.alicdn.com/sail-web/maas/1.10.1/static/modelscopeIcon.cd89353f.svg" height="12">]](https://modelscope.cn/models/neukg01/TechGPT-2.0-Qwen1.5-7b) |
|
||||
|
||||
### 模型说明
|
||||
**TechGPT2** 在不同规模且经过扩充后的**TechKG**大规模的中文学术语料支持下训练完成。
|
||||
|
|
|
|||
|
|
@ -0,0 +1,97 @@
|
|||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import gradio as gr
|
||||
|
||||
model_id = "your path"
|
||||
|
||||
# 加载模型和分词器
|
||||
device = "cuda" if torch.cuda.is_available() else "cpu"
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_id,
|
||||
device_map="auto",
|
||||
torch_dtype=torch.float16,
|
||||
# load_in_8bit=True,
|
||||
# trust_remote_code=True,
|
||||
# use_flash_attention_2=True
|
||||
).eval()
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=False)
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
# 定义默认系统提示和模板
|
||||
DEFAULT_SYSTEM_PROMPT = "You are a helpful assistant. 你是一个乐于助人的助手。请你提供专业、有逻辑、内容真实、有价值的详细回复。"
|
||||
TEMPLATE = (
|
||||
"<s> A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: [INST] <<SYS>>\n"
|
||||
"{system_prompt}\n"
|
||||
"<</SYS>>\n\n"
|
||||
"{instruction} [/INST] ASSISTANT: "
|
||||
)
|
||||
|
||||
# 定义生成函数
|
||||
|
||||
|
||||
def generate_response(input_text, max_new_tokens, min_new_tokens, top_k, top_p, temperature, repetition_penalty, do_sample, num_beams):
|
||||
# 使用新的模板和系统提示
|
||||
prompt = TEMPLATE.format(
|
||||
system_prompt=DEFAULT_SYSTEM_PROMPT, instruction=input_text)
|
||||
input_ids = tokenizer(prompt, return_tensors="pt",
|
||||
add_special_tokens=False).input_ids
|
||||
input_ids = input_ids.to(model.device)
|
||||
|
||||
generate_input = {
|
||||
"input_ids": input_ids,
|
||||
"max_new_tokens": max_new_tokens,
|
||||
"min_new_tokens": min_new_tokens,
|
||||
"do_sample": do_sample,
|
||||
"top_k": top_k,
|
||||
"top_p": top_p,
|
||||
"temperature": temperature,
|
||||
"repetition_penalty": repetition_penalty,
|
||||
"num_beams": num_beams,
|
||||
"eos_token_id": tokenizer.eos_token_id,
|
||||
"bos_token_id": tokenizer.bos_token_id,
|
||||
"pad_token_id": tokenizer.pad_token_id
|
||||
}
|
||||
generate_ids = model.generate(**generate_input)
|
||||
text = tokenizer.decode(generate_ids[0], skip_special_tokens=True)
|
||||
text = text.split("ASSISTANT:")[-1].strip()
|
||||
return text
|
||||
|
||||
|
||||
# 使用 gr.Blocks 创建界面
|
||||
with gr.Blocks() as demo:
|
||||
gr.Markdown("# 文本生成应用")
|
||||
gr.Markdown("使用 Alpaca 模型生成文本。")
|
||||
|
||||
with gr.Row():
|
||||
with gr.Column():
|
||||
prompt_input = gr.Textbox(
|
||||
lines=5, placeholder="输入提示文本...", label="提示")
|
||||
max_new_tokens = gr.Slider(
|
||||
10, 1024, value=512, step=1, label="最大新生成的 tokens 数")
|
||||
min_new_tokens = gr.Slider(
|
||||
0, 100, value=0, step=1, label="最小生成的 tokens 数")
|
||||
top_k = gr.Slider(1, 100, value=50, step=1, label="Top-K")
|
||||
top_p = gr.Slider(0.0, 1.0, value=0.95, step=0.01, label="Top-P")
|
||||
temperature = gr.Slider(
|
||||
0.0, 2.0, value=1.0, step=0.01, label="Temperature")
|
||||
repetition_penalty = gr.Slider(
|
||||
1.0, 2.0, value=1.3, step=0.1, label="重复惩罚")
|
||||
do_sample = gr.Checkbox(label="使用采样生成 (do_sample)", value=False)
|
||||
num_beams = gr.Slider(1, 10, value=1, step=1,
|
||||
label="束搜索数 (num_beams)")
|
||||
|
||||
with gr.Column():
|
||||
output_text = gr.Textbox(label="生成文本")
|
||||
|
||||
generate_button = gr.Button("生成文本")
|
||||
|
||||
# 按钮点击事件
|
||||
generate_button.click(
|
||||
fn=generate_response,
|
||||
inputs=[prompt_input, max_new_tokens, min_new_tokens, top_k,
|
||||
top_p, temperature, repetition_penalty, do_sample, num_beams],
|
||||
outputs=output_text
|
||||
)
|
||||
|
||||
# 启动 Gradio 接口
|
||||
demo.launch(server_name="0.0.0.0", server_port=31814)
|
||||
|
|
@ -0,0 +1,85 @@
|
|||
import torch
|
||||
from transformers import AutoTokenizer, AutoModelForCausalLM
|
||||
import gradio as gr
|
||||
|
||||
# 加载模型和分词器
|
||||
device = "cuda" if torch.cuda.is_available() else "cpu"
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
'your path',
|
||||
device_map="auto",
|
||||
torch_dtype=torch.float16,
|
||||
# load_in_8bit=True,
|
||||
# trust_remote_code=True,
|
||||
# use_flash_attention_2=True
|
||||
).eval()
|
||||
tokenizer = AutoTokenizer.from_pretrained(
|
||||
'your path', use_fast=False)
|
||||
tokenizer.pad_token = tokenizer.eos_token
|
||||
|
||||
# 定义生成函数
|
||||
|
||||
|
||||
def generate_response(input_text, max_new_tokens, min_new_tokens, top_k, top_p, temperature, repetition_penalty, do_sample, num_beams):
|
||||
prompt = f"<s>Human: {input_text}\n</s><s>Assistant: "
|
||||
input_ids = tokenizer(prompt, return_tensors="pt",
|
||||
add_special_tokens=False).input_ids
|
||||
input_ids = input_ids.to(model.device)
|
||||
|
||||
generate_input = {
|
||||
"input_ids": input_ids,
|
||||
"max_new_tokens": max_new_tokens,
|
||||
"min_new_tokens": min_new_tokens,
|
||||
"do_sample": do_sample,
|
||||
"top_k": top_k,
|
||||
"top_p": top_p,
|
||||
"temperature": temperature,
|
||||
"repetition_penalty": repetition_penalty,
|
||||
"num_beams": num_beams,
|
||||
"eos_token_id": tokenizer.eos_token_id,
|
||||
"bos_token_id": tokenizer.bos_token_id,
|
||||
"pad_token_id": tokenizer.pad_token_id
|
||||
}
|
||||
generate_ids = model.generate(**generate_input)
|
||||
text = tokenizer.decode(generate_ids[0], skip_special_tokens=True)
|
||||
text = text.split("Assistant:")[-1]
|
||||
return text
|
||||
|
||||
|
||||
# 使用 gr.Blocks 创建界面
|
||||
with gr.Blocks() as demo:
|
||||
gr.Markdown("# 文本生成应用")
|
||||
gr.Markdown("使用 Atom-7B 模型生成文本。")
|
||||
|
||||
with gr.Row():
|
||||
with gr.Column():
|
||||
prompt_input = gr.Textbox(
|
||||
lines=5, placeholder="输入提示文本...", label="提示")
|
||||
max_new_tokens = gr.Slider(
|
||||
10, 1024, value=512, step=1, label="最大新生成的 tokens 数")
|
||||
min_new_tokens = gr.Slider(
|
||||
0, 100, value=0, step=1, label="最小生成的 tokens 数")
|
||||
top_k = gr.Slider(1, 100, value=50, step=1, label="Top-K")
|
||||
top_p = gr.Slider(0.0, 1.0, value=0.95, step=0.01, label="Top-P")
|
||||
temperature = gr.Slider(
|
||||
0.0, 2.0, value=1.0, step=0.01, label="Temperature")
|
||||
repetition_penalty = gr.Slider(
|
||||
1.0, 2.0, value=1.3, step=0.1, label="重复惩罚")
|
||||
do_sample = gr.Checkbox(label="使用采样生成 (do_sample)", value=False)
|
||||
num_beams = gr.Slider(1, 10, value=1, step=1,
|
||||
label="束搜索数 (num_beams)")
|
||||
|
||||
with gr.Column():
|
||||
output_text = gr.Textbox(label="生成文本")
|
||||
|
||||
generate_button = gr.Button("生成文本")
|
||||
|
||||
# 按钮点击事件
|
||||
generate_button.click(
|
||||
fn=generate_response,
|
||||
inputs=[prompt_input, max_new_tokens, min_new_tokens, top_k,
|
||||
top_p, temperature, repetition_penalty, do_sample, num_beams],
|
||||
outputs=output_text
|
||||
)
|
||||
|
||||
# 启动 Gradio 接口
|
||||
demo.launch(server_name="0.0.0.0", server_port=31814)
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
# Copyright (c) Alibaba Cloud.
|
||||
#
|
||||
# This source code is licensed under the license found in the
|
||||
# LICENSE file in the root directory of this source tree.
|
||||
|
||||
"""A simple web interactive chat demo based on gradio."""
|
||||
|
||||
from argparse import ArgumentParser
|
||||
from threading import Thread
|
||||
|
||||
import gradio as gr
|
||||
import torch
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer, TextIteratorStreamer
|
||||
|
||||
DEFAULT_CKPT_PATH = 'your path'
|
||||
|
||||
|
||||
def _get_args():
|
||||
parser = ArgumentParser()
|
||||
parser.add_argument("-c", "--checkpoint-path", type=str, default=DEFAULT_CKPT_PATH,
|
||||
help="Checkpoint name or path, default to %(default)r")
|
||||
parser.add_argument("--cpu-only", action="store_true", help="Run demo with CPU only")
|
||||
|
||||
parser.add_argument("--share", action="store_true", default=False,
|
||||
help="Create a publicly shareable link for the interface.")
|
||||
parser.add_argument("--inbrowser", action="store_true", default=False,
|
||||
help="Automatically launch the interface in a new tab on the default browser.")
|
||||
parser.add_argument("--server-port", type=int, default=31814,
|
||||
help="Demo server port.")
|
||||
parser.add_argument("--server-name", type=str, default="0.0.0.0",
|
||||
help="Demo server name.")
|
||||
|
||||
args = parser.parse_args()
|
||||
return args
|
||||
|
||||
|
||||
def _load_model_tokenizer(args):
|
||||
tokenizer = AutoTokenizer.from_pretrained(
|
||||
args.checkpoint_path, resume_download=True,
|
||||
)
|
||||
|
||||
if args.cpu_only:
|
||||
device_map = "cpu"
|
||||
else:
|
||||
device_map = "auto"
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
args.checkpoint_path,
|
||||
torch_dtype="auto",
|
||||
device_map=device_map,
|
||||
resume_download=True,
|
||||
).eval()
|
||||
model.generation_config.max_new_tokens = 2048 # For chat.
|
||||
|
||||
return model, tokenizer
|
||||
|
||||
|
||||
def _chat_stream(model, tokenizer, query, history):
|
||||
conversation = [
|
||||
{'role': 'system', 'content': 'You are a helpful assistant.'},
|
||||
]
|
||||
for query_h, response_h in history:
|
||||
conversation.append({'role': 'user', 'content': query_h})
|
||||
conversation.append({'role': 'assistant', 'content': response_h})
|
||||
conversation.append({'role': 'user', 'content': query})
|
||||
inputs = tokenizer.apply_chat_template(
|
||||
conversation,
|
||||
add_generation_prompt=True,
|
||||
return_tensors='pt',
|
||||
)
|
||||
inputs = inputs.to(model.device)
|
||||
streamer = TextIteratorStreamer(tokenizer=tokenizer, skip_prompt=True, timeout=60.0, skip_special_tokens=True)
|
||||
generation_kwargs = dict(
|
||||
input_ids=inputs,
|
||||
streamer=streamer,
|
||||
)
|
||||
thread = Thread(target=model.generate, kwargs=generation_kwargs)
|
||||
thread.start()
|
||||
|
||||
for new_text in streamer:
|
||||
yield new_text
|
||||
|
||||
|
||||
def _gc():
|
||||
import gc
|
||||
gc.collect()
|
||||
if torch.cuda.is_available():
|
||||
torch.cuda.empty_cache()
|
||||
|
||||
|
||||
def _launch_demo(args, model, tokenizer):
|
||||
|
||||
def predict(_query, _chatbot, _task_history):
|
||||
print(f"User: {_query}")
|
||||
_chatbot.append((_query, ""))
|
||||
full_response = ""
|
||||
response = ""
|
||||
for new_text in _chat_stream(model, tokenizer, _query, history=_task_history):
|
||||
response += new_text
|
||||
_chatbot[-1] = (_query, response)
|
||||
|
||||
yield _chatbot
|
||||
full_response = response
|
||||
|
||||
print(f"History: {_task_history}")
|
||||
_task_history.append((_query, full_response))
|
||||
print(f"Qwen2-Instruct: {full_response}")
|
||||
|
||||
def regenerate(_chatbot, _task_history):
|
||||
if not _task_history:
|
||||
yield _chatbot
|
||||
return
|
||||
item = _task_history.pop(-1)
|
||||
_chatbot.pop(-1)
|
||||
yield from predict(item[0], _chatbot, _task_history)
|
||||
|
||||
def reset_user_input():
|
||||
return gr.update(value="")
|
||||
|
||||
def reset_state(_chatbot, _task_history):
|
||||
_task_history.clear()
|
||||
_chatbot.clear()
|
||||
_gc()
|
||||
return _chatbot
|
||||
|
||||
with gr.Blocks() as demo:
|
||||
gr.Markdown("""\
|
||||
<p align="center"><img src="https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/logo_qwen2.png" style="height: 80px"/><p>""")
|
||||
gr.Markdown("""<center><font size=8>Qwen2 Chat Bot</center>""")
|
||||
gr.Markdown(
|
||||
"""\
|
||||
<center><font size=3>This WebUI is based on Qwen2-Instruct, developed by Alibaba Cloud. \
|
||||
(本WebUI基于Qwen2-Instruct打造，实现聊天机器人功能。)</center>""")
|
||||
gr.Markdown("""\
|
||||
<center><font size=4>
|
||||
Qwen2-7B-Instruct <a href="https://modelscope.cn/models/qwen/Qwen2-7B-Instruct/summary">🤖 </a> |
|
||||
<a href="https://huggingface.co/Qwen/Qwen2-7B-Instruct">🤗</a>  ｜
|
||||
Qwen2-72B-Instruct <a href="https://modelscope.cn/models/qwen/Qwen2-72B-Instruct/summary">🤖 </a> |
|
||||
<a href="https://huggingface.co/Qwen/Qwen2-72B-Instruct">🤗</a>  ｜
|
||||
 <a href="https://github.com/QwenLM/Qwen2">Github</a></center>""")
|
||||
|
||||
chatbot = gr.Chatbot(label='Qwen2-Instruct', elem_classes="control-height")
|
||||
query = gr.Textbox(lines=2, label='Input')
|
||||
task_history = gr.State([])
|
||||
|
||||
with gr.Row():
|
||||
empty_btn = gr.Button("🧹 Clear History (清除历史)")
|
||||
submit_btn = gr.Button("🚀 Submit (发送)")
|
||||
regen_btn = gr.Button("🤔️ Regenerate (重试)")
|
||||
|
||||
submit_btn.click(predict, [query, chatbot, task_history], [chatbot], show_progress=True)
|
||||
submit_btn.click(reset_user_input, [], [query])
|
||||
empty_btn.click(reset_state, [chatbot, task_history], outputs=[chatbot], show_progress=True)
|
||||
regen_btn.click(regenerate, [chatbot, task_history], [chatbot], show_progress=True)
|
||||
|
||||
gr.Markdown("""\
|
||||
<font size=2>Note: This demo is governed by the original license of Qwen2. \
|
||||
We strongly advise users not to knowingly generate or allow others to knowingly generate harmful content, \
|
||||
including hate speech, violence, pornography, deception, etc. \
|
||||
(注：本演示受Qwen2的许可协议限制。我们强烈建议，用户不应传播及不应允许他人传播以下内容，\
|
||||
包括但不限于仇恨言论、暴力、色情、欺诈相关的有害信息。)""")
|
||||
|
||||
demo.queue().launch(
|
||||
share=args.share,
|
||||
inbrowser=args.inbrowser,
|
||||
server_port=args.server_port,
|
||||
server_name=args.server_name,
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
args = _get_args()
|
||||
|
||||
model, tokenizer = _load_model_tokenizer(args)
|
||||
|
||||
_launch_demo(args, model, tokenizer)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Loading…
Reference in New Issue