mirror of https://github.com/alibaba/MNN.git
Compare commits
6 Commits
b980184b41
...
3a57d17324
| Author | SHA1 | Date |
|---|---|---|
Jenni Vandervort
|
3a57d17324 | |
jxt1234
|
75484d1efe | |
若遗
|
982189831f | |
|
若遗
|
07af494c11 | |
|
若遗
|
02fbee79c0 | |
shunlibest
|
30b463fe52 |
|
|
@ -7,11 +7,11 @@
|
|||
[](http://www.mnn.zone)
|
||||
|
||||
[](./apps/Android/MnnLlmChat/README.md)
|
||||
[](./apps/Android/Mnn3dAvatar/README.md)
|
||||
[](./apps/Android/MnnTaoAvatar/README.md)
|
||||
|
||||
|
||||
## News 🔥
|
||||
- [2025/06/11] New App MNN TaoAvatar released, you can talk with 3DAvatar offline with LLM, ASR, TTS, A2BS and NNR models all run local on your device!! [MNN TaoAvatar](./apps/Android/Mnn3dAvatar/README.md)
|
||||
- [2025/06/11] New App MNN TaoAvatar released, you can talk with 3DAvatar offline with LLM, ASR, TTS, A2BS and NNR models all run local on your device!! [MNN TaoAvatar](./apps/Android/MnnTaoAvatar/README.md)
|
||||
<p align="center">
|
||||
<img width="20%" alt="Icon" src="https://meta.alicdn.com/data/mnn/avatar/avatar_demo.gif" style="margin: 0 10px;">
|
||||
</p>
|
||||
|
|
|
|||
|
|
@ -1,13 +1,16 @@
|
|||
# MNN-TaoAvatar Android - Local 3D Avatar Intelligence
|
||||
|
||||
# MNN TaoAvatar Android - Local 3D Avatar Intelligence
|
||||
|
||||
<div align="center">
|
||||
<b style="color:red;">⚠️ Attention: this folder has been moved to: </b>
|
||||
<a href="../MnnTaoAvatar/README.md">apps/Android/MnnTaoAvatar</a>
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<img width="20%" alt="Icon" src="https://meta.alicdn.com/data/mnn/avatar/avatar_demo.gif" style="margin: 0 10px;">
|
||||
</p>
|
||||
|
||||
+ [中文](./README_CN.md)
|
||||
+ [Download](#releases)
|
||||
+ [中文](../MnnTaoAvatar/README_CN.md)
|
||||
+ [Download](../MnnTaoAvatar/README.md#releases)
|
||||
|
||||
Welcome to **MNN TaoAvatar App**, a full-featured mobile app about the [TaoAvatar paper (arXiv:2503.17032v1)](https://arxiv.org/html/2503.17032v1). This project brings multimodal AI avatars to life directly on Android devices, running **all models locally**, including:
|
||||
|
||||
|
|
@ -29,7 +32,7 @@ Welcome to **MNN TaoAvatar App**, a full-featured mobile app about the [TaoAvata
|
|||
- **100% offline** and privacy-focused
|
||||
|
||||
## How it Works
|
||||
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
|
|
@ -52,7 +55,7 @@ Because **all AI models are executed locally on-device**, this project requires
|
|||
|
||||
```bash
|
||||
git clone https://github.com/alibaba/MNN.git
|
||||
cd apps/Android/Mnn3dAvatar
|
||||
cd apps/Android/MnnTaoAvatar
|
||||
```
|
||||
|
||||
### 2. Build and Deploy
|
||||
|
|
@ -64,12 +67,6 @@ cd apps/Android/Mnn3dAvatar
|
|||
./gradlew installDebug
|
||||
```
|
||||
|
||||
## Releases
|
||||
## Version 0.0.1
|
||||
+ Click here to [download](https://meta.alicdn.com/data/mnn/avatar/mnn_avatar_0_0_1.apk)
|
||||
+ this is our first public released version; you can chat with 3d avatar in the app with asr and tts; if you have any questions, please feel free to open an issue for assistance.
|
||||
|
||||
|
||||
## Links And References
|
||||
|
||||
- [TaoAvatar Paper](https://arxiv.org/html/2503.17032v1)
|
||||
|
|
|
|||
|
|
@ -1,10 +1,14 @@
|
|||
# MNN TaoAvatar Android - 本地智能数字人
|
||||
# MNN-TaoAvatar Android - 本地智能数字人
|
||||
|
||||
<div align="center">
|
||||
<b style="color:red;">⚠️ 注意:此文件夹已经移动到： </b>
|
||||
<a href="../MnnTaoAvatar/README_CN.md">apps/Android/MnnTaoAvatar</a>
|
||||
</div>
|
||||
<p align="center">
|
||||
<img width="20%" alt="Icon" src="https://meta.alicdn.com/data/mnn/avatar/avatar_demo.gif" style="margin: 0 10px;">
|
||||
</p>
|
||||
|
||||
+ [下载](#releases)
|
||||
+ [下载](../MnnTaoAvatar/README_CN.md#releases)
|
||||
|
||||
这是一个一个本地运行、完全离线、支持多模态交互的智能数字人App！
|
||||
|
||||
|
|
@ -22,7 +26,7 @@
|
|||
* **100%离线运行**：完全本地运行，守护隐私更放心
|
||||
|
||||
## 工作原理
|
||||
|
||||
|
||||
|
||||
## 硬件要求
|
||||
|
||||
|
|
@ -41,7 +45,7 @@
|
|||
|
||||
```bash
|
||||
git clone https://github.com/alibaba/MNN.git
|
||||
cd apps/Android/Mnn3dAvatar
|
||||
cd apps/Android/MnnTaoAvatar
|
||||
```
|
||||
|
||||
2. 构建并部署
|
||||
|
|
@ -53,11 +57,6 @@ cd apps/Android/Mnn3dAvatar
|
|||
```
|
||||
|
||||
|
||||
## Releases
|
||||
## Version 0.0.1
|
||||
+ 点击这里[下载](https://meta.alicdn.com/data/mnn/avatar/mnn_avatar_0_0_1.apk)
|
||||
+ 这是我们首次公开发布的版本；您可以在应用程序中通过语音识别（ASR）和语音合成（TTS）与3D虚拟形象进行聊天；如果您有任何问题，请随时提交Issue以获得帮助。
|
||||
|
||||
## 更多相关资源
|
||||
|
||||
* [TaoAvatar 论文](https://arxiv.org/html/2503.17032v1)
|
||||
|
|
|
|||
|
|
@ -181,7 +181,7 @@ class ModelDownloadManager private constructor(private val context: Context) {
|
|||
if (getDownloadedFile(modelId) != null) {
|
||||
downloadInfo.downlodaState = DownloadInfo.DownloadSate.COMPLETED
|
||||
downloadInfo.progress = 1.0
|
||||
} else if (getDownloadSizeTotal(ApplicationProvider.get(), modelId) > 0) {
|
||||
} else if (getDownloadSizeSaved(ApplicationProvider.get(), modelId) > 0) {
|
||||
val totalSize = getDownloadSizeTotal(ApplicationProvider.get(), modelId)
|
||||
val savedSize = getDownloadSizeSaved(ApplicationProvider.get(), modelId)
|
||||
downloadInfo.totalSize = totalSize
|
||||
|
|
|
|||
|
|
@ -0,0 +1,80 @@
|
|||
|
||||
# MNN-TaoAvatar Android - Local 3D Avatar Intelligence
|
||||
|
||||
<p align="center">
|
||||
<img width="20%" alt="Icon" src="https://meta.alicdn.com/data/mnn/avatar/avatar_demo.gif" style="margin: 0 10px;">
|
||||
</p>
|
||||
|
||||
+ [中文](./README_CN.md)
|
||||
+ [Download](#releases)
|
||||
|
||||
Welcome to **MNN TaoAvatar App**, a full-featured mobile app about the [TaoAvatar paper (arXiv:2503.17032v1)](https://arxiv.org/html/2503.17032v1). This project brings multimodal AI avatars to life directly on Android devices, running **all models locally**, including:
|
||||
|
||||
- LLM (Large Language Model)
|
||||
- ASR (Automatic Speech Recognition)
|
||||
- TTS (Text-to-Speech)
|
||||
- A2BS (Audio-to-BlendShape)
|
||||
- NNR (Neural Rendering)
|
||||
|
||||
> The iOS App will be coming later, stay tuned for updates!
|
||||
|
||||
## Features
|
||||
|
||||
- **Conversational AI** powered by a local LLM
|
||||
- **Speech-to-text** with embedded ASR models
|
||||
- **Voice synthesis** with TTS on-device
|
||||
- **Avatar behavior animation** via A2BS(Audio-to-BlendShape)
|
||||
- **Real-time neural rendering** for expressive avatars
|
||||
- **100% offline** and privacy-focused
|
||||
|
||||
## How it Works
|
||||
|
||||
|
||||
## Requirements
|
||||
|
||||
Because **all AI models are executed locally on-device**, this project requires **high-performance hardware** to run smoothly.
|
||||
|
||||
### Minimum Device Requirements
|
||||
|
||||
- **Snapdragon 8 Gen 3** or **equivalent flagship SoC**
|
||||
_Examples: Snapdragon 8 Gen 3, Dimensity 9200 to have smooth experience._
|
||||
- **8 GB RAM** or more
|
||||
- **5 GB free disk space** for model files
|
||||
- **ARM64 architecture**
|
||||
|
||||
> ⚠️ Devices below these specs may experience lag, audio stutter, or limited functionality.
|
||||
|
||||
|
||||
## Setup and Run
|
||||
|
||||
### 1. Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/alibaba/MNN.git
|
||||
cd apps/Android/MnnTaoAvatar
|
||||
```
|
||||
|
||||
### 2. Build and Deploy
|
||||
|
||||
- Connect an Android device
|
||||
- Hit **Run** in Android Studio or use:
|
||||
|
||||
```bash
|
||||
./gradlew installDebug
|
||||
```
|
||||
|
||||
## Releases
|
||||
## Version 0.0.1
|
||||
+ Click here to [download](https://meta.alicdn.com/data/mnn/avatar/mnn_avatar_0_0_1.apk)
|
||||
+ this is our first public released version; you can chat with 3d avatar in the app with asr and tts; if you have any questions, please feel free to open an issue for assistance.
|
||||
|
||||
|
||||
## Links And References
|
||||
|
||||
- [TaoAvatar Paper](https://arxiv.org/html/2503.17032v1)
|
||||
- [Models Collection](https://modelscope.cn/collections/TaoAvatar-68d8a46f2e554a)
|
||||
- [LLM Model: Qwen2.5-1.5B MNN](https://github.com/alibaba/MNN/tree/master/3rd_party/NNR)
|
||||
- [TTS Model: bert-vits2-MNN](https://modelscope.cn/models/MNN/bert-vits2-MNN)
|
||||
- [A2BS Model: UniTalker-MNN](https://modelscope.cn/models/MNN/UniTalker-MNN)
|
||||
- [NNR Model: TaoAvatar-NNR-MNN](https://modelscope.cn/models/MNN/TaoAvatar-NNR-MNN)
|
||||
- [ASR Model: sherpa-mnn-streaming-zipformer-bilingual-zh-en-2023-02-20](https://modelscope.cn/models/MNN/sherpa-mnn-streaming-zipformer-bilingual-zh-en-2023-02-20)
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
# MNN-TaoAvatar Android - 本地智能数字人
|
||||
|
||||
<p align="center">
|
||||
<img width="20%" alt="Icon" src="https://meta.alicdn.com/data/mnn/avatar/avatar_demo.gif" style="margin: 0 10px;">
|
||||
</p>
|
||||
|
||||
+ [下载](#releases)
|
||||
|
||||
这是一个一个本地运行、完全离线、支持多模态交互的智能数字人App！
|
||||
|
||||
什么是 TaoAvatar？它是阿里最新研究成果的落地应用（详见 [TaoAvatar论文](https://arxiv.org/html/2503.17032v1)），将大语言模型（LLM）、语音识别（ASR）、语音合成（TTS）、声音到动作合成（A2BS）、神经渲染（NNR）统统搬到手机端，全本地运行，无需联网！
|
||||
|
||||
> 📢 iOS版稍后上线，敬请期待！
|
||||
|
||||
## 特色功能一览
|
||||
|
||||
* **本地聊天机器人**：基于本地运行的LLM，实时与数字人畅聊
|
||||
* **语音识别更智能**：内置ASR模型，即说即转文字
|
||||
* **随心所欲合成语音**：TTS模型，让你的数字人发声自然真实
|
||||
* **声音驱动表情动作**：A2BS技术，通过声音自动生成数字人丰富的面部表情和动作
|
||||
* **实时神经渲染**：让数字人表情细腻逼真，互动感更强
|
||||
* **100%离线运行**：完全本地运行，守护隐私更放心
|
||||
|
||||
## 工作原理
|
||||
|
||||
|
||||
## 硬件要求
|
||||
|
||||
由于需要将多个模型同时运行在手机上，需要**性能高的芯片**和**内存足够大**：
|
||||
|
||||
* **旗舰芯片级性能**：高通骁龙8 Gen 3或联发科天玑9200以上级别
|
||||
* **内存至少8GB**
|
||||
* **手机存储需至少5GB空间**用于存放模型文件
|
||||
* **ARM64架构**
|
||||
|
||||
> ⚠️ 性能不足的设备可能会遇到卡顿、声音断续或功能受限哦。
|
||||
|
||||
## 安装与体验步骤
|
||||
|
||||
1. 克隆项目代码
|
||||
|
||||
```bash
|
||||
git clone https://github.com/alibaba/MNN.git
|
||||
cd apps/Android/MnnTaoAvatar
|
||||
```
|
||||
|
||||
2. 构建并部署
|
||||
|
||||
* 连接你的安卓手机，打开Android Studio点击“Run”，或执行：
|
||||
|
||||
```bash
|
||||
./gradlew installDebug
|
||||
```
|
||||
|
||||
|
||||
## Releases
|
||||
## Version 0.0.1
|
||||
+ 点击这里[下载](https://meta.alicdn.com/data/mnn/avatar/mnn_avatar_0_0_1.apk)
|
||||
+ 这是我们首次公开发布的版本；您可以在应用程序中通过语音识别（ASR）和语音合成（TTS）与3D虚拟形象进行聊天；如果您有任何问题，请随时提交Issue以获得帮助。
|
||||
|
||||
## 更多相关资源
|
||||
|
||||
* [TaoAvatar 论文](https://arxiv.org/html/2503.17032v1)
|
||||
* [模型合集](https://modelscope.cn/collections/TaoAvatar-68d8a46f2e554a)
|
||||
* [LLM模型：Qwen2.5-1.5B MNN](https://github.com/alibaba/MNN/tree/master/3rd_party/NNR)
|
||||
* [TTS模型：bert-vits2-MNN](https://modelscope.cn/models/MNN/bert-vits2-MNN)
|
||||
* [声音动作模型：UniTalker-MNN](https://modelscope.cn/models/MNN/UniTalker-MNN)
|
||||
* [神经渲染模型：TaoAvatar-NNR-MNN](https://modelscope.cn/models/MNN/TaoAvatar-NNR-MNN)
|
||||
* [ASR模型：Sherpa 双语流式识别模型](https://modelscope.cn/models/MNN/sherpa-mnn-streaming-zipformer-bilingual-zh-en-2023-02-20)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue