Compare commits

...

281 Commits

Author SHA1 Message Date
Coltea 1c30e16fd6
Merge pull request #1576 from KuaiYu95/fe/fallback
fix: 修复 diff 组件导致前台页面无法展示的问题
2025-11-28 19:08:20 +08:00
yu.kuai cd804dbddd fix: 修复 diff 组件导致前台页面无法展示的问题 2025-11-28 19:06:41 +08:00
Coltea 574fc999a4
Merge pull request #1575 from KuaiYu95/fe/iframe
fix: 修复 iframe 渲染问题
2025-11-28 18:32:06 +08:00
yu.kuai ab24f7b78b fix: 修复 iframe 渲染问题 2025-11-28 18:27:41 +08:00
Coltea 5347726535
Merge pull request #1574 from coltea/fix-update-app
fix app update copyrightInfo
2025-11-28 18:10:18 +08:00
coltea 7c03aa7f6d fix app update copyrightInfo 2025-11-28 18:08:50 +08:00
Coltea a5e649a1bf
Merge pull request #1573 from KuaiYu95/fix/table
修复 iframe 水平对齐不生效的问题、解决控制台editor diff 组件重复扩展的警告
2025-11-28 17:50:40 +08:00
Coltea 59afc05284
Merge pull request #1572 from guanweiwang/feature/weichat_bot
fix: 侧边栏展示不齐问题, 重置提示词样式穿透问题
2025-11-28 17:50:31 +08:00
yu.kuai 1dd222f759 fix: 修复 iframe 水平对齐不生效的问题
fix: 解决控制台editor diff 组件重复扩展的警告
2025-11-28 17:48:34 +08:00
Gavan 409b69dda1 fix: 侧边栏展示不齐问题, 重置提示词样式穿透问题 2025-11-28 17:40:57 +08:00
Coltea a76dbc4d62
Merge pull request #1570 from coltea/feat-wecom-setting
feat 企微机器人高级配置
2025-11-28 17:17:29 +08:00
coltea 94da5431ad feat wecom advanced setting 2025-11-28 17:14:15 +08:00
Coltea 4e2db886e2
Merge pull request #1568 from jiangwel/feat-more-date-report
feat(telemetry): 添加每日数据上报功能并扩展客户端配置
2025-11-28 17:09:43 +08:00
jiangwel 5c901aa6b3 feat(telemetry): 添加每日数据上报功能并扩展客户端配置
扩展telemetry客户端以支持每日数据上报,包括知识库数量、模型配置模式、管理员登录状态、文档数量、各类型对话次数等指标
新增MCPRepository用于获取MCP调用次数
修改配置添加pandawiki_env字段用于区分环境
在NodeRepository和ConversationRepository中添加新方法支持数据统计
2025-11-28 17:04:00 +08:00
Coltea f80d240944
Merge pull request #1569 from guanweiwang/feature/weichat_bot
feat: 企微机器人问答设置
2025-11-28 17:00:51 +08:00
Coltea 189aa2e286
Merge pull request #1571 from KuaiYu95/fix/table
修复嵌套表格问题、优化 iframe 体验、支持插入 B 站视频
2025-11-28 16:34:58 +08:00
yu.kuai a8999c3dba fix: 修复了嵌套表格 handle & selection overlay 不显示的问题
feat: 支持删除表格
feat: 添加插入 bilibili 视频入口
pref: Iframe 编辑体验优化
2025-11-28 16:09:05 +08:00
Gavan 5ba454e084 feat: 企微机器人问答设置 2025-11-28 15:32:18 +08:00
Coltea 24a5a1574b
Merge pull request #1566 from KuaiYu95/fe/stopPropagation
fix: 编辑器快捷键阻止冒泡事件问题
2025-11-26 18:35:45 +08:00
yu.kuai 9f9e6388fe fix: 编辑器快捷键阻止冒泡事件问题 2025-11-26 18:32:35 +08:00
Coltea 5ba7c6b73a
Merge pull request #1565 from KuaiYu95/fix/editor
修复了编辑器的一些问题
2025-11-26 17:57:41 +08:00
yu.kuai 9350e631a4 fix: 表格中已合并单元格中间无法正常插入行/列的问题
fix: 移除编辑器扩展不必要的 z-index
fix: 修复无序列表图标移位的问题
fix: 修复任务列表中子任务受父任务影响的问题
fix: 代码块样式更新
2025-11-26 17:52:45 +08:00
Coltea 039b6b3061
Merge pull request #1563 from coltea/update-pro2
pro update
2025-11-26 15:26:51 +08:00
coltea 9fa8ad5177 pro update 2025-11-26 15:24:08 +08:00
Coltea 2d7c20c799
Merge pull request #1560 from coltea/feat-pv
feat: pv 支持浏览量展示
2025-11-26 14:56:01 +08:00
coltea 4a980b92c5 feat: pv 2025-11-26 14:35:59 +08:00
Coltea fe2d70bfa9
Merge pull request #1559 from KuaiYu95/fix/editor
修复了编辑器一些问题
2025-11-25 18:15:05 +08:00
yu.kuai c9f3a61e86 fix: 导出 markdown 错误
fix: 图片预览底部呈现工具栏
fix: 表格 100% 宽度出现滚动条的问题
2025-11-25 18:04:29 +08:00
Coltea 7b75f1bf55
Merge pull request #1558 from jiangwel/chore-mcp-placeholder
docs(设置页面): 更新MCP Tool输入框的默认值提示文本
2025-11-25 17:36:12 +08:00
jiangwel 07646f0823 chore: 设置默认的tool_name和tool_desc值
设置CardMCP组件中tool_name和tool_desc的默认值,确保功能正常运行
2025-11-25 16:26:52 +08:00
Coltea 63a8d8a743
Merge pull request #1557 from guanweiwang/feature/create_wiki_model
pref: 创建wiki站点,模型配置保存按钮合并到下一步按钮
2025-11-25 16:10:19 +08:00
Coltea 587c2842be
Merge pull request #1555 from coltea/feat-sitemap
feat: auto sitemap
2025-11-25 16:09:44 +08:00
coltea fd34bb4c55 feat: auto sitemap 2025-11-25 16:04:09 +08:00
Gavan 46afba778f pref: 创建wiki站点,模型配置保存按钮合并到下一步按钮 2025-11-25 11:38:55 +08:00
Coltea 79234c2394
Merge pull request #1547 from jiangwel/feat-mcp
feat: mcp server
2025-11-24 20:40:36 +08:00
jiangwel 67e796dbb7 feat(chat): 添加 ChatRagOnlyRequset 结构并重构 ChatRagOnly 逻辑
重构 ChatRagOnly 方法,使用新的 ChatRagOnlyRequset 结构作为参数
简化聊天逻辑,直接获取并返回相关文档内容
移除不必要的会话管理代码
2025-11-24 20:21:17 +08:00
Coltea 55c563cc48
Merge pull request #1554 from KuaiYu95/fix/key
fix: 修复ctrlKey+b 富文本模式下加粗同时收起展示目录的问题
2025-11-24 19:02:33 +08:00
yu.kuai cfdc546d20 fix: 修复ctrlKey+b 富文本模式下加粗同时收起展示目录的问题 2025-11-24 18:56:45 +08:00
Coltea 7f58308dae
Merge pull request #1553 from KuaiYu95/fe/editor-update
表格功能丰富,优化编辑器体验
2025-11-24 18:25:13 +08:00
yu.kuai f34864621a feat: 表格功能丰富
----
1. hover 展示 table handle,点击可对表格的行/列进行操作
2. hover 展示 insert button,点击可相对当前行左右插入行,当前列左右插入列
3. 多选选中单元格支持对选中单元格设置
4. hover 最后一行/列展示 insert handle,点击可最后一行/列后面插入行/列
----

fix: 修复了编辑器的一些问题
----
1. 修复了编辑模式图片不能预览的问题
2. 修复了设置文字大小后,行高未能自动变化的问题
3. 修复了创建标题,输入拼音被打断 IME 问题
4. 修复了编辑页面编辑器聚焦时 cmd+s 保存无反应
5. 修复了空格缩进导致代码块不展示的问题
6. 视频支持设置自适应宽度,四角拖拽改变宽高,支持水平对齐设置
----
2025-11-24 18:14:14 +08:00
jiangwel da9039ff37 feat: mcp server 2025-11-24 15:48:47 +08:00
Coltea 797e0c033d
Merge pull request #1549 from guanweiwang/feature/pref
pref: 优化统计 tab 展示
2025-11-24 14:18:58 +08:00
Gavan ea6f958d24 pref: 优化统计 tab 展示 2025-11-21 17:31:50 +08:00
Coltea d3502e105a
Merge pull request #1546 from coltea/fix-conversation-copyright
fix: conversation copyright valid
2025-11-21 15:25:41 +08:00
coltea df4937aeb2 fix: conversation copyright valid 2025-11-21 15:22:57 +08:00
Coltea d8c869198e
Merge pull request #1542 from coltea/feat-copyright
Feat 前台问答版权配置 && 支持超级管理员修改普通用户密码
2025-11-20 18:19:44 +08:00
Coltea f04f96d894
Merge pull request #1544 from guanweiwang/feature/pref
feat: 添加智能问答版权信息, 修改超级管理员权限, 优化 banner 移动端样式, 优化 header 移动端样式
2025-11-20 18:19:28 +08:00
Gavan 74e87540e0 feat: 添加智能问答版权信息, 修改超级管理员权限, 优化 banner 移动端样式, 优化 header 移动端样式 2025-11-20 18:15:30 +08:00
coltea 4a011aa1d2 feat conversation setting 2025-11-20 16:12:03 +08:00
coltea acf17e94b2 feat admin reset user password 2025-11-20 15:46:44 +08:00
Coltea 59ca885518
Merge pull request #1539 from KuaiYu95/fe/doc
文档前台查看功能
2025-11-19 18:48:14 +08:00
Coltea 0e64ff946f
Merge pull request #1540 from coltea/feat-node-list-publisher
feat 文档相关页面支持直接跳转至前台
2025-11-19 18:47:43 +08:00
coltea 19e6a66809 feat node list publish id 2025-11-19 18:43:10 +08:00
yu.kuai c15272aeb2 feat: 分享文档 2025-11-19 18:39:18 +08:00
yu.kuai 32ed999b48 fix: 挂件 esc 快捷关闭 2025-11-19 17:24:15 +08:00
Coltea 98e4a917e0
Merge pull request #1538 from guanweiwang/feature/icon
pref: 统一图标地址, 去除无用文件
2025-11-19 17:13:59 +08:00
Gavan 6e5f780771 pref: 统一图标地址, 去除无用文件 2025-11-19 17:08:28 +08:00
Coltea 95bd31b8ed
Merge pull request #1535 from KuaiYu95/fe/widget4
feat: 添加版本控制
2025-11-18 18:16:13 +08:00
yu.kuai 8457544a30 feat: 添加版本控制 2025-11-18 18:09:47 +08:00
Coltea 55abd7452c
Merge pull request #1532 from KuaiYu95/fe/widget3
挂件优化
2025-11-18 17:18:04 +08:00
Coltea 6224d713b6
Merge pull request #1534 from coltea/feat-widget-copyright
feat widget copyright
2025-11-18 17:17:14 +08:00
coltea 3f93246d85 feat widget copyright 2025-11-18 17:02:41 +08:00
yu.kuai 3196f2d130 feat: 挂件新增版权配置 2025-11-18 16:38:10 +08:00
yu.kuai 4a9a1ff78b fix: 修复挂件点击位置偏移的问题
fix: markdown 格式下适配 附件和链接
2025-11-18 10:46:18 +08:00
Coltea e80cbf9f47
Merge pull request #1530 from guanweiwang/hotfix/bug
fix: 暗黑模式样式, 优化首屏渲染闪烁
2025-11-17 18:56:54 +08:00
Gavan 76f9878d55 fix: 暗黑模式样式, 优化首屏渲染闪烁 2025-11-17 18:51:05 +08:00
Coltea eb662208dc
Merge pull request #1528 from guanweiwang/feature/catalog
feat: 支持文档目录页
2025-11-17 18:23:38 +08:00
Coltea 0dcd65961b
Merge pull request #1529 from KuaiYu95/fe/widget-2
feat: 挂件支持暗黑模式
2025-11-17 18:16:37 +08:00
Coltea dafb8de41e
Merge pull request #1527 from coltea/fix-folder-detail
feat 前台文档目录页 && 前台问答支持流程图
2025-11-17 18:12:26 +08:00
coltea de86adf90f feat html2md data-code 2025-11-17 18:11:35 +08:00
yu.kuai 4c0990df8c feat: 挂件支持暗黑模式
fix: 粘贴 excel 复制的 表格被认为是图片
2025-11-17 17:43:31 +08:00
Gavan 87d24e06c4 feat: 目录可以点击 2025-11-17 17:21:20 +08:00
coltea b1b354b785 fix share folder detail 2025-11-17 14:16:46 +08:00
xiaomakuaiz e9d30eb3d4
Merge pull request #1512 from xiaomakuaiz/fix/openai-api-compatibility
修复 /share/v1/chat/completions 接口 OpenAI 兼容性问题
2025-11-14 20:36:32 +08:00
xiaomakuaiz b8f2b95f22 完善 OpenAI API 兼容性:增强 OpenAIContentPart 结构体
- 添加 ImageURL 字段支持 image_url 类型的内容部分
- 新增 OpenAIContentPartURL 结构体处理嵌套的 URL 对象
- 修复 MarshalJSON 使用指针接收器以保持与 UnmarshalJSON 的一致性
- 修复 String 方法的空格逻辑,使用 builder.Len() 替代索引判断

修复了测试用例中 image_url 数据被静默忽略的问题,确保正确解析包含图片 URL 的混合内容。

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
2025-11-14 19:41:37 +08:00
xiaomakuaiz 98c602819f lint: make golangci-lint happy
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
2025-11-14 19:41:37 +08:00
xiaomakuaiz be163a5f80 优化 OpenAI API 兼容性实现
根据 PR #1512 代码审查意见,进行以下优化:

**安全性修复 (P0)**
- 修复 JSON 注入安全漏洞:移除不安全的字符串拼接构造 JSON 的方式
- 添加 NewStringContent 和 NewArrayContent 构造函数,直接构造对象而非通过 JSON 序列化

**性能优化**
- String() 方法使用 strings.Builder 替代字符串拼接,提升性能
- 多个 text 部分之间添加空格分隔符,避免语义错误

**测试覆盖**
- 添加完整的单元测试覆盖 MessageContent 类型
- 测试字符串格式解析(包括特殊字符、Unicode、换行符等)
- 测试数组格式解析(单个/多个 text 部分、混合类型)
- 测试无效输入处理
- 测试序列化/反序列化往返

**代码改进**
- 更新 handler/share/chat.go 使用安全的构造函数
- 所有测试通过验证

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>

Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
2025-11-14 19:26:29 +08:00
xiaomakuaiz 4c03078103 修复 /share/v1/chat/completions 接口 OpenAI 兼容性问题
现有的 OpenAI API 兼容接口不支持标准的 OpenAI messages 格式,特别是当 content 字段为数组格式时会解析失败。

1. **扩展 MessageContent 类型**:实现自定义的 JSON 序列化/反序列化,支持 content 既可以是字符串,也可以是包含 text/type 的对象数组
2. **添加 stream_options 支持**:支持 OpenAI 标准的 stream_options 参数(如 include_usage)
3. **更新响应格式**:在流式响应中添加 usage 字段支持,符合 OpenAI 标准

- `domain/openai.go`:
  - 新增 `MessageContent` 类型及其 JSON 序列化方法
  - 新增 `OpenAIStreamOptions` 结构体
  - 更新 `OpenAIMessage.Content` 类型从 string 改为 *MessageContent
  - 在流式响应中添加 usage 字段

- `handler/share/chat.go`:
  - 更新消息内容提取逻辑,使用 MessageContent.String() 方法
  - 修复流式和非流式响应中的 content 序列化

- 已通过单元测试验证 MessageContent 可以正确解析字符串和数组格式
- 编译通过,无语法错误

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>

Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
Co-authored-by: MonkeyCode-AI <monkeycode-ai@chaitin.com>
2025-11-14 19:26:29 +08:00
Coltea 2b372fd81d
Merge pull request #1521 from coltea/fix-widget-search-path
fix widget search path
2025-11-14 19:01:45 +08:00
Coltea 74c6ba131d
Merge pull request #1520 from guanweiwang/hotfix/bug
fix: 修复 浏览器将http自动升级为 https
2025-11-14 19:01:32 +08:00
Coltea 6c77246e34
Merge pull request #1522 from KuaiYu95/fe/apiurl
fix: api url
2025-11-14 19:01:09 +08:00
coltea d347482d31 fix widget search path 2025-11-14 19:00:15 +08:00
yu.kuai 92443dfafe fix: api url 2025-11-14 18:59:52 +08:00
Gavan 1dd2d93010 fix: 修复 浏览器将http自动升级为 https 2025-11-14 18:49:25 +08:00
Coltea 54bf1fd108
Merge pull request #1518 from guanweiwang/feature/more_feat
feat: 退出登录 & 对话记录 & md 渲染允许 img 标签 & 样式优化 & 升级nextjs 依赖
2025-11-14 18:34:28 +08:00
Coltea c23ce398d7
Merge pull request #1517 from coltea/feat-widget
feat 新版挂件机器人 && 用户登出 &&  文档目录页
2025-11-14 18:34:14 +08:00
coltea 2c90932f60 feat node detail folder 2025-11-14 18:33:30 +08:00
Gavan 035ce0284d feat: 退出登录 & 对话记录 & d 渲染允许 img 标签 & 样式优化 & 升级nextjs 依赖 2025-11-14 18:30:49 +08:00
Coltea 93559125c2
Merge pull request #1519 from KuaiYu95/fe/widget
网页挂件支持三种模式
2025-11-14 18:23:04 +08:00
yu.kuai 6c5cf256ac feat: 后台网页挂件新增配置项
feat: 前台网页挂件支持三种模式:悬浮球,侧边吸附,自定义按钮
fix: markdown 编辑器新增配置项:showAutocomplete,highlightActiveLine
fix: markdown 编辑器输入中午拼音与 placeholder 重叠
fix: 使用官方修复后的 migrateMathStrings.ts
fix: 使用官方修复后的 table-of-contents
2025-11-14 17:54:37 +08:00
coltea 23bdfc3d50 feat widget search 2025-11-14 15:53:21 +08:00
coltea 9008c537cd feat logout 2025-11-14 10:44:00 +08:00
Coltea 63a4a964b1
Merge pull request #1516 from guanweiwang/feature/perm
feat: 权限
2025-11-13 19:00:28 +08:00
Coltea 62f2b2eaf5
Merge pull request #1514 from guanweiwang/feature/ai_search
feat: 优化 ai 搜索弹窗
2025-11-13 18:59:54 +08:00
Gavan 5c1c6368b8 feat: add expandable sections for results and thinking content in AiQaContent; update styles and remove unused hooks 2025-11-13 18:59:13 +08:00
Gavan 7282503acf feat: 权限 2025-11-13 18:55:25 +08:00
Coltea 60a4177229
Merge pull request #1511 from coltea/feat-edition
feat edition
2025-11-13 18:52:20 +08:00
coltea 2f706a6100 fix share nodes position 2025-11-13 18:40:33 +08:00
coltea febcb06654 feat edition 2025-11-13 15:50:05 +08:00
xiaomakuaiz ca323de9b2
Merge pull request #1485 from jiangwel/feat-modelsetting3
实现模型设置模式(手动/自动)切换功能
2025-11-12 17:26:06 +08:00
jiangwel 5c81d714b1 fix: 修复前端样式问题 2025-11-12 17:15:44 +08:00
jiangwel 2a123cf8b1 chore: 更新模型设置模式及相关配置,移除冗余迁移功能 2025-11-12 17:15:44 +08:00
jiangwel b1074f0956 chore: 添加模型设置模式及相关配置 2025-11-12 17:15:44 +08:00
jiangwel b98bf6664c feat: 添加关闭弹窗时未应用更改的确认提示 2025-11-12 17:15:44 +08:00
jiangwel 2b02d85fb3 feat(model): 添加模型设置模式切换功能
实现模型设置模式(手动/自动)切换功能,包括:
1. 新增SettingRepository及相关数据库操作
2. 添加模型模式相关常量定义
3. 实现模式切换API接口及业务逻辑
4. 添加数据库迁移脚本初始化模型设置
5. 更新swagger文档

feat(模型): 添加百智云自动模式配置功能

实现百智云模型自动模式的API Key和对话模型配置功能
重构相关代码,将BaiZhiCloud重命名为AutoMode
更新迁移脚本和模型设置接口

feat: 添加模型模式设置和自动模式支持

feat(系统设置): 重构系统设置模块,迁移至新表结构

- 新增system_settings表及相关迁移文件
- 将原setting表功能迁移至system_setting表
- 更新模型设置相关逻辑,支持自动模式配置
- 优化Makefile构建目标,分离pro和pro_consumer
- 更新API文档,修正模型设置相关接口路径

feat: 前端支持自动配置模型

feat: 在教程添加模型配置

feat: 在教程中添加配置模型

chore

feat: 修改前端样式

feat: 修改前端样式

feat: 前端流程跑通

feat: 跑通前端流程

feat: 优化接口
2025-11-12 17:15:44 +08:00
xiaomakuaiz 21f5a776df
Merge pull request #1509 from KuaiYu95/fe/start
修复了一些问题
2025-11-12 16:00:14 +08:00
yu.kuai 3999621981 feat: markdown 编辑模式支持拖拽/粘贴文件资源 2025-11-12 15:46:12 +08:00
yu.kuai da17b21387 fix: ts 报错问题修复 2025-11-12 11:43:07 +08:00
yu.kuai e361644f01 feat: 编辑页添加快捷键 ctrl+B 控制展开和收起目录 2025-11-12 11:23:56 +08:00
yu.kuai b8a1a130ac fix: 保存 markdown 时提交 ace 编辑器文本内容 2025-11-12 11:08:17 +08:00
yu.kuai 385c21a36c feat: 批量处理待学习文档 2025-11-12 10:39:08 +08:00
xiaomakuaiz a5c99fca95
Merge pull request #1507 from KuaiYu95/fe/table
fix: 文档页超宽可以横向滚动的问题
2025-11-11 19:37:15 +08:00
yu.kuai 7b0d71b4c5 fix: 文档页超宽可以横向滚动的问题 2025-11-11 19:33:04 +08:00
xiaomakuaiz 61688c86c9
Merge pull request #1506 from coltea/chore-pro-update
update pro
2025-11-11 18:34:13 +08:00
coltea c69e74d15d update pro 2025-11-11 18:28:59 +08:00
xiaomakuaiz 26e06e69a7
Merge pull request #1503 from coltea/fix-wecom-app-auth
feat 企业微信应用内自动登录
2025-11-11 18:22:29 +08:00
xiaomakuaiz 74e8b03975
Merge pull request #1505 from KuaiYu95/fe/error
fix: 修复无法插入链接的 bug
2025-11-11 18:22:04 +08:00
xiaomakuaiz f91a8fb38f
Merge pull request #1504 from guanweiwang/hotfix/bug
fix: 兼容企业微信客户端打开授权
2025-11-11 18:21:46 +08:00
Gavan 8e6f7ae77c fix: 兼容企业微信客户端打开授权 2025-11-11 18:13:21 +08:00
yu.kuai cefd3fe3a2 fix: 修复无法插入链接的 bug
feat: 支持 markdown 编辑器新增配置项
2025-11-11 18:05:22 +08:00
coltea 8d70727d0a fix wecom app auth 2025-11-11 17:37:26 +08:00
xiaomakuaiz 4a787a3a6c
Merge pull request #1502 from guanweiwang/pref/feat
pref: 优化评论
2025-11-11 16:18:52 +08:00
xiaomakuaiz da16f5b335
Merge pull request #1501 from guanweiwang/hotfix/bug
优化和修复bug
2025-11-11 16:18:41 +08:00
xiaomakuaiz 7e770de4df
Merge pull request #1500 from KuaiYu95/fe/mdeidtor
feat: markdwon 工具栏点击插入逻辑优化
2025-11-11 16:14:33 +08:00
Gavan 681b250296 pref: 优化评论 2025-11-11 15:52:06 +08:00
yu.kuai 3597afcc2b feat: 支持文件上传,显示上传进度
chore: 更新 tiptap 版本
2025-11-11 15:50:06 +08:00
Gavan 284392c379 pref: 添加对缓存的清理 2025-11-11 12:02:59 +08:00
Gavan 4b54cdf4ac fix: 修复无缓存图片闪烁问题, 修复图片引起的不自动滚动问题, 优化手动小距离滚动抖动问题, footer 移动端样式问题 2025-11-11 11:54:05 +08:00
yu.kuai c48b13366d feat: markdwon 工具栏点击插入逻辑优化 2025-11-11 10:38:18 +08:00
xiaomakuaiz c31f229483
Merge pull request #1499 from KuaiYu95/fe/editor-li-maker
feat: 列表项maker 左对齐
2025-11-10 21:50:59 +08:00
yu.kuai 8fad4d6262 fix: 支持上传‘’
fix: 支持上传
2025-11-10 21:36:27 +08:00
yu.kuai 712e2f8af8 feat: 列表项maker 左对齐 2025-11-10 21:23:12 +08:00
xiaomakuaiz b990b00df0
Merge pull request #1498 from KuaiYu95/fe/markdown-pref
feat: 优化 markdown 模式样式
2025-11-10 21:07:48 +08:00
yu.kuai bb8337a33e fix: ts error 2025-11-10 20:11:11 +08:00
holly 2f56ad7f6b
Update web/admin/src/pages/document/editor/edit/Wrap.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-10 20:04:50 +08:00
holly d7948ddecc
Update web/admin/src/pages/document/editor/edit/Wrap.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-10 20:03:59 +08:00
yu.kuai 9d329d21fb feat: 优化 markdown 模式样式
fix: 修复了一些编辑器问题
2025-11-10 19:37:31 +08:00
xiaomakuaiz e4dbfcb9fb
Merge pull request #1492 from KuaiYu95/fe/contribute
feat: 支持批量重新学习失败的文档
2025-11-10 19:33:56 +08:00
xiaomakuaiz 40c395400d
Merge pull request #1483 from coltea/feat-restudy
feat restudy
2025-11-10 19:33:43 +08:00
coltea b7cdec0d4a feat restudy 2025-11-10 14:13:09 +08:00
xiaomakuaiz 44126e0a11
Merge pull request #1493 from guanweiwang/hotfix/bug
fix: 拖拽组件输入文字 失焦问题
2025-11-10 12:14:36 +08:00
Gavan 3375fcb643 fix: 拖拽组件输入文字 失焦问题 2025-11-10 11:29:42 +08:00
yu.kuai ef3bae6336 feat: 支持批量重新学习失败的文档 2025-11-10 11:15:36 +08:00
xiaomakuaiz d9d3bc4911
Merge pull request #1491 from KuaiYu95/fe/editor-ui
feat: 编辑器部分组件样式更新
2025-11-10 10:37:29 +08:00
xiaomakuaiz 546062470b
Merge pull request #1478 from xiaomakuaiz/feature/summary-optimization
Feature/summary optimization
2025-11-10 10:37:10 +08:00
yu.kuai 5a3b23ac75 feat: 编辑器部分组件样式更新 2025-11-10 10:12:54 +08:00
xiaomakuaiz 35cd94e342
Merge pull request #1484 from guanweiwang/pref/landing_drag
pref: 优化和bug修复
2025-11-07 17:06:15 +08:00
monkeycode-ai f7c0fe273b Improve summary optimization with simplified aggregation
优化摘要生成逻辑:
1. 将chunk token限制从16KB提升到30KB,更合理地利用模型上下文
2. 简化摘要聚合逻辑,移除复杂的分批聚合,直接合并所有summaries
3. 保留fallback机制,当最终摘要生成失败时返回已聚合的摘要

这些改进确保了:
- 长文档能够更充分地被摘要(30KB vs 16KB)
- 代码更简洁,避免不必要的迭代聚合和额外LLM调用
- 即使最终摘要失败也能返回有用的结果

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-07 14:21:36 +08:00
Gavan 0175624c84 pref: 删除无用代码,提取公共拖拽函数,修复 footer 2025-11-07 12:00:48 +08:00
xiaomakuaiz 3032384457
Merge pull request #1479 from xiaomakuaiz/fix/qa-modal-logo
Fix QA modal logo rendering
2025-11-06 23:01:16 +08:00
monkeycode-ai 50ed0c6794 Fix QA modal logo rendering
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 22:44:18 +08:00
monkeycode-ai a6f4688b88 Run goimports on llm summary
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 20:07:30 +08:00
monkeycode-ai 575f51f0ea Simplify final summary aggregation
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 19:52:03 +08:00
monkeycode-ai 83f6853716 Iteratively reduce summary chunks
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 19:21:26 +08:00
monkeycode-ai 3dae8e8d01 Raise summary chunk limit to 16k
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 19:17:27 +08:00
monkeycode-ai 2e1e1848c4 Adjust summary chunking and concurrency
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-11-06 19:09:15 +08:00
xiaomakuaiz 45238c3dfa
Merge pull request #1476 from guanweiwang/feature/qa_modal
feat: 优化问答弹窗,完善 ts 配置
2025-11-06 17:43:33 +08:00
Gavan ea87d5ef7e pref: 优化 ts 配置 2025-11-06 15:37:56 +08:00
Gavan b1aefd8cfd feat: ui调整 2025-11-06 11:46:37 +08:00
xiaomakuaiz f3394a65b9
Merge pull request #1460 from KuaiYu95/fe/contribute
文档显示人员操作记录 & 网页挂件配置支持推荐问题和推荐文档 & 记录学习状态
2025-11-05 17:10:41 +08:00
xiaomakuaiz 051d2589dd
Merge pull request #1452 from coltea/feat-doc-release-user-info
feat: node release user id
2025-11-05 16:56:16 +08:00
coltea 1300682454 feat: node release user info 2025-11-05 16:51:52 +08:00
xiaomakuaiz b915dc8459
Merge pull request #1472 from guanweiwang/pref/landing
style: 样式修改
2025-11-05 16:35:05 +08:00
yu.kuai e5bad16b3c pref: code 2025-11-05 16:07:02 +08:00
yu.kuai 1a621205ef feat: 记录学习状态 2025-11-05 15:51:40 +08:00
Gavan dc14f5280b feat: 添加配色方案 2025-11-05 15:24:51 +08:00
Gavan 0ce0f9eb52 style: 样式问题 2025-11-05 14:21:33 +08:00
yu.kuai 0081f05cd9 feat: 网页挂件配置支持推荐问题和推荐文档 2025-11-05 14:07:28 +08:00
yu.kuai 1d0e4857a9 fix: 发布人员 icon 调整大小 2025-11-05 11:15:43 +08:00
yu.kuai edb7e01085 feat: 历史版本展示 markdown 文本 2025-11-05 10:40:41 +08:00
yu.kuai a4679f0ada feat: 前台支持展示更新人/创建人 2025-11-05 10:40:41 +08:00
yu.kuai 5e93e9da73 feat: 文档显示人员操作记录 2025-11-05 10:40:41 +08:00
coltea 0aeda02985 feat widget setting 2025-11-05 10:30:47 +08:00
xiaomakuaiz d630567a3c
Merge pull request #1463 from guanweiwang/pref/landing
pref: 优化主题
2025-11-04 20:03:09 +08:00
Gavan 20a0a4ded4 style: 样式问题 2025-11-04 19:56:44 +08:00
Gavan 38383b983d pref: 美化样式 2025-11-04 19:46:36 +08:00
Gavan 030a8ac25d pref: 优化主题 2025-11-04 19:13:31 +08:00
xiaomakuaiz 24d1ed1bcd
Merge pull request #1461 from guanweiwang/pref/landing
feat: 常见问题,九宫格
2025-11-04 18:55:07 +08:00
Gavan 487db8e944 feat: add DomainBlockGridConfig and DomainQuestionConfig interfaces; update BlockGridConfig and DragList components 2025-11-04 18:47:13 +08:00
Gavan 2638fcdc0c feat: 常见问题,九宫格 2025-11-04 18:32:29 +08:00
xiaomakuaiz 1aa2855e00
Merge pull request #1459 from KuaiYu95/fe/custom-md
优化 markdown 文档模式
2025-11-04 18:31:18 +08:00
xiaomakuaiz fd81e83807
Merge pull request #1462 from xiaomakuaiz/feat-contribute-content-type
Feat contribute content type
2025-11-04 18:28:52 +08:00
yu.kuai f121494416 fix: pref code 2025-11-04 18:26:21 +08:00
xiaomakuaiz b04aa2d472 feat: add content-type for contribute 2025-11-04 10:19:50 +00:00
xiaomakuaiz 69bf9cbf0e feat: add more landing components 2025-11-04 10:19:26 +00:00
yu.kuai 940282a521 feat: 后台贡献新增 markdown diff view modal 2025-11-04 16:38:22 +08:00
yu.kuai 171cc6c632 feat: 前台文档贡献支持选择文档类型 2025-11-04 16:38:22 +08:00
yu.kuai 78e5e1d70d feat: 前台编辑贡献支持 markdown 2025-11-04 16:38:22 +08:00
yu.kuai c5151ee7fe feat: 优化 markdown 编辑器
feat: subscript, supscript, alert 支持 markdown
2025-11-04 16:38:22 +08:00
yu.kuai c7f764199e feat: 编辑页支持创建最外层文件夹/文件
fix: 修复 alert 展示问题,适配 tiptap 的 markdown 代码设计
2025-11-04 16:38:22 +08:00
yu.kuai 5588a46752 feat: 自定义markdown 解析上标,下标和警告提示 2025-11-04 16:38:22 +08:00
yu.kuai 5fba15654f feat: 点击大纲标题,markdown 和 tiptap 预览同步滚动 2025-11-04 16:38:22 +08:00
xiaomakuaiz 028b872349
Merge pull request #1455 from guanweiwang/hotfix/bug
fix: 暗黑模式下, md渲染问题
2025-11-04 15:01:20 +08:00
Gavan 02d17cb48f fix: 暗黑模式下, md渲染问题 2025-11-04 14:37:48 +08:00
xiaomakuaiz c762fa3899
Merge pull request #1450 from coltea/fix-consumer-rag-info
fix consumer rag info
2025-11-03 21:16:16 +08:00
coltea 75d4149d0e fix consumer rag info 2025-11-03 21:14:04 +08:00
xiaomakuaiz ab21ad8b1f
Merge pull request #1448 from guanweiwang/pref/landing
style: 样式优化
2025-11-03 21:10:11 +08:00
xiaomakuaiz 2fea4c268d
Merge pull request #1449 from coltea/fix-consumer-deliver-policy
fix consumer deliver plicy
2025-11-03 20:53:37 +08:00
coltea ee4df8da62 fix consumer deliver plicy 2025-11-03 20:51:29 +08:00
Gavan 79e3606bb0 style: 样式优化 2025-11-03 19:34:14 +08:00
xiaomakuaiz c2086a39ea
Merge pull request #1418 from coltea/feat-release-status
feat rag doc update sync
2025-11-03 19:12:23 +08:00
coltea 8e9fbd237e feat rag doc update task 2025-11-03 19:11:50 +08:00
xiaomakuaiz 652a8385c0
Merge pull request #1446 from guanweiwang/pref/landing
feat: 添加产品特性,图文,点评卡片
2025-11-03 19:03:58 +08:00
xiaomakuaiz e620f0bdcd
Merge pull request #1447 from xiaomakuaiz/feat-more-landing-components
feat: add more components for landing page
2025-11-03 19:03:43 +08:00
Gavan ed1ef4a038 feat: 添加产品特性,图文,点评卡片 2025-11-03 18:24:26 +08:00
xiaomakuaiz 186a0c25ef feat: add more components for landing page 2025-11-03 10:16:02 +00:00
xiaomakuaiz 9d79e7645b
Merge pull request #1440 from guanweiwang/pref/landing
fix: 编辑导航和footer, 优化轮播图和指标卡片
2025-11-03 12:07:21 +08:00
Gavan d27dcc7ca6 style: 兼容暗色模式样式 2025-11-03 12:02:53 +08:00
Gavan 2a8bcb69b7 fix: 编辑导航和footer, 修改轮播图和指标卡片 2025-11-03 11:43:05 +08:00
xiaomakuaiz 2027e878af
Merge pull request #1437 from guanweiwang/pref/landing
fix: update background URL and translate FAQ title in landing data
2025-10-31 19:58:10 +08:00
Gavan 2f11a62572 fix: update background URL and translate FAQ title in landing data 2025-10-31 19:56:16 +08:00
xiaomakuaiz 322ae41fb8
Merge pull request #1435 from xiaomakuaiz/feat-new-landing-components
feat: add new landing components
2025-10-31 19:32:42 +08:00
xiaomakuaiz 09585f9b01
Merge pull request #1434 from guanweiwang/pref/landing
feat: 卡片
2025-10-31 19:32:31 +08:00
xiaomakuaiz 75f1c1b903
Merge pull request #1436 from xiaomakuaiz/add-project-structure-doc
docs: add project structure documentation
2025-10-31 19:26:45 +08:00
Gavan eef3e8ffbc fix: ts error 2025-10-31 19:23:17 +08:00
monkeycode-ai c104f57631 docs: add project structure documentation
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Co-authored-by: monkeycode-ai <monkeycode-ai@chaitin.com>
2025-10-31 11:14:39 +00:00
Gavan 148e13c921 feat: 卡片 2025-10-31 19:13:45 +08:00
xiaomakuaiz 9911f08240 feat: add new landing components 2025-10-31 11:03:32 +00:00
xiaomakuaiz 10be87ef45
Merge pull request #1408 from jiangwel/feat-unbind-lic
feat:解绑授权
2025-10-31 17:57:31 +08:00
jiangwel ca8be63542 feat:解绑授权 2025-10-31 17:53:45 +08:00
xiaomakuaiz 2788054de0
Merge pull request #1433 from xiaomakuaiz/feat-node-content-type
feat: add content type for node
2025-10-31 17:42:27 +08:00
xiaomakuaiz 5af9961eeb
Merge pull request #1420 from KuaiYu95/fe/markdown
feat: 支持markdown编辑器
2025-10-31 17:41:01 +08:00
yu.kuai 75144382bb feat: 支持markdown编辑器 2025-10-31 17:30:18 +08:00
xiaomakuaiz 4eb94928d1 feat: add content type for node 2025-10-31 07:54:58 +00:00
xiaomakuaiz 34e3588ba9
Merge pull request #1423 from guanweiwang/pref/landing
fix: 主题变量缺失
2025-10-30 21:46:35 +08:00
Gavan 88aca2cfcd fix: 主题变量缺失 2025-10-30 21:38:47 +08:00
xiaomakuaiz ef5ae4384a
Merge pull request #1422 from xiaomakuaiz/feat-landing-theme
feat: add landing theme
2025-10-30 21:00:42 +08:00
xiaomakuaiz c28e125c33
Merge pull request #1421 from guanweiwang/pref/landing
feat: 添加多套主题
2025-10-30 20:58:48 +08:00
Gavan 6e101b973d feat: 添加多套主题 2025-10-30 20:36:25 +08:00
xiaomakuaiz 88230f7255
Merge pull request #1414 from guanweiwang/pref/landing
pref: footer, 卡片文案
2025-10-30 18:55:25 +08:00
xiaomakuaiz bf882342d8
Merge pull request #1413 from guanweiwang/pref/qa
pref: 优化部分功能,修复暗黑模式问题
2025-10-30 18:54:58 +08:00
xiaomakuaiz 73c2a1fee7 feat: add landing theme 2025-10-30 08:50:37 +00:00
Gavan 0e37cc0e03 pref: 优化部分功能,修复暗黑模式问题 2025-10-29 18:49:41 +08:00
Gavan 316f377661 pref: footer, 卡片文案 2025-10-29 18:38:36 +08:00
xiaomakuaiz f39b5b49fd
Merge pull request #1409 from guanweiwang/hotfix/bug
fix: 快捷键展示问题
2025-10-28 20:08:06 +08:00
Gavan b4c0b3ca2c fix: 快捷键展示问题 2025-10-28 19:04:23 +08:00
xiaomakuaiz bbf86b11bb
Merge pull request #1406 from guanweiwang/feature/ai_qa
perf: 添加思考, 美化样式
2025-10-28 18:32:20 +08:00
Gavan da9de87b8e pref: 添加智能自动滚动机制 2025-10-28 18:25:43 +08:00
xiaomakuaiz 1e2032b50d
Merge pull request #1407 from KuaiYu95/fe/feishuclouddisk
修改云盘文件夹名称
2025-10-28 18:20:54 +08:00
holly d0d7dd3cfe
Merge branch 'chaitin:main' into fe/feishuclouddisk 2025-10-28 18:15:20 +08:00
yu.kuai 82605197f2 fix: 修改云盘文件夹名称 2025-10-28 18:14:32 +08:00
xiaomakuaiz cb9183aa52
Merge pull request #1405 from KuaiYu95/fe/feishuclouddisk
fix: 导入时未拉取文档的知识库不导入
2025-10-28 18:00:38 +08:00
Gavan 187b14dc3e perf: 添加思考, 美化样式 2025-10-28 17:44:20 +08:00
yu.kuai b518857170 fix: 导入时未拉取文档的知识库不导入 2025-10-28 17:40:33 +08:00
xiaomakuaiz b67707e552
Merge pull request #1404 from KuaiYu95/fe/doc-publish
fix: 编辑文档保存并发布时,弹框中无可发布数据
2025-10-28 16:58:27 +08:00
yu.kuai 44163ac2fb fix: 编辑文档保存并发布时,弹框中无可发布数据 2025-10-28 15:55:51 +08:00
xiaomakuaiz 6b3eb3efa0
Merge pull request #1403 from KuaiYu95/fe/import-docs
feat: 导入文档支持保留文档树结构
2025-10-28 15:39:52 +08:00
yu.kuai 7decde160b fix: 设置服务监听方式保存后,保存按钮未消失 2025-10-28 15:30:51 +08:00
yu.kuai 0e4176489a feat: 导入文档支持保留文档树结构 2025-10-28 15:30:45 +08:00
xiaomakuaiz bbef07f779
Merge pull request #1388 from coltea/feat-doc-tree
feat doc tree
2025-10-28 15:23:01 +08:00
coltea 8416484488 feat delete folder 2025-10-28 14:47:06 +08:00
xiaomakuaiz fd28505e5f
Merge pull request #1402 from xiaomakuaiz/feat-search-simlarity
feat: change search similarity to 0.2
2025-10-27 19:26:12 +08:00
xiaomakuaiz 70a7b1b788 feat: change search similarity to 0.2 2025-10-27 11:21:29 +00:00
xiaomakuaiz cb3399944d
Merge pull request #1401 from guanweiwang/feature/ai_qa
feat: ai qa
2025-10-27 19:20:22 +08:00
Gavan 4bd58714d2 feat: ai qa 2025-10-27 19:12:31 +08:00
xiaomakuaiz b3346d8a89
Merge pull request #1397 from xiaomakuaiz/feat-node-path-names
feat: return path of node
2025-10-27 17:56:50 +08:00
xiaomakuaiz b17832a148
Merge pull request #1398 from jiangwel/feat-update-modelkit
feat: modelkit支持自定义文档url
2025-10-27 17:56:40 +08:00
jiangwel 75a091d99e feat: modelkit支持自定义文档url 2025-10-27 17:16:39 +08:00
coltea 0306b4816c feat doc tree 2025-10-27 17:11:44 +08:00
xiaomakuaiz cdb1dcbbc5 feat: return path of node 2025-10-27 08:29:58 +00:00
xiaomakuaiz 10b50b6d29
Merge pull request #1395 from jiangwel/feat-update-modelkit
添加配置模型教程url, 修复模型名称被清空的bug
2025-10-27 14:37:26 +08:00
xiaomakuaiz 02a4a741d5
Merge pull request #1396 from xiaomakuaiz/fix-ranked-nodes
fix: shadow rankedNodes
2025-10-27 12:25:41 +08:00
xiaomakuaiz c6589ee906 fix: shadow rankedNodes 2025-10-27 04:21:28 +00:00
jiangwel 5716f70fce fix: 配置其它模型时修改baseurl modelname被清空的问题 2025-10-27 11:57:49 +08:00
jiangwel 4587c934b3 feat: 在添加模型弹窗添加教程url 2025-10-27 11:40:49 +08:00
xiaomakuaiz b9937a986c
Merge pull request #1387 from xiaomakuaiz/feat-trust-proxies
feat: support multi trusted proxies
2025-10-24 15:39:19 +08:00
xiaomakuaiz 88f51bad10 feat: support multi trusted proxies 2025-10-24 07:22:32 +00:00
xiaomakuaiz e19a7971e2
Merge pull request #1381 from xiaomakuaiz/feat-support
feat(web/admin): add bbs support
2025-10-23 20:42:56 +08:00
xiaomakuaiz cc3d946f01 feat(web/admin): add bbs support 2025-10-23 12:24:41 +00:00
xiaomakuaiz a2195a389e
Merge pull request #1376 from coltea/fix-contribute-setting
fix contribute setting
2025-10-23 18:40:41 +08:00
coltea 7e084766f4 fix contribute setting 2025-10-23 16:53:56 +08:00
xiaomakuaiz 68bfe1ba4b
Merge pull request #1374 from KuaiYu95/fe/update-tiptap-version-3.7.1
chore: 更新编辑器版本至最新版
2025-10-23 16:09:07 +08:00
yu.kuai bbd9df9e61 chore: 更新编辑器版本至最新版
fix(fe/editor): 优化编辑器
2025-10-23 16:07:32 +08:00
xiaomakuaiz b2aecd3674
Merge pull request #1368 from guanweiwang/feature/comment
feat: 评论区添加图片支持
2025-10-23 16:03:36 +08:00
xiaomakuaiz 86f622a795
Merge pull request #1373 from guanweiwang/hotfix/bug
pref: 优化新装用户首次使用弹窗
2025-10-23 16:03:21 +08:00
Gavan e28d26d0cc pref: 修改文案 2025-10-23 15:58:26 +08:00
xiaomakuaiz 9561ca0eaa
Merge pull request #1364 from coltea/feat-comment-pic
feat comment pic
2025-10-23 15:54:34 +08:00
Gavan fab4fb185b pref: 兼容 pic_urls 可能为空的情况 2025-10-23 15:52:59 +08:00
coltea d36892fe69 feat comment pic 2025-10-23 15:50:12 +08:00
Gavan a32d1b7dba feat: 评论区添加图片支持 2025-10-23 14:46:24 +08:00
Gavan 78ca41100c pref: 优化初始化安装 2025-10-23 10:29:57 +08:00
520 changed files with 35982 additions and 29382 deletions

92
PROJECT_STRUCTURE.md Normal file
View File

@ -0,0 +1,92 @@
# PandaWiki 项目结构文档
## 项目概述
PandaWiki 是一个由 AI 大模型驱动的开源知识库搭建系统。该项目采用前后端分离的架构,包含后端服务、前端管理界面、前端用户界面以及 SDK。
## 根目录结构
```
/workspace/
├── .github/ # GitHub 相关配置 (如 workflows, issue templates)
├── backend/ # 后端服务代码 (Go 语言)
├── images/ # 项目相关的图片资源 (如 README 中使用的图片)
├── sdk/ # 软件开发工具包 (SDK)
├── web/ # 前端代码 (Node.js/React)
├── .gitattributes # Git 属性配置
├── .gitignore # Git 忽略文件配置
├── .gitmodules # Git 子模块配置
├── CODE_OF_CONDUCT.md # 行为准则
├── CONTRIBUTING.md # 贡献指南
├── LICENSE # 许可证 (AGPL-3.0)
├── README.md # 项目介绍和使用指南
└── SECURITY.md # 安全策略
```
## 后端 (backend/) 结构
后端服务使用 Go 语言编写,主要负责 API 提供、业务逻辑处理、数据存储等。
```
/workspace/backend/
├── api/ # API 定义和接口实现
├── apm/ # 应用性能管理 (APM) 相关代码
├── cmd/ # 应用程序入口点 (main 函数)
├── config/ # 配置文件解析和管理
├── consts/ # 常量定义
├── docs/ # 项目内部文档
├── domain/ # 领域模型和核心业务逻辑
├── handler/ # HTTP 请求处理器
├── log/ # 日志管理
├── middleware/ # 中间件 (如认证、日志记录)
├── migration/ # 数据库迁移脚本
├── mq/ # 消息队列相关代码
├── pkg/ # 公共包和工具库
├── pro/ # 专业版功能相关代码
├── repo/ # 数据访问层 (Repository)
├── server/ # 服务器初始化和启动逻辑
├── setup/ # 安装和初始化相关代码
├── store/ # 存储层抽象和实现
├── telemetry/ # 遥测和监控相关代码
├── usecase/ # 用例层 (业务逻辑的具体实现)
├── utils/ # 工具函数
├── .dockerignore # Docker 构建忽略文件
├── .golangci.toml # Go 语言 lint 工具配置
├── cSpell.json # 拼写检查配置
├── Dockerfile.api # API 服务的 Dockerfile
├── Dockerfile.api.pro # 专业版 API 服务的 Dockerfile
├── Dockerfile.consumer # 消费者服务的 Dockerfile
├── Dockerfile.consumer.pro # 专业版消费者服务的 Dockerfile
├── go.mod # Go 模块依赖管理
├── go.sum # Go 模块依赖校验
├── Makefile # 构建脚本
├── pro_imports.go # 专业版功能导入
└── project-words.txt # 项目特定词汇列表 (用于拼写检查)
```
## 前端 (web/) 结构
前端使用 Node.js 和 React 构建,采用 monorepo 结构管理多个应用。
```
/workspace/web/
├── .husky/ # Git hooks 配置
├── admin/ # 管理后台前端代码
├── app/ # 用户端 Wiki 网站前端代码
├── packages/ # 共享的组件库和工具包
├── .gitignore # Git 忽略文件配置
├── .prettierignore # Prettier 格式化忽略文件
├── package.json # Node.js 项目配置
├── pnpm-lock.yaml # pnpm 依赖锁定文件
├── pnpm-workspace.yaml # pnpm 工作区配置
└── prettier.config.js # Prettier 代码格式化配置
```
## SDK (sdk/) 结构
SDK 提供了与 PandaWiki 系统交互的工具包。
```
/workspace/sdk/
└── rag/ # RAG (Retrieval-Augmented Generation) 相关 SDK
```

View File

@ -1,15 +1,40 @@
package v1
import "github.com/chaitin/panda-wiki/consts"
import (
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/pkg/anydoc"
)
type ScrapeReq struct {
URL string `json:"url" validate:"required"`
KbID string `json:"kb_id" validate:"required"`
type CrawlerParseReq struct {
Key string `json:"key"`
KbID string `json:"kb_id" validate:"required"`
CrawlerSource consts.CrawlerSource `json:"crawler_source" validate:"required"`
Filename string `json:"filename"`
FeishuSetting FeishuSetting `json:"feishu_setting"`
}
type ScrapeResp struct {
type FeishuSetting struct {
UserAccessToken string `json:"user_access_token"`
AppID string `json:"app_id"`
AppSecret string `json:"app_secret"`
SpaceId string `json:"space_id"`
}
type CrawlerParseResp struct {
ID string `json:"id"`
Docs anydoc.Child `json:"docs"`
}
type CrawlerExportReq struct {
KbID string `json:"kb_id" validate:"required"`
ID string `json:"id" validate:"required"`
DocID string `json:"doc_id" validate:"required"`
SpaceId string `json:"space_id"`
FileType string `json:"file_type"`
}
type CrawlerExportResp struct {
TaskId string `json:"task_id"`
Title string `json:"title"`
}
type CrawlerResultReq struct {
@ -34,52 +59,3 @@ type CrawlerResultItem struct {
Status consts.CrawlerStatus `json:"status"`
Content string `json:"content"`
}
type SitemapParseReq struct {
URL string `json:"url" validate:"required"`
}
type SitemapParseResp struct {
ID string `json:"id"`
List []SitemapParseItem `json:"list"`
}
type SitemapParseItem struct {
URL string `json:"url"`
Title string `json:"title"`
}
type SitemapScrapeReq struct {
KbID string `json:"kb_id" validate:"required"`
ID string `json:"id" validate:"required"`
URL string `json:"url" validate:"required"`
}
type SitemapScrapeResp struct {
Content string `json:"content"`
}
type RssParseReq struct {
URL string `json:"url" validate:"required"`
}
type RssParseResp struct {
ID string `json:"id"`
List []RssParseItem `json:"list"`
}
type RssParseItem struct {
URL string `json:"url"`
Title string `json:"title"`
Desc string `json:"desc"`
}
type RssScrapeReq struct {
KbID string `json:"kb_id" validate:"required"`
ID string `json:"id" validate:"required"`
URL string `json:"url" validate:"required"`
}
type RssScrapeResp struct {
Content string `json:"content"`
}

View File

@ -13,17 +13,24 @@ type GetNodeDetailReq struct {
}
type NodeDetailResp struct {
ID string `json:"id"`
KbID string `json:"kb_id"`
Type domain.NodeType `json:"type"`
Status domain.NodeStatus `json:"status"`
Name string `json:"name"`
Content string `json:"content"`
Meta domain.NodeMeta `json:"meta"`
ParentID string `json:"parent_id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
Permissions domain.NodePermissions `json:"permissions"`
ID string `json:"id"`
KbID string `json:"kb_id"`
Type domain.NodeType `json:"type"`
Status domain.NodeStatus `json:"status"`
Name string `json:"name"`
Content string `json:"content"`
Meta domain.NodeMeta `json:"meta"`
ParentID string `json:"parent_id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
Permissions domain.NodePermissions `json:"permissions"`
CreatorId string `json:"creator_id"`
EditorId string `json:"editor_id"`
PublisherId string `json:"publisher_id" gorm:"-"`
CreatorAccount string `json:"creator_account"`
EditorAccount string `json:"editor_account"`
PublisherAccount string `json:"publisher_account" gorm:"-"`
PV int64 `json:"pv" gorm:"-"`
}
type NodePermissionReq struct {
@ -50,3 +57,11 @@ type NodePermissionEditReq struct {
type NodePermissionEditResp struct {
}
type NodeRestudyReq struct {
NodeIds []string `json:"node_ids" validate:"required,min=1"`
KbId string `json:"kb_id" validate:"required"`
}
type NodeRestudyResp struct {
}

View File

@ -0,0 +1,11 @@
package v1
type FileUploadReq struct {
KbId string `form:"kb_id" json:"kb_id" validate:"required"`
File string `form:"file"`
CaptchaToken string `form:"captcha_token" json:"captcha_token" validate:"required"`
}
type FileUploadResp struct {
Key string `json:"key"`
}

View File

@ -0,0 +1,29 @@
package v1
import (
"time"
"github.com/chaitin/panda-wiki/domain"
)
type ShareNodeDetailResp struct {
ID string `json:"id"`
KbID string `json:"kb_id"`
Type domain.NodeType `json:"type"`
Status domain.NodeStatus `json:"status"`
Name string `json:"name"`
Content string `json:"content"`
Meta domain.NodeMeta `json:"meta"`
ParentID string `json:"parent_id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
Permissions domain.NodePermissions `json:"permissions"`
CreatorId string `json:"creator_id"`
EditorId string `json:"editor_id"`
PublisherId string `json:"publisher_id"`
CreatorAccount string `json:"creator_account"`
EditorAccount string `json:"editor_account"`
PublisherAccount string `json:"publisher_account"`
List []*domain.ShareNodeDetailItem `json:"list" gorm:"-"`
PV int64 `json:"pv" gorm:"-"`
}

View File

@ -0,0 +1,8 @@
package v1
type WechatAppInfoResp struct {
WeChatAppIsEnabled bool `json:"wechat_app_is_enabled"`
FeedbackEnable bool `json:"feedback_enable"`
FeedbackType []string `json:"feedback_type"`
DisclaimerContent string `json:"disclaimer_content"`
}

View File

@ -1,5 +1,4 @@
//go:build wireinject
// +build wireinject
package main

View File

@ -96,7 +96,9 @@ func createApp() (*App, error) {
return nil, err
}
authRepo := pg2.NewAuthRepo(db, logger, cacheCache)
nodeUsecase := usecase.NewNodeUsecase(nodeRepository, appRepository, ragRepository, knowledgeBaseRepository, llmUsecase, logger, minioClient, modelRepository, authRepo)
systemSettingRepo := pg2.NewSystemSettingRepo(db, logger)
modelUsecase := usecase.NewModelUsecase(modelRepository, nodeRepository, ragRepository, ragService, logger, configConfig, knowledgeBaseRepository, systemSettingRepo)
nodeUsecase := usecase.NewNodeUsecase(nodeRepository, appRepository, ragRepository, userRepository, knowledgeBaseRepository, llmUsecase, ragService, logger, minioClient, modelRepository, authRepo, modelUsecase)
nodeHandler := v1.NewNodeHandler(baseHandler, echo, nodeUsecase, authMiddleware, logger)
geoRepo := cache2.NewGeoCache(cacheCache, db, logger)
ipdbIPDB, err := ipdb.NewIPDB(configConfig, logger)
@ -105,7 +107,6 @@ func createApp() (*App, error) {
}
ipAddressRepo := ipdb2.NewIPAddressRepo(ipdbIPDB, logger)
conversationUsecase := usecase.NewConversationUsecase(conversationRepository, nodeRepository, geoRepo, logger, ipAddressRepo, authRepo)
modelUsecase := usecase.NewModelUsecase(modelRepository, nodeRepository, ragRepository, ragService, logger, configConfig, knowledgeBaseRepository)
blockWordRepo := pg2.NewBlockWordRepo(db, logger)
chatUsecase, err := usecase.NewChatUsecase(llmUsecase, knowledgeBaseRepository, conversationUsecase, modelUsecase, appRepository, blockWordRepo, authRepo, logger)
if err != nil {
@ -165,10 +166,11 @@ func createApp() (*App, error) {
wechatRepository := pg2.NewWechatRepository(db, logger)
wechatUsecase := usecase.NewWechatUsecase(logger, appUsecase, chatUsecase, wechatRepository, authRepo)
wecomUsecase := usecase.NewWecomUsecase(logger, cacheCache, appUsecase, chatUsecase, authRepo)
wechatAppUsecase := usecase.NewWechatAppUsecase(logger, appUsecase, chatUsecase, wechatRepository, authRepo)
wechatAppUsecase := usecase.NewWechatAppUsecase(logger, appUsecase, chatUsecase, wechatRepository, authRepo, appRepository)
shareWechatHandler := share.NewShareWechatHandler(echo, baseHandler, logger, appUsecase, conversationUsecase, wechatUsecase, wecomUsecase, wechatAppUsecase)
shareCaptchaHandler := share.NewShareCaptchaHandler(baseHandler, echo, logger)
openapiV1Handler := share.NewOpenapiV1Handler(echo, baseHandler, logger, authUsecase, appUsecase)
shareCommonHandler := share.NewShareCommonHandler(echo, baseHandler, logger, fileUsecase)
shareHandler := &share.ShareHandler{
ShareNodeHandler: shareNodeHandler,
ShareAppHandler: shareAppHandler,
@ -181,8 +183,10 @@ func createApp() (*App, error) {
ShareWechatHandler: shareWechatHandler,
ShareCaptchaHandler: shareCaptchaHandler,
OpenapiV1Handler: openapiV1Handler,
ShareCommonHandler: shareCommonHandler,
}
client, err := telemetry.NewClient(logger, knowledgeBaseRepository)
mcpRepository := pg2.NewMCPRepository(db, logger)
client, err := telemetry.NewClient(logger, knowledgeBaseRepository, modelUsecase, userUsecase, nodeRepository, conversationRepository, mcpRepository, configConfig)
if err != nil {
return nil, err
}

View File

@ -1,5 +1,4 @@
//go:build wireinject
// +build wireinject
package main
@ -28,5 +27,5 @@ type App struct {
MQConsumer mq.MQConsumer
Config *config.Config
MQHandlers *handler.MQHandlers
StatCronHandler *handler.StatCronHandler
StatCronHandler *handler.CronHandler
}

View File

@ -8,16 +8,18 @@ package main
import (
"github.com/chaitin/panda-wiki/config"
mq2 "github.com/chaitin/panda-wiki/handler/mq"
mq3 "github.com/chaitin/panda-wiki/handler/mq"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/mq"
cache2 "github.com/chaitin/panda-wiki/repo/cache"
ipdb2 "github.com/chaitin/panda-wiki/repo/ipdb"
mq2 "github.com/chaitin/panda-wiki/repo/mq"
pg2 "github.com/chaitin/panda-wiki/repo/pg"
"github.com/chaitin/panda-wiki/store/cache"
"github.com/chaitin/panda-wiki/store/ipdb"
"github.com/chaitin/panda-wiki/store/pg"
"github.com/chaitin/panda-wiki/store/rag"
"github.com/chaitin/panda-wiki/store/s3"
"github.com/chaitin/panda-wiki/usecase"
)
@ -47,7 +49,18 @@ func createApp() (*App, error) {
modelRepository := pg2.NewModelRepository(db, logger)
promptRepo := pg2.NewPromptRepo(db, logger)
llmUsecase := usecase.NewLLMUsecase(configConfig, ragService, conversationRepository, knowledgeBaseRepository, nodeRepository, modelRepository, promptRepo, logger)
ragmqHandler, err := mq2.NewRAGMQHandler(mqConsumer, logger, ragService, nodeRepository, knowledgeBaseRepository, llmUsecase, modelRepository)
mqProducer, err := mq.NewMQProducer(configConfig, logger)
if err != nil {
return nil, err
}
ragRepository := mq2.NewRAGRepository(mqProducer)
systemSettingRepo := pg2.NewSystemSettingRepo(db, logger)
modelUsecase := usecase.NewModelUsecase(modelRepository, nodeRepository, ragRepository, ragService, logger, configConfig, knowledgeBaseRepository, systemSettingRepo)
ragmqHandler, err := mq3.NewRAGMQHandler(mqConsumer, logger, ragService, nodeRepository, knowledgeBaseRepository, llmUsecase, modelUsecase)
if err != nil {
return nil, err
}
ragDocUpdateHandler, err := mq3.NewRagDocUpdateHandler(mqConsumer, logger, nodeRepository)
if err != nil {
return nil, err
}
@ -65,19 +78,26 @@ func createApp() (*App, error) {
geoRepo := cache2.NewGeoCache(cacheCache, db, logger)
authRepo := pg2.NewAuthRepo(db, logger, cacheCache)
statUseCase := usecase.NewStatUseCase(statRepository, nodeRepository, conversationRepository, appRepository, ipAddressRepo, geoRepo, authRepo, knowledgeBaseRepository, logger)
statCronHandler, err := mq2.NewStatCronHandler(logger, statRepository, statUseCase)
userRepository := pg2.NewUserRepository(db, logger)
minioClient, err := s3.NewMinioClient(configConfig)
if err != nil {
return nil, err
}
mqHandlers := &mq2.MQHandlers{
RAGMQHandler: ragmqHandler,
StatCronHandler: statCronHandler,
nodeUsecase := usecase.NewNodeUsecase(nodeRepository, appRepository, ragRepository, userRepository, knowledgeBaseRepository, llmUsecase, ragService, logger, minioClient, modelRepository, authRepo, modelUsecase)
cronHandler, err := mq3.NewStatCronHandler(logger, statRepository, statUseCase, nodeUsecase)
if err != nil {
return nil, err
}
mqHandlers := &mq3.MQHandlers{
RAGMQHandler: ragmqHandler,
RagDocUpdateHandler: ragDocUpdateHandler,
StatCronHandler: cronHandler,
}
app := &App{
MQConsumer: mqConsumer,
Config: configConfig,
MQHandlers: mqHandlers,
StatCronHandler: statCronHandler,
StatCronHandler: cronHandler,
}
return app, nil
}
@ -87,6 +107,6 @@ func createApp() (*App, error) {
type App struct {
MQConsumer mq.MQConsumer
Config *config.Config
MQHandlers *mq2.MQHandlers
StatCronHandler *mq2.StatCronHandler
MQHandlers *mq3.MQHandlers
StatCronHandler *mq3.CronHandler
}

View File

@ -1,5 +1,4 @@
//go:build wireinject
// +build wireinject
package main

View File

@ -41,6 +41,7 @@ func createApp() (*App, error) {
return nil, err
}
ragRepository := mq2.NewRAGRepository(mqProducer)
userRepository := pg2.NewUserRepository(db, logger)
ragService, err := rag.NewRAGService(configConfig, logger)
if err != nil {
return nil, err
@ -59,8 +60,9 @@ func createApp() (*App, error) {
return nil, err
}
authRepo := pg2.NewAuthRepo(db, logger, cacheCache)
nodeUsecase := usecase.NewNodeUsecase(nodeRepository, appRepository, ragRepository, knowledgeBaseRepository, llmUsecase, logger, minioClient, modelRepository, authRepo)
userRepository := pg2.NewUserRepository(db, logger)
systemSettingRepo := pg2.NewSystemSettingRepo(db, logger)
modelUsecase := usecase.NewModelUsecase(modelRepository, nodeRepository, ragRepository, ragService, logger, configConfig, knowledgeBaseRepository, systemSettingRepo)
nodeUsecase := usecase.NewNodeUsecase(nodeRepository, appRepository, ragRepository, userRepository, knowledgeBaseRepository, llmUsecase, ragService, logger, minioClient, modelRepository, authRepo, modelUsecase)
kbRepo := cache2.NewKBRepo(cacheCache)
knowledgeBaseUsecase, err := usecase.NewKnowledgeBaseUsecase(knowledgeBaseRepository, nodeRepository, ragRepository, userRepository, ragService, kbRepo, logger, configConfig)
if err != nil {

View File

@ -24,6 +24,7 @@ const (
SourceTypeDiscordBot SourceType = "discord_bot"
SourceTypeWechatOfficialAccount SourceType = "wechat_official_account"
SourceTypeOpenAIAPI SourceType = "openai_api"
SourceTypeMcpServer SourceType = "mcp_server"
)
func (s SourceType) Name() string {
@ -46,6 +47,8 @@ func (s SourceType) Name() string {
return "Discord 机器人"
case SourceTypeWechatOfficialAccount:
return "微信公众号"
case SourceTypeMcpServer:
return "MCP 服务器"
default:
return ""
}

View File

@ -1,8 +1,6 @@
package consts
import (
"math"
"github.com/labstack/echo/v4"
)
@ -13,28 +11,13 @@ const ContextKeyEdition contextKey = "edition"
type LicenseEdition int32
const (
LicenseEditionFree LicenseEdition = 0 // 开源版
LicenseEditionContributor LicenseEdition = 1 // 联创版
LicenseEditionEnterprise LicenseEdition = 2 // 企业版
LicenseEditionFree LicenseEdition = 0 // 开源版
LicenseEditionProfession LicenseEdition = 1 // 专业版
LicenseEditionEnterprise LicenseEdition = 2 // 企业版
LicenseEditionBusiness LicenseEdition = 3 // 商业版
)
func GetLicenseEdition(c echo.Context) LicenseEdition {
edition, _ := c.Get("edition").(LicenseEdition)
return edition
}
func (e LicenseEdition) GetMaxAuth(sourceType SourceType) int {
switch e {
case LicenseEditionFree:
if sourceType == SourceTypeGitHub {
return 10
}
return 0
case LicenseEditionContributor:
return 10
case LicenseEditionEnterprise:
return math.MaxInt
default:
return 0
}
}

39
backend/consts/model.go Normal file
View File

@ -0,0 +1,39 @@
package consts
type AutoModeDefaultModel string
const (
AutoModeDefaultChatModel AutoModeDefaultModel = "deepseek-chat"
AutoModeDefaultEmbeddingModel AutoModeDefaultModel = "bge-m3"
AutoModeDefaultRerankModel AutoModeDefaultModel = "bge-reranker-v2-m3"
AutoModeDefaultAnalysisModel AutoModeDefaultModel = "qwen2.5-3b-instruct"
AutoModeDefaultAnalysisVLModel AutoModeDefaultModel = "qwen3-vl-max"
)
func GetAutoModeDefaultModel(modelType string) string {
switch modelType {
case "chat":
return string(AutoModeDefaultChatModel)
case "embedding":
return string(AutoModeDefaultEmbeddingModel)
case "rerank":
return string(AutoModeDefaultRerankModel)
case "analysis":
return string(AutoModeDefaultAnalysisModel)
case "analysis-vl":
return string(AutoModeDefaultAnalysisVLModel)
default:
return string(AutoModeDefaultChatModel)
}
}
type ModelSettingMode string
const (
ModelSettingModeManual ModelSettingMode = "manual"
ModelSettingModeAuto ModelSettingMode = "auto"
)
const (
AutoModeBaseURL = "https://model-square.app.baizhi.cloud/v1"
)

View File

@ -15,3 +15,17 @@ const (
NodePermNameVisitable NodePermName = "visitable" // 可被访问
NodePermNameAnswerable NodePermName = "answerable" // 可被问答
)
type NodeRagInfoStatus string
const (
NodeRagStatusBasicPending NodeRagInfoStatus = "BASIC_PENDING" // 等待基础处理
NodeRagStatusBasicRunning NodeRagInfoStatus = "BASIC_RUNNING" // 正在进行基础处理(文本分割、向量化等)
NodeRagStatusBasicFailed NodeRagInfoStatus = "BASIC_FAILED" // 基础处理失败
NodeRagStatusBasicSucceeded NodeRagInfoStatus = "BASIC_SUCCEEDED" // 基础处理成功
NodeRagStatusEnhancePending NodeRagInfoStatus = "ENHANCE_PENDING" // 基础处理完成,等待增强处理
NodeRagStatusEnhanceRunning NodeRagInfoStatus = "ENHANCE_RUNNING" // 正在进行增强处理(关键词提取等)
NodeRagStatusEnhanceFailed NodeRagInfoStatus = "ENHANCE_FAILED" // 增强处理失败
NodeRagStatusEnhanceSucceeded NodeRagInfoStatus = "ENHANCE_SUCCEEDED" // 增强处理成功
)

42
backend/consts/parse.go Normal file
View File

@ -0,0 +1,42 @@
package consts
type CrawlerSource string
const (
// CrawlerSourceUrl key或url形式 直接走parse接口
CrawlerSourceUrl CrawlerSource = "url"
CrawlerSourceRSS CrawlerSource = "rss"
CrawlerSourceSitemap CrawlerSource = "sitemap"
CrawlerSourceNotion CrawlerSource = "notion"
CrawlerSourceFeishu CrawlerSource = "feishu"
// CrawlerSourceFile file形式 需要先走upload接口先上传文件
CrawlerSourceFile CrawlerSource = "file"
CrawlerSourceEpub CrawlerSource = "epub"
CrawlerSourceYuque CrawlerSource = "yuque"
CrawlerSourceSiyuan CrawlerSource = "siyuan"
CrawlerSourceMindoc CrawlerSource = "mindoc"
CrawlerSourceWikijs CrawlerSource = "wikijs"
CrawlerSourceConfluence CrawlerSource = "confluence"
)
type CrawlerSourceType string
const (
CrawlerSourceTypeFile CrawlerSourceType = "file"
CrawlerSourceTypeUrl CrawlerSourceType = "url"
CrawlerSourceTypeKey CrawlerSourceType = "key"
)
func (c CrawlerSource) Type() CrawlerSourceType {
switch c {
case CrawlerSourceNotion, CrawlerSourceFeishu:
return CrawlerSourceTypeKey
case CrawlerSourceUrl, CrawlerSourceRSS, CrawlerSourceSitemap:
return CrawlerSourceTypeUrl
case CrawlerSourceFile, CrawlerSourceEpub, CrawlerSourceYuque, CrawlerSourceSiyuan, CrawlerSourceMindoc, CrawlerSourceWikijs, CrawlerSourceConfluence:
return CrawlerSourceTypeFile
default:
return ""
}
}

View File

@ -0,0 +1,7 @@
package consts
type SystemSettingKey string
const (
SystemSettingModelMode SystemSettingKey = "model_setting_mode"
)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -24,6 +24,7 @@ const (
AppTypeOpenAIAPI
AppTypeWecomAIBot
AppTypeLarkBot
AppTypeMcpServer
)
var AppTypes = []AppType{
@ -38,6 +39,7 @@ var AppTypes = []AppType{
AppTypeOpenAIAPI,
AppTypeWecomAIBot,
AppTypeLarkBot,
AppTypeMcpServer,
}
func (t AppType) ToSourceType() consts.SourceType {
@ -92,9 +94,8 @@ type AppSettings struct {
RecommendQuestions []string `json:"recommend_questions,omitempty"`
RecommendNodeIDs []string `json:"recommend_node_ids,omitempty"`
// seo
Desc string `json:"desc,omitempty"`
Keyword string `json:"keyword,omitempty"`
AutoSitemap bool `json:"auto_sitemap,omitempty"`
Desc string `json:"desc,omitempty"`
Keyword string `json:"keyword,omitempty"`
// inject code
HeadCode string `json:"head_code,omitempty"`
BodyCode string `json:"body_code,omitempty"`
@ -110,12 +111,13 @@ type AppSettings struct {
// LarkBot
LarkBotSettings LarkBotSettings `json:"lark_bot_settings,omitempty"`
// WechatAppBot 企业微信机器人
WeChatAppIsEnabled *bool `json:"wechat_app_is_enabled,omitempty"`
WeChatAppToken string `json:"wechat_app_token,omitempty"`
WeChatAppEncodingAESKey string `json:"wechat_app_encodingaeskey,omitempty"`
WeChatAppCorpID string `json:"wechat_app_corpid,omitempty"`
WeChatAppSecret string `json:"wechat_app_secret,omitempty"`
WeChatAppAgentID string `json:"wechat_app_agent_id,omitempty"`
WeChatAppIsEnabled *bool `json:"wechat_app_is_enabled,omitempty"`
WeChatAppToken string `json:"wechat_app_token,omitempty"`
WeChatAppEncodingAESKey string `json:"wechat_app_encodingaeskey,omitempty"`
WeChatAppCorpID string `json:"wechat_app_corpid,omitempty"`
WeChatAppSecret string `json:"wechat_app_secret,omitempty"`
WeChatAppAgentID string `json:"wechat_app_agent_id,omitempty"`
WeChatAppAdvancedSetting WeChatAppAdvancedSetting `json:"wechat_app_advanced_setting"`
// WecomAIBotSettings 企业微信智能机器人
WecomAIBotSettings WecomAIBotSettings `json:"wecom_ai_bot_settings"`
// WechatServiceBot
@ -159,12 +161,49 @@ type AppSettings struct {
DisclaimerSettings DisclaimerSettings `json:"disclaimer_settings"`
// WebAppLandingConfigs
WebAppLandingConfigs []WebAppLandingConfig `json:"web_app_landing_configs,omitempty"`
WebAppLandingTheme WebAppLandingTheme `json:"web_app_landing_theme"`
WatermarkContent string `json:"watermark_content"`
WatermarkSetting consts.WatermarkSetting `json:"watermark_setting" validate:"omitempty,oneof='' hidden visible"`
CopySetting consts.CopySetting `json:"copy_setting" validate:"omitempty,oneof='' append disabled"`
ContributeSettings ContributeSettings `json:"contribute_settings"`
HomePageSetting consts.HomePageSetting `json:"home_page_setting"`
WatermarkContent string `json:"watermark_content"`
WatermarkSetting consts.WatermarkSetting `json:"watermark_setting" validate:"omitempty,oneof='' hidden visible"`
CopySetting consts.CopySetting `json:"copy_setting" validate:"omitempty,oneof='' append disabled"`
ContributeSettings ContributeSettings `json:"contribute_settings"`
HomePageSetting consts.HomePageSetting `json:"home_page_setting"`
ConversationSetting ConversationSetting `json:"conversation_setting"`
// MCP Server Settings
MCPServerSettings MCPServerSettings `json:"mcp_server_settings,omitempty"`
StatsSetting StatsSetting `json:"stats_setting"`
}
type WeChatAppAdvancedSetting struct {
TextResponseEnable bool `json:"text_response_enable,omitempty"`
FeedbackEnable bool `json:"feedback_enable,omitempty"`
FeedbackType []string `json:"feedback_type,omitempty"`
DisclaimerContent string `json:"disclaimer_content,omitempty"`
Prompt string `json:"prompt,omitempty"`
}
type StatsSetting struct {
PVEnable bool `json:"pv_enable"`
}
type ConversationSetting struct {
CopyrightInfo string `json:"copyright_info"`
CopyrightHideEnabled bool `json:"copyright_hide_enabled"`
}
type WebAppLandingTheme struct {
Name string `json:"name"`
}
type MCPServerSettings struct {
IsEnabled bool `json:"is_enabled"`
DocsToolSettings MCPToolSettings `json:"docs_tool_settings"`
SampleAuth SimpleAuth `json:"sample_auth"`
}
type MCPToolSettings struct {
Name string `json:"name"`
Desc string `json:"desc"`
}
type LarkBotSettings struct {
@ -228,6 +267,84 @@ type FaqConfig struct {
Link string `json:"link"`
} `json:"list"`
}
type TextConfig struct {
Type string `json:"type"`
Title string `json:"title"`
}
type MetricsConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Name string `json:"name"`
Number string `json:"number"`
} `json:"list"`
}
type CaseConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Name string `json:"name"`
Link string `json:"link"`
} `json:"list"`
}
type CommentConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Avatar string `json:"avatar"`
UserName string `json:"user_name"`
Profession string `json:"profession"`
Comment string `json:"comment"`
} `json:"list"`
}
type FeatureConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Name string `json:"name"`
Desc string `json:"desc"`
} `json:"list"`
}
type ImgTextConfig struct {
Type string `json:"type"`
Title string `json:"title"`
Item struct {
URL string `json:"url"`
Name string `json:"name"`
Desc string `json:"desc"`
} `json:"item"`
}
type TextImgConfig struct {
Type string `json:"type"`
Title string `json:"title"`
Item struct {
URL string `json:"url"`
Name string `json:"name"`
Desc string `json:"desc"`
} `json:"item"`
}
type QuestionConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Question string `json:"question"`
} `json:"list"`
}
type BlockGridConfig struct {
Type string `json:"type"`
Title string `json:"title"`
List []struct {
ID string `json:"id"`
Name string `json:"name"`
URL string `json:"url"`
} `json:"list"`
}
type WebAppLandingConfig struct {
Type string `json:"type"`
NodeIds []string `json:"node_ids"`
@ -237,6 +354,15 @@ type WebAppLandingConfig struct {
SimpleDocConfig *SimpleDocConfig `json:"simple_doc_config,omitempty"`
CarouselConfig *CarouselConfig `json:"carousel_config,omitempty"`
FaqConfig *FaqConfig `json:"faq_config,omitempty"`
MetricsConfig *MetricsConfig `json:"metrics_config,omitempty"`
CaseConfig *CaseConfig `json:"case_config,omitempty"`
TextConfig *TextConfig `json:"text_config,omitempty"`
CommentConfig *CommentConfig `json:"comment_config,omitempty"`
FeatureConfig *FeatureConfig `json:"feature_config,omitempty"`
ImgTextConfig *ImgTextConfig `json:"img_text_config,omitempty"`
TextImgConfig *TextImgConfig `json:"text_img_config,omitempty"`
QuestionConfig *QuestionConfig `json:"question_config,omitempty"`
BlockGridConfig *BlockGridConfig `json:"block_grid_config,omitempty"`
ComConfigOrder []string `json:"com_config_order"`
}
@ -307,10 +433,21 @@ type FooterSettings struct {
}
type WidgetBotSettings struct {
IsOpen bool `json:"is_open,omitempty"`
ThemeMode string `json:"theme_mode,omitempty"`
BtnText string `json:"btn_text,omitempty"`
BtnLogo string `json:"btn_logo,omitempty"`
IsOpen bool `json:"is_open,omitempty"`
ThemeMode string `json:"theme_mode,omitempty"`
BtnText string `json:"btn_text,omitempty"`
BtnLogo string `json:"btn_logo,omitempty"`
RecommendQuestions []string `json:"recommend_questions,omitempty"`
RecommendNodeIDs []string `json:"recommend_node_ids,omitempty"`
BtnStyle string `json:"btn_style,omitempty"`
BtnID string `json:"btn_id,omitempty"`
BtnPosition string `json:"btn_position,omitempty"`
ModalPosition string `json:"modal_position,omitempty"`
SearchMode string `json:"search_mode,omitempty"`
Placeholder string `json:"placeholder,omitempty"`
Disclaimer string `json:"disclaimer,omitempty"`
CopyrightInfo string `json:"copyright_info,omitempty"`
CopyrightHideEnabled bool `json:"copyright_hide_enabled,omitempty"`
}
type BrandGroup struct {
@ -358,9 +495,8 @@ type AppSettingsResp struct {
RecommendQuestions []string `json:"recommend_questions,omitempty"`
RecommendNodeIDs []string `json:"recommend_node_ids,omitempty"`
// seo
Desc string `json:"desc,omitempty"`
Keyword string `json:"keyword,omitempty"`
AutoSitemap bool `json:"auto_sitemap,omitempty"`
Desc string `json:"desc,omitempty"`
Keyword string `json:"keyword,omitempty"`
// inject code
HeadCode string `json:"head_code,omitempty"`
BodyCode string `json:"body_code,omitempty"`
@ -376,12 +512,13 @@ type AppSettingsResp struct {
// LarkBot
LarkBotSettings LarkBotSettings `json:"lark_bot_settings,omitempty"`
// WechatAppBot
WeChatAppIsEnabled *bool `json:"wechat_app_is_enabled,omitempty"`
WeChatAppToken string `json:"wechat_app_token,omitempty"`
WeChatAppEncodingAESKey string `json:"wechat_app_encodingaeskey,omitempty"`
WeChatAppCorpID string `json:"wechat_app_corpid,omitempty"`
WeChatAppSecret string `json:"wechat_app_secret,omitempty"`
WeChatAppAgentID string `json:"wechat_app_agent_id,omitempty"`
WeChatAppIsEnabled *bool `json:"wechat_app_is_enabled,omitempty"`
WeChatAppToken string `json:"wechat_app_token,omitempty"`
WeChatAppEncodingAESKey string `json:"wechat_app_encodingaeskey,omitempty"`
WeChatAppCorpID string `json:"wechat_app_corpid,omitempty"`
WeChatAppSecret string `json:"wechat_app_secret,omitempty"`
WeChatAppAgentID string `json:"wechat_app_agent_id,omitempty"`
WeChatAppAdvancedSetting WeChatAppAdvancedSetting `json:"wechat_app_advanced_setting"`
// WechatServiceBot
WeChatServiceIsEnabled *bool `json:"wechat_service_is_enabled,omitempty"`
WeChatServiceToken string `json:"wechat_service_token,omitempty"`
@ -432,7 +569,12 @@ type AppSettingsResp struct {
DisclaimerSettings DisclaimerSettings `json:"disclaimer_settings"`
// WebApp Landing Settings
WebAppLandingConfigs []WebAppLandingConfigResp `json:"web_app_landing_configs,omitempty"`
WebAppLandingTheme WebAppLandingTheme `json:"web_app_landing_theme"`
HomePageSetting consts.HomePageSetting `json:"home_page_setting"`
ConversationSetting ConversationSetting `json:"conversation_setting"`
// MCP Server Settings
MCPServerSettings MCPServerSettings `json:"mcp_server_settings,omitempty"`
StatsSetting StatsSetting `json:"stats_setting"`
}
type WebAppLandingConfigResp struct {
@ -443,6 +585,15 @@ type WebAppLandingConfigResp struct {
SimpleDocConfig *SimpleDocConfig `json:"simple_doc_config,omitempty"`
CarouselConfig *CarouselConfig `json:"carousel_config,omitempty"`
FaqConfig *FaqConfig `json:"faq_config,omitempty"`
MetricsConfig *MetricsConfig `json:"metrics_config,omitempty"`
CaseConfig *CaseConfig `json:"case_config,omitempty"`
TextConfig *TextConfig `json:"text_config,omitempty"`
CommentConfig *CommentConfig `json:"comment_config,omitempty"`
FeatureConfig *FeatureConfig `json:"feature_config,omitempty"`
ImgTextConfig *ImgTextConfig `json:"img_text_config,omitempty"`
TextImgConfig *TextImgConfig `json:"text_img_config,omitempty"`
QuestionConfig *QuestionConfig `json:"question_config,omitempty"`
BlockGridConfig *BlockGridConfig `json:"block_grid_config,omitempty"`
ComConfigOrder []string `json:"com_config_order"`
NodeIds []string `json:"node_ids"`
Nodes []*RecommendNodeListResp `json:"nodes" gorm:"-"`

View File

@ -21,6 +21,16 @@ type ChatRequest struct {
RemoteIP string `json:"-"`
Info ConversationInfo `json:"-"`
Prompt string `json:"-"`
}
type ChatRagOnlyRequest struct {
Message string `json:"message" validate:"required"`
KBID string `json:"-" validate:"required"`
UserInfo UserInfo `json:"user_info"`
AppType AppType `json:"app_type" validate:"required,oneof=1 2"`
}
type ConversationInfo struct {

View File

@ -6,20 +6,26 @@ import (
"errors"
"fmt"
"time"
"github.com/lib/pq"
)
type Comment struct {
ID string `json:"id" gorm:"primaryKey"`
ID string `json:"id" gorm:"primaryKey"`
KbID string `json:"kb_id"`
UserID string `json:"user_id"`
NodeID string `json:"node_id" gorm:"index"`
Info CommentInfo `json:"info" gorm:"type:jsonb"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
Content string `json:"content"`
Status CommentStatus `json:"status"` // status : -1 reject 0 pending 1 accept
PicUrls pq.StringArray `json:"pic_urls" gorm:"type:text[];not null;default:{}"`
CreatedAt time.Time `json:"created_at"`
}
KbID string `json:"kb_id"`
UserID string `json:"user_id"`
NodeID string `json:"node_id" gorm:"index"`
Info CommentInfo `json:"info" gorm:"type:jsonb"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
Content string `json:"content"`
Status CommentStatus `json:"status"` // status : -1 reject 0 pending 1 accept
CreatedAt time.Time `json:"created_at"`
func (Comment) TableName() string {
return "comments"
}
type CommentInfo struct {
@ -50,14 +56,14 @@ func (d *CommentInfo) Scan(value any) error {
return json.Unmarshal(bytes, d)
}
// 前端请求
type CommentReq struct {
NodeID string `json:"node_id" validate:"required"`
Content string `json:"content" validate:"required"`
UserName string `json:"user_name"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
CaptchaToken string `json:"captcha_token"`
NodeID string `json:"node_id" validate:"required"`
Content string `json:"content" validate:"required"`
UserName string `json:"user_name"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
CaptchaToken string `json:"captcha_token"`
PicUrls []string `json:"pic_urls" validate:"required"`
}
type CommentListReq struct {
@ -84,14 +90,14 @@ type DeleteCommentListReq struct {
}
type ShareCommentListItem struct {
ID string `json:"id" gorm:"primaryKey"`
KbID string `json:"kb_id"`
NodeID string `json:"node_id" gorm:"index"`
Info CommentInfo `json:"info" gorm:"type:jsonb"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
Content string `json:"content"`
IPAddress *IPAddress `json:"ip_address" gorm:"-"` // ip地址
CreatedAt time.Time `json:"created_at"`
ID string `json:"id" gorm:"primaryKey"`
KbID string `json:"kb_id"`
NodeID string `json:"node_id" gorm:"index"`
Info CommentInfo `json:"info" gorm:"type:jsonb"`
ParentID string `json:"parent_id"`
RootID string `json:"root_id"`
Content string `json:"content"`
PicUrls pq.StringArray `json:"pic_urls" gorm:"type:text[]"`
IPAddress *IPAddress `json:"ip_address" gorm:"-"` // ip地址
CreatedAt time.Time `json:"created_at"`
}

View File

@ -129,6 +129,10 @@ type KBReleaseNodeRelease struct {
CreatedAt time.Time `json:"created_at"`
}
func (KBReleaseNodeRelease) TableName() string {
return "kb_release_node_releases"
}
type CreateKBReleaseReq struct {
KBID string `json:"kb_id" validate:"required"`
Message string `json:"message" validate:"required"`

45
backend/domain/license.go Normal file
View File

@ -0,0 +1,45 @@
package domain
import (
"context"
"encoding/json"
)
const ContextKeyEditionLimitation contextKey = "edition_limitation"
type BaseEditionLimitation struct {
MaxKb int `json:"max_kb"` // 知识库站点数量
MaxNode int `json:"max_node"` // 单个知识库下文档数量
MaxSSOUser int `json:"max_sso_users"` // SSO认证用户数量
MaxAdmin int64 `json:"max_admin"` // 后台管理员数量
AllowAdminPerm bool `json:"allow_admin_perm"` // 支持管理员分权控制
AllowCustomCopyright bool `json:"allow_custom_copyright"` // 支持自定义版权信息
AllowCommentAudit bool `json:"allow_comment_audit"` // 支持评论审核
AllowAdvancedBot bool `json:"allow_advanced_bot"` // 支持高级机器人配置
AllowWatermark bool `json:"allow_watermark"` // 支持水印
AllowCopyProtection bool `json:"allow_copy_protection"` // 支持内容复制保护
AllowOpenAIBotSettings bool `json:"allow_open_ai_bot_settings"` // 支持问答机器人
AllowMCPServer bool `json:"allow_mcp_server"` // 支持创建MCP Server
AllowNodeStats bool `json:"allow_node_stats"` // 支持文档统计
}
var baseEditionLimitationDefault = BaseEditionLimitation{
MaxKb: 1,
MaxAdmin: 1,
MaxNode: 300,
}
func GetBaseEditionLimitation(c context.Context) BaseEditionLimitation {
edition, ok := c.Value(ContextKeyEditionLimitation).([]byte)
if !ok {
return baseEditionLimitationDefault
}
var editionLimitation BaseEditionLimitation
if err := json.Unmarshal(edition, &editionLimitation); err != nil {
return baseEditionLimitationDefault
}
return editionLimitation
}

View File

@ -6,7 +6,7 @@ import (
"strings"
)
var SystemPrompt = `
var SystemDefaultPrompt = `
你是一个专业的AI知识库问答助手要按照以下步骤回答用户问题
请仔细阅读以下信息

View File

@ -165,3 +165,13 @@ type ProviderModelListItem struct {
type ActivateModelReq struct {
ModelID string `json:"model_id" validate:"required"`
}
type SwitchModeReq struct {
Mode string `json:"mode" validate:"required,oneof=manual auto"`
AutoModeAPIKey string `json:"auto_mode_api_key"` // 百智云 API Key
ChatModel string `json:"chat_model"` // 自定义对话模型名称
}
type SwitchModeResp struct {
Message string `json:"message"`
}

View File

@ -3,11 +3,13 @@ package domain
const (
VectorTaskTopic = "apps.panda-wiki.vector.task"
AnydocTaskExportTopic = "anydoc.persistence.doc.task.export"
RagDocUpdateTopic = "rag.doc.update"
)
var TopicConsumerName = map[string]string{
VectorTaskTopic: "panda-wiki-vector-consumer",
AnydocTaskExportTopic: "anydoc-task-export-consumer",
RagDocUpdateTopic: "rag-doc-update-consumer",
}
type NodeReleaseVectorRequest struct {
@ -29,3 +31,9 @@ type AnydocTaskExportEvent struct {
Markdown string `json:"markdown"`
JSON string `json:"json"`
}
type RagDocInfoUpdateEvent struct {
ID string `json:"id"`
Status string `json:"status"`
Message string `json:"message"`
}

View File

@ -31,32 +31,51 @@ const (
NodeStatusReleased NodeStatus = 2
)
const (
ContentTypeMD string = "md"
ContentTypeHTML string = "html"
)
// table: nodes
type Node struct {
ID string `json:"id" gorm:"primaryKey"`
KBID string `json:"kb_id" gorm:"index"`
Type NodeType `json:"type"`
Status NodeStatus `json:"status"`
Name string `json:"name"`
Content string `json:"content"`
Meta NodeMeta `json:"meta" gorm:"type:jsonb"` // summary
ParentID string `json:"parent_id"`
Position float64 `json:"position"`
DocID string `json:"doc_id"` // DEPRECATED: for rag service
CreatorId string `json:"creator_id"`
EditorId string `json:"editor_id"`
EditTime time.Time `json:"edit_time"`
ID string `json:"id" gorm:"primaryKey"`
KBID string `json:"kb_id" gorm:"index"`
Type NodeType `json:"type"`
Status NodeStatus `json:"status"`
RagInfo RagInfo `json:"rag_info" gorm:"type:jsonb"`
Name string `json:"name"`
Content string `json:"content"`
Meta NodeMeta `json:"meta" gorm:"type:jsonb"` // summary
ParentID string `json:"parent_id"`
Position float64 `json:"position"`
DocID string `json:"doc_id"` // DEPRECATED: for rag service
CreatorId string `json:"creator_id"`
EditorId string `json:"editor_id"`
EditTime time.Time `json:"edit_time"`
Permissions NodePermissions `json:"permissions" gorm:"type:jsonb"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
func (Node) TableName() string {
return "nodes"
}
type RagInfo struct {
Status consts.NodeRagInfoStatus `json:"status"`
Message string `json:"message"`
}
func (d *RagInfo) Value() (driver.Value, error) {
return json.Marshal(d)
}
func (d *RagInfo) Scan(value any) error {
bytes, ok := value.([]byte)
if !ok {
return errors.New(fmt.Sprint("invalid node meta type:", value))
}
return json.Unmarshal(bytes, d)
}
type NodePermissions struct {
@ -99,8 +118,9 @@ type NodeGroupDetail struct {
}
type NodeMeta struct {
Summary string `json:"summary"`
Emoji string `json:"emoji"`
Summary string `json:"summary"`
Emoji string `json:"emoji"`
ContentType string `json:"content_type"`
}
func (d *NodeMeta) Value() (driver.Value, error) {
@ -123,8 +143,9 @@ type CreateNodeReq struct {
Name string `json:"name" validate:"required"`
Content string `json:"content"`
Emoji string `json:"emoji"`
Summary *string `json:"summary"`
Emoji string `json:"emoji"`
Summary *string `json:"summary"`
ContentType *string `json:"content_type"`
MaxNode int `json:"-"`
@ -140,9 +161,11 @@ type NodeListItemResp struct {
ID string `json:"id"`
Type NodeType `json:"type"`
Status NodeStatus `json:"status"`
RagInfo RagInfo `json:"rag_info"`
Name string `json:"name"`
Summary string `json:"summary"`
Emoji string `json:"emoji"`
ContentType string `json:"content_type"`
Position float64 `json:"position"`
ParentID string `json:"parent_id"`
CreatedAt time.Time `json:"created_at"`
@ -151,6 +174,7 @@ type NodeListItemResp struct {
EditorId string `json:"editor_id"`
Creator string `json:"creator"`
Editor string `json:"editor"`
PublisherId string `json:"publisher_id" gorm:"-"`
Permissions NodePermissions `json:"permissions" gorm:"type:jsonb"`
}
@ -165,11 +189,12 @@ type NodeContentChunk struct {
}
type RankedNodeChunks struct {
NodeID string
NodeName string
NodeSummary string
NodeEmoji string
Chunks []*NodeContentChunk
NodeID string
NodeName string
NodeSummary string
NodeEmoji string
NodePathNames []string
Chunks []*NodeContentChunk
}
func (n *RankedNodeChunks) GetURL(baseURL string) string {
@ -184,10 +209,11 @@ type ChunkListItemResp struct {
}
type NodeContentChunkSSE struct {
NodeID string `json:"node_id"`
Name string `json:"name"`
Summary string `json:"summary"`
Emoji string `json:"emoji"`
NodeID string `json:"node_id"`
Name string `json:"name"`
Summary string `json:"summary"`
Emoji string `json:"emoji"`
NodePathNames []string `json:"node_path_names"`
}
type RecommendNodeListResp struct {
@ -209,13 +235,14 @@ type NodeActionReq struct {
}
type UpdateNodeReq struct {
ID string `json:"id" validate:"required"`
KBID string `json:"kb_id" validate:"required"`
Name *string `json:"name"`
Content *string `json:"content"`
Emoji *string `json:"emoji"`
Summary *string `json:"summary"`
Position *float64 `json:"position"`
ID string `json:"id" validate:"required"`
KBID string `json:"kb_id" validate:"required"`
Name *string `json:"name"`
Content *string `json:"content"`
Emoji *string `json:"emoji"`
Summary *string `json:"summary"`
Position *float64 `json:"position"`
ContentType *string `json:"content_type"`
}
type ShareNodeListItemResp struct {
@ -225,10 +252,24 @@ type ShareNodeListItemResp struct {
ParentID string `json:"parent_id"`
Position float64 `json:"position"`
Emoji string `json:"emoji"`
Meta NodeMeta `json:"meta"`
UpdatedAt time.Time `json:"updated_at"`
Permissions NodePermissions `json:"permissions" gorm:"type:jsonb"`
}
type ShareNodeDetailItem struct {
ID string `json:"id"`
Name string `json:"name"`
Type NodeType `json:"type"`
ParentID string `json:"parent_id"`
Position float64 `json:"position"`
Emoji string `json:"emoji"`
Meta NodeMeta `json:"meta"`
UpdatedAt time.Time `json:"updated_at"`
Permissions NodePermissions `json:"permissions" gorm:"type:jsonb"`
Children []*ShareNodeDetailItem `json:"children,omitempty"`
}
func (n *ShareNodeListItemResp) GetURL(baseURL string) string {
return fmt.Sprintf("%s/node/%s", baseURL, n.ID)
}
@ -253,10 +294,12 @@ type GetRecommendNodeListReq struct {
// table: node_releases
type NodeRelease struct {
ID string `json:"id" gorm:"primaryKey"`
KBID string `json:"kb_id" gorm:"index"`
NodeID string `json:"node_id" gorm:"index"`
DocID string `json:"doc_id" gorm:"index"` // for rag service
ID string `json:"id" gorm:"primaryKey"`
KBID string `json:"kb_id" gorm:"index"`
PublisherId string `json:"publisher_id"`
EditorId string `json:"editor_id"`
NodeID string `json:"node_id" gorm:"index"`
DocID string `json:"doc_id" gorm:"index"` // for rag service
Type NodeType `json:"type"`
@ -271,6 +314,10 @@ type NodeRelease struct {
UpdatedAt time.Time `json:"updated_at"`
}
func (NodeRelease) TableName() string {
return "node_releases"
}
// NodeReleaseWithDirPath extends NodeRelease with directory path information
type NodeReleaseWithDirPath struct {
*NodeRelease
@ -282,3 +329,15 @@ type BatchMoveReq struct {
KBID string `json:"kb_id" validate:"required"`
ParentID string `json:"parent_id"`
}
type NodeCreateInfo struct {
ID string `json:"id"`
Account string `json:"account"`
CreatorId string `json:"creator_id"`
}
type NodeReleaseWithPublisher struct {
ID string `json:"id" gorm:"primaryKey"`
PublisherId string `json:"publisher_id"`
PublisherAccount string `json:"publisher_account"`
}

View File

@ -1,10 +1,17 @@
package domain
import (
"encoding/json"
"fmt"
"strings"
)
// OpenAI API 请求结构体
type OpenAICompletionsRequest struct {
Model string `json:"model" validate:"required"`
Messages []OpenAIMessage `json:"messages" validate:"required"`
Stream bool `json:"stream,omitempty"`
StreamOptions *OpenAIStreamOptions `json:"stream_options,omitempty"`
Temperature *float64 `json:"temperature,omitempty"`
MaxTokens *int `json:"max_tokens,omitempty"`
TopP *float64 `json:"top_p,omitempty"`
@ -17,9 +24,95 @@ type OpenAICompletionsRequest struct {
ResponseFormat *OpenAIResponseFormat `json:"response_format,omitempty"`
}
type OpenAIStreamOptions struct {
IncludeUsage bool `json:"include_usage,omitempty"`
}
// MessageContent 支持字符串或内容数组
type MessageContent struct {
isString bool
strValue string
arrValue []OpenAIContentPart
}
// OpenAIContentPart 表示内容数组中的单个元素
type OpenAIContentPart struct {
Type string `json:"type"`
Text string `json:"text,omitempty"`
ImageURL *OpenAIContentPartURL `json:"image_url,omitempty"`
}
// OpenAIContentPartURL represents the image_url field in content parts
type OpenAIContentPartURL struct {
URL string `json:"url"`
}
// UnmarshalJSON 自定义解析,支持 string 或 array 格式
func (mc *MessageContent) UnmarshalJSON(data []byte) error {
// 尝试解析为字符串
var str string
if err := json.Unmarshal(data, &str); err == nil {
mc.isString = true
mc.strValue = str
return nil
}
// 尝试解析为数组
var arr []OpenAIContentPart
if err := json.Unmarshal(data, &arr); err == nil {
mc.isString = false
mc.arrValue = arr
return nil
}
return fmt.Errorf("content must be string or array")
}
// MarshalJSON 自定义序列化
func (mc *MessageContent) MarshalJSON() ([]byte, error) {
if mc.isString {
return json.Marshal(mc.strValue)
}
return json.Marshal(mc.arrValue)
}
// NewStringContent 创建字符串类型的 MessageContent
func NewStringContent(s string) *MessageContent {
return &MessageContent{
isString: true,
strValue: s,
}
}
// NewArrayContent 创建数组类型的 MessageContent
func NewArrayContent(parts []OpenAIContentPart) *MessageContent {
return &MessageContent{
isString: false,
arrValue: parts,
}
}
// String 获取文本内容
func (mc *MessageContent) String() string {
if mc.isString {
return mc.strValue
}
// 从数组中提取文本
var builder strings.Builder
for _, part := range mc.arrValue {
if part.Type == "text" {
if builder.Len() > 0 && part.Text != "" {
builder.WriteString(" ")
}
builder.WriteString(part.Text)
}
}
return builder.String()
}
type OpenAIMessage struct {
Role string `json:"role" validate:"required"`
Content string `json:"content,omitempty"`
Content *MessageContent `json:"content,omitempty"`
Name string `json:"name,omitempty"`
ToolCalls []OpenAIToolCall `json:"tool_calls,omitempty"`
ToolCallID string `json:"tool_call_id,omitempty"`
@ -90,6 +183,7 @@ type OpenAIStreamResponse struct {
Created int64 `json:"created"`
Model string `json:"model"`
Choices []OpenAIStreamChoice `json:"choices"`
Usage *OpenAIUsage `json:"usage,omitempty"`
}
type OpenAIStreamChoice struct {

View File

@ -0,0 +1,186 @@
package domain
import (
"encoding/json"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestMessageContent_UnmarshalJSON_String(t *testing.T) {
tests := []struct {
name string
json string
expected string
}{
{"simple string", `"hello"`, "hello"},
{"with quotes", `"say \"hello\""`, `say "hello"`},
{"with newline", `"line1\nline2"`, "line1\nline2"},
{"empty string", `""`, ""},
{"unicode", `"你好 🌍"`, "你好 🌍"},
{"special chars", `"Hello \"World\"\nNew Line\tTab"`, "Hello \"World\"\nNew Line\tTab"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var mc MessageContent
err := json.Unmarshal([]byte(tt.json), &mc)
require.NoError(t, err)
assert.Equal(t, tt.expected, mc.String())
assert.True(t, mc.isString)
})
}
}
func TestMessageContent_UnmarshalJSON_Array(t *testing.T) {
tests := []struct {
name string
json string
expected string
}{
{
"single text part",
`[{"type":"text","text":"Hello"}]`,
"Hello",
},
{
"multiple text parts",
`[{"type":"text","text":"Hello"},{"type":"text","text":"World"}]`,
"Hello World",
},
{
"mixed types with image",
`[{"type":"text","text":"Look at this"},{"type":"image_url","image_url":{"url":"https://example.com/img.png"}},{"type":"text","text":"image"}]`,
"Look at this image",
},
{
"empty array",
`[]`,
"",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var mc MessageContent
err := json.Unmarshal([]byte(tt.json), &mc)
require.NoError(t, err)
assert.Equal(t, tt.expected, mc.String())
assert.False(t, mc.isString)
})
}
}
func TestMessageContent_UnmarshalJSON_Invalid(t *testing.T) {
tests := []struct {
name string
json string
}{
{"number", `123`},
{"boolean", `true`},
{"object", `{"key":"value"}`},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var mc MessageContent
err := json.Unmarshal([]byte(tt.json), &mc)
assert.Error(t, err)
assert.Contains(t, err.Error(), "content must be string or array")
})
}
}
func TestMessageContent_UnmarshalJSON_Null(t *testing.T) {
var mc *MessageContent
err := json.Unmarshal([]byte(`null`), &mc)
assert.NoError(t, err)
assert.Nil(t, mc)
}
func TestMessageContent_MarshalJSON_String(t *testing.T) {
mc := NewStringContent("Hello World")
data, err := json.Marshal(mc)
require.NoError(t, err)
assert.Equal(t, `"Hello World"`, string(data))
}
func TestMessageContent_MarshalJSON_Array(t *testing.T) {
mc := NewArrayContent([]OpenAIContentPart{
{Type: "text", Text: "Hello"},
{Type: "text", Text: "World"},
})
data, err := json.Marshal(mc)
require.NoError(t, err)
assert.JSONEq(t, `[{"type":"text","text":"Hello"},{"type":"text","text":"World"}]`, string(data))
}
func TestMessageContent_Roundtrip_String(t *testing.T) {
original := NewStringContent("Test message with \"quotes\" and \nnewlines")
// Marshal
data, err := json.Marshal(original)
require.NoError(t, err)
// Unmarshal
var decoded MessageContent
err = json.Unmarshal(data, &decoded)
require.NoError(t, err)
// Verify
assert.Equal(t, original.String(), decoded.String())
assert.Equal(t, original.isString, decoded.isString)
}
func TestMessageContent_Roundtrip_Array(t *testing.T) {
parts := []OpenAIContentPart{
{Type: "text", Text: "Part 1"},
{Type: "text", Text: "Part 2"},
}
original := NewArrayContent(parts)
// Marshal
data, err := json.Marshal(original)
require.NoError(t, err)
// Unmarshal
var decoded MessageContent
err = json.Unmarshal(data, &decoded)
require.NoError(t, err)
// Verify
assert.Equal(t, original.String(), decoded.String())
assert.Equal(t, original.isString, decoded.isString)
}
func TestNewStringContent(t *testing.T) {
mc := NewStringContent("test")
assert.NotNil(t, mc)
assert.True(t, mc.isString)
assert.Equal(t, "test", mc.strValue)
assert.Equal(t, "test", mc.String())
}
func TestNewArrayContent(t *testing.T) {
parts := []OpenAIContentPart{
{Type: "text", Text: "Hello"},
}
mc := NewArrayContent(parts)
assert.NotNil(t, mc)
assert.False(t, mc.isString)
assert.Equal(t, parts, mc.arrValue)
assert.Equal(t, "Hello", mc.String())
}
func TestMessageContent_String_EmptyArray(t *testing.T) {
mc := NewArrayContent([]OpenAIContentPart{})
assert.Equal(t, "", mc.String())
}
func TestMessageContent_String_NoTextParts(t *testing.T) {
mc := NewArrayContent([]OpenAIContentPart{
{Type: "image_url", Text: ""},
})
assert.Equal(t, "", mc.String())
}

View File

@ -8,6 +8,7 @@ import (
const (
SettingKeySystemPrompt = "system_prompt"
SettingBlockWords = "block_words"
SettingCopyrightInfo = "本网站由 PandaWiki 提供技术支持"
)
// table: settings

View File

@ -105,3 +105,14 @@ type StatPageHour struct {
func (StatPageHour) TableName() string {
return "stat_page_hours"
}
// NodeStats node表统计数据
type NodeStats struct {
ID int64 `json:"id" gorm:"primaryKey;autoIncrement"`
NodeID string `json:"node_id" gorm:"uniqueIndex"`
PV int64 `json:"pv"`
}
func (NodeStats) TableName() string {
return "node_stats"
}

View File

@ -0,0 +1,29 @@
package domain
import (
"time"
"github.com/chaitin/panda-wiki/consts"
)
// table: settings
type SystemSetting struct {
ID int `json:"id" gorm:"primary_key"`
Key consts.SystemSettingKey `json:"key"`
Value []byte `json:"value" gorm:"type:jsonb"` // JSON string
Description string `json:"description"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
}
func (SystemSetting) TableName() string {
return "system_settings"
}
// ModelModeSetting 模型配置结构体
type ModelModeSetting struct {
Mode consts.ModelSettingMode `json:"mode"` // 模式: manual 或 auto
AutoModeAPIKey string `json:"auto_mode_api_key"` // 百智云 API Key
ChatModel string `json:"chat_model"` // 自定义对话模型名称
IsManualEmbeddingUpdated bool `json:"is_manual_embedding_updated"` // 手动模式下嵌入模型是否更新
}

View File

@ -35,6 +35,7 @@ require (
github.com/larksuite/oapi-sdk-go/v3 v3.4.20
github.com/lib/pq v1.10.9
github.com/lionsoul2014/ip2region/binding/golang v0.0.0-20250508043914-ed57fa5c5274
github.com/mark3labs/mcp-go v0.43.0
github.com/microcosm-cc/bluemonday v1.0.27
github.com/mileusna/useragent v1.3.5
github.com/minio/minio-go/v7 v7.0.91
@ -49,6 +50,7 @@ require (
github.com/sbzhu/weworkapi_golang v0.0.0-20210525081115-1799804a7c8d
github.com/silenceper/wechat/v2 v2.1.9
github.com/spf13/viper v1.20.1
github.com/stretchr/testify v1.10.0
github.com/swaggo/echo-swagger v1.4.1
github.com/swaggo/swag v1.16.5
github.com/tidwall/gjson v1.14.1
@ -98,6 +100,7 @@ require (
github.com/cloudwego/eino-ext/components/model/openai v0.0.0-20250710065240-482d48888f25 // indirect
github.com/cloudwego/eino-ext/libs/acl/openai v0.0.0-20250626133421-3c142631c961 // indirect
github.com/cohesion-org/deepseek-go v1.2.8 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
@ -135,6 +138,7 @@ require (
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.3 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/invopop/yaml v0.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
@ -165,6 +169,7 @@ require (
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/perimeterx/marshmallow v1.1.5 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/rs/xid v1.6.0 // indirect
github.com/sagikazarmark/locafero v0.9.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
@ -183,6 +188,7 @@ require (
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/yargevad/filepathx v1.0.0 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.61.0 // indirect

View File

@ -321,6 +321,8 @@ github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/C
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/invopop/jsonschema v0.13.0 h1:KvpoAJWEjR3uD9Kbm2HWJmqsEaHt8lBUpd0qHcIi21E=
github.com/invopop/jsonschema v0.13.0/go.mod h1:ffZ5Km5SWWRAIN6wbDXItl95euhFz2uON45H2qjYt+0=
github.com/invopop/yaml v0.1.0 h1:YW3WGUoJEXYfzWBjn00zIlrw7brGVD0fUKRYDPAPhrc=
github.com/invopop/yaml v0.1.0/go.mod h1:2XuRLgs/ouIrW3XNzuNj7J3Nvu/Dig5MXvbCEdiBN3Q=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
@ -396,6 +398,8 @@ github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
github.com/mailru/easyjson v0.9.0 h1:PrnmzHw7262yW8sTBwxi1PdJA3Iw/EKBa8psRf7d9a4=
github.com/mailru/easyjson v0.9.0/go.mod h1:1+xMtQp2MRNVL/V1bOzuP3aP8VNwRW55fQUto+XFtTU=
github.com/mark3labs/mcp-go v0.43.0 h1:lgiKcWMddh4sngbU+hoWOZ9iAe/qp/m851RQpj3Y7jA=
github.com/mark3labs/mcp-go v0.43.0/go.mod h1:YnJfOL382MIWDx1kMY+2zsRHU/q78dBg9aFb8W6Thdw=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
@ -576,6 +580,8 @@ github.com/x-cray/logrus-prefixed-formatter v0.5.2 h1:00txxvfBM9muc0jiLIEAkAcIMJ
github.com/x-cray/logrus-prefixed-formatter v0.5.2/go.mod h1:2duySbKsL6M18s5GU7VPsoEPHyzalCE06qoARUCeBBE=
github.com/yargevad/filepathx v1.0.0 h1:SYcT+N3tYGi+NvazubCNlvgIPbzAk7i7y2dwg3I5FYc=
github.com/yargevad/filepathx v1.0.0/go.mod h1:BprfX/gpYNJHJfc35GjRRpVcwWXS89gGulUIU5tK3tA=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.30/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=

View File

@ -2,6 +2,7 @@ package mq
import (
"context"
"time"
"github.com/robfig/cron/v3"
@ -10,17 +11,19 @@ import (
"github.com/chaitin/panda-wiki/usecase"
)
type StatCronHandler struct {
type CronHandler struct {
logger *log.Logger
statRepo *pg.StatRepository
statUseCase *usecase.StatUseCase
nodeUseCase *usecase.NodeUsecase
}
func NewStatCronHandler(logger *log.Logger, statRepo *pg.StatRepository, statUseCase *usecase.StatUseCase) (*StatCronHandler, error) {
h := &StatCronHandler{
func NewStatCronHandler(logger *log.Logger, statRepo *pg.StatRepository, statUseCase *usecase.StatUseCase, nodeUseCase *usecase.NodeUsecase) (*CronHandler, error) {
h := &CronHandler{
statRepo: statRepo,
statUseCase: statUseCase,
logger: logger.WithModule("handler.mq.stat"),
nodeUseCase: nodeUseCase,
logger: logger.WithModule("handler.mq.cron"),
}
cron := cron.New()
@ -45,13 +48,35 @@ func NewStatCronHandler(logger *log.Logger, statRepo *pg.StatRepository, statUse
}
h.logger.Info("add cron job", log.String("cron_id", "cleanup_old_hourly_stats"))
// 启动时先异步跑一次
go func() {
if err := h.nodeUseCase.SyncRagNodeStatus(context.Background()); err != nil {
h.logger.Error("initial sync rag node status failed", log.Error(err))
}
}()
if _, err := cron.AddFunc("26 * * * *", h.SyncRagNodeStatus); err != nil {
h.logger.Error("failed to sync rag node status", log.Error(err))
return nil, err
}
h.logger.Info("add cron job", log.String("cron_id", "sync_rag_node_status"))
cron.Start()
h.logger.Info("start cron jobs")
return h, nil
}
func (h *StatCronHandler) RemoveOldStatData() {
func (h *CronHandler) RemoveOldStatData() {
h.logger.Info("remove old stat data start")
// 零点时同步数据至node_stats持久化
if time.Now().Hour() == 0 {
if err := h.statUseCase.MigrateYesterdayPVToNodeStats(context.Background()); err != nil {
h.logger.Error("migrate yesterday PV data to node_stats failed", log.Error(err))
} else {
h.logger.Info("migrate yesterday PV data to node_stats successful")
}
}
err := h.statRepo.RemoveOldData(context.Background())
if err != nil {
h.logger.Error("remove old stat data failed", log.Error(err))
@ -59,7 +84,7 @@ func (h *StatCronHandler) RemoveOldStatData() {
h.logger.Info("remove old stat data successful")
}
func (h *StatCronHandler) AggregateHourlyStats() {
func (h *CronHandler) AggregateHourlyStats() {
h.logger.Info("aggregate hourly stats start")
err := h.statUseCase.AggregateHourlyStats(context.Background())
if err != nil {
@ -69,7 +94,7 @@ func (h *StatCronHandler) AggregateHourlyStats() {
h.logger.Info("aggregate hourly stats successful")
}
func (h *StatCronHandler) CleanupOldHourlyStats() {
func (h *CronHandler) CleanupOldHourlyStats() {
h.logger.Info("cleanup old hourly stats start")
err := h.statUseCase.CleanupOldHourlyStats(context.Background())
if err != nil {
@ -78,3 +103,13 @@ func (h *StatCronHandler) CleanupOldHourlyStats() {
}
h.logger.Info("cleanup old hourly stats successful")
}
func (h *CronHandler) SyncRagNodeStatus() {
h.logger.Info("sync rag node status")
err := h.nodeUseCase.SyncRagNodeStatus(context.Background())
if err != nil {
h.logger.Error("sync rag node status failed", log.Error(err))
return
}
h.logger.Info("sync rag node status successful")
}

View File

@ -7,12 +7,14 @@ import (
"github.com/chaitin/panda-wiki/repo/mq"
"github.com/chaitin/panda-wiki/repo/pg"
"github.com/chaitin/panda-wiki/store/rag"
"github.com/chaitin/panda-wiki/store/s3"
"github.com/chaitin/panda-wiki/usecase"
)
type MQHandlers struct {
RAGMQHandler *RAGMQHandler
StatCronHandler *StatCronHandler
RAGMQHandler *RAGMQHandler
RagDocUpdateHandler *RagDocUpdateHandler
StatCronHandler *CronHandler
}
var ProviderSet = wire.NewSet(
@ -20,11 +22,15 @@ var ProviderSet = wire.NewSet(
rag.ProviderSet,
mq.ProviderSet,
ipdb.ProviderSet,
s3.ProviderSet,
usecase.NewLLMUsecase,
usecase.NewStatUseCase,
usecase.NewNodeUsecase,
usecase.NewModelUsecase,
NewRAGMQHandler,
NewRagDocUpdateHandler,
NewStatCronHandler,
wire.Struct(new(MQHandlers), "*"),

View File

@ -15,24 +15,24 @@ import (
)
type RAGMQHandler struct {
consumer mq.MQConsumer
logger *log.Logger
rag rag.RAGService
nodeRepo *pg.NodeRepository
kbRepo *pg.KnowledgeBaseRepository
modelRepo *pg.ModelRepository
llmUsecase *usecase.LLMUsecase
consumer mq.MQConsumer
logger *log.Logger
rag rag.RAGService
nodeRepo *pg.NodeRepository
kbRepo *pg.KnowledgeBaseRepository
llmUsecase *usecase.LLMUsecase
modelUsecase *usecase.ModelUsecase
}
func NewRAGMQHandler(consumer mq.MQConsumer, logger *log.Logger, rag rag.RAGService, nodeRepo *pg.NodeRepository, kbRepo *pg.KnowledgeBaseRepository, llmUsecase *usecase.LLMUsecase, modelRepo *pg.ModelRepository) (*RAGMQHandler, error) {
func NewRAGMQHandler(consumer mq.MQConsumer, logger *log.Logger, rag rag.RAGService, nodeRepo *pg.NodeRepository, kbRepo *pg.KnowledgeBaseRepository, llmUsecase *usecase.LLMUsecase, modelUsecase *usecase.ModelUsecase) (*RAGMQHandler, error) {
h := &RAGMQHandler{
consumer: consumer,
logger: logger.WithModule("mq.rag"),
rag: rag,
nodeRepo: nodeRepo,
kbRepo: kbRepo,
llmUsecase: llmUsecase,
modelRepo: modelRepo,
consumer: consumer,
logger: logger.WithModule("mq.rag"),
rag: rag,
nodeRepo: nodeRepo,
kbRepo: kbRepo,
llmUsecase: llmUsecase,
modelUsecase: modelUsecase,
}
if err := consumer.RegisterHandler(domain.VectorTaskTopic, h.HandleNodeContentVectorRequest); err != nil {
return nil, err
@ -134,11 +134,13 @@ func (h *RAGMQHandler) HandleNodeContentVectorRequest(ctx context.Context, msg t
h.logger.Info("node is folder, skip summary", log.Any("node_id", request.NodeID))
return nil
}
model, err := h.modelRepo.GetChatModel(ctx)
model, err := h.modelUsecase.GetChatModel(ctx)
if err != nil {
h.logger.Error("get chat model failed", log.Error(err))
return nil
}
summary, err := h.llmUsecase.SummaryNode(ctx, model, node.Name, node.Content)
if err != nil {
h.logger.Error("summary node content failed", log.Error(err))

View File

@ -0,0 +1,65 @@
package mq
import (
"context"
"encoding/json"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/mq"
"github.com/chaitin/panda-wiki/mq/types"
"github.com/chaitin/panda-wiki/repo/pg"
)
type RagDocUpdateHandler struct {
consumer mq.MQConsumer
logger *log.Logger
nodeRepo *pg.NodeRepository
}
func NewRagDocUpdateHandler(consumer mq.MQConsumer, logger *log.Logger, nodeRepo *pg.NodeRepository) (*RagDocUpdateHandler, error) {
h := &RagDocUpdateHandler{
consumer: consumer,
logger: logger.WithModule("mq.rag_doc_update"),
nodeRepo: nodeRepo,
}
if err := consumer.RegisterHandler(domain.RagDocUpdateTopic, h.HandleRagDocUpdate); err != nil {
return nil, err
}
return h, nil
}
func (h *RagDocUpdateHandler) HandleRagDocUpdate(ctx context.Context, msg types.Message) error {
var event domain.RagDocInfoUpdateEvent
err := json.Unmarshal(msg.GetData(), &event)
if err != nil {
h.logger.Error("unmarshal rag doc update event failed", log.Error(err))
return err
}
h.logger.Info("received rag doc update event",
log.String("doc_id", event.ID),
log.String("status", event.Status),
log.String("message", event.Message))
nodeId, err := h.nodeRepo.GetNodeIdByDocId(ctx, event.ID)
if err != nil {
h.logger.Error("failed to get node id by doc id",
log.String("doc_id", event.ID),
log.Error(err))
return err
}
if err := h.nodeRepo.Update(ctx, nodeId, map[string]interface{}{
"rag_info": domain.RagInfo{
Status: consts.NodeRagInfoStatus(event.Status),
Message: event.Message,
},
}); err != nil {
return err
}
h.logger.Debug("node rag update success", log.String("doc_id", event.ID))
return nil
}

View File

@ -49,6 +49,7 @@ func NewShareAppHandler(
})
share.GET("/web/info", h.GetWebAppInfo)
share.GET("/widget/info", h.GetWidgetAppInfo)
share.GET("/wechat/info", h.WechatAppInfo)
// wechat official account
share.GET("/wechat/official_account", h.VerifyUrlWechatOfficialAccount)
@ -101,6 +102,28 @@ func (h *ShareAppHandler) GetWidgetAppInfo(c echo.Context) error {
return h.NewResponseWithData(c, appInfo)
}
// WechatAppInfo
//
// @Summary WechatAppInfo
// @Description WechatAppInfo
// @Tags share_chat
// @Accept json
// @Produce json
// @Param X-KB-ID header string true "kb id"
// @Success 200 {object} domain.Response{data=v1.WechatAppInfoResp}
// @Router /share/v1/app/wechat/info [get]
func (h *ShareAppHandler) WechatAppInfo(c echo.Context) error {
kbID := c.Request().Header.Get("X-KB-ID")
if kbID == "" {
return h.NewResponseWithError(c, "kb_id is required", nil)
}
appInfo, err := h.usecase.GetWechatAppInfo(c.Request().Context(), kbID)
if err != nil {
return h.NewResponseWithError(c, err.Error(), err)
}
return h.NewResponseWithData(c, appInfo)
}
func (h *ShareAppHandler) VerifyUrlWechatOfficialAccount(c echo.Context) error {
kbID := c.Request().Header.Get("X-KB-ID")

View File

@ -61,6 +61,7 @@ func NewShareChatHandler(
share.POST("/search", h.ChatSearch, h.ShareAuthMiddleware.Authorize)
share.POST("/completions", h.ChatCompletions)
share.POST("/widget", h.ChatWidget)
share.POST("/widget/search", h.WidgetSearch)
share.POST("/feedback", h.FeedBack)
return h
}
@ -131,7 +132,7 @@ func (h *ShareChatHandler) ChatMessage(c echo.Context) error {
//
// @Summary ChatWidget
// @Description ChatWidget
// @Tags share_chat
// @Tags Widget
// @Accept json
// @Produce json
// @Param app_type query string true "app type"
@ -268,7 +269,9 @@ func (h *ShareChatHandler) ChatCompletions(c echo.Context) error {
var lastUserMessage string
for i := len(req.Messages) - 1; i >= 0; i-- {
if req.Messages[i].Role == "user" {
lastUserMessage = req.Messages[i].Content
if req.Messages[i].Content != nil {
lastUserMessage = req.Messages[i].Content.String()
}
break
}
}
@ -345,11 +348,12 @@ func (h *ShareChatHandler) handleOpenAIStreamResponse(c echo.Context, eventCh <-
Index: 0,
Delta: domain.OpenAIMessage{
Role: "assistant",
Content: event.Content,
Content: domain.NewStringContent(event.Content),
},
},
},
}
if err := h.writeOpenAIStreamEvent(c, streamResp); err != nil {
return err
}
@ -397,7 +401,7 @@ func (h *ShareChatHandler) handleOpenAINonStreamResponse(c echo.Context, eventCh
Index: 0,
Message: domain.OpenAIMessage{
Role: "assistant",
Content: content,
Content: domain.NewStringContent(content),
},
FinishReason: "stop",
},
@ -441,7 +445,7 @@ func stringPtr(s string) *string {
return &s
}
// ChatMessage chat search
// ChatSearch searches chat messages in shared knowledge base
//
// @Summary ChatSearch
// @Description ChatSearch
@ -484,3 +488,43 @@ func (h *ShareChatHandler) ChatSearch(c echo.Context) error {
}
return h.NewResponseWithData(c, resp)
}
// WidgetSearch
//
// @Summary WidgetSearch
// @Description WidgetSearch
// @Tags Widget
// @Accept json
// @Produce json
// @Param request body domain.ChatSearchReq true "Comment"
// @Success 200 {object} domain.Response{data=domain.ChatSearchResp}
// @Router /share/v1/chat/widget/search [post]
func (h *ShareChatHandler) WidgetSearch(c echo.Context) error {
var req domain.ChatSearchReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "parse request failed", err)
}
req.KBID = c.Request().Header.Get("X-KB-ID")
if err := c.Validate(&req); err != nil {
return h.NewResponseWithError(c, "validate request failed", err)
}
ctx := c.Request().Context()
// validate widget info
widgetAppInfo, err := h.appUsecase.GetWidgetAppInfo(c.Request().Context(), req.KBID)
if err != nil {
h.logger.Error("get widget app info failed", log.Error(err))
return h.sendErrMsg(c, "get app info error")
}
if !widgetAppInfo.Settings.WidgetBotSettings.IsOpen {
return h.sendErrMsg(c, "widget is not open")
}
req.RemoteIP = c.RealIP()
resp, err := h.chatUsecase.Search(ctx, &req)
if err != nil {
return h.NewResponseWithError(c, "failed to search docs", err)
}
return h.NewResponseWithData(c, resp)
}

View File

@ -2,10 +2,10 @@ package share
import (
"net/http"
"strings"
"github.com/labstack/echo/v4"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/handler"
"github.com/chaitin/panda-wiki/log"
@ -89,6 +89,12 @@ func (h *ShareCommentHandler) CreateComment(c echo.Context) error {
return h.NewResponseWithError(c, "failed to validate captcha token", nil)
}
for _, url := range req.PicUrls {
if !strings.HasPrefix(url, "/static-file/") {
return h.NewResponseWithError(c, "validate param pic_urls failed", err)
}
}
remoteIP := c.RealIP()
// get user info --> no enterprise is nil
@ -150,7 +156,7 @@ func (h *ShareCommentHandler) GetCommentList(c echo.Context) error {
}
// 查询数据库获取所有评论-->0 所有, 12 为需要审核的评论
commentsList, err := h.usecase.GetCommentListByNodeID(ctx, nodeID, consts.GetLicenseEdition(c))
commentsList, err := h.usecase.GetCommentListByNodeID(ctx, nodeID)
if err != nil {
return h.NewResponseWithError(c, "failed to get comment list", err)
}

View File

@ -0,0 +1,93 @@
package share
import (
"fmt"
"net/http"
"github.com/labstack/echo/v4"
v1 "github.com/chaitin/panda-wiki/api/share/v1"
"github.com/chaitin/panda-wiki/handler"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/usecase"
"github.com/chaitin/panda-wiki/utils"
)
type ShareCommonHandler struct {
*handler.BaseHandler
logger *log.Logger
fileUsecase *usecase.FileUsecase
}
func NewShareCommonHandler(
e *echo.Echo,
baseHandler *handler.BaseHandler,
logger *log.Logger,
fileUsecase *usecase.FileUsecase,
) *ShareCommonHandler {
h := &ShareCommonHandler{
BaseHandler: baseHandler,
logger: logger,
fileUsecase: fileUsecase,
}
share := e.Group("share/v1/common",
func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
c.Response().Header().Set("Access-Control-Allow-Origin", "*")
c.Response().Header().Set("Access-Control-Allow-Methods", "GET, POST, OPTIONS")
c.Response().Header().Set("Access-Control-Allow-Headers", "Content-Type, Origin, Accept")
if c.Request().Method == "OPTIONS" {
return c.NoContent(http.StatusOK)
}
return next(c)
}
})
share.POST("/file/upload", h.FileUpload, h.ShareAuthMiddleware.Authorize)
return h
}
// FileUpload 文件上传
//
// @Tags ShareFile
// @Summary 文件上传
// @Description 前台用户上传文件,目前只支持图片文件上传
// @ID share-FileUpload
// @Accept multipart/form-data
// @Produce json
// @Param X-KB-ID header string true "kb id"
// @Param file formData file true "File"
// @Param captcha_token formData string true "captcha_token"
// @Success 200 {object} domain.Response{data=v1.FileUploadResp}
// @Router /share/v1/common/file/upload [post]
func (h *ShareCommonHandler) FileUpload(c echo.Context) error {
ctx := c.Request().Context()
var req v1.FileUploadReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "invalid request parameters", err)
}
file, err := c.FormFile("file")
if err != nil {
return h.NewResponseWithError(c, "failed to get file", err)
}
if !utils.IsImageFile(file.Filename) {
return h.NewResponseWithError(c, "只支持图片文件上传", fmt.Errorf("unsupported file type: %s", file.Filename))
}
// validate captcha token
if !h.Captcha.ValidateToken(ctx, req.CaptchaToken) {
return h.NewResponseWithError(c, "failed to validate captcha token", nil)
}
key, err := h.fileUsecase.UploadFile(ctx, req.KbId, file)
if err != nil {
return h.NewResponseWithError(c, "upload failed", err)
}
return h.NewResponseWithData(c, v1.FileUploadResp{
Key: key,
})
}

View File

@ -70,7 +70,7 @@ func (h *ShareNodeHandler) GetNodeList(c echo.Context) error {
// @Param X-KB-ID header string true "kb id"
// @Param id query string true "node id"
// @Param format query string true "format"
// @Success 200 {object} domain.Response
// @Success 200 {object} domain.Response{data=v1.ShareNodeDetailResp}
// @Router /share/v1/node/detail [get]
func (h *ShareNodeHandler) GetNodeDetail(c echo.Context) error {
kbID := c.Request().Header.Get("X-KB-ID")
@ -91,5 +91,15 @@ func (h *ShareNodeHandler) GetNodeDetail(c echo.Context) error {
if err != nil {
return h.NewResponseWithError(c, "failed to get node detail", err)
}
// If the node is a folder, return the list of child nodes
if node.Type == domain.NodeTypeFolder {
childNodes, err := h.usecase.GetNodeReleaseListByParentID(c.Request().Context(), kbID, id, domain.GetAuthID(c))
if err != nil {
return h.NewResponseWithError(c, "failed to get child nodes", err)
}
node.List = childNodes
}
return h.NewResponseWithData(c, node)
}

View File

@ -18,6 +18,7 @@ type ShareHandler struct {
ShareWechatHandler *ShareWechatHandler
ShareCaptchaHandler *ShareCaptchaHandler
OpenapiV1Handler *OpenapiV1Handler
ShareCommonHandler *ShareCommonHandler
}
var ProviderSet = wire.NewSet(
@ -33,6 +34,7 @@ var ProviderSet = wire.NewSet(
NewShareConversationHandler,
NewShareWechatHandler,
NewShareCaptchaHandler,
NewShareCommonHandler,
NewOpenapiV1Handler,
wire.Struct(new(ShareHandler), "*"),

View File

@ -5,7 +5,6 @@ import (
"github.com/labstack/echo/v4"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/handler"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/usecase"
@ -37,13 +36,6 @@ func (h *ShareSitemapHandler) GetSitemap(c echo.Context) error {
if kbID == "" {
return h.NewResponseWithError(c, "kb_id is required", nil)
}
appInfo, err := h.appUsecase.ShareGetWebAppInfo(c.Request().Context(), kbID, domain.GetAuthID(c))
if err != nil {
return h.NewResponseWithError(c, "web app not found", err)
}
if !appInfo.Settings.AutoSitemap {
return h.NewResponseWithError(c, "未开启自动生成站点地图功能", nil)
}
xml, err := h.sitemapUsecase.GetSitemap(c.Request().Context(), kbID)
if err != nil {

View File

@ -7,6 +7,7 @@ import (
"fmt"
"io"
"net/http"
"strconv"
"github.com/labstack/echo/v4"
"github.com/sbzhu/weworkapi_golang/wxbizmsgcrypt"
@ -111,6 +112,12 @@ func (h *ShareWechatHandler) GetWechatAnswer(c echo.Context) error {
return err
}
//2.answer
if err := h.writeSSEEvent(c, domain.SSEEvent{Type: "feedback_score", Content: strconv.Itoa(int(conversation.Messages[1].Info.Score))}); err != nil {
return err
}
if err := h.writeSSEEvent(c, domain.SSEEvent{Type: "message_id", Content: conversation.Messages[1].ID}); err != nil {
return err
}
if err := h.writeSSEEvent(c, domain.SSEEvent{Type: "answer", Content: conversation.Messages[1].Content}); err != nil {
return err
}
@ -331,6 +338,7 @@ func (h *ShareWechatHandler) VerifyUrlWechatApp(c echo.Context) error {
return c.String(http.StatusOK, string(req))
}
// WechatHandlerApp /share/v1/app/wechat/app
func (h *ShareWechatHandler) WechatHandlerApp(c echo.Context) error {
signature := c.QueryParam("msg_signature")
timestamp := c.QueryParam("timestamp")
@ -383,20 +391,22 @@ func (h *ShareWechatHandler) WechatHandlerApp(c echo.Context) error {
return c.String(http.StatusOK, "")
}
immediateResponse, err := wechatConfig.SendResponse(*msg, "正在思考您的问题,请稍候...")
if err != nil {
return h.NewResponseWithError(c, "Failed to send immediate response", err)
var immediateResponse []byte
if domain.GetBaseEditionLimitation(ctx).AllowAdvancedBot && appInfo.Settings.WeChatAppAdvancedSetting.TextResponseEnable {
immediateResponse, err = wechatConfig.SendResponse(*msg, "正在思考您的问题,请稍候...")
if err != nil {
return h.NewResponseWithError(c, "Failed to send immediate response", err)
}
}
go func(msg *wechat.ReceivedMessage, wechatConfig *wechat.WechatConfig, kbId string) {
ctx := context.Background()
err := h.wechatAppUsecase.Wechat(ctx, msg, wechatConfig, kbId)
go func(ctx context.Context, msg *wechat.ReceivedMessage, wechatConfig *wechat.WechatConfig, kbId string, appInfo *domain.AppDetailResp) {
err := h.wechatAppUsecase.Wechat(ctx, msg, wechatConfig, kbId, &appInfo.Settings.WeChatAppAdvancedSetting)
if err != nil {
h.logger.Error("wechat async failed")
}
}(msg, wechatConfig, kbID)
}(ctx, msg, wechatConfig, kbID, appInfo)
return c.XMLBlob(http.StatusOK, []byte(immediateResponse))
return c.XMLBlob(http.StatusOK, immediateResponse)
}
func (h *ShareWechatHandler) WecomAIBotVerify(c echo.Context) error {

View File

@ -100,8 +100,8 @@ func (h *AppHandler) UpdateApp(c echo.Context) error {
}
ctx := c.Request().Context()
if err := h.usecase.ValidateUpdateApp(ctx, id, &appRequest, consts.GetLicenseEdition(c)); err != nil {
h.logger.Error("UpdateApp", log.Any("req:", appRequest), log.Any("err:", err))
if err := h.usecase.ValidateUpdateApp(ctx, id, &appRequest); err != nil {
h.logger.Error("UpdateApp", log.Any("req:", appRequest.Settings), log.Any("err:", err))
return h.NewResponseWithErrCode(c, domain.ErrCodePermissionDenied)
}

View File

@ -1,12 +1,11 @@
package v1
import (
"fmt"
"github.com/labstack/echo/v4"
v1 "github.com/chaitin/panda-wiki/api/crawler/v1"
"github.com/chaitin/panda-wiki/config"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/handler"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/middleware"
@ -37,116 +36,70 @@ func NewCrawlerHandler(echo *echo.Echo,
fileUsecase: fileUsecase,
}
group := echo.Group("/api/v1/crawler", auth.Authorize)
group.POST("/scrape", h.Scrape)
group.POST("/parse", h.CrawlerParse)
group.POST("/export", h.CrawlerExport)
group.GET("/result", h.CrawlerResult)
group.POST("/results", h.CrawlerResults)
// feishu
group.POST("/feishu/list_spaces", h.FeishuListSpaces)
group.POST("/feishu/list_doc", h.FeishuListCloudDoc)
group.POST("/feishu/search_wiki", h.FeishuWikiSearch)
group.POST("/feishu/get_doc", h.FeishuDoc)
// epub
group.POST("/epub/parse", h.EpubParse)
// yuque
group.POST("/yuque/parse", h.YuqueParse)
// rss
group.POST("/rss/parse", h.RSSParse)
group.POST("/rss/scrape", h.RSSScrape)
// sitemap
group.POST("/sitemap/parse", h.SitemapParse)
group.POST("/sitemap/scrape", h.SitemapScrape)
// notion
group.POST("/notion/parse", h.NotionParse)
group.POST("/notion/scrape", h.NotionScrape)
// confluence
group.POST("/confluence/parse", h.ConfluenceParse)
group.POST("/confluence/scrape", h.ConfluenceScrape)
// siyuan
group.POST("/siyuan/parse", h.SiyuanParse)
group.POST("/siyuan/scrape", h.SiyuanScrape)
// mindoc
group.POST("/mindoc/parse", h.MindocParse)
group.POST("/mindoc/scrape", h.MindocScrape)
// wikijs
group.POST("/wikijs/parse", h.WikijsParse)
group.POST("/wikijs/scrape", h.WikijsScrape)
return h
}
// NotionParse
// CrawlerParse 解析文档树
//
// @Summary NotionParse
// @Description NotionParse
// @Summary 解析文档树
// @Description 解析文档树
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.NotionParseReq true "Scrape"
// @Success 200 {object} domain.PWResponse{data=v1.NotionParseResp}
// @Router /api/v1/crawler/notion/parse [post]
func (h *CrawlerHandler) NotionParse(c echo.Context) error {
var req v1.NotionParseReq
// @Param body body v1.CrawlerParseReq true "Scrape"
// @Success 200 {object} domain.PWResponse{data=v1.CrawlerParseResp}
// @Router /api/v1/crawler/parse [post]
func (h *CrawlerHandler) CrawlerParse(c echo.Context) error {
var req v1.CrawlerParseReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
if req.CrawlerSource == consts.CrawlerSourceFeishu {
if req.FeishuSetting.AppID == "" || req.FeishuSetting.AppSecret == "" || req.FeishuSetting.UserAccessToken == "" {
return h.NewResponseWithError(c, "validate request param feishu failed", nil)
}
} else {
if req.Key == "" {
return h.NewResponseWithError(c, "validate request param key failed", nil)
resp, err := h.usecase.NotionGetDocList(c.Request().Context(), req.Integration)
}
}
resp, err := h.usecase.ParseUrl(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "parse notion failed", err)
h.logger.Error("scrape url failed", log.Error(err))
return h.NewResponseWithError(c, "scrape url failed", err)
}
return h.NewResponseWithData(c, resp)
}
// NotionScrape
// CrawlerExport
//
// @Summary NotionScrape
// @Description NotionScrape
// @Summary CrawlerExport
// @Description CrawlerExport
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.NotionScrapeReq true "Get Docs"
// @Success 200 {object} domain.PWResponse{data=v1.NotionScrapeResp}
// @Router /api/v1/crawler/notion/scrape [post]
func (h *CrawlerHandler) NotionScrape(c echo.Context) error {
var req v1.NotionScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body failed", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.NotionGetDoc(c.Request().Context(), req)
if err != nil {
return h.NewResponseWithError(c, "get Docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// Scrape
//
// @Summary Scrape
// @Description Scrape
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.ScrapeReq true "Scrape"
// @Success 200 {object} domain.PWResponse{data=v1.ScrapeResp}
// @Router /api/v1/crawler/scrape [post]
func (h *CrawlerHandler) Scrape(c echo.Context) error {
var req v1.ScrapeReq
// @Param body body v1.CrawlerExportReq true "Scrape"
// @Success 200 {object} domain.PWResponse{data=v1.CrawlerExportResp}
// @Router /api/v1/crawler/export [post]
func (h *CrawlerHandler) CrawlerExport(c echo.Context) error {
var req v1.CrawlerExportReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.ScrapeURL(c.Request().Context(), req.URL, req.KbID)
resp, err := h.usecase.ExportDoc(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "scrape url failed", err)
}
@ -160,7 +113,7 @@ func (h *CrawlerHandler) Scrape(c echo.Context) error {
// @Tags crawler
// @Accept json
// @Produce json
// @Param param query v1.CrawlerResultReq true "Crawler Result Request"
// @Param body body v1.CrawlerResultReq true "Crawler Result Request"
// @Success 200 {object} domain.PWResponse{data=v1.CrawlerResultResp}
// @Router /api/v1/crawler/result [get]
func (h *CrawlerHandler) CrawlerResult(c echo.Context) error {
@ -204,520 +157,3 @@ func (h *CrawlerHandler) CrawlerResults(c echo.Context) error {
}
return h.NewResponseWithData(c, resp)
}
// EpubParse
//
// @Tags crawler
// @Summary EpubParse
// @Description EpubParse
// @Accept json
// @Produce json
// @Param body body v1.EpubParseReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=v1.EpubParseResp}
// @Router /api/v1/crawler/epub/parse [post]
func (h *CrawlerHandler) EpubParse(c echo.Context) error {
var req v1.EpubParseReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.EpubParse(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "get Docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// YuqueParse
//
// @Tags crawler
// @Summary YuqueParse
// @Description YuqueParse
// @Accept json
// @Produce json
// @Param body body v1.YuqueParseReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=v1.YuqueParseResp}
// @Router /api/v1/crawler/yuque/parse [post]
func (h *CrawlerHandler) YuqueParse(c echo.Context) error {
var req v1.YuqueParseReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.YuqueParse(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "get Docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// FeishuListSpaces
//
// @Summary FeishuListSpaces
// @Description List All Feishu Spaces
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.FeishuSpaceListReq true "List Spaces"
// @Success 200 {object} domain.PWResponse{data=[]v1.FeishuSpaceListResp}
// @Router /api/v1/crawler/feishu/list_spaces [post]
func (h *CrawlerHandler) FeishuListSpaces(c echo.Context) error {
var req *v1.FeishuSpaceListReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.FeishuListSpace(c.Request().Context(), req)
if err != nil {
return h.NewResponseWithError(c, fmt.Sprintf("list spaces failed %s", err.Error()), err)
}
return h.NewResponseWithData(c, resp)
}
// FeishuListCloudDoc
//
// @Summary FeishuListCloudDoc
// @Description List Docx in Feishu Spaces
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.FeishuListCloudDocReq true "Search Docx"
// @Success 200 {object} domain.PWResponse{data=[]v1.FeishuListCloudDocResp}
// @Router /api/v1/crawler/feishu/list_doc [post]
func (h *CrawlerHandler) FeishuListCloudDoc(c echo.Context) error {
var req *v1.FeishuListCloudDocReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.FeishuListCloudDoc(c.Request().Context(), req)
if err != nil {
return h.NewResponseWithError(c, fmt.Sprintf("list spaces failed %s", err.Error()), err)
}
return h.NewResponseWithData(c, resp)
}
// FeishuWikiSearch
//
// @Summary FeishuWikiSearch
// @Description Search Wiki in Feishu Spaces
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.FeishuSearchWikiReq true "Search Wiki"
// @Success 200 {object} domain.PWResponse{data=[]v1.FeishuSearchWikiResp}
// @Router /api/v1/crawler/feishu/search_wiki [post]
func (h *CrawlerHandler) FeishuWikiSearch(c echo.Context) error {
var req *v1.FeishuSearchWikiReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
resp, err := h.usecase.FeishuSearchWiki(c.Request().Context(), req)
if err != nil {
return h.NewResponseWithError(c, fmt.Sprintf("search wiki failed %s", err.Error()), err)
}
return h.NewResponseWithData(c, resp)
}
// FeishuDoc
//
// @Summary FeishuDoc
// @Description Get Docx in Feishu Spaces
// @Tags crawler
// @Accept json
// @Produce json
// @Param body body v1.FeishuGetDocReq true "Get Docx"
// @Success 200 {object} domain.PWResponse{data=v1.FeishuGetDocResp}
// @Router /api/v1/crawler/feishu/get_doc [post]
func (h *CrawlerHandler) FeishuDoc(c echo.Context) error {
var req *v1.FeishuGetDocReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.FeishuGetDoc(c.Request().Context(), req)
if err != nil {
return h.NewResponseWithError(c, fmt.Sprintf("get docx failed %s", err.Error()), err)
}
return h.NewResponseWithData(c, resp)
}
// ConfluenceParse
//
// @Summary ConfluenceParse
// @Description Parse Confluence Export File and return document list
// @Tags crawler
// @Accept multipart/form-data
// @Produce json
// @Param file formData file true "file"
// @Param kb_id formData string true "kb_id"
// @Success 200 {object} domain.PWResponse{data=v1.ConfluenceParseResp}
// @Router /api/v1/crawler/confluence/parse [post]
func (h *CrawlerHandler) ConfluenceParse(c echo.Context) error {
ctx := c.Request().Context()
file, err := c.FormFile("file")
if err != nil {
return h.NewResponseWithError(c, "get file failed", err)
}
var req v1.ConfluenceParseReq
req.KbID = c.FormValue("kb_id")
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate failed", err)
}
fileUrl, err := h.fileUsecase.UploadFileGetUrl(ctx, req.KbID, file)
if err != nil {
return h.NewResponseWithError(c, "upload failed", err)
}
h.logger.Info("ConfluenceParse UploadFile successfully", "fileUrl", fileUrl)
resp, err := h.usecase.ConfluenceParse(c.Request().Context(), fileUrl, file.Filename)
if err != nil {
return h.NewResponseWithError(c, "parse confluence export file failed", err)
}
return h.NewResponseWithData(c, resp)
}
// ConfluenceScrape
//
// @Tags crawler
// @Summary ConfluenceScrape
// @Description Scrape specific Confluence documents by ID
// @Accept json
// @Produce json
// @Param body body v1.ConfluenceScrapeReq true "Scrape Request"
// @Success 200 {object} domain.PWResponse{data=v1.ConfluenceScrapeResp}
// @Router /api/v1/crawler/confluence/scrape [post]
func (h *CrawlerHandler) ConfluenceScrape(c echo.Context) error {
var req v1.ConfluenceScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.ConfluenceScrape(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "scrape confluence docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// RSSParse
//
// @Tags crawler
// @Summary Parse RSS
// @Description Parse RSS
// @Accept json
// @Produce json
// @Param body body v1.RssParseReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=v1.RssParseResp}
// @Router /api/v1/crawler/rss/parse [post]
func (h *CrawlerHandler) RSSParse(c echo.Context) error {
var req v1.RssParseReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.GetRSSParse(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "get Docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// RSSScrape
//
// @Tags crawler
// @Summary RSSScrape
// @Description RSSScrape
// @Accept json
// @Produce json
// @Param body body v1.RssScrapeReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=v1.RssScrapeResp}
// @Router /api/v1/crawler/rss/scrape [post]
func (h *CrawlerHandler) RSSScrape(c echo.Context) error {
var req v1.RssScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.GetRssDoc(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "get Docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// SitemapParse
//
// @Tags crawler
// @Summary Parse Sitemap
// @Description Parse Sitemap
// @Accept json
// @Produce json
// @Param body body v1.SitemapParseReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=SitemapParseResp}
// @Router /api/v1/crawler/sitemap/parse [post]
func (h *CrawlerHandler) SitemapParse(c echo.Context) error {
var req v1.SitemapParseReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.SitemapGetUrls(c.Request().Context(), req.URL)
if err != nil {
return h.NewResponseWithError(c, "parse sitemap url failed", err)
}
return h.NewResponseWithData(c, resp)
}
// SitemapScrape
//
// @Tags crawler
// @Summary SitemapScrape
// @Description SitemapScrape
// @Accept json
// @Produce json
// @Param body body v1.SitemapScrapeReq true "Parse URL"
// @Success 200 {object} domain.PWResponse{data=v1.SitemapScrapeResp}
// @Router /api/v1/crawler/sitemap/scrape [post]
func (h *CrawlerHandler) SitemapScrape(c echo.Context) error {
var req v1.SitemapScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.SitemapGetDoc(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "parse sitemap url failed", err)
}
return h.NewResponseWithData(c, resp)
}
// SiyuanParse
//
// @Summary SiyuanParse
// @Description Parse Siyuan Export File and return document list
// @Tags crawler
// @Accept multipart/form-data
// @Produce json
// @Param file formData file true "file"
// @Param kb_id formData string true "kb_id"
// @Success 200 {object} domain.PWResponse{data=v1.SiyuanParseResp}
// @Router /api/v1/crawler/siyuan/parse [post]
func (h *CrawlerHandler) SiyuanParse(c echo.Context) error {
ctx := c.Request().Context()
file, err := c.FormFile("file")
if err != nil {
return h.NewResponseWithError(c, "get file failed", err)
}
var req v1.SiyuanParseReq
req.KbID = c.FormValue("kb_id")
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate failed", err)
}
fileUrl, err := h.fileUsecase.UploadFileGetUrl(ctx, req.KbID, file)
if err != nil {
return h.NewResponseWithError(c, "upload failed", err)
}
h.logger.Info("SiyuanParse UploadFile successfully", "fileUrl", fileUrl)
resp, err := h.usecase.SiyuanParse(c.Request().Context(), fileUrl, file.Filename)
if err != nil {
return h.NewResponseWithError(c, "parse Siyuan export file failed", err)
}
return h.NewResponseWithData(c, resp)
}
// SiyuanScrape
//
// @Tags crawler
// @Summary SiyuanScrape
// @Description Scrape specific Siyuan documents by ID
// @Accept json
// @Produce json
// @Param body body v1.SiyuanScrapeReq true "Scrape Request"
// @Success 200 {object} domain.PWResponse{data=v1.SiyuanScrapeResp}
// @Router /api/v1/crawler/siyuan/scrape [post]
func (h *CrawlerHandler) SiyuanScrape(c echo.Context) error {
var req v1.SiyuanScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.SiyuanScrape(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "scrape Siyuan docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// MindocParse
//
// @Summary MindocParse
// @Description Parse Mindoc Export File and return document list
// @Tags crawler
// @Accept multipart/form-data
// @Produce json
// @Param file formData file true "file"
// @Param kb_id formData string true "kb_id"
// @Success 200 {object} domain.PWResponse{data=v1.MindocParseResp}
// @Router /api/v1/crawler/mindoc/parse [post]
func (h *CrawlerHandler) MindocParse(c echo.Context) error {
ctx := c.Request().Context()
file, err := c.FormFile("file")
if err != nil {
return h.NewResponseWithError(c, "get file failed", err)
}
var req v1.MindocParseReq
req.KbID = c.FormValue("kb_id")
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate failed", err)
}
fileUrl, err := h.fileUsecase.UploadFileGetUrl(ctx, req.KbID, file)
if err != nil {
return h.NewResponseWithError(c, "upload failed", err)
}
h.logger.Info("MindocParse UploadFile successfully", "fileUrl", fileUrl)
resp, err := h.usecase.MindocParse(c.Request().Context(), fileUrl, file.Filename)
if err != nil {
return h.NewResponseWithError(c, "parse Mindoc export file failed", err)
}
return h.NewResponseWithData(c, resp)
}
// MindocScrape
//
// @Tags crawler
// @Summary MindocScrape
// @Description Scrape specific Mindoc documents by ID
// @Accept json
// @Produce json
// @Param body body v1.MindocScrapeReq true "Scrape Request"
// @Success 200 {object} domain.PWResponse{data=v1.MindocScrapeResp}
// @Router /api/v1/crawler/mindoc/scrape [post]
func (h *CrawlerHandler) MindocScrape(c echo.Context) error {
var req v1.MindocScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.MindocScrape(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "scrape Mindoc docs failed", err)
}
return h.NewResponseWithData(c, resp)
}
// WikijsParse
//
// @Summary WikijsParse
// @Description Parse Wikijs Export File and return document list
// @Tags crawler
// @Accept multipart/form-data
// @Produce json
// @Param file formData file true "file"
// @Param kb_id formData string true "kb_id"
// @Success 200 {object} domain.PWResponse{data=v1.WikijsParseResp}
// @Router /api/v1/crawler/wikijs/parse [post]
func (h *CrawlerHandler) WikijsParse(c echo.Context) error {
ctx := c.Request().Context()
file, err := c.FormFile("file")
if err != nil {
return h.NewResponseWithError(c, "get file failed", err)
}
var req v1.WikijsParseReq
req.KbID = c.FormValue("kb_id")
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate failed", err)
}
fileUrl, err := h.fileUsecase.UploadFileGetUrl(ctx, req.KbID, file)
if err != nil {
return h.NewResponseWithError(c, "upload failed", err)
}
h.logger.Info("WikijsParse UploadFile successfully", "fileUrl", fileUrl)
resp, err := h.usecase.WikijsParse(c.Request().Context(), fileUrl, file.Filename)
if err != nil {
return h.NewResponseWithError(c, "parse Wikijs export file failed", err)
}
return h.NewResponseWithData(c, resp)
}
// WikijsScrape
//
// @Tags crawler
// @Summary WikijsScrape
// @Description Scrape specific Wikijs documents by ID
// @Accept json
// @Produce json
// @Param body body v1.WikijsScrapeReq true "Scrape Request"
// @Success 200 {object} domain.PWResponse{data=v1.WikijsScrapeResp}
// @Router /api/v1/crawler/wikijs/scrape [post]
func (h *CrawlerHandler) WikijsScrape(c echo.Context) error {
var req v1.WikijsScrapeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
resp, err := h.usecase.WikijsScrape(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "scrape Wikijs docs failed", err)
}
return h.NewResponseWithData(c, resp)
}

View File

@ -5,6 +5,7 @@ import (
v1 "github.com/chaitin/panda-wiki/api/kb/v1"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/domain"
)
// KBUserList
@ -55,8 +56,8 @@ func (h *KnowledgeBaseHandler) KBUserInvite(c echo.Context) error {
return h.NewResponseWithError(c, "validate request failed", err)
}
if consts.GetLicenseEdition(c) != consts.LicenseEditionEnterprise && req.Perm != consts.UserKBPermissionFullControl {
return h.NewResponseWithError(c, "非企业版本只能使用完全控制权限", nil)
if !domain.GetBaseEditionLimitation(c.Request().Context()).AllowAdminPerm && req.Perm != consts.UserKBPermissionFullControl {
return h.NewResponseWithError(c, "当前版本不支持管理员分权控制", nil)
}
err := h.usecase.KBUserInvite(c.Request().Context(), req)
@ -87,8 +88,8 @@ func (h *KnowledgeBaseHandler) KBUserUpdate(c echo.Context) error {
return h.NewResponseWithError(c, "validate request failed", err)
}
if consts.GetLicenseEdition(c) != consts.LicenseEditionEnterprise && req.Perm != consts.UserKBPermissionFullControl {
return h.NewResponseWithError(c, "非企业版本只能使用完全控制权限", nil)
if !domain.GetBaseEditionLimitation(c.Request().Context()).AllowAdminPerm && req.Perm != consts.UserKBPermissionFullControl {
return h.NewResponseWithError(c, "当前版本不支持管理员分权控制", nil)
}
err := h.usecase.UpdateUserKB(c.Request().Context(), req)

View File

@ -91,11 +91,7 @@ func (h *KnowledgeBaseHandler) CreateKnowledgeBase(c echo.Context) error {
return h.NewResponseWithError(c, "ports is required", nil)
}
req.MaxKB = 1
maxKB := c.Get("max_kb")
if maxKB != nil {
req.MaxKB = maxKB.(int)
}
req.MaxKB = domain.GetBaseEditionLimitation(c.Request().Context()).MaxKb
did, err := h.usecase.CreateKnowledgeBase(c.Request().Context(), &req)
if err != nil {
@ -244,6 +240,12 @@ func (h *KnowledgeBaseHandler) DeleteKnowledgeBase(c echo.Context) error {
// @Success 200 {object} domain.Response
// @Router /api/v1/knowledge_base/release [post]
func (h *KnowledgeBaseHandler) CreateKBRelease(c echo.Context) error {
ctx := c.Request().Context()
authInfo := domain.GetAuthInfoFromCtx(ctx)
if authInfo == nil {
return h.NewResponseWithError(c, "authInfo not found in context", nil)
}
req := &domain.CreateKBReleaseReq{}
if err := c.Bind(req); err != nil {
return h.NewResponseWithError(c, "request body is invalid", err)
@ -252,7 +254,7 @@ func (h *KnowledgeBaseHandler) CreateKBRelease(c echo.Context) error {
return h.NewResponseWithError(c, "validate request body failed", err)
}
id, err := h.usecase.CreateKBRelease(c.Request().Context(), req)
id, err := h.usecase.CreateKBRelease(ctx, req, authInfo.UserId)
if err != nil {
return h.NewResponseWithError(c, "create kb release failed", err)
}

View File

@ -40,11 +40,13 @@ func NewModelHandler(echo *echo.Echo, baseHandler *handler.BaseHandler, logger *
group.POST("/check", handler.CheckModel)
group.POST("/provider/supported", handler.GetProviderSupportedModelList)
group.PUT("", handler.UpdateModel)
group.POST("/switch-mode", handler.SwitchMode)
group.GET("/mode-setting", handler.GetModelModeSetting)
return handler
}
// get model list
// GetModelList
//
// @Summary get model list
// @Description get model list
@ -64,7 +66,7 @@ func (h *ModelHandler) GetModelList(c echo.Context) error {
return h.NewResponseWithData(c, models)
}
// create model
// CreateModel
//
// @Summary create model
// @Description create model
@ -83,9 +85,6 @@ func (h *ModelHandler) CreateModel(c echo.Context) error {
return h.NewResponseWithError(c, "invalid request", err)
}
if consts.GetLicenseEdition(c) == consts.LicenseEditionContributor && req.Provider != domain.ModelProviderBrandBaiZhiCloud {
return h.NewResponseWithError(c, "联创版只能使用百智云模型哦~", nil)
}
ctx := c.Request().Context()
param := domain.ModelParam{}
@ -110,7 +109,7 @@ func (h *ModelHandler) CreateModel(c echo.Context) error {
return h.NewResponseWithData(c, model)
}
// update model
// UpdateModel
//
// @Description update model
// @Tags model
@ -128,9 +127,6 @@ func (h *ModelHandler) UpdateModel(c echo.Context) error {
return h.NewResponseWithError(c, "invalid request", err)
}
if consts.GetLicenseEdition(c) == consts.LicenseEditionContributor && req.Provider != domain.ModelProviderBrandBaiZhiCloud {
return h.NewResponseWithError(c, "联创版只能使用百智云模型哦~", nil)
}
ctx := c.Request().Context()
if err := h.usecase.Update(ctx, &req); err != nil {
return h.NewResponseWithError(c, "update model failed", err)
@ -138,7 +134,7 @@ func (h *ModelHandler) UpdateModel(c echo.Context) error {
return h.NewResponseWithData(c, nil)
}
// check model
// CheckModel
//
// @Summary check model
// @Description check model
@ -211,3 +207,58 @@ func (h *ModelHandler) GetProviderSupportedModelList(c echo.Context) error {
}
return h.NewResponseWithData(c, models)
}
// SwitchMode
//
// @Summary switch mode
// @Description switch model mode between manual and auto
// @Tags model
// @Accept json
// @Produce json
// @Param request body domain.SwitchModeReq true "switch mode request"
// @Success 200 {object} domain.Response{data=domain.SwitchModeResp}
// @Router /api/v1/model/switch-mode [post]
func (h *ModelHandler) SwitchMode(c echo.Context) error {
var req domain.SwitchModeReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "bind request failed", err)
}
if err := c.Validate(&req); err != nil {
return h.NewResponseWithError(c, "validate request failed", err)
}
ctx := c.Request().Context()
if err := h.usecase.SwitchMode(ctx, &req); err != nil {
return h.NewResponseWithError(c, err.Error(), err)
}
resp := &domain.SwitchModeResp{
Message: "模式切换成功",
}
return h.NewResponseWithData(c, resp)
}
// GetModelModeSetting
//
// @Summary get model mode setting
// @Description get current model mode setting including mode, API key and chat model
// @Tags model
// @Accept json
// @Produce json
// @Success 200 {object} domain.Response{data=domain.ModelModeSetting}
// @Router /api/v1/model/mode-setting [get]
func (h *ModelHandler) GetModelModeSetting(c echo.Context) error {
ctx := c.Request().Context()
setting, err := h.usecase.GetModelModeSetting(ctx)
if err != nil {
// 如果获取失败,返回默认值(手动模式)
h.logger.Warn("failed to get model mode setting, return default", log.Error(err))
defaultSetting := domain.ModelModeSetting{
Mode: consts.ModelSettingModeManual,
AutoModeAPIKey: "",
ChatModel: "",
}
return h.NewResponseWithData(c, defaultSetting)
}
return h.NewResponseWithData(c, setting)
}

View File

@ -47,6 +47,7 @@ func NewNodeHandler(
group.POST("/batch_move", h.BatchMoveNode)
group.GET("/recommend_nodes", h.RecommendNodes)
group.POST("/restudy", h.NodeRestudy)
// node permission
group.GET("/permission", h.NodePermission)
@ -80,15 +81,13 @@ func (h *NodeHandler) CreateNode(c echo.Context) error {
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request body failed", err)
}
req.MaxNode = 300
if maxNode := c.Get("max_node"); maxNode != nil {
req.MaxNode = maxNode.(int)
}
req.MaxNode = domain.GetBaseEditionLimitation(ctx).MaxNode
id, err := h.usecase.Create(c.Request().Context(), req, authInfo.UserId)
if err != nil {
if errors.Is(err, domain.ErrMaxNodeLimitReached) {
return h.NewResponseWithError(c, "已达到最大文档数量限制,请升级到联创版或企业版", nil)
return h.NewResponseWithError(c, "已达到最大文档数量限制,请升级到更高版本", nil)
}
return h.NewResponseWithError(c, "create node failed", err)
}
@ -147,6 +146,7 @@ func (h *NodeHandler) GetNodeDetail(c echo.Context) error {
node, err := h.usecase.GetNodeByKBID(c.Request().Context(), req.ID, req.KbId, req.Format)
if err != nil {
h.logger.Error("get node by kb id failed", log.Error(err))
return h.NewResponseWithError(c, "get node detail failed", err)
}
return h.NewResponseWithData(c, node)
@ -173,9 +173,6 @@ func (h *NodeHandler) NodeAction(c echo.Context) error {
}
ctx := c.Request().Context()
if err := h.usecase.NodeAction(ctx, req); err != nil {
if err == domain.ErrNodeParentIDInIDs {
return h.NewResponseWithError(c, "文件夹下有子文件,不能删除~", nil)
}
return h.NewResponseWithError(c, "node action failed", err)
}
return h.NewResponseWithData(c, nil)
@ -387,3 +384,32 @@ func (h *NodeHandler) NodePermissionEdit(c echo.Context) error {
}
return h.NewResponseWithData(c, nil)
}
// NodeRestudy 文档重新学习
//
// @Tags Node
// @Summary 文档重新学习
// @Description 文档重新学习
// @ID v1-NodeRestudy
// @Accept json
// @Produce json
// @Security bearerAuth
// @Param param body v1.NodeRestudyReq true "para"
// @Success 200 {object} domain.Response{data=v1.NodeRestudyResp}
// @Router /api/v1/node/restudy [post]
func (h *NodeHandler) NodeRestudy(c echo.Context) error {
var req v1.NodeRestudyReq
if err := c.Bind(&req); err != nil {
return h.NewResponseWithError(c, "request params is invalid", err)
}
if err := c.Validate(req); err != nil {
return h.NewResponseWithError(c, "validate request params failed", err)
}
if err := h.usecase.NodeRestudy(c.Request().Context(), &req); err != nil {
return h.NewResponseWithError(c, "node restudy failed", err)
}
return h.NewResponseWithData(c, nil)
}

View File

@ -192,12 +192,29 @@ func (h *UserHandler) ResetPassword(c echo.Context) error {
if err != nil {
return h.NewResponseWithError(c, "failed to get user", err)
}
if user.Account == "admin" && authInfo.UserId == req.ID {
return h.NewResponseWithError(c, "请修改安装目录下 .env 文件中的 ADMIN_PASSWORD并重启 panda-wiki-api 容器使更改生效。", nil)
// 非超级管理员没有改密码权限
if user.Role != consts.UserRoleAdmin {
return h.NewResponseWithErrCode(c, domain.ErrCodePermissionDenied)
}
if user.Account != "admin" && authInfo.UserId != req.ID {
return h.NewResponseWithError(c, "只有管理员可以重置其他用户密码", nil)
if user.Account == "admin" {
// admin 改不了自己的密码
if authInfo.UserId == req.ID {
return h.NewResponseWithError(c, "请修改安装目录下 .env 文件中的 ADMIN_PASSWORD并重启 panda-wiki-api 容器使更改生效。", nil)
}
} else {
targetUser, err := h.usecase.GetUser(ctx, req.ID)
if err != nil {
return h.NewResponseWithError(c, "failed to get target user", err)
}
// 超级管理员不能改其他超级管理员密码
if targetUser.Role == consts.UserRoleAdmin && targetUser.ID != authInfo.UserId {
return h.NewResponseWithError(c, "无法修改其他超级管理员密码", nil)
}
}
err = h.usecase.ResetPassword(c.Request().Context(), &req)
if err != nil {
return h.NewResponseWithError(c, "failed to reset password", err)

View File

@ -15,7 +15,7 @@ type AuthMiddleware interface {
Authorize(next echo.HandlerFunc) echo.HandlerFunc
ValidateUserRole(role consts.UserRole) echo.MiddlewareFunc
ValidateKBUserPerm(role consts.UserKBPermission) echo.MiddlewareFunc
ValidateLicenseEdition(edition consts.LicenseEdition) echo.MiddlewareFunc
ValidateLicenseEdition(edition ...consts.LicenseEdition) echo.MiddlewareFunc
MustGetUserID(c echo.Context) (string, bool)
}

View File

@ -6,6 +6,7 @@ import (
"encoding/json"
"io"
"net/http"
"slices"
"strings"
"github.com/golang-jwt/jwt/v5"
@ -194,7 +195,7 @@ func (m *JWTMiddleware) ValidateKBUserPerm(perm consts.UserKBPermission) echo.Mi
}
}
func (m *JWTMiddleware) ValidateLicenseEdition(needEdition consts.LicenseEdition) echo.MiddlewareFunc {
func (m *JWTMiddleware) ValidateLicenseEdition(needEditions ...consts.LicenseEdition) echo.MiddlewareFunc {
return func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c echo.Context) error {
@ -206,7 +207,7 @@ func (m *JWTMiddleware) ValidateLicenseEdition(needEdition consts.LicenseEdition
})
}
if edition < needEdition {
if !slices.Contains(needEditions, edition) {
return c.JSON(http.StatusForbidden, domain.PWResponse{
Success: false,
Message: "Unauthorized ValidateLicenseEdition",

View File

@ -53,7 +53,7 @@ func (m *MigrationNodeVersion) Execute(tx *gorm.DB) error {
Message: "release all old nodes",
Tag: "init",
NodeIDs: nodeIDs,
})
}, "")
if err != nil {
return fmt.Errorf("create kb release failed: %w", err)
}

View File

@ -94,6 +94,16 @@ func (c *MQConsumer) registerCoreNATSHandler(topic string, handler func(ctx cont
// registerJetStreamHandler 使用 JetStream 订阅主题
func (c *MQConsumer) registerJetStreamHandler(topic string, handler func(ctx context.Context, msg types.Message) error) error {
consumerName := domain.TopicConsumerName[topic]
// Choose deliver policy based on topic
var deliverPolicy nats.SubOpt
if topic == domain.VectorTaskTopic {
deliverPolicy = nats.DeliverNew()
} else {
deliverPolicy = nats.DeliverAll()
}
sub, err := c.js.Subscribe(topic, func(msg *nats.Msg) {
c.logger.Debug("received message via JetStream",
log.String("topic", topic),
@ -111,7 +121,7 @@ func (c *MQConsumer) registerJetStreamHandler(topic string, handler func(ctx con
log.String("topic", topic),
log.Error(err))
}
}, nats.DeliverNew(), nats.AckExplicit(), nats.Durable(domain.TopicConsumerName[topic]), nats.ConsumerName(domain.TopicConsumerName[topic]))
}, deliverPolicy, nats.AckExplicit(), nats.Durable(consumerName), nats.ConsumerName(consumerName))
if err != nil {
c.logger.Error("failed to subscribe to topic via JetStream",
log.String("topic", topic),

View File

@ -30,6 +30,10 @@ func (p *MQProducer) EnsureStreams() error {
name: "scraper",
subjects: []string{"apps.panda-wiki.scraper.>"},
},
{
name: "rag",
subjects: []string{"rag.doc.update"},
},
}
for _, stream := range streams {

View File

@ -72,7 +72,7 @@ func NewClient(logger *log.Logger, mqConsumer mq.MQConsumer) (*Client, error) {
return client, nil
}
func (c *Client) GetUrlList(ctx context.Context, targetURL, id string) (*GetUrlListData, error) {
func (c *Client) GetUrlList(ctx context.Context, targetURL, id string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
@ -99,7 +99,7 @@ func (c *Client) GetUrlList(ctx context.Context, targetURL, id string) (*GetUrlL
return nil, err
}
c.logger.Info("scrape url", "requestURL:", requestURL, "resp", string(respBody))
var scrapeResp GetUrlListResponse
var scrapeResp ListDocResponse
err = json.Unmarshal(respBody, &scrapeResp)
if err != nil {
return nil, err
@ -109,11 +109,7 @@ func (c *Client) GetUrlList(ctx context.Context, targetURL, id string) (*GetUrlL
return nil, errors.New(scrapeResp.Msg)
}
if len(scrapeResp.Data.Docs) == 0 {
return nil, errors.New("data list is empty")
}
return &scrapeResp.Data, nil
return &scrapeResp, nil
}
func (c *Client) UrlExport(ctx context.Context, id, docID, kbId string) (*UrlExportRes, error) {

View File

@ -23,25 +23,6 @@ type ConfluenceListDocsRequest struct {
UUID string `json:"uuid"` // 必填的唯一标识符
}
// ConfluenceListDocsResponse Confluence 获取文档列表响应
type ConfluenceListDocsResponse struct {
Success bool `json:"success"`
Msg string `json:"msg"`
Data ConfluenceListDocsData `json:"data"`
}
// ConfluenceListDocsData Confluence 文档列表数据
type ConfluenceListDocsData struct {
Docs []ConfluenceDoc `json:"docs"`
}
// ConfluenceDoc Confluence 文档信息
type ConfluenceDoc struct {
ID string `json:"id"`
Title string `json:"title"`
URL string `json:"url"`
}
// ConfluenceExportDocRequest Confluence 导出文档请求
type ConfluenceExportDocRequest struct {
UUID string `json:"uuid"` // 必须与 list 接口使用的 uuid 相同
@ -63,7 +44,7 @@ type ConfluenceExportDocData struct {
}
// ConfluenceListDocs 获取 Confluence 文档列表
func (c *Client) ConfluenceListDocs(ctx context.Context, confluenceURL, filename, uuid string) (*ConfluenceListDocsResponse, error) {
func (c *Client) ConfluenceListDocs(ctx context.Context, confluenceURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +82,7 @@ func (c *Client) ConfluenceListDocs(ctx context.Context, confluenceURL, filename
c.logger.Info("ConfluenceListDocs", "requestURL:", requestURL, "resp", string(respBody))
var confluenceResp ConfluenceListDocsResponse
var confluenceResp ListDocResponse
err = json.Unmarshal(respBody, &confluenceResp)
if err != nil {
return nil, err

View File

@ -63,7 +63,7 @@ type EpubpExportDocData struct {
}
// EpubpListDocs 获取 Epubp 文档列表
func (c *Client) EpubpListDocs(ctx context.Context, epubpURL, filename, uuid string) (*EpubpListDocsResponse, error) {
func (c *Client) EpubpListDocs(ctx context.Context, epubpURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +101,7 @@ func (c *Client) EpubpListDocs(ctx context.Context, epubpURL, filename, uuid str
c.logger.Info("EpubpListDocs", "requestURL:", requestURL, "resp", string(respBody))
var epubpResp EpubpListDocsResponse
var epubpResp ListDocResponse
err = json.Unmarshal(respBody, &epubpResp)
if err != nil {
return nil, err

View File

@ -64,7 +64,7 @@ type FeishuExportDocData struct {
}
// FeishuListDocs 获取 Feishu 文档列表
func (c *Client) FeishuListDocs(ctx context.Context, uuid, appId, appSecret, accessToken, spaceId string) (*FeishuListDocsResponse, error) {
func (c *Client) FeishuListDocs(ctx context.Context, uuid, appId, appSecret, accessToken, spaceId string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +101,7 @@ func (c *Client) FeishuListDocs(ctx context.Context, uuid, appId, appSecret, acc
c.logger.Info("FeishuListDocs", "requestURL:", requestURL, "resp", string(respBody))
var feishuResp FeishuListDocsResponse
var feishuResp ListDocResponse
err = json.Unmarshal(respBody, &feishuResp)
if err != nil {
return nil, err
@ -115,7 +115,7 @@ func (c *Client) FeishuListDocs(ctx context.Context, uuid, appId, appSecret, acc
}
// FeishuExportDoc 导出 Feishu 文档
func (c *Client) FeishuExportDoc(ctx context.Context, uuid, docID, fileType, spaceId, kbId string) (*FeishuExportDocResponse, error) {
func (c *Client) FeishuExportDoc(ctx context.Context, uuid, docID, fileType, spaceId, kbId string) (*UrlExportRes, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -161,7 +161,7 @@ func (c *Client) FeishuExportDoc(ctx context.Context, uuid, docID, fileType, spa
c.logger.Info("FeishuDoc", "requestURL:", requestURL, "body", string(jsonData), "resp", string(respBody))
var exportResp FeishuExportDocResponse
var exportResp UrlExportRes
err = json.Unmarshal(respBody, &exportResp)
if err != nil {
return nil, err

View File

@ -63,7 +63,7 @@ type MindocExportDocData struct {
}
// MindocListDocs 获取 Mindoc 文档列表
func (c *Client) MindocListDocs(ctx context.Context, mindocURL, filename, uuid string) (*MindocListDocsResponse, error) {
func (c *Client) MindocListDocs(ctx context.Context, mindocURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +101,7 @@ func (c *Client) MindocListDocs(ctx context.Context, mindocURL, filename, uuid s
c.logger.Info("MindocListDocs", "requestURL:", requestURL, "resp", string(respBody))
var mindocResp MindocListDocsResponse
var mindocResp ListDocResponse
err = json.Unmarshal(respBody, &mindocResp)
if err != nil {
return nil, err

View File

@ -43,7 +43,7 @@ type NotionExportDocResponse struct {
}
// NotionListDocs 获取 Notion 文档列表
func (c *Client) NotionListDocs(ctx context.Context, secret, uuid string) (*NotionListDocsResponse, error) {
func (c *Client) NotionListDocs(ctx context.Context, secret, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -76,7 +76,7 @@ func (c *Client) NotionListDocs(ctx context.Context, secret, uuid string) (*Noti
c.logger.Info("NotionListDocs", "requestURL:", requestURL, "resp", string(respBody))
var notionResp NotionListDocsResponse
var notionResp ListDocResponse
err = json.Unmarshal(respBody, &notionResp)
if err != nil {
return nil, err

View File

@ -36,3 +36,28 @@ type TaskRes struct {
} `json:"data"`
Msg string `json:"msg"`
}
type ListDocResponse struct {
Success bool `json:"success"`
Data ListDocsData `json:"data"`
Msg string `json:"msg"`
Err string `json:"err"`
TraceID string `json:"trace_id"`
}
type ListDocsData struct {
Docs Child `json:"docs"`
}
type Value struct {
ID string `json:"id"`
File bool `json:"file"`
FileType string `json:"file_type"`
Title string `json:"title"`
Summary string `json:"summary"`
}
type Child struct {
Value Value `json:"value"`
Children []Child `json:"children"`
}

View File

@ -57,7 +57,7 @@ type RssExportDocData struct {
}
// RssListDocs 获取 Rss 文档列表
func (c *Client) RssListDocs(ctx context.Context, uuid, xmlUrl string) (*RssListDocsResponse, error) {
func (c *Client) RssListDocs(ctx context.Context, xmlUrl, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -89,7 +89,7 @@ func (c *Client) RssListDocs(ctx context.Context, uuid, xmlUrl string) (*RssList
c.logger.Info("RssListDocs", "requestURL:", requestURL, "resp", string(respBody))
var rssResp RssListDocsResponse
var rssResp ListDocResponse
err = json.Unmarshal(respBody, &rssResp)
if err != nil {
return nil, err

View File

@ -57,7 +57,7 @@ type SitemapExportDocData struct {
}
// SitemapListDocs 获取 Sitemap 文档列表
func (c *Client) SitemapListDocs(ctx context.Context, uuid, xmlUrl string) (*SitemapListDocsResponse, error) {
func (c *Client) SitemapListDocs(ctx context.Context, xmlUrl, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -89,7 +89,7 @@ func (c *Client) SitemapListDocs(ctx context.Context, uuid, xmlUrl string) (*Sit
c.logger.Info("SitemapListDocs", "requestURL:", requestURL, "resp", string(respBody))
var sitemapResp SitemapListDocsResponse
var sitemapResp ListDocResponse
err = json.Unmarshal(respBody, &sitemapResp)
if err != nil {
return nil, err

View File

@ -63,7 +63,7 @@ type SiyuanExportDocData struct {
}
// SiyuanListDocs 获取 Siyuan 文档列表
func (c *Client) SiyuanListDocs(ctx context.Context, siyuanURL, filename, uuid string) (*SiyuanListDocsResponse, error) {
func (c *Client) SiyuanListDocs(ctx context.Context, siyuanURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +101,7 @@ func (c *Client) SiyuanListDocs(ctx context.Context, siyuanURL, filename, uuid s
c.logger.Info("SiyuanListDocs", "requestURL:", requestURL, "resp", string(respBody))
var siyuanResp SiyuanListDocsResponse
var siyuanResp ListDocResponse
err = json.Unmarshal(respBody, &siyuanResp)
if err != nil {
return nil, err

View File

@ -23,25 +23,6 @@ type WikijsListDocsRequest struct {
UUID string `json:"uuid"` // 必填的唯一标识符
}
// WikijsListDocsResponse Wikijs 获取文档列表响应
type WikijsListDocsResponse struct {
Success bool `json:"success"`
Msg string `json:"msg"`
Data WikijsListDocsData `json:"data"`
}
// WikijsListDocsData Wikijs 文档列表数据
type WikijsListDocsData struct {
Docs []WikijsDoc `json:"docs"`
}
// WikijsDoc Wikijs 文档信息
type WikijsDoc struct {
ID string `json:"id"`
Title string `json:"title"`
URL string `json:"url"`
}
// WikijsExportDocRequest Wikijs 导出文档请求
type WikijsExportDocRequest struct {
UUID string `json:"uuid"` // 必须与 list 接口使用的 uuid 相同
@ -63,7 +44,7 @@ type WikijsExportDocData struct {
}
// WikijsListDocs 获取 Wikijs 文档列表
func (c *Client) WikijsListDocs(ctx context.Context, wikijsURL, filename, uuid string) (*WikijsListDocsResponse, error) {
func (c *Client) WikijsListDocs(ctx context.Context, wikijsURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -101,7 +82,7 @@ func (c *Client) WikijsListDocs(ctx context.Context, wikijsURL, filename, uuid s
c.logger.Info("WikijsListDocs", "requestURL:", requestURL, "resp", string(respBody))
var wikijsResp WikijsListDocsResponse
var wikijsResp ListDocResponse
err = json.Unmarshal(respBody, &wikijsResp)
if err != nil {
return nil, err

View File

@ -55,7 +55,7 @@ type YuqueExportDocResponse struct {
}
// YuqueListDocs 获取 Yuque 文档列表
func (c *Client) YuqueListDocs(ctx context.Context, yuqueURL, filename, uuid string) (*YuqueListDocsResponse, error) {
func (c *Client) YuqueListDocs(ctx context.Context, yuqueURL, filename, uuid string) (*ListDocResponse, error) {
u, err := url.Parse(crawlerServiceHost)
if err != nil {
return nil, err
@ -93,7 +93,7 @@ func (c *Client) YuqueListDocs(ctx context.Context, yuqueURL, filename, uuid str
c.logger.Info("YuqueListDocs", "requestURL:", requestURL, "resp", string(respBody))
var yuqueResp YuqueListDocsResponse
var yuqueResp ListDocResponse
err = json.Unmarshal(respBody, &yuqueResp)
if err != nil {
return nil, err

View File

@ -14,11 +14,14 @@ import (
"github.com/google/uuid"
"github.com/sbzhu/weworkapi_golang/wxbizmsgcrypt"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/pkg/bot"
)
const wechatMessageMaxBytes = 2000
func NewWechatConfig(ctx context.Context, CorpID, Token, EncodingAESKey string, kbid string, secret string, agentID string, logger *log.Logger) (*WechatConfig, error) {
return &WechatConfig{
Ctx: ctx,
@ -49,22 +52,30 @@ func (cfg *WechatConfig) VerifyUrlWechatAPP(signature, timestamp, nonce, echostr
return decryptEchoStr, nil
}
func (cfg *WechatConfig) Wechat(msg ReceivedMessage, getQA bot.GetQAFun, userinfo *UserInfo) error {
func (cfg *WechatConfig) Wechat(msg ReceivedMessage, getQA bot.GetQAFun, userinfo *UserInfo, useTextResponse bool, weChatAppAdvancedSetting *domain.WeChatAppAdvancedSetting) error {
token, err := cfg.GetAccessToken()
if err != nil {
return err
}
err = cfg.ProcessMessage(msg, getQA, token, userinfo)
if err != nil {
cfg.logger.Error("send to ai failed!", log.Error(err))
return err
if useTextResponse {
err = cfg.ProcessTextMessage(msg, getQA, token, userinfo, weChatAppAdvancedSetting.DisclaimerContent)
if err != nil {
cfg.logger.Error("send to ai failed!", log.Error(err))
return err
}
} else {
if err := cfg.ProcessUrlMessage(msg, getQA, token, userinfo); err != nil {
cfg.logger.Error("send to ai failed!", log.Error(err))
return err
}
}
return nil
}
// forwardToBackend
func (cfg *WechatConfig) ProcessMessage(msg ReceivedMessage, GetQA bot.GetQAFun, token string, userinfo *UserInfo) error {
func (cfg *WechatConfig) ProcessUrlMessage(msg ReceivedMessage, GetQA bot.GetQAFun, token string, userinfo *UserInfo) error {
// 1. get ai channel
id, err := uuid.NewV7()
if err != nil {
@ -73,7 +84,7 @@ func (cfg *WechatConfig) ProcessMessage(msg ReceivedMessage, GetQA bot.GetQAFun,
}
conversationID := id.String()
wccontent, err := GetQA(cfg.Ctx, msg.Content, domain.ConversationInfo{
contentChan, err := GetQA(cfg.Ctx, msg.Content, domain.ConversationInfo{
UserInfo: domain.UserInfo{
UserID: userinfo.UserID,
NickName: userinfo.Name,
@ -93,7 +104,7 @@ func (cfg *WechatConfig) ProcessMessage(msg ReceivedMessage, GetQA bot.GetQAFun,
}
domain.ConversationManager.Store(conversationID, state)
go cfg.SendQuestionToAI(conversationID, wccontent)
go cfg.SendQuestionToAI(conversationID, contentChan)
}
baseUrl, err := cfg.WeRepo.GetWechatBaseURL(cfg.Ctx, cfg.kbID)
@ -112,6 +123,54 @@ func (cfg *WechatConfig) ProcessMessage(msg ReceivedMessage, GetQA bot.GetQAFun,
return nil
}
func (cfg *WechatConfig) ProcessTextMessage(msg ReceivedMessage, GetQA bot.GetQAFun, token string, userinfo *UserInfo, disclaimerContent string) error {
// 1. get ai channel
id, err := uuid.NewV7()
if err != nil {
cfg.logger.Error("failed to generate conversation uuid", log.Error(err))
id = uuid.New()
}
conversationID := id.String()
contentChan, err := GetQA(cfg.Ctx, msg.Content, domain.ConversationInfo{
UserInfo: domain.UserInfo{
UserID: userinfo.UserID,
NickName: userinfo.Name,
From: domain.MessageFromPrivate,
}}, conversationID)
if err != nil {
return err
}
var fullResponse string
for content := range contentChan {
fullResponse += content
if len([]byte(fullResponse)) > wechatMessageMaxBytes { // wechat limit 2048 byte
if _, _, err := cfg.SendResponseToUser(fullResponse, msg.FromUserName, token); err != nil {
return err
}
fullResponse = ""
}
}
if len([]byte(fullResponse+disclaimerContent)) > wechatMessageMaxBytes {
if _, _, err := cfg.SendResponseToUser(fullResponse, msg.FromUserName, token); err != nil {
return err
}
if _, _, err := cfg.SendResponseToUser(disclaimerContent, msg.FromUserName, token); err != nil {
return err
}
} else {
if disclaimerContent != "" {
fullResponse += fmt.Sprintf("\n%s", disclaimerContent)
}
if _, _, err := cfg.SendResponseToUser(fullResponse, msg.FromUserName, token); err != nil {
return err
}
}
return nil
}
// SendResponseToUser
func (cfg *WechatConfig) SendURLToUser(touser, question, token, conversationID, baseUrl string) (int, string, error) {
msgData := map[string]interface{}{
@ -121,7 +180,7 @@ func (cfg *WechatConfig) SendURLToUser(touser, question, token, conversationID,
"textcard": map[string]interface{}{
"title": question,
"description": "<div class = \"highlight\">本回答由 PandaWiki 基于 AI 生成,仅供参考。</div>",
"url": fmt.Sprintf("%s/h5-chat?id=%s", baseUrl, conversationID),
"url": fmt.Sprintf("%s/h5-chat?id=%s&source_type=%s", baseUrl, conversationID, consts.SourceTypeWechatBot),
},
}
@ -150,7 +209,6 @@ func (cfg *WechatConfig) SendURLToUser(touser, question, token, conversationID,
return result.Errcode, result.Errmsg, nil
}
// SendResponseToUser
func (cfg *WechatConfig) SendResponseToUser(response string, touser string, token string) (int, string, error) {
msgData := map[string]interface{}{
@ -184,6 +242,9 @@ func (cfg *WechatConfig) SendResponseToUser(response string, touser string, toke
if err := json.Unmarshal(body, &result); err != nil {
return 0, "", err
}
if result.Errcode != 0 {
return result.Errcode, result.Errmsg, fmt.Errorf("wechat Api failed : %s (code: %d)", result.Errmsg, result.Errcode)
}
return result.Errcode, result.Errmsg, nil
}
@ -235,7 +296,7 @@ func (cfg *WechatConfig) GetAccessToken() (string, error) {
defer tokenCache.Mutex.Unlock()
if tokenCache.AccessToken != "" && time.Now().Before(tokenCache.TokenExpire) {
cfg.logger.Info("access token has existed and is valid")
cfg.logger.Debug("access token has existed and is valid")
return tokenCache.AccessToken, nil
}

View File

@ -297,7 +297,7 @@ func (cfg *WechatServiceConfig) GetAccessToken() (string, error) {
defer tokenCache.Mutex.Unlock()
if tokenCache.AccessToken != "" && time.Now().Before(tokenCache.TokenExpire) {
cfg.logger.Info("access token has existed and is valid")
cfg.logger.Debug("access token has existed and is valid")
return tokenCache.AccessToken, nil
}

View File

@ -18,7 +18,8 @@ import (
const (
// AuthURL api doc https://developer.work.weixin.qq.com/document/path/98152
AuthURL = "https://login.work.weixin.qq.com/wwlogin/sso/login"
AuthWebURL = "https://login.work.weixin.qq.com/wwlogin/sso/login"
AuthAPPURL = "https://open.weixin.qq.com/connect/oauth2/authorize"
TokenURL = "https://qyapi.weixin.qq.com/cgi-bin/gettoken"
UserInfoURL = "https://qyapi.weixin.qq.com/cgi-bin/auth/getuserinfo"
UserDetailURL = "https://qyapi.weixin.qq.com/cgi-bin/user/get"
@ -29,11 +30,6 @@ const (
callbackPath = "/share/pro/v1/openapi/wecom/callback"
)
var oauthEndpoint = oauth2.Endpoint{
AuthURL: AuthURL,
TokenURL: TokenURL,
}
// Client 企业微信客户端
type Client struct {
context context.Context
@ -115,17 +111,24 @@ type UserListResponse struct {
} `json:"userlist"`
}
func NewClient(ctx context.Context, logger *log.Logger, corpID, corpSecret, agentID, redirectURI string, cache *cache.Cache) (*Client, error) {
func NewClient(ctx context.Context, logger *log.Logger, corpID, corpSecret, agentID, redirectURI string, cache *cache.Cache, isApp bool) (*Client, error) {
redirectURL, _ := url.Parse(redirectURI)
redirectURL.Path = callbackPath
redirectURI = redirectURL.String()
authUrl := AuthWebURL
if isApp {
authUrl = AuthAPPURL
}
oauthConfig := &oauth2.Config{
ClientID: corpID,
ClientSecret: corpSecret,
RedirectURL: redirectURI,
Endpoint: oauthEndpoint,
Scopes: []string{"snsapi_privateinfo"},
Endpoint: oauth2.Endpoint{
AuthURL: authUrl,
TokenURL: TokenURL,
},
Scopes: []string{"snsapi_privateinfo"},
}
return &Client{
@ -150,7 +153,11 @@ func (c *Client) GenerateAuthURL(state string) string {
params.Set("agentid", c.agentID)
params.Set("state", state)
return fmt.Sprintf("%s?%s", AuthURL, params.Encode())
authUrl := fmt.Sprintf("%s?%s", c.oauthConfig.Endpoint.AuthURL, params.Encode())
if c.oauthConfig.Endpoint.AuthURL == AuthAPPURL {
authUrl += "#wechat_redirect"
}
return authUrl
}
// GetAccessToken 获取企业微信访问令牌

@ -1 +1 @@
Subproject commit 32ec7a54b3cb14ac214827cd6cc0905dd3e4ca78
Subproject commit 530a059db3c8b1ef86c3a43eaf70d75c46c75df9

View File

@ -2,5 +2,7 @@ package backend
import (
_ "github.com/jinzhu/copier"
_ "github.com/mark3labs/mcp-go/mcp"
_ "github.com/mark3labs/mcp-go/server"
_ "google.golang.org/protobuf/types/known/emptypb"
)

View File

@ -300,8 +300,8 @@ func (r *AuthRepo) GetOrCreateAuth(ctx context.Context, auth *domain.Auth, sourc
return err
}
if int(count) >= licenseEdition.GetMaxAuth(sourceType) {
return fmt.Errorf("exceed max auth limit for kb %s, current count: %d, max limit: %d", auth.KBID, count, licenseEdition.GetMaxAuth(sourceType))
if int(count) >= domain.GetBaseEditionLimitation(ctx).MaxSSOUser {
return fmt.Errorf("exceed max auth limit for kb %s, current count: %d, max limit: %d", auth.KBID, count, domain.GetBaseEditionLimitation(ctx).MaxSSOUser)
}
auth.LastLoginTime = time.Now()

View File

@ -26,12 +26,12 @@ func (r *CommentRepository) CreateComment(ctx context.Context, comment *domain.C
return nil
}
func (r *CommentRepository) GetCommentList(ctx context.Context, nodeID string, edition consts.LicenseEdition) ([]*domain.ShareCommentListItem, int64, error) {
func (r *CommentRepository) GetCommentList(ctx context.Context, nodeID string) ([]*domain.ShareCommentListItem, int64, error) {
// 按照时间排序来查询node_id的comments
comments := []*domain.ShareCommentListItem{}
query := r.db.Model(&domain.Comment{}).Where("node_id = ?", nodeID)
var comments []*domain.ShareCommentListItem
query := r.db.WithContext(ctx).Model(&domain.Comment{}).Where("node_id = ?", nodeID)
if edition == consts.LicenseEditionContributor || edition == consts.LicenseEditionEnterprise {
if domain.GetBaseEditionLimitation(ctx).AllowCommentAudit {
query = query.Where("status = ?", domain.CommentStatusAccepted) //accepted
}
@ -50,14 +50,14 @@ func (r *CommentRepository) GetCommentList(ctx context.Context, nodeID string, e
func (r *CommentRepository) GetCommentListByKbID(ctx context.Context, req *domain.CommentListReq, edition consts.LicenseEdition) ([]*domain.CommentListItem, int64, error) {
comments := []*domain.CommentListItem{}
query := r.db.Model(&domain.Comment{}).Where("comments.kb_id = ?", req.KbID)
query := r.db.WithContext(ctx).Model(&domain.Comment{}).Where("comments.kb_id = ?", req.KbID)
var count int64
if req.Status == nil {
if err := query.Count(&count).Error; err != nil {
return nil, 0, err
}
} else {
if edition == consts.LicenseEditionContributor || edition == consts.LicenseEditionEnterprise {
if domain.GetBaseEditionLimitation(ctx).AllowCommentAudit {
query = query.Where("comments.status = ?", *req.Status)
}
// 按照时间排序来查询kb_id的comments ->reject pending accepted
@ -84,7 +84,7 @@ func (r *CommentRepository) GetCommentListByKbID(ctx context.Context, req *domai
func (r *CommentRepository) DeleteCommentList(ctx context.Context, commentID []string) error {
// 批量删除指定id的comment,获取删除的总的数量、
query := r.db.Model(&domain.Comment{}).Where("id IN (?)", commentID)
query := r.db.WithContext(ctx).Model(&domain.Comment{}).Where("id IN (?)", commentID)
if err := query.Delete(&domain.Comment{}).Error; err != nil {
return err

View File

@ -263,3 +263,27 @@ func (r *ConversationRepository) GetConversationDistributionByHour(ctx context.C
return counts, nil
}
func (r *ConversationRepository) GetConversationCountByAppType(ctx context.Context) (map[domain.AppType]int64, error) {
type row struct {
AppType int `gorm:"column:app_type"`
Count int64 `gorm:"column:count"`
}
var rows []row
if err := r.db.WithContext(ctx).
Model(&domain.Conversation{}).
Joins("JOIN apps ON conversations.app_id = apps.id").
Select("apps.type as app_type, COUNT(*) as count").
Group("apps.type").
Find(&rows).Error; err != nil {
return nil, err
}
result := make(map[domain.AppType]int64)
for _, t := range domain.AppTypes {
result[t] = 0
}
for _, rrow := range rows {
result[domain.AppType(rrow.AppType)] = rrow.Count
}
return result, nil
}

View File

@ -169,7 +169,7 @@ func (r *KnowledgeBaseRepository) SyncKBAccessSettingsToCaddy(ctx context.Contex
{
"match": []map[string]any{
{
"path": []string{"/share/v1/chat/completions", "/share/v1/app/wechat/app", "/share/v1/app/wechat/service", "/sitemap.xml", "/share/v1/app/wechat/official_account", "/share/v1/app/wechat/service/answer"},
"path": []string{"/share/v1/chat/completions", "/share/v1/app/wechat/app", "/share/v1/app/wechat/service", "/sitemap.xml", "/share/v1/app/wechat/official_account", "/share/v1/app/wechat/service/answer", "/mcp"},
},
},
"handle": []map[string]any{

25
backend/repo/pg/mcp.go Normal file
View File

@ -0,0 +1,25 @@
package pg
import (
"context"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/store/pg"
)
type MCPRepository struct {
db *pg.DB
logger *log.Logger
}
func NewMCPRepository(db *pg.DB, logger *log.Logger) *MCPRepository {
return &MCPRepository{db: db, logger: logger}
}
func (r *MCPRepository) GetMCPCallCount(ctx context.Context) (int64, error) {
var count int64
if err := r.db.WithContext(ctx).Table("mcp_calls").Count(&count).Error; err != nil {
return 0, err
}
return count, nil
}

View File

@ -11,10 +11,12 @@ import (
"gorm.io/gorm/clause"
"github.com/google/uuid"
"github.com/lib/pq"
"github.com/samber/lo"
"github.com/samber/lo/mutable"
v1 "github.com/chaitin/panda-wiki/api/node/v1"
shareV1 "github.com/chaitin/panda-wiki/api/share/v1"
"github.com/chaitin/panda-wiki/consts"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/log"
@ -84,27 +86,34 @@ func (r *NodeRepository) Create(ctx context.Context, req *domain.CreateNodeReq,
if req.Summary != nil {
meta.Summary = *req.Summary
}
if req.ContentType != nil {
meta.ContentType = *req.ContentType
}
node := &domain.Node{
ID: nodeIDStr,
KBID: req.KBID,
Name: req.Name,
Content: req.Content,
Meta: meta,
Type: req.Type,
ParentID: req.ParentID,
Position: newPos,
Status: domain.NodeStatusDraft,
Permissions: domain.NodePermissions{
Answerable: consts.NodeAccessPermOpen,
Visitable: consts.NodeAccessPermOpen,
Visible: consts.NodeAccessPermOpen,
},
ID: nodeIDStr,
KBID: req.KBID,
Name: req.Name,
Content: req.Content,
Meta: meta,
Type: req.Type,
ParentID: req.ParentID,
Position: newPos,
Status: domain.NodeStatusDraft,
CreatorId: userId,
EditorId: userId,
CreatedAt: now,
UpdatedAt: now,
EditTime: now,
RagInfo: domain.RagInfo{
Status: consts.NodeRagStatusBasicPending,
Message: "",
},
Permissions: domain.NodePermissions{
Answerable: consts.NodeAccessPermOpen,
Visitable: consts.NodeAccessPermOpen,
Visible: consts.NodeAccessPermOpen,
},
}
return tx.Create(node).Error
@ -123,7 +132,7 @@ func (r *NodeRepository) GetList(ctx context.Context, req *domain.GetNodeListReq
Joins("LEFT JOIN users cu ON nodes.creator_id = cu.id").
Joins("LEFT JOIN users eu ON nodes.editor_id = eu.id").
Where("nodes.kb_id = ?", req.KBID).
Select("cu.account AS creator, eu.account AS editor, nodes.editor_id, nodes.creator_id, nodes.id, nodes.permissions, nodes.type, nodes.status, nodes.name, nodes.parent_id, nodes.position, nodes.created_at, nodes.edit_time as updated_at, nodes.meta->>'summary' as summary, nodes.meta->>'emoji' as emoji")
Select("cu.account AS creator, eu.account AS editor, nodes.editor_id, nodes.rag_info, nodes.creator_id, nodes.id, nodes.permissions, nodes.type, nodes.status, nodes.name, nodes.parent_id, nodes.position, nodes.created_at, nodes.edit_time as updated_at, nodes.meta->>'summary' as summary, nodes.meta->>'emoji' as emoji, nodes.meta->>'content_type' as content_type")
if req.Search != "" {
searchPattern := "%" + req.Search + "%"
query = query.Where("name LIKE ? OR content LIKE ?", searchPattern, searchPattern)
@ -148,6 +157,32 @@ func (r *NodeRepository) GetLatestNodeReleaseByNodeIDs(ctx context.Context, kbID
return nodeReleases, nil
}
func (r *NodeRepository) GetNodeReleasePublisherMap(ctx context.Context, kbID string) (map[string]string, error) {
type Result struct {
NodeID string `gorm:"column:node_id"`
PublisherID string `gorm:"column:publisher_id"`
}
var results []Result
if err := r.db.WithContext(ctx).
Model(&domain.NodeRelease{}).
Select("node_id, publisher_id").
Where("kb_id = ?", kbID).
Where("node_releases.doc_id != '' ").
Find(&results).Error; err != nil {
return nil, err
}
publisherMap := make(map[string]string)
for _, result := range results {
if result.PublisherID != "" {
publisherMap[result.NodeID] = result.PublisherID
}
}
return publisherMap, nil
}
func (r *NodeRepository) UpdateNodeContent(ctx context.Context, req *domain.UpdateNodeReq, userId string) error {
// Use transaction to ensure data consistency
err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
@ -188,7 +223,7 @@ func (r *NodeRepository) UpdateNodeContent(ctx context.Context, req *domain.Upda
}
// Handle multiple meta field updates
if req.Emoji != nil || req.Summary != nil {
if req.Emoji != nil || req.Summary != nil || req.ContentType != nil {
metaExpr := "meta"
var args []any
metaUpdated := false
@ -209,6 +244,16 @@ func (r *NodeRepository) UpdateNodeContent(ctx context.Context, req *domain.Upda
metaUpdated = true
}
// Compare and update ContentType
if currentNode.Meta.ContentType == "" { // can only modify content_type if it was empty before
if req.ContentType != nil && *req.ContentType != currentNode.Meta.ContentType {
// Second jsonb_set: jsonb_set(previous_expr, '{content_type}', to_jsonb(?::text))
metaExpr = "jsonb_set(" + metaExpr + ", '{content_type}', to_jsonb(?::text))"
args = append(args, *req.ContentType) // Second parameter for content_type
metaUpdated = true
}
}
if metaUpdated {
updateMap["meta"] = gorm.Expr(metaExpr, args...)
updateStatus = true
@ -240,8 +285,11 @@ func (r *NodeRepository) GetByID(ctx context.Context, id, kbId string) (*v1.Node
var node *v1.NodeDetailResp
if err := r.db.WithContext(ctx).
Model(&domain.Node{}).
Where("id = ?", id).
Where("kb_id = ?", kbId).
Select("nodes.*, creator.id as creator_id, creator.account as creator_account, editor.id as editor_id, editor.account as editor_account").
Joins("left join users creator on creator.id = nodes.creator_id").
Joins("left join users editor on editor.id = nodes.editor_id").
Where("nodes.id = ?", id).
Where("nodes.kb_id = ?", kbId).
First(&node).Error; err != nil {
return nil, err
}
@ -251,20 +299,12 @@ func (r *NodeRepository) GetByID(ctx context.Context, id, kbId string) (*v1.Node
func (r *NodeRepository) Delete(ctx context.Context, kbID string, ids []string) ([]string, error) {
docIDs := make([]string, 0)
if err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// check if node.parent_id in ids
var parentIDs []string
if err := tx.Model(&domain.Node{}).
Where("parent_id IN ?", ids).
Select("parent_id").
Find(&parentIDs).Error; err != nil {
return err
}
if len(parentIDs) > 0 {
return domain.ErrNodeParentIDInIDs
}
// recursively collect all child node IDs
allIDs := r.collectAllChildNodeIDs(tx, kbID, ids)
var nodes []*domain.Node
if err := tx.Model(&domain.Node{}).
Where("id IN ?", ids).
Where("id IN ?", allIDs).
Where("kb_id = ?", kbID).
Clauses(clause.Returning{Columns: []clause.Column{{Name: "doc_id"}}}).
Delete(&nodes).Error; err != nil {
@ -273,7 +313,7 @@ func (r *NodeRepository) Delete(ctx context.Context, kbID string, ids []string)
// delete node release
var nodeReleases []*domain.NodeRelease
if err := tx.Model(&domain.NodeRelease{}).
Where("node_id IN ?", ids).
Where("node_id IN ?", allIDs).
Clauses(clause.Returning{Columns: []clause.Column{{Name: "doc_id"}}}).
Delete(&nodeReleases).Error; err != nil {
return err
@ -295,6 +335,33 @@ func (r *NodeRepository) Delete(ctx context.Context, kbID string, ids []string)
return lo.Uniq(docIDs), nil
}
// collectAllChildNodeIDs recursively collects all child node IDs for the given parent IDs
func (r *NodeRepository) collectAllChildNodeIDs(tx *gorm.DB, kbID string, parentIDs []string) []string {
allIDs := make([]string, 0)
allIDs = append(allIDs, parentIDs...)
currentParentIDs := parentIDs
for len(currentParentIDs) > 0 {
var childIDs []string
if err := tx.Model(&domain.Node{}).
Where("parent_id IN ?", currentParentIDs).
Where("kb_id = ?", kbID).
Select("id").
Find(&childIDs).Error; err != nil {
break
}
if len(childIDs) == 0 {
break
}
allIDs = append(allIDs, childIDs...)
currentParentIDs = childIDs
}
return lo.Uniq(allIDs)
}
func (r *NodeRepository) GetNodeByID(ctx context.Context, id string) (*domain.Node, error) {
var node *domain.Node
if err := r.db.WithContext(ctx).
@ -388,6 +455,20 @@ func (r *NodeRepository) GetLatestNodeReleaseByNodeID(ctx context.Context, nodeI
return nodeRelease, nil
}
func (r *NodeRepository) GetLatestNodeReleaseWithPublishAccount(ctx context.Context, nodeID string) (*domain.NodeReleaseWithPublisher, error) {
var nodeRelease *domain.NodeReleaseWithPublisher
if err := r.db.WithContext(ctx).
Model(&domain.NodeRelease{}).
Select("node_releases.id, node_releases.publisher_id, users.account as publisher_account").
Joins("left join users on users.id = node_releases.publisher_id").
Where("node_releases.node_id = ?", nodeID).
Order("node_releases.updated_at DESC").
Find(&nodeRelease).Error; err != nil {
return nil, err
}
return nodeRelease, nil
}
// GetNodeReleaseWithDirPathByID gets a node release by ID and includes its directory path
func (r *NodeRepository) GetNodeReleaseWithDirPathByID(ctx context.Context, id string) (*domain.NodeReleaseWithDirPath, error) {
// First get the node release
@ -433,6 +514,137 @@ func (r *NodeRepository) GetNodeReleasesByDocIDs(ctx context.Context, ids []stri
return nodesMap, nil
}
// NodeReleaseWithPath represents a node release with path information
type NodeReleaseWithPath struct {
*domain.NodeRelease
PathIDs []string `json:"path_ids"`
PathNames []string `json:"path_names"`
Depth int `json:"depth"`
}
// GetNodeReleasesWithPathsByDocIDs retrieving node releases with path information
func (r *NodeRepository) GetNodeReleasesWithPathsByDocIDs(ctx context.Context, ids []string) (map[string]*NodeReleaseWithPath, error) {
if len(ids) == 0 {
return make(map[string]*NodeReleaseWithPath), nil
}
// 1. 查询节点基本信息
var nodeReleases []*domain.NodeRelease
if err := r.db.WithContext(ctx).
Model(&domain.NodeRelease{}).
Where("doc_id IN ?", ids).
Find(&nodeReleases).Error; err != nil {
return nil, err
}
if len(nodeReleases) == 0 {
return make(map[string]*NodeReleaseWithPath), nil
}
docIDs := lo.Map(nodeReleases, func(release *domain.NodeRelease, i int) string {
return release.DocID
})
// 2. 批量查询路径
paths, err := r.getNodePathsBatch(ctx, docIDs)
if err != nil {
return nil, fmt.Errorf("failed to get paths: %w", err)
}
// 3. 组装结果
result := make(map[string]*NodeReleaseWithPath, len(nodeReleases))
for _, nr := range nodeReleases {
nrWithPath := &NodeReleaseWithPath{
NodeRelease: nr,
}
if path, ok := paths[nr.DocID]; ok {
nrWithPath.PathIDs = path.PathIDs
nrWithPath.PathNames = path.PathNames
nrWithPath.Depth = path.Depth
}
result[nr.DocID] = nrWithPath
}
return result, nil
}
// NodePathInfo contains path information for a node
type NodePathInfo struct {
DocID string
PathIDs []string
PathNames []string
Depth int
}
// getNodePathsBatch batch query node paths
func (r *NodeRepository) getNodePathsBatch(ctx context.Context, docIDs []string) (map[string]*NodePathInfo, error) {
type pathResult struct {
DocID string `gorm:"column:doc_id"`
PathIDs pq.StringArray `gorm:"column:path_ids;type:text[]"`
PathNames pq.StringArray `gorm:"column:path_names;type:text[]"`
Depth int `gorm:"column:depth"`
}
var results []pathResult
query := `
WITH RECURSIVE node_paths AS (
SELECT
node_id,
parent_id,
name,
doc_id as root_doc_id,
ARRAY[node_id] as path_ids,
ARRAY[name] as path_names,
1 as depth
FROM node_releases
WHERE doc_id = ANY($1)
UNION ALL
SELECT
n.node_id,
n.parent_id,
n.name,
np.root_doc_id,
n.node_id || np.path_ids,
n.name || np.path_names,
np.depth + 1
FROM node_releases n
INNER JOIN node_paths np ON n.node_id = np.parent_id
WHERE np.depth < 20 AND n.doc_id != ''
)
SELECT
root_doc_id as doc_id,
path_ids,
path_names,
depth
FROM node_paths
WHERE parent_id IS NULL OR parent_id = ''
`
if err := r.db.WithContext(ctx).
Raw(query, pq.Array(docIDs)).
Scan(&results).Error; err != nil {
return nil, err
}
// 转换为map
pathMap := make(map[string]*NodePathInfo, len(results))
for _, res := range results {
pathMap[res.DocID] = &NodePathInfo{
DocID: res.DocID,
PathIDs: res.PathIDs,
PathNames: res.PathNames,
Depth: res.Depth,
}
}
return pathMap, nil
}
// GetRecommendNodeListByIDs get node list by ids
func (r *NodeRepository) GetRecommendNodeListByIDs(ctx context.Context, kbID string, releaseID string, ids []string) ([]*domain.RecommendNodeListResp, error) {
var nodes []*domain.RecommendNodeListResp
@ -497,14 +709,14 @@ func (r *NodeRepository) GetNodeReleaseListByKBID(ctx context.Context, kbID stri
Where("kb_release_node_releases.kb_id = ?", kbID).
Where("kb_release_node_releases.release_id = ?", kbRelease.ID).
Where("nodes.permissions->>'visible' != ?", consts.NodeAccessPermClosed).
Select("node_releases.node_id as id, node_releases.name, node_releases.type, node_releases.parent_id, node_releases.position, node_releases.meta->>'emoji' as emoji, node_releases.updated_at, nodes.permissions").
Select("node_releases.node_id as id, node_releases.name, node_releases.type, node_releases.parent_id, nodes.position, node_releases.meta->>'emoji' as emoji, node_releases.updated_at, nodes.permissions, nodes.meta").
Find(&nodes).Error; err != nil {
return nil, err
}
return nodes, nil
}
func (r *NodeRepository) GetNodeReleaseDetailByKBIDAndID(ctx context.Context, kbID, id string) (*v1.NodeDetailResp, error) {
func (r *NodeRepository) GetNodeReleaseDetailByKBIDAndID(ctx context.Context, kbID, id string) (*shareV1.ShareNodeDetailResp, error) {
// get kb release
var kbRelease *domain.KBRelease
if err := r.db.WithContext(ctx).
@ -515,16 +727,16 @@ func (r *NodeRepository) GetNodeReleaseDetailByKBIDAndID(ctx context.Context, kb
return nil, err
}
var node *v1.NodeDetailResp
var node *shareV1.ShareNodeDetailResp
if err := r.db.WithContext(ctx).
Model(&domain.KBReleaseNodeRelease{}).
Select("node_releases.*, nodes.permissions, nodes.creator_id").
Joins("LEFT JOIN node_releases ON node_releases.id = kb_release_node_releases.node_release_id").
Joins("LEFT JOIN nodes ON nodes.id = kb_release_node_releases.node_id").
Where("kb_release_node_releases.release_id = ?", kbRelease.ID).
Where("node_releases.node_id = ?", id).
Where("node_releases.kb_id = ?", kbID).
Where("nodes.permissions->>'visitable' != ?", consts.NodeAccessPermClosed).
Select("node_releases.*, nodes.permissions").
First(&node).Error; err != nil {
return nil, err
}
@ -642,7 +854,7 @@ func (r *NodeRepository) TraverseNodesByCursor(ctx context.Context, callback fun
}
// CreateNodeReleases create node releases
func (r *NodeRepository) CreateNodeReleases(ctx context.Context, kbID string, nodeIDs []string) ([]string, error) {
func (r *NodeRepository) CreateNodeReleases(ctx context.Context, kbID, userId string, nodeIDs []string) ([]string, error) {
releaseIDs := make([]string, 0)
if err := r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// update node status to published and return node ids
@ -661,17 +873,19 @@ func (r *NodeRepository) CreateNodeReleases(ctx context.Context, kbID string, no
for i, updatedNode := range updatedNodes {
// create node release
nodeRelease := &domain.NodeRelease{
ID: uuid.New().String(),
KBID: kbID,
NodeID: updatedNode.ID,
Type: updatedNode.Type,
Name: updatedNode.Name,
Meta: updatedNode.Meta,
Content: updatedNode.Content,
ParentID: updatedNode.ParentID,
Position: updatedNode.Position,
CreatedAt: updatedNode.CreatedAt,
UpdatedAt: time.Now(),
ID: uuid.New().String(),
KBID: kbID,
PublisherId: userId,
EditorId: updatedNode.EditorId,
NodeID: updatedNode.ID,
Type: updatedNode.Type,
Name: updatedNode.Name,
Meta: updatedNode.Meta,
Content: updatedNode.Content,
ParentID: updatedNode.ParentID,
Position: updatedNode.Position,
CreatedAt: updatedNode.CreatedAt,
UpdatedAt: time.Now(),
}
nodeReleases[i] = nodeRelease
releaseIDs = append(releaseIDs, nodeRelease.ID)
@ -903,3 +1117,75 @@ func (r *NodeRepository) GetNodeGroupByNodeId(ctx context.Context, nodeId string
}
return nodeGroup, nil
}
func (r *NodeRepository) Update(ctx context.Context, id string, m map[string]interface{}) error {
return r.db.WithContext(ctx).Model(domain.Node{}).Where("id = ?", id).Updates(m).Error
}
func (r *NodeRepository) GetNodeIdByDocId(ctx context.Context, docId string) (string, error) {
nodeIds := make([]string, 0)
if err := r.db.WithContext(ctx).Model(domain.NodeRelease{}).
Where("doc_id = ?", docId).
Pluck("node_id", &nodeIds).Error; err != nil {
return "", err
}
if len(nodeIds) < 1 {
return "", fmt.Errorf("node not found for doc_id: %s", docId)
}
return nodeIds[0], nil
}
func (r *NodeRepository) GetNodeIdsWithoutStatusByKbId(ctx context.Context, kbId string) ([]string, error) {
docIds := make([]string, 0)
if err := r.db.WithContext(ctx).
Model(&domain.Node{}).
Joins("left join node_releases on node_releases.node_id = nodes.id").
Where("(nodes.rag_info ->> 'status' IS NULL OR nodes.rag_info ->> 'status' = '')").
Where("nodes.kb_id = ? ", kbId).
Where("nodes.type = ? ", domain.NodeTypeDocument).
Where("node_releases.doc_id != '' ").
Pluck("node_releases.doc_id", &docIds).Error; err != nil {
return nil, err
}
return docIds, nil
}
// GetNodeIdsByDocIds 批量获取 doc_id 到 node_id 的映射
func (r *NodeRepository) GetNodeIdsByDocIds(ctx context.Context, docIds []string) (map[string]string, error) {
if len(docIds) == 0 {
return make(map[string]string), nil
}
type Result struct {
DocID string `gorm:"column:doc_id"`
NodeID string `gorm:"column:node_id"`
}
results := make([]Result, 0)
if err := r.db.WithContext(ctx).
Model(&domain.NodeRelease{}).
Select("doc_id, node_id").
Where("doc_id IN (?)", docIds).
Find(&results).Error; err != nil {
return nil, err
}
// 构建 doc_id -> node_id 的映射
docToNodeMap := make(map[string]string, len(results))
for _, result := range results {
docToNodeMap[result.DocID] = result.NodeID
}
return docToNodeMap, nil
}
func (r *NodeRepository) GetNodeCount(ctx context.Context) (int, error) {
var count int64
err := r.db.WithContext(ctx).
Model(&domain.Node{}).
Count(&count).Error
if err != nil {
return 0, err
}
return int(count), nil
}

View File

@ -0,0 +1,39 @@
package pg
import (
"context"
"errors"
"gorm.io/gorm"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/utils"
)
func (r *NodeRepository) GetNodeStatsByNodeId(ctx context.Context, nodeId string) (*domain.NodeStats, error) {
var nodeStats *domain.NodeStats
if err := r.db.WithContext(ctx).
Model(&domain.NodeStats{}).
Where("node_id = ?", nodeId).
First(&nodeStats).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
nodeStats = &domain.NodeStats{
ID: 0,
NodeID: nodeId,
PV: 0,
}
} else {
return nil, err
}
}
var todayStats int64
if err := r.db.WithContext(ctx).Model(&domain.StatPage{}).
Where("created_at >= ?", utils.GetTimeHourOffset(-24)).
Where("node_id = ?", nodeId).Count(&todayStats).Error; err != nil {
return nil, err
}
nodeStats.PV += todayStats
return nodeStats, nil
}

View File

@ -23,4 +23,6 @@ var ProviderSet = wire.NewSet(
NewAuthRepo,
NewWechatRepository,
NewAPITokenRepo,
NewSystemSettingRepo,
NewMCPRepository,
)

View File

@ -3,6 +3,9 @@ package pg
import (
"context"
"gorm.io/gorm"
"gorm.io/gorm/clause"
v1 "github.com/chaitin/panda-wiki/api/stat/v1"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/store/cache"
@ -156,3 +159,46 @@ func (r *StatRepository) RemoveOldData(ctx context.Context) error {
}
return nil
}
// GetYesterdayPVByNode 获取昨天的PV数据按node_id分组
func (r *StatRepository) GetYesterdayPVByNode(ctx context.Context) (map[string]int64, error) {
type PVResult struct {
NodeID string
Count int64
}
var results []PVResult
if err := r.db.WithContext(ctx).Model(&domain.StatPage{}).
Where("created_at < ?", utils.GetTimeHourOffset(0)).
Where("created_at >= ?", utils.GetTimeHourOffset(-24)).
Where("node_id != ?", "").
Group("node_id").
Select("node_id, COUNT(*) as count").
Find(&results).Error; err != nil {
return nil, err
}
pvMap := make(map[string]int64)
for _, result := range results {
pvMap[result.NodeID] = result.Count
}
return pvMap, nil
}
// UpsertNodeStats 插入或更新node_stats表
func (r *StatRepository) UpsertNodeStats(ctx context.Context, nodeID string, pvCount int64) error {
nodeStats := &domain.NodeStats{
NodeID: nodeID,
PV: pvCount,
}
// 使用GORM的Clauses进行upsert操作
return r.db.WithContext(ctx).
Clauses(clause.OnConflict{
Columns: []clause.Column{{Name: "node_id"}},
DoUpdates: clause.Assignments(map[string]interface{}{
"pv": gorm.Expr("node_stats.pv + ?", pvCount),
}),
}).
Create(nodeStats).Error
}

View File

@ -0,0 +1,35 @@
package pg
import (
"context"
"github.com/chaitin/panda-wiki/domain"
"github.com/chaitin/panda-wiki/log"
"github.com/chaitin/panda-wiki/store/pg"
)
type SystemSettingRepo struct {
db *pg.DB
logger *log.Logger
}
func NewSystemSettingRepo(db *pg.DB, logger *log.Logger) *SystemSettingRepo {
return &SystemSettingRepo{
db: db,
logger: logger.WithModule("repo.pg.system_setting"),
}
}
func (r *SystemSettingRepo) GetSystemSetting(ctx context.Context, key string) (*domain.SystemSetting, error) {
var setting domain.SystemSetting
result := r.db.WithContext(ctx).Where("key = ?", key).First(&setting)
if result.Error != nil {
return nil, result.Error
}
return &setting, nil
}
func (r *SystemSettingRepo) UpdateSystemSetting(ctx context.Context, key, value string) error {
return r.db.WithContext(ctx).Model(&domain.SystemSetting{}).Where("key = ?", key).Update("value", value).Error
}

View File

@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"github.com/samber/lo"
"golang.org/x/crypto/bcrypt"
"gorm.io/gorm"
@ -59,18 +60,14 @@ func (r *UserRepository) CreateUser(ctx context.Context, user *domain.User, edit
}
user.Password = string(hashedPassword)
return r.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
if edition == consts.LicenseEditionContributor || edition == consts.LicenseEditionFree {
var count int64
if err := tx.Model(&domain.User{}).Count(&count).Error; err != nil {
return err
}
if edition == consts.LicenseEditionFree && count >= 1 {
return errors.New("free edition only allows 1 user")
}
if edition == consts.LicenseEditionContributor && count >= 5 {
return errors.New("contributor edition only allows 5 user")
}
var count int64
if err := tx.Model(&domain.User{}).Count(&count).Error; err != nil {
return err
}
if count >= domain.GetBaseEditionLimitation(ctx).MaxAdmin {
return fmt.Errorf("exceed max admin limit, current count: %d, max limit: %d", count, domain.GetBaseEditionLimitation(ctx).MaxAdmin)
}
if err := tx.Create(user).Error; err != nil {
return err
}
@ -113,6 +110,22 @@ func (r *UserRepository) ListUsers(ctx context.Context) ([]v1.UserListItemResp,
return users, nil
}
func (r *UserRepository) GetUsersAccountMap(ctx context.Context) (map[string]string, error) {
var users []v1.UserListItemResp
err := r.db.WithContext(ctx).
Model(&domain.User{}).
Find(&users).Error
if err != nil {
return nil, err
}
m := lo.SliceToMap(users, func(user v1.UserListItemResp) (string, string) {
return user.ID, user.Account
})
return m, nil
}
func (r *UserRepository) UpdateUserPassword(ctx context.Context, userID string, newPassword string) error {
hashedPassword, err := bcrypt.GenerateFromPassword([]byte(newPassword), bcrypt.DefaultCost)
if err != nil {

View File

@ -0,0 +1 @@
ALTER TABLE comments DROP COLUMN IF EXISTS pic_urls;

View File

@ -0,0 +1 @@
ALTER TABLE comments ADD COLUMN IF NOT EXISTS pic_urls text[] not null default ARRAY[]::text[];

View File

@ -0,0 +1 @@
ALTER TABLE nodes DROP COLUMN IF EXISTS rag_info;

View File

@ -0,0 +1 @@
ALTER TABLE nodes ADD COLUMN IF NOT EXISTS rag_info jsonb default '{}';

View File

@ -0,0 +1,2 @@
ALTER TABLE node_releases DROP COLUMN IF EXISTS publisher_id;
ALTER TABLE node_releases DROP COLUMN IF EXISTS editor_id;

View File

@ -0,0 +1,2 @@
ALTER TABLE node_releases ADD COLUMN IF NOT EXISTS publisher_id text default '';
ALTER TABLE node_releases ADD COLUMN IF NOT EXISTS editor_id text default '';

View File

@ -0,0 +1,4 @@
-- Drop settings table
DROP TABLE IF EXISTS system_settings;
-- drop index
DROP INDEX IF EXISTS idx_system_settings_key;

View File

@ -0,0 +1,30 @@
-- Create settings table
CREATE TABLE IF NOT EXISTS system_settings (
id SERIAL PRIMARY KEY,
key TEXT NOT NULL,
value JSONB NOT NULL,
description TEXT,
created_at timestamptz NOT NULL DEFAULT NOW(),
updated_at timestamptz NOT NULL DEFAULT NOW()
);
CREATE UNIQUE INDEX idx_uniq_system_settings_key ON system_settings(key);
-- Insert model_setting_mode setting
-- If there are existing knowledge bases, set mode to 'manual', otherwise set to 'auto'
INSERT INTO system_settings (key, value, description)
SELECT
'model_setting_mode',
jsonb_build_object(
'mode', CASE
WHEN EXISTS (SELECT 1 FROM knowledge_bases LIMIT 1) THEN 'manual'
ELSE 'auto'
END,
'auto_mode_api_key', '',
'chat_model', '',
'is_manual_embedding_updated', false
),
'Model setting mode configuration'
WHERE NOT EXISTS (
SELECT 1 FROM system_settings WHERE key = 'model_setting_mode'
);

View File

@ -0,0 +1 @@
DROP TABLE IF EXISTS mcp_calls;

View File

@ -0,0 +1,11 @@
CREATE TABLE IF NOT EXISTS mcp_calls (
id SERIAL PRIMARY KEY,
mcp_session_id TEXT NOT NULL,
kb_id TEXT NOT NULL,
remote_ip TEXT,
initialize_req JSONB,
initialize_resp JSONB,
tool_call_req JSONB,
tool_call_resp TEXT,
created_at timestamptz NOT NULL DEFAULT NOW()
);

View File

@ -0,0 +1 @@
DROP TABLE IF EXISTS node_stats;

Some files were not shown because too many files have changed in this diff Show More