Author SHA1 Message Date
19904408334 73f296731c refactor(workflow): 隔离逻辑运行键为 node_group_id
将 checkpoint、异步任务与段结果缓存的唯一键从 execution_id 改为 node_group_id,区分换参重跑与续跑语义;恢复/续跑复用执行记录的组标识,换参重跑则换新组并软删旧组残留,消除软删墓碑导致同键重存失效的问题。
2026-09-04 17:36:46 +08:00
19904408334 8c6305f267 fix(workflow): 修复取消/中断漏扣费与终态提前落库问题
- launchExecution 失败时返回真实 execId,避免 wrapper 兜底失效
- summary 节点移除执行终态与计费写库,统一由 recordWorkflow 结算
- 更新数据库连接凭据
2026-09-04 13:29:47 +08:00
19904408334 f7957761f0 fix(workflow): 修复子流程批量执行缓存键冲突
递归识别子工作流中的视频模型以支持 per_second 计费,并为子流程批量子执行添加 SubFlowScope 隔离 async/segment 缓存键。
2026-09-03 16:50:18 +08:00
19904408334 fc629ee493 fix(ai-agent): 支持按条计费模式逐条上报视频时长 2026-09-03 15:48:46 +08:00
19904408334 d699f7ce14 feat(workflow): 增加工作流计费与执行生命周期管理
- 新增计费模块:执行开始建单、终态结算/取消/失败处理,支持按条/按秒/按token计费
- 新增执行生命周期跟踪:优雅关停时取消运行中执行并等待落库
- 新增异步任务等待/通知机制(Wait/Notify)
- 重构执行记录落库与进度上报,统一失败分类与重试语义
- 重命名文件:async_task.go→async.go、flow_checkpoint_store.go→exec_checkpoint.go、flow_graph_util.go→exec_record.go
- 更新 .gitignore 与数据库密码配置
2026-09-03 13:22:22 +08:00
19904408334 67d049e586 refactor(workflow): 统一 OSS 接口并重构恢复锁
- 将文件地址前缀与上传逻辑迁移至 common/oss
- 恢复执行改用 utils.WithLock 自动续期锁
- 转写提示词增加单镜头最小时长约束
2026-08-29 11:51:30 +08:00
19904408334 740003a192 build(ai-agent): 更新 common 和 gmq 依赖版本 2026-08-26 15:01:58 +08:00
19904408334 5c5f97af17 feat(workflow): 支持运行中执行事件中枢与优雅关停 2026-08-26 14:52:15 +08:00
19904408334 e0f7ce61a1 feat(workflow): main 启动工作流恢复扫描 2026-08-25 17:06:15 +08:00
19904408334 36b10ae8e7 fix(workflow): 补 reExecute 后再次 Insert 分支的 LastHeartbeat,消除 30s 僵尸双跑窗口 2026-08-25 17:04:49 +08:00
19904408334andClaude Opus 4.7 ce2407bb2d feat(workflow): 错误分类 + in-process 自动重试 + status=1 防重复执行
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 16:58:38 +08:00
19904408334 95dc0d6052 feat(workflow): 恢复例程 + 心跳 + 启动/周期扫描 2026-08-25 16:44:29 +08:00
19904408334 788c835580 feat(workflow): lambda 节点接入统一异步恢复入口 2026-08-25 16:28:28 +08:00
19904408334 e3c299412f feat(workflow): 统一异步调用 memo 包装 AsyncModelCallWithRecovery 2026-08-25 16:17:08 +08:00
19904408334andClaude Opus 4.7 b039a70547 feat(workflow): Redis 分布式锁工具(SETNX + Lua 原子释放)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 16:10:13 +08:00
19904408334andClaude Opus 4.7 3620e1ea61 refactor(gateway): ModelCallResult 拆分为 SubmitModelCall + WaitModelCallResult
- SubmitModelCall: 提交段(异步模型生成唯一 msgTopic),返回 (res, msgTopic, err)
- WaitModelCallResult: 订阅段(重订阅可恢复结果,供 Task5 AsyncModelCallWithRecovery 复用)
- ModelCallResult 签名不变,内部复用二者;订阅参数与拆分前一致

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 16:04:13 +08:00
19904408334andClaude Opus 4.7 5672c8272e feat(workflow): flow_async_task 统一异步任务表实体与 DAO(物理删除)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 15:55:03 +08:00
19904408334 d5b2d90a27 feat(workflow): exec_workflow 增加重试/心跳三列与可恢复查询 DAO 2026-08-25 15:41:50 +08:00
19904408334 b3f4b94b21 feat: 支持模板子流程递归拷贝与段级续跑优化
* 模板拷贝用户流程时递归复制子流程,并重写 sub_flow 节点引用
* 段级续跑改用列表位置作为段身份,移除对 segment_index 的依赖
* 段结果保存移到每段生成完成时立即落库,降低崩溃丢失风险
* 移除视频分段续跑设计与对应测试
2026-08-25 13:56:55 +08:00
19904408334 855d0cca72 fix: 表名常量对齐短名约定 + segment_index 去重守卫防视频批量节点续跑静默损坏 2026-08-25 10:59:09 +08:00
19904408334 01a73e5153 fix: DeleteByExecution 改为物理删除,消除软删后同键重存无法复活的续跑失效 2026-08-25 10:38:22 +08:00
19904408334andClaude Opus 4.7 d984cba714 test: 段结果 DAO 补 pgsql 驱动空导入并加软删复活路径验证
单测包不经 main,缺驱动空导入导致 go test ./workflow/dao/flow/ 报
"cannot find database driver for specified database type pgsql"。

新增 TestFlowSegmentResultDaoResurrect:验证 DeleteByExecution 软删后,
对同一 (execution_id, node_id, segment_index) 重新 Save 能否清除
deleted_at 使 ListByNode 再次可见。实测 upsert 的 DO UPDATE SET 子句
不含 deleted_at,段2 不复活 —— 留作失败证据供控制器裁决。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 10:34:14 +08:00
19904408334 bae916e019 feat: ModelLambda 段级续跑——只生成缺失段、失败重试1次、成功落库、按序合并 2026-08-25 10:20:47 +08:00
19904408334 2d7f6888a6 feat: 视频段级续跑纯逻辑(复用决策+按序合并)及单测 2026-08-25 10:16:54 +08:00
19904408334andClaude Opus 4.7 0de29988b4 feat: FlowExecutionInput 增加 ForceNewRun,BuildExecution 全新生前清旧段/成功后清理
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 10:09:58 +08:00
19904408334andClaude Opus 4.7 035c467a62 feat: 新增视频段级结果表及 DAO(flow_segment_result)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-25 10:01:48 +08:00
19904408334 ac7936829f chore: ignore .superpowers SDD workspace 2026-08-25 09:56:28 +08:00
19904408334 e0529c4ced docs: 补充段结果表执行成功后清理策略
工作流执行成功时(BuildExecution 成功返回)与删 checkpoint 并列清理该执行的
flow_segment_result 段结果,执行失败则保留供 reExecute 复用,避免数据残留。
2026-08-25 09:44:07 +08:00
19904408334 a357ca860f docs: 添加视频节点段级断点续跑设计文档
视频生成节点把长视频拆段生成,当前任意段失败即整节点失败,重跑全部段重新生成。
设计段级断点续跑:flow_segment_result 表存已成功段,执行内段失败重试 1 次,
reExecute 时只重生成失败段,按 segment_index 升序合并后拼接,保证顺序。
2026-08-25 09:37:15 +08:00
19904408334 8936a4c921 fix: 修复更新流程用户时的返回值和逻辑
- 调整 Update 接口返回创建结果 ID
- 修复管理员更新流程模板和普通用户更新流程的归属逻辑
2026-08-24 17:18:53 +08:00
19904408334 f0f8724bd9 fix: 修复并行分支续跑输出丢失及模型空指针
合并续跑时各分支独立的 ConfigMap 副本,避免汇合节点输出丢失;并对模型配置缺失和空 responseType 增加保护,防止空指针崩溃。
2026-08-24 16:04:45 +08:00
19904408334 75fb40274d fix: 修复工作流断点续跑时执行记录复用逻辑 2026-08-24 08:59:29 +08:00
19904408334 f34566d9ad fix: 修复模板字段取值逻辑并补充调试日志 2026-08-22 10:55:04 +08:00
19904408334 76b899133e fix: 添加httpCallResultLambda入参调试日志 2026-08-22 10:36:28 +08:00
19904408334 294b8b229e fix: 修复断连或取消时工作流断点保存失败 2026-08-22 09:58:56 +08:00
19904408334 c7091e3702 fix: 修复执行结果列表依赖结果表查询 2026-08-22 09:15:05 +08:00
19904408334 670fe61033 fix: 修复工作流执行记录卡在运行中的问题 2026-08-22 08:46:57 +08:00
19904408334 aae4ebe199 fix: 修复工作流执行状态比较与落库逻辑
- 解引用状态指针按值比较,避免执行记录误显示为运行中
- panic 时兜底写入失败状态,防止记录卡在 Running
- 保存上下文提前至异步执行入口,确保断连后仍能落库
- 移除结果文件地址前缀拼接,使用原始 URL
2026-08-22 08:28:04 +08:00
19904408334 5b93a7a507 fix: 修复工作流执行记录创建与静音模式约束
- 执行记录仅在合法 execId 时写入,查询失败时返回错误
- 字幕构建改为词级精确对齐并增加比例兜底,避免整句被吞
- 静音模式下从转写与段级 prompt 双重杜绝口播/字幕/口型
- 清理静音事件描述中的说话类动词
2026-08-21 19:38:25 +08:00
19904408334 a06766db6d fix: 复用失败工作流执行记录并统一节点触发模式 2026-08-21 13:30:07 +08:00
19904408334 9bc388018b fix: 重试成功后清空执行记录中的旧报错信息 2026-08-21 10:56:31 +08:00
19904408334 cef837a35c feat: 支持工作流断点续跑并拆分错误信息存储
- 新增同会话+同工作流最近执行失败且参数一致时断点续跑逻辑
- exec_workflow/exec_chat 新增 error 字段存储原始错误,error_message 仅存友好提示
- 新增 UpdateExecChatReq 与 exec_chat_dao Update 方法
- 新增 GetLatestBySessionAndFlow 查询最近执行记录
- 修正 ListDates 分组与排序 SQL 表达式
- 新增 pipeline 配置结构,删除旧设计文档
2026-08-21 09:54:06 +08:00
19904408334 a939c45508 feat: 新增执行记录实体与文件上传能力 2026-08-18 09:47:51 +08:00
19904408334 155a5cad0c fix: execWorkflowDao.Insert 形参改为 CreateWorkflowReq 2026-08-13 16:51:20 +08:00
19904408334 1c0f1a7707 feat: 会话 Get 按 exec_chat/exec_workflow 混排返回结果 2026-08-13 16:49:59 +08:00
19904408334 1dce62921f feat: 新增会话管理 service/controller 并注册路由 2026-08-13 13:04:00 +08:00
19904408334 ab7b8889ed feat: 新增会话/结果 DAO(含软删与级联删除) 2026-08-13 12:34:54 +08:00
19904408334 114563b4f0 feat: 新增会话管理 DTO 2026-08-13 12:30:35 +08:00
19904408334 358c5ca3a7 feat: 新增会话/工作流结果/普通对话结果表 DDL 2026-08-13 12:27:34 +08:00
19904408334 7ea3abbefe style: gofmt 对齐 table_name.go 常量块 2026-08-13 12:23:31 +08:00
19904408334 49674576ac feat: 新增会话/工作流结果/普通对话结果表实体 2026-08-13 12:22:09 +08:00
19904408334andClaude Opus 4.7 9b23f69d9c docs: 新增分镜→时间线→模型语言 pipeline 设计方案
独立纯函数包 video/pipeline/:重建时间线+回写对齐、prompt 实体名→token 替换、按可配置 schema 组装完整视频 API 请求体。参考 video-factory 实现,方案 B 从零设计。

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-08-11 11:29:14 +08:00
19904408334 cb9510c13b fix: 继续执行 2026-06-25 14:54:32 +08:00
19904408334 2f6f994730 fix: 继续执行 2026-06-25 14:53:45 +08:00
96 changed files with 11477 additions and 4093 deletions
+2
View File
@@ -1 +1,3 @@
/.idea/*
/.superpowers/
/docs/superpowers/
+13 -6
View File
@@ -7,8 +7,8 @@ database:
- type: "pgsql"
host: "192.168.0.83"
port: "15432"
user: "postgres"
pass: "Bjang09@686^*^"
user: "sql9f15b63fd203b36e"
pass: "1ec94b1acdaf57b66030242d418fad5a"
name: "digital-human"
prefix: "digital_human_" # (可选)表名前缀
role: "master" # (可选)数据库主从角色(master/slave),默认为master。如果不使用应用主从机制请不配置或留空即可。
@@ -28,8 +28,8 @@ database:
- type: "pgsql"
host: "192.168.0.83"
port: "15432"
user: "postgres"
pass: "Bjang09@686^*^"
user: "sql9f15b63fd203b36e"
pass: "1ec94b1acdaf57b66030242d418fad5a"
name: "black-deacon"
prefix: "black_deacon_" # (可选)表名前缀
role: "master"
@@ -64,5 +64,12 @@ consul:
jaeger:
addr: 192.168.0.83:4318
# 文件上传服务地址,与oss模块minio中的endpoint一致
filePrefix: "http://192.168.0.83:9000"
nats:
addr: 192.168.0.83
port: 4222
# 文件上传服务地址,cdn访问地址
filePrefix: "http://cdn.redpowerfuture.com"
# 文件上传服务地址,minio内网访问地址
minioPrefix: "http://192.168.0.83:9000"
+34
View File
@@ -0,0 +1,34 @@
package dto
import (
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
)
// TestModelStreamReq 测试模型流式调用请求(流式调用上游 → 全量一次返回)
type TestModelStreamReq struct {
g.Meta `path:"/testModelStream" method:"post" tags:"模型流测试" summary:"测试模型流式调用" dc:"透传调用model-gateway的流式接口(上游流式调用,缓冲后全量一次返回)"`
ModelName string `json:"modelName" v:"required#modelName不能为空" dc:"模型名称"`
BizName string `json:"bizName" dc:"业务名称"`
SessionId string `json:"sessionId" v:"required#sessionId不能为空" dc:"会话ID"`
RequestParams map[string]any `json:"requestParams" v:"required#请求参数不能为空" dc:"模型请求参数"`
}
// TestModelStreamAAReq 测试模型流式调用请求(流式调用上游 → SSE 分块返回)
type TestModelStreamAAReq struct {
g.Meta `path:"/testModelStreamAA" method:"post" tags:"模型流测试" summary:"测试模型流式调用AA" dc:"透传调用model-gateway的SSE流式接口(上游流式调用,逐分片SSE返回)"`
ModelName string `json:"modelName" v:"required#modelName不能为空" dc:"模型名称"`
BizName string `json:"bizName" dc:"业务名称"`
SessionId string `json:"sessionId" v:"required#sessionId不能为空" dc:"会话ID"`
RequestParams map[string]any `json:"requestParams" v:"required#请求参数不能为空" dc:"模型请求参数"`
}
// TestModelStreamRes 测试模型流式调用响应(实际通过RawWriter直接写出,该结构体仅占位)
type TestModelStreamRes struct {
*beans.ResponseEmpty
}
// TestModelStreamAARes 测试模型流式调用AA响应(实际通过RawWriter直接写出,该结构体仅占位)
type TestModelStreamAARes struct {
*beans.ResponseEmpty
}
+47
View File
@@ -0,0 +1,47 @@
package gateway
import (
"context"
"io"
"net/http"
"strings"
"gitea.redpowerfuture.com/red-future/common/oss"
"github.com/gogf/gf/v2/errors/gerror"
"github.com/gogf/gf/v2/frame/g"
)
// Upload 上传文件字节到 OSS,返回可访问 URL。
// 原签名基于 workflow/model/dto 的 UploadFileBytesReq/Res,抽离时简化为直接传文件名与字节,便于业务侧解耦 workflow。
// 统一走 common/ossmultipart field=file、X-User-Info 三态注入(透传请求头 / ctx 注入 user / 解析 token)。
func Upload(ctx context.Context, fileName string, fileBytes []byte) (string, error) {
res, err := oss.UploadFileBytes(ctx, fileName, fileBytes)
if err != nil {
return "", err
}
return res.FileURL, nil
}
// GetFileBytesFromURL 下载远程文件内容(把 filePrefix 前缀替换为 minioPrefix 后经 GoFrame 客户端下载)
func GetFileBytesFromURL(ctx context.Context, fileUrl string) ([]byte, error) {
newS := strings.ReplaceAll(fileUrl, g.Cfg().MustGet(ctx, "filePrefix").String(), g.Cfg().MustGet(ctx, "minioPrefix").String())
// 使用 GoFrame 客户端(自带超时、追踪、日志等能力)
resp, err := g.Client().Get(ctx, newS)
if err != nil {
return nil, gerror.Wrapf(err, "failed to request url: %s", newS)
}
defer resp.Close()
// 校验状态码
if resp.StatusCode != http.StatusOK {
return nil, gerror.Newf("request failed with status code: %d, url: %s", resp.StatusCode, newS)
}
// 读取全部内容
allBytes, err := io.ReadAll(resp.Body)
if err != nil {
return nil, gerror.Wrapf(err, "failed to read response body, url: %s", fileUrl)
}
return allBytes, nil
}
+209
View File
@@ -0,0 +1,209 @@
// Package gateway ai-agent 对外部服务的调用层:模型网关(model-gateway)的查询/提交/聊天调用,以及文件下载/上传(OSS)。
// 从 workflow/service/flow 抽离为独立目录,供工作流节点与视频管线(video/)共用,业务侧不再直接依赖 workflow 引擎。
package gateway
import (
"context"
"fmt"
"net/http"
"time"
"ai-agent/workflow/consts/model"
"ai-agent/workflow/consts/public"
"gitea.redpowerfuture.com/red-future/common/beans"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/utils"
gmq "github.com/bjang03/gmq/core/gmq"
"github.com/bjang03/gmq/mq"
"github.com/bjang03/gmq/types"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
)
// ListModelManageReq 配置列表
type ListModelManageReq struct {
*beans.Page `json:"page"`
ModelName string `p:"modelName" json:"modelName" dc:"模型名称(模糊查询,可选)"`
ModelType int `p:"modelType" json:"modelType" dc:"模型类型"`
}
// ModelManageListItem 模型配置列表项。listModelManage 返回平铺的 model-gateway entityid 在顶层),
// 与 getModelManage 的 {modelManage:{...}} 嵌套不同,故独立成结构而非复用 GetModelInfoByIdRes。
type ModelManageListItem struct {
Id int64 `json:"id" dc:"配置ID"`
ModelName string `json:"modelName" dc:"模型名称"`
}
type ListModelManageRes struct {
List []*ModelManageListItem `json:"list" dc:"列表数据"`
Total int `json:"total" dc:"总数"`
}
// GetModelInfoByIdReq 查询模型配置
type GetModelInfoByIdReq struct {
ModelId int64 `json:"id" dc:"模型ID"`
}
type GetModelInfoByIdRes struct {
ModelManage struct {
Id int64 `json:"id" dc:"模型ID"`
ModelType model.ModelType `json:"modelType" description:"模型类型"`
ModelName string `json:"modelName" description:"模型名称"`
MaxTokens int `json:"maxTokens" description:"最大token数"`
ChatModel *bool `json:"chatModel" description:"是否聊天模型"`
ResponseType model.ResponseType `json:"responseType" description:"返回类型:1同步,2异步,3流"`
Enabled *bool `json:"enabled" description:"是否启用"`
RequestBodyMapping map[string]any `json:"requestBodyMapping" description:"请求体映射(约等于视频模型 schema)"`
RequestBusinessFieldMapping map[string]string `json:"requestBusinessFieldMapping" description:"请求业务字段映射"`
MinDuration int `json:"minDuration" description:"最小时长(秒)"`
MaxDuration int `json:"maxDuration" description:"最大时长(秒)"`
} `json:"modelManage" dc:"模型配置"`
}
// modelCallTopic 生成异步模型消息主题:带业务标识(bizName/modelId/sessionId)便于排查,
// uuid 保证每次调用唯一,避免并发调用共用同一主题导致结果串扰
func modelCallTopic(bizName string, modelId int64, sessionId string) string {
return fmt.Sprintf("model-call-%s-%d-%s-%s", bizName, modelId, sessionId, uuid.NewString())
}
// ModelCallReq 调用模型网关
type ModelCallReq struct {
ModelId int64 `json:"modelId" dc:"模型ID"`
BizName string `json:"bizName" dc:"业务名称(调用方模块/系统,用于统计)"`
SessionId string `json:"sessionId" dc:"会话ID"`
RequestParams map[string]any `json:"requestParams" dc:"请求参数"`
BusinessParams map[string]any `json:"businessParams" dc:"业务参数"`
MsgTopic string `json:"msgTopic" dc:"消息主题(异步必要参数)"`
}
type ModelCallRes struct {
TaskId int64 `json:"id" dc:"任务ID"`
State int8 `json:"state" dc:"状态"`
ModelId int64 `json:"modelId" dc:"生效模型ID(引用行=解析后的系统模型ID,计价按此)"`
MediaType string `json:"mediaType" dc:"输入媒体类型(shop词汇: text/audio/video"`
TotalTokens int64 `json:"totalTokens" dc:"总token"`
PromptTokens int64 `json:"promptTokens" dc:"输入token"`
CompletionTokens int64 `json:"completionTokens" dc:"输出token"`
Tools []ModelTool `json:"tools" dc:"工具"`
Content map[string]any `json:"content" dc:"内容"`
Cost float64 `json:"cost" dc:"费用(元)"`
Duration int64 `json:"duration" dc:"时长(秒)"`
ErrorMsg string `json:"errorMsg" dc:"错误消息"`
}
type ModelTool struct {
Id string `json:"id" dc:"工具ID"`
Type string `json:"type" dc:"工具类型"`
Function struct {
Name string `json:"name"`
Arguments string `json:"arguments"`
} `json:"function"`
}
// ListModelManage 配置列表
func ListModelManage(ctx context.Context, req *ListModelManageReq) (res *ListModelManageRes, err error) {
res = new(ListModelManageRes)
err = commonHttp.Get(ctx, "model-gateway/model/manage/listModelManage", utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), res, req)
return
}
// GetModelInfoById 查询模型配置
func GetModelInfoById(ctx context.Context, req *GetModelInfoByIdReq) (res *GetModelInfoByIdRes, err error) {
res = new(GetModelInfoByIdRes)
err = commonHttp.Get(ctx, "model-gateway/model/manage/getModelManage", utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), res, req)
return
}
// SubmitModelCall 提交模型调用(异步模型自动生成唯一 msgTopic),返回提交响应与消息主题。
// taskId 在 res.TaskId;同步模型 msgTopic 为空串。结果等待由 WaitModelCallResult 完成。
func SubmitModelCall(ctx context.Context, modelId int64, responseType model.ResponseType, sessionId string, requestParams map[string]any, businessParams map[string]any) (res *ModelCallRes, msgTopic string, err error) {
// 异步模型必须绑定消息主题接收结果:自动生成唯一主题,
// 带业务标识(bizName/modelId/sessionId)便于排查,每次调用唯一避免并发串结果
if responseType != nil && *responseType == *model.ResponseTypeAsync.Code() {
msgTopic = modelCallTopic(g.Cfg().MustGet(ctx, "server.name").String(), modelId, sessionId)
}
req := ModelCallReq{
ModelId: modelId,
BizName: g.Cfg().MustGet(ctx, "server.name").String(),
SessionId: sessionId,
RequestParams: requestParams,
BusinessParams: businessParams,
MsgTopic: msgTopic,
}
// 克隆 commonHttp 客户端(保留 Consul 服务发现),显式设置超时和 ResponseHeaderTimeout
client := commonHttp.Httpclient.Clone()
client.SetTimeout(30 * time.Minute)
if tr, ok := client.Transport.(*http.Transport); ok {
tr.ResponseHeaderTimeout = 30 * time.Minute
}
res = new(ModelCallRes)
err = commonHttp.Post(ctx, "model-gateway/model/call/modelCall", utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), res, &req)
if err != nil {
return nil, "", err
}
if g.IsEmpty(res.TaskId) || !g.IsEmpty(res.ErrorMsg) {
return nil, "", fmt.Errorf("创建模型任务失败:%v", res.ErrorMsg)
}
return res, msgTopic, nil
}
// WaitModelCallResult 订阅 msgTopic 等待异步模型结果。
// 重订阅可拿回已发布结果:gmq NATS 侧 Retention=LimitsPolicy+MaxAge=7天(消息不随 ack 删除、保留 7 天),
// 订阅用 DeliverPolicy=DeliverAllPolicy(新订阅从 stream 序头重放),崩溃后重订阅同一 msgTopic 即可恢复。
func WaitModelCallResult(ctx context.Context, msgTopic string) (responseParams *ModelCallRes, err error) {
resultCh := make(chan *ModelCallRes, 1)
errCh := make(chan error, 1)
_, err = gmq.GetGmq(public.GmqMsgPluginsName).GmqSubscribe(ctx, &mq.NatsSubMessage{
SubMessage: types.SubMessage{
Topic: msgTopic,
ConsumerName: fmt.Sprintf("model-call-result-%s", uuid.NewString()),
AutoAck: false,
AutoUnsubscribe: true,
FetchCount: 1,
HandleFunc: func(ctx context.Context, msg any) error {
r := new(ModelCallRes)
if err := gconv.Struct(msg, r); err != nil {
errCh <- err
return nil
}
if g.IsEmpty(r.TaskId) || !g.IsEmpty(r.ErrorMsg) {
errCh <- fmt.Errorf("创建模型任务失败:%v", r.ErrorMsg)
return nil
}
resultCh <- r
return nil
},
},
Durable: true,
})
if err != nil {
return
}
select {
case responseParams = <-resultCh:
case err = <-errCh:
return nil, err
case <-ctx.Done():
return nil, ctx.Err()
}
return
}
// ModelCallResult 调模型网关生成一段内容并等待结果(内部复用 SubmitModelCall + WaitModelCallResult)。
// 同步模型提交即返回;异步模型为"提交+等待"一体。
func ModelCallResult(ctx context.Context, modelId int64, responseType model.ResponseType, sessionId string, requestParams map[string]any, businessParams map[string]any) (responseParams *ModelCallRes, err error) {
res, msgTopic, err := SubmitModelCall(ctx, modelId, responseType, sessionId, requestParams, businessParams)
if err != nil {
return nil, err
}
if responseType != nil && *responseType == *model.ResponseTypeAsync.Code() {
return WaitModelCallResult(ctx, msgTopic)
}
return res, nil
}
+158
View File
@@ -0,0 +1,158 @@
package gateway
import (
"bufio"
"context"
"encoding/json"
"io"
"strings"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
// ModelCallStreamReq 调用模型网关流式(client 侧请求体)
type ModelCallStreamReq struct {
ModelId int64 `json:"modelId" dc:"模型ID"`
BizName string `json:"bizName" dc:"业务名称"`
SessionId string `json:"sessionId" dc:"会话ID"`
RequestParams map[string]any `json:"requestParams" dc:"请求参数(模板字段)"`
BusinessParams map[string]any `json:"businessParams" dc:"业务参数"`
}
// ModelCallStream 流式调用模型网关:POST /modelCallStream,逐 chunk 调 onChunk(文本增量转发),
// 流结束返回 ModelCallResContent 为累加全量文本({"respBody": ...}),Tools 为流末工具列表。
func ModelCallStream(ctx context.Context, modelId int64, sessionId string, requestParams, businessParams map[string]any, onChunk func(chunk map[string]any) error) (*ModelCallRes, error) {
req := ModelCallStreamReq{
ModelId: modelId,
BizName: g.Cfg().MustGet(ctx, "server.name").String(),
SessionId: sessionId,
RequestParams: requestParams,
BusinessParams: businessParams,
}
body, err := commonHttp.PostStream(ctx, "model-gateway/model/call/modelCallStream", utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), &req)
if err != nil {
return nil, err
}
defer body.Close()
res := new(ModelCallRes)
var contentBuf strings.Builder
scanErr := parseSSEStream(ctx, body, func(chunk map[string]any) error {
tools, done := processStreamChunk(chunk)
if done {
res.Tools = tools
// 流末 done 事件携带该次调用累计 token 与费用:纯工具调用步骤无文本 chunk,靠它取数(>0 才覆盖,兼容旧网关无 token/cost 的 done 事件)
if total := gconv.Int64(chunk["totalTokens"]); total > 0 {
res.TotalTokens = total
res.PromptTokens = gconv.Int64(chunk["promptTokens"])
res.CompletionTokens = gconv.Int64(chunk["completionTokens"])
}
if cost := gconv.Float64(chunk["cost"]); cost > 0 {
res.Cost = cost
}
return nil
}
if onChunk != nil {
if err := onChunk(chunk); err != nil {
return err
}
}
contentBuf.WriteString(streamTextDelta(chunk))
// 网关端每个 chunk 携带累计 token 值,取最后一个文本 chunk 即为该步总消耗
res.TotalTokens = gconv.Int64(chunk["totalTokens"])
res.PromptTokens = gconv.Int64(chunk["promptTokens"])
res.CompletionTokens = gconv.Int64(chunk["completionTokens"])
res.Cost = gconv.Float64(chunk["cost"])
return nil
})
if scanErr != nil {
// 流被中断(如 ctx 取消)时仍返回已累计的 res,调用方可读取已产生的 token 正常落库
return res, scanErr
}
if contentBuf.Len() > 0 {
res.Content = map[string]any{"respBody": contentBuf.String()}
}
return res, nil
}
// processStreamChunk 处理单个流式 chunk。done 事件({"type":"done","tools":[...]})返回解析出的
// tools(可为空)且 done=true;其余为文本增量 chunk,返回 done=false。
func processStreamChunk(chunk map[string]any) (tools []ModelTool, done bool) {
if typ, _ := chunk["type"].(string); typ == "done" {
raw, _ := chunk["tools"].([]any)
for _, t := range raw {
m := gconv.Map(t)
if m == nil {
continue
}
var tool ModelTool
tool.Id = gconv.String(m["id"])
tool.Type = gconv.String(m["type"])
if fn := gconv.Map(m["function"]); fn != nil {
tool.Function.Name = gconv.String(fn["name"])
tool.Function.Arguments = gconv.String(fn["arguments"])
}
tools = append(tools, tool)
}
return tools, true
}
return nil, false
}
// streamTextDelta 提取文本增量 chunk 的文本(读 content 子对象,字段名由网关端结构体统一管理)
func streamTextDelta(chunk map[string]any) string {
var sb strings.Builder
if content, ok := chunk["content"].(map[string]any); ok {
for _, v := range content {
if s := gconv.String(v); s != "" {
sb.WriteString(s)
}
}
}
return sb.String()
}
// parseSSEStream 标准 SSE 解析:逐 data 行 JSON 回调,支持 [DONE] 与上下文取消。
// 对齐 model-gateway ParseSSEStream 语义:跳过空行、[DONE]、非 data 行。
func parseSSEStream(ctx context.Context, reader io.Reader, onChunk func(chunk map[string]any) error) error {
scanner := bufio.NewScanner(reader)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
var sb strings.Builder
for scanner.Scan() {
select {
case <-ctx.Done():
return ctx.Err()
default:
}
line := scanner.Text()
trimmed := strings.TrimSpace(line)
if trimmed == "" {
if sb.Len() == 0 {
continue
}
data := sb.String()
sb.Reset()
if data == "[DONE]" {
continue
}
var chunk map[string]any
if err := json.Unmarshal([]byte(data), &chunk); err != nil {
continue
}
if onChunk != nil {
if err := onChunk(chunk); err != nil {
return err
}
}
continue
}
if strings.HasPrefix(line, "data:") {
sb.WriteString(strings.TrimSpace(strings.TrimPrefix(line, "data:")))
}
}
return scanner.Err()
}
+24 -14
View File
@@ -3,7 +3,8 @@ module ai-agent
go 1.26.0
require (
gitea.redpowerfuture.com/red-future/common v0.0.24
gitea.redpowerfuture.com/red-future/common v0.0.33
github.com/bjang03/gmq v0.0.3
github.com/cloudwego/eino v0.9.5
github.com/cloudwego/eino-examples v0.0.0-20260611092511-bd64846fbc1d
github.com/cloudwego/eino-ext/components/model/qwen v0.1.9
@@ -11,10 +12,9 @@ require (
github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.2
github.com/gogf/gf/v2 v2.10.2
github.com/google/uuid v1.6.0
github.com/stretchr/testify v1.11.1
github.com/tidwall/gjson v1.19.0
github.com/tidwall/sjson v1.2.5
go.opentelemetry.io/otel/trace v1.44.0
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674
github.com/tidwall/gjson v1.18.0
github.com/tiger1103/gfast-token v1.0.10
)
require (
@@ -31,7 +31,6 @@ require (
github.com/clbanning/mxj/v2 v2.7.0 // indirect
github.com/cloudwego/base64x v0.1.6 // indirect
github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/dgraph-io/badger/v4 v4.2.0 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
@@ -41,9 +40,13 @@ require (
github.com/evanphx/json-patch v0.5.2 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.13 // indirect
github.com/go-ego/gse v1.0.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.30.1 // indirect
github.com/gogf/gf/contrib/registry/consul/v2 v2.9.5 // indirect
github.com/gogf/gf/contrib/trace/otlphttp/v2 v2.9.5 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
@@ -54,7 +57,6 @@ require (
github.com/golang/snappy v1.0.0 // indirect
github.com/google/flatbuffers v1.12.1 // indirect
github.com/goph/emperror v0.17.2 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
github.com/grokify/html-strip-tags-go v0.1.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect
github.com/hashicorp/consul/api v1.26.1 // indirect
@@ -67,8 +69,9 @@ require (
github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/serf v0.10.1 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.18.2 // indirect
github.com/klauspost/compress v1.18.4 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/lib/pq v1.10.9 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
@@ -80,21 +83,25 @@ require (
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/nats-io/nats.go v1.49.0 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
github.com/nikolalohinski/gonja v1.5.3 // indirect
github.com/olekukonko/errors v1.1.0 // indirect
github.com/olekukonko/ll v0.0.9 // indirect
github.com/olekukonko/tablewriter v1.1.0 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/pkg/errors v0.9.2-0.20201214064552-5dd12d0cfe7f // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/r3labs/diff/v2 v2.15.1 // indirect
github.com/redis/go-redis/v9 v9.17.2 // indirect
github.com/rabbitmq/amqp091-go v1.10.0 // indirect
github.com/redis/go-redis/v9 v9.18.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/slongfield/pyfmt v0.0.0-20220222012616-ea85ff4c361f // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tiger1103/gfast-token v1.0.10 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/vcaesar/cedar v0.30.0 // indirect
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
@@ -108,12 +115,15 @@ require (
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 // indirect
go.opentelemetry.io/otel/metric v1.44.0 // indirect
go.opentelemetry.io/otel/sdk v1.38.0 // indirect
go.opentelemetry.io/otel/trace v1.44.0 // indirect
go.opentelemetry.io/proto/otlp v1.7.1 // indirect
go.uber.org/atomic v1.11.0 // indirect
golang.org/x/arch v0.19.0 // indirect
golang.org/x/crypto v0.48.0 // indirect
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 // indirect
golang.org/x/net v0.48.0 // indirect
golang.org/x/sys v0.39.0 // indirect
golang.org/x/text v0.32.0 // indirect
golang.org/x/net v0.49.0 // indirect
golang.org/x/sys v0.41.0 // indirect
golang.org/x/text v0.34.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250825161204-c5933d9347a5 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5 // indirect
+52 -28
View File
@@ -1,8 +1,6 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
gitea.redpowerfuture.com/red-future/common v0.0.23 h1:xieoA00iKOCDm5SO9iXn+cSyMKBAlZwI0fuEVPWrHLg=
gitea.redpowerfuture.com/red-future/common v0.0.23/go.mod h1:50U1Xi+Ie56z09S5LQbZvaken0Mxv3OeS9LgR7U/ZRY=
gitea.redpowerfuture.com/red-future/common v0.0.24 h1:sXxhnmDmCgn+KwH/3gDnhAtAQ7FCmf/5AsMfvxRmri0=
gitea.redpowerfuture.com/red-future/common v0.0.24/go.mod h1:50U1Xi+Ie56z09S5LQbZvaken0Mxv3OeS9LgR7U/ZRY=
gitea.redpowerfuture.com/red-future/common v0.0.33 h1:AhWJ6l9zrjc1U0UEfyIZu8wkkVFxNe0hfuA51vOnOIo=
gitea.redpowerfuture.com/red-future/common v0.0.33/go.mod h1:FtI9KJJSKo4/K0emjVkbL8yoSIPHJdZXr27vnScQpmM=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
@@ -25,6 +23,8 @@ github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+Ce
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/bitly/go-simplejson v0.5.0/go.mod h1:cXHtHw4XUPsvGaxgjIAn8PhEWG9NfngEKAMDJEczWVA=
github.com/bjang03/gmq v0.0.3 h1:Yn9GZP1okOc8uh0f/1FFTooV5/mbO4pKrkcK9mTMjok=
github.com/bjang03/gmq v0.0.3/go.mod h1:Y7TwWGuV4Cw97WUDaM7x+NC4kyFx1z44WAvNwJV3HV8=
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
@@ -36,8 +36,6 @@ github.com/bugsnag/bugsnag-go v1.4.0/go.mod h1:2oa8nejYd4cQ/b0hMIopN0lCRxU0bueqR
github.com/bugsnag/panicwrap v1.2.0/go.mod h1:D/8v3kj0zr8ZAKg1AQ6crr+5VwKN5eIywRkfhyM/+dE=
github.com/bwmarrin/snowflake v0.3.0 h1:xm67bEhkKh6ij1790JB83OujPR5CzNe8QuQqAgISZN0=
github.com/bwmarrin/snowflake v0.3.0/go.mod h1:NdZxfVWX+oR6y2K0o6qAYv6gIOP9rjG0/E9WsDpxqwE=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/gopkg v0.1.4 h1:oZnQwnX82KAIWb7033bEwtxvTqXcYMxDBaQxo5JJHWM=
github.com/bytedance/gopkg v0.1.4/go.mod h1:v1zWfPm21Fb+OsyXN2VAHdL6TBb2L88anLQgdyje6R4=
github.com/bytedance/mockey v1.3.0 h1:ONLRdvhqmCfr9rTasUB8ZKCfvbdD2tohOg4u+4Q/ed0=
@@ -68,8 +66,6 @@ github.com/cloudwego/eino-ext/components/model/qwen v0.1.9 h1:xCz/mp43JeWqupjPR3
github.com/cloudwego/eino-ext/components/model/qwen v0.1.9/go.mod h1:slTGTuhzkzhNavf+1UtUg1FvUSA31iNAF+rq1mT4SnI=
github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17 h1:EeVcR1TslRA2IdNW1h/2LaGbPlffwGhQm99jM3zWZiI=
github.com/cloudwego/eino-ext/libs/acl/openai v0.1.17/go.mod h1:Zkcx6DPTR2NfWmtSXbhItswGw6hqUezNPhNcke0pOG8=
github.com/cloudwego/hertz v0.10.5 h1:N4oBqAJShSjYQm2Jfr0ryTzzJ9fnY9qSvIvUBMkoFWg=
github.com/cloudwego/hertz v0.10.5/go.mod h1:Im9u6rUa1v2mL2HiDKKJoof/CPQ3mPBBpT92v67Cetg=
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -101,9 +97,13 @@ github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.13 h1:46nXokslUBsAJE/wMsp5gtO500a4F3Nkz9Ufpk2AcUM=
github.com/gabriel-vasile/mimetype v1.4.13/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ=
github.com/go-check/check v0.0.0-20180628173108-788fd7840127 h1:0gkP6mzaMqkmpcJYCFOLkIBwI7xFExG03bbkOkCvUPI=
github.com/go-check/check v0.0.0-20180628173108-788fd7840127/go.mod h1:9ES+weclKsC9YodN5RgxqK/VD9HM9JsCSh7rNhMZE98=
@@ -118,6 +118,14 @@ github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.30.1 h1:f3zDSN/zOma+w6+1Wswgd9fLkdwy06ntQJp0BBvFG0w=
github.com/go-playground/validator/v10 v10.30.1/go.mod h1:oSuBIQzuJxL//3MelwSLD5hc2Tu889bF0Idm9Dg26cM=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
github.com/gogf/gf/contrib/drivers/pgsql/v2 v2.10.2 h1:u8EpP24GkprogROnJ7htMov9Fc66pTP1eVYrWxiCYOs=
@@ -239,8 +247,8 @@ github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7V
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0/go.mod h1:1NbS8ALrpOvjt0rHPNLyCIeMtbizbir8U//inJ+zuB8=
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.18.2 h1:iiPHWW0YrcFgpBYhsA6D1+fqHssJscY/Tm/y2Uqnapk=
github.com/klauspost/compress v1.18.2/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/compress v1.18.4 h1:RPhnKRAQ4Fh8zU2FY/6ZFDwTVTxgJ/EMydqSTzE9a2c=
github.com/klauspost/compress v1.18.4/go.mod h1:R0h/fSBs8DE4ENlcrlib3PsXS61voFxhIs2DeRhCvJ4=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
@@ -252,6 +260,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
@@ -298,6 +308,12 @@ github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjY
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee h1:W5t00kpgFdJifH4BDsTlE89Zl93FEloxaWZfGcifgq8=
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nats-io/nats.go v1.49.0 h1:yh/WvY59gXqYpgl33ZI+XoVPKyut/IcEaqtsiuTJpoE=
github.com/nats-io/nats.go v1.49.0/go.mod h1:fDCn3mN5cY8HooHwE2ukiLb4p4G4ImmzvXyJt+tGwdw=
github.com/nats-io/nkeys v0.4.15 h1:JACV5jRVO9V856KOapQ7x+EY8Jo3qw1vJt/9Jpwzkk4=
github.com/nats-io/nkeys v0.4.15/go.mod h1:CpMchTXC9fxA5zrMo4KpySxNjiDVvr8ANOSZdiNfUrs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/nikolalohinski/gonja v1.5.3 h1:GsA+EEaZDZPGJ8JtpeGN78jidhOlxeJROpqMT9fTj9c=
github.com/nikolalohinski/gonja v1.5.3/go.mod h1:RmjwxNiXAEqcq1HeK5SSMmqFJvKOfTfXhkJv6YBtPa4=
github.com/olekukonko/errors v1.1.0 h1:RNuGIh15QdDenh+hNvKrJkmxxjV4hcS50Db478Ou5sM=
@@ -338,8 +354,10 @@ github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsT
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/r3labs/diff/v2 v2.15.1 h1:EOrVqPUzi+njlumoqJwiS/TgGgmZo83619FNDB9xQUg=
github.com/r3labs/diff/v2 v2.15.1/go.mod h1:I8noH9Fc2fjSaMxqF3G2lhDdC0b+JXCfyx85tWFM9kc=
github.com/redis/go-redis/v9 v9.17.2 h1:P2EGsA4qVIM3Pp+aPocCJ7DguDHhqrXNhVcEp4ViluI=
github.com/redis/go-redis/v9 v9.17.2/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
github.com/rabbitmq/amqp091-go v1.10.0 h1:STpn5XsHlHGcecLmMFCtg7mqq0RnD+zFr4uzukfVhBw=
github.com/rabbitmq/amqp091-go v1.10.0/go.mod h1:Hy4jKW5kQART1u+JkDTF9YYOQUHXqMuhrgxOEeS7G4o=
github.com/redis/go-redis/v9 v9.18.0 h1:pMkxYPkEbMPwRdenAzUNyFNrDgHx9U+DrBabWNfSRQs=
github.com/redis/go-redis/v9 v9.18.0/go.mod h1:k3ufPphLU5YXwNTUcCRXGxUoF1fqxnhFQmscfkCoDA0=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
@@ -359,6 +377,8 @@ github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGB
github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
@@ -379,8 +399,8 @@ github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.19.0 h1:xwxm7n691Uf3u5OFjzngavjGTh55KX5q/9w9xHW88JU=
github.com/tidwall/gjson v1.19.0/go.mod h1:V37/opeE/JbLUOfH0QTXiNez2l0RUjYUhpT4szFQAfc=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
@@ -407,6 +427,8 @@ github.com/yargevad/filepathx v1.0.0 h1:SYcT+N3tYGi+NvazubCNlvgIPbzAk7i7y2dwg3I5
github.com/yargevad/filepathx v1.0.0/go.mod h1:BprfX/gpYNJHJfc35GjRRpVcwWXS89gGulUIU5tK3tA=
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M=
@@ -429,6 +451,8 @@ go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.opentelemetry.io/proto/otlp v1.7.1 h1:gTOMpGDb0WTBOP8JaO72iL3auEZhVmAQg4ipjOVAtj4=
go.opentelemetry.io/proto/otlp v1.7.1/go.mod h1:b2rVh6rfI/s2pHWNlB7ILJcRALpcNDzKhACevjI+ZnE=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.5.0 h1:KAMbZvZPyBPWgD14IrIQ38QCyjwpvVVV6K/bHl1IwQU=
@@ -440,8 +464,8 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3H3cr1v9wB50oz8l4C4h62xy7jSTY=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 h1:R9PFI6EUdfVKgwKjZef7QIwGcBKu86OEFpJ9nUEP2l4=
golang.org/x/exp v0.0.0-20250718183923-645b1fa84792/go.mod h1:A+z0yzpGtvnG90cToK5n2tu8UJVP2XUATh+r+sfOOOc=
@@ -450,8 +474,8 @@ golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvx
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.30.0 h1:fDEXFVZ/fmCKProc/yAXXUijritrDzahmwwefnjoPFk=
golang.org/x/mod v0.30.0/go.mod h1:lAsf5O2EvJeSFMiBxXDki7sCgAxEUcZHXoXMKT4GJKc=
golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c=
golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -468,8 +492,8 @@ golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwY
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8=
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -506,17 +530,17 @@ golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.38.0 h1:PQ5pkm/rLO6HnxFR7N2lJHOZX6Kez5Y1gDSJla6jo7Q=
golang.org/x/term v0.38.0/go.mod h1:bSEAKrOT1W+VSu9TSCMtoGEOUcKxOKgl3LE5QEF/xVg=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg=
golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
@@ -526,8 +550,8 @@ golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtn
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
golang.org/x/tools v0.39.0 h1:ik4ho21kwuQln40uelmciQPp9SipgNDdrafrYA4TmQQ=
golang.org/x/tools v0.39.0/go.mod h1:JnefbkDPyD8UU2kI5fuf8ZX4/yUeh9W877ZeBONxUqQ=
golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc=
golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
+56 -3
View File
@@ -2,23 +2,43 @@ package main
import (
digitalhumanController "ai-agent/digital-human/controller"
"ai-agent/workflow/service/flow"
// 空导入:加载内置模型工具与工作流处理器(组合根统一装载,业务侧不感知实现位置)
_ "ai-agent/tools/builtin/current_time"
"ai-agent/workflow/consts/public"
workController "ai-agent/workflow/controller"
workflowController "ai-agent/workflow/controller/flow"
workflowNodeController "ai-agent/workflow/controller/node"
sessionController "ai-agent/workflow/controller/session"
workflowSkillController "ai-agent/workflow/controller/skill"
toolController "ai-agent/workflow/controller/tool"
_ "ai-agent/workflow/service/flow/processor/builtin/media"
_ "ai-agent/workflow/service/flow/processor/builtin/split_batch"
_ "ai-agent/workflow/service/flow/processor/builtin/split_segment"
_ "ai-agent/workflow/service/flow/processor/builtin/split_shots_pipeline"
"context"
"os"
"os/signal"
"syscall"
"time"
"gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/jaeger"
gmq "github.com/bjang03/gmq/core/gmq"
"github.com/bjang03/gmq/mq"
_ "github.com/gogf/gf/contrib/drivers/pgsql/v2"
_ "github.com/gogf/gf/contrib/nosql/redis/v2"
"github.com/gogf/gf/v2/frame/g"
)
func main() {
ctx := context.Background()
defer jaeger.ShutDown(ctx)
// 注册路由
// 注册HTTP路由
http.Httpserver.BindHandler("/httpNodeCallback", workflowController.FlowCallBack.HttpNodeCallback)
http.RouteRegister([]interface{}{
//digitalhuman相关接口
digitalhumanController.Audio, // 语音相关接口
@@ -34,12 +54,45 @@ func main() {
workflowNodeController.NodePrompt,
workflowSkillController.SkillTemplate,
workflowSkillController.SkillUser,
toolController.Tool,
sessionController.Session,
})
//workflow.ExternalInterruptDemo()
//err := activePullService.ActivePullService.AllList(ctx)
//if err != nil {
// g.Log().Error(ctx, "ActivePullService err: %v", err)
//}
// 保持应用运行
select {}
gmq.GmqRegister(public.GmqMsgPluginsName, &mq.NatsConn{
NatsConfig: mq.NatsConfig{
Addr: g.Config().MustGet(ctx, "nats.addr").String(),
Port: g.Config().MustGet(ctx, "nats.port").String(),
Username: g.Config().MustGet(ctx, "nats.username").String(),
Password: g.Config().MustGet(ctx, "nats.password").String(),
},
})
// 启动工作流恢复扫描(首次立即扫 + 周期扫,多节点靠 Redis 锁去重)
flow.StartRecoveryLoop(ctx)
// 监听退出信号,执行优雅关闭
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM, syscall.SIGQUIT)
shutdownDone := make(chan struct{})
go func() {
<-sigCh
// 先置关停标记再 Close:Close 会取消各连接 ctx,正在执行的 exec 会收到 context.Canceled
// 标记让错误分类识别为"程序关停中断"retryable=1),下次启动恢复扫描捞起续跑,
// 而非误判为"用户已终止执行"retryable=0,永不恢复)
// SetShuttingDown 同时取消所有运行中执行(含脱离连接的恢复执行),保证全部落终态
flow.SetShuttingDown()
flow.SessionWsService.Close()
// 等待运行中执行落完终态再退出,避免进程退出时 exec 仍卡 status=1
// 结束后返回 main 触发 deferred 资源清理,进程自然退出(不再 select{} 挂死)
flow.WaitExecRunsDrain(15 * time.Second)
close(shutdownDone)
}()
// 保持应用运行;收到关停信号并落完终态后自然退出
<-shutdownDone
}
@@ -0,0 +1,32 @@
// Package current_time 内置示例工具:返回服务器当前时间。
// 仅用于验证独立工具对话入口(/tool/agent 将全部注册工具交给模型 function calling),可按需移除。
package current_time
import (
"context"
"time"
"gitea.redpowerfuture.com/red-future/common/tools"
)
func init() {
tools.Register(CurrentTimeTool())
}
// CurrentTimeTool 返回服务器当前时间,作为 model 作用域示例工具
func CurrentTimeTool() *tools.Tool {
return &tools.Tool{
Name: "current_time",
Description: "返回服务器当前时间。当用户询问时间/日期时调用。",
Parameters: map[string]any{
"type": "object",
"properties": map[string]any{},
"required": false,
},
Func: func(ctx context.Context, args map[string]any) (tools.ToolResult, error) {
return tools.OK(map[string]any{
"now": time.Now().Format("2006-01-02 15:04:05"),
}), nil
},
}
}
+259
View File
@@ -0,0 +1,259 @@
// Package runner 工具框架的"使用面"ReAct 执行循环。
// 模型调用按模型响应类型分流:流式走 gateway.ModelCallStream(逐 chunk 推增量),同步/异步走 gateway.ModelCallResult
// 本包只保留"模型思考→调用工具→观察结果"的循环编排,工具的"使用"统一走 tools.Default.Call。
package runner
import (
"ai-agent/gateway"
"ai-agent/workflow/consts/model"
"context"
"encoding/json"
"errors"
"fmt"
"github.com/gogf/gf/v2/util/gconv"
"gitea.redpowerfuture.com/red-future/common/tools"
)
// ReActEventType ReAct 过程事件类型(WebSocket 场景逐条推送,HTTP 同步场景不设置 OnEvent 则忽略)
type ReActEventType string
const (
ReActEventRoundStart ReActEventType = "round_start"
ReActEventModelCall ReActEventType = "model_call" // 模型思考(step 开始)
ReActEventToolCall ReActEventType = "tool_call" // 模型请求调用工具
ReActEventToolResult ReActEventType = "tool_result" // 工具返回结果
ReActEventAnswer ReActEventType = "answer" // 最终回答
ReActEventAnswerChunk ReActEventType = "answer_chunk" // 回答内容增量(逐 chunk
ReActEventReasoningChunk ReActEventType = "reasoning_chunk" // 思考内容增量(逐 chunk
ReActEventError ReActEventType = "error"
)
// ReActEvent ReAct 过程事件,按发生顺序回调
type ReActEvent struct {
Type ReActEventType
Id int64
Step int
MaxStep int
Description string // 工具用途说明(tool_call/tool_result 推给前端展示,不暴露工具名/参数)
Answer string
Delta string // 流式文本增量(ReActEventAnswerChunk / ReActEventReasoningChunk 用)
Message string
Error string
}
// ReActAgent 实现 ReAct 模式的智能体:模型思考→调用工具→观察工具结果→重复→最终回答。
// 工具的"使用"统一走 tools.Default.Call:未知工具由 Server 返回结构化 not_found
// 无需在 runner 内维护工具查找表。
type ReActAgent struct {
ModelId int64
SessionId string
ToolList []*tools.Tool
SystemPrompt string
MaxStep int
// OnEvent 过程回调(WebSocket 流式推送用;HTTP 同步调用不设置,忽略事件)
OnEvent func(ReActEvent)
// TotalTokens 本次 ReAct 循环累计 token 消耗(供调用方落库)
TotalTokens int64
// TotalCost 本次 ReAct 循环累计费用(元,各步模型调用返回的 cost 求和,供调用方落库)
TotalCost float64
}
// NewReActAgent 创建 ReAct 智能体
func NewReActAgent(modelId int64, sessionId string, toolList []*tools.Tool, systemPrompt string, maxStep int) *ReActAgent {
return &ReActAgent{
ModelId: modelId,
SessionId: sessionId,
ToolList: toolList,
SystemPrompt: systemPrompt,
MaxStep: maxStep,
}
}
// Run 执行 ReAct 循环
// 标准流程: 思考 → 行动(调用工具) → 观察(工具结果) → 重复 → 最终回答
func (a *ReActAgent) Run(ctx context.Context, userInput string) (string, error) {
// 模型须为已启用且勾选「聊天模型」的对话模型,再按其响应类型分流调用:
// 流式走 ModelCallStream(逐 chunk 推增量),同步/异步走 ModelCallResult(一次返回)
modelInfo, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: a.ModelId})
if err != nil {
return "", fmt.Errorf("获取模型配置失败: %w", err)
}
if modelInfo.ModelManage.Enabled == nil || !*modelInfo.ModelManage.Enabled {
return "", fmt.Errorf("模型 [%s] 未启用", modelInfo.ModelManage.ModelName)
}
if modelInfo.ModelManage.ChatModel == nil || !*modelInfo.ModelManage.ChatModel {
return "", fmt.Errorf("模型 [%s] 不是对话模型,请在模型配置中勾选「聊天模型」", modelInfo.ModelManage.ModelName)
}
isStream := *modelInfo.ModelManage.ResponseType == *model.ResponseTypeStream.Code()
messages := map[string]any{
"user_prompt": userInput,
"system_prompt": a.SystemPrompt,
"tools": a.rawTools(a.ToolList),
}
for step := 0; step < a.MaxStep; step++ {
a.emit(ReActEvent{Type: ReActEventModelCall, Step: step + 1, MaxStep: a.MaxStep})
var result *gateway.ModelCallRes
if isStream {
result, err = gateway.ModelCallStream(ctx, a.ModelId, a.SessionId, nil, messages, func(chunk map[string]any) error {
// 文本增量从 content 子对象取,字段名由网关端结构体统一管理
if content, ok := chunk["content"].(map[string]any); ok {
for _, v := range content {
if s := gconv.String(v); s != "" {
a.emit(ReActEvent{Type: ReActEventAnswerChunk, Delta: s})
}
}
}
// 思考内容增量从 reasoningContent 字段取(model-gateway 按 ResponseBusinessFieldMapping 配置返回)
if s := gconv.String(chunk["reasoningContent"]); s != "" {
a.emit(ReActEvent{Type: ReActEventReasoningChunk, Delta: s})
}
return nil
})
} else {
// 同步/异步对话模型:结果一次返回,无增量事件(异步 msgTopic 为空时由 gateway.ModelCallResult 自动生成)
result, err = gateway.ModelCallResult(ctx, a.ModelId, modelInfo.ModelManage.ResponseType, a.SessionId, nil, messages)
}
// 出错时 result 仍可能携带已累计的 token/cost(流中断返回 res),先累加再判错
if result != nil {
a.TotalTokens += result.TotalTokens
a.TotalCost += result.Cost
}
if err != nil {
// 前端终止/上下文取消:静默停止,不推错误事件
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return "", err
}
a.emit(ReActEvent{Type: ReActEventError, Message: "模型调用失败", Error: err.Error()})
return "", fmt.Errorf("step %d: model call failed: %w", step, err)
}
var content string
var toolCalls []gateway.ModelTool
if isStream {
for _, v := range gconv.Map(result.Content) {
content += v.(string)
}
toolCalls = result.Tools
} else {
// 同步响应 Content 按 responseBodyMapping 抽取(content/toolCalls/finishReason),工具在 toolCalls 里
content, toolCalls = syncChatResult(result.Content)
}
// 无工具调用 → 最终回答
if len(toolCalls) == 0 {
return content, nil
}
messages["assistant_prompt"] = content
for _, tool := range toolCalls {
fn := gconv.Map(tool.Function)
name := gconv.String(fn["name"])
arguments := gconv.String(fn["arguments"])
desc := a.toolDescription(name)
a.emit(ReActEvent{Type: ReActEventToolCall, Description: desc})
// 执行每个工具调用
toolMsg := a.executeToolCall(ctx, name, arguments)
a.emit(ReActEvent{Type: ReActEventToolResult, Description: desc})
messages["tool_prompt"] = toolMsg
messages["tool_id"] = tool.Id
}
}
err = fmt.Errorf("max steps reached %d, generation incomplete", a.MaxStep)
a.emit(ReActEvent{Type: ReActEventError, Message: "最大步数达到,生成失败", Error: err.Error()})
return "", err
}
// syncChatResult 解析同步对话响应 Contentmodel-gateway 按 responseBodyMapping 抽取,固定
// content/toolCalls/finishReason 三个 key):返回最终文本与工具调用列表(同步响应工具在 toolCalls 里,res.Tools 为空)。
func syncChatResult(content map[string]any) (string, []gateway.ModelTool) {
text := gconv.String(content["content"])
raw := content["toolCalls"]
if raw == nil {
raw = content["tools"]
}
var toolCalls []gateway.ModelTool
if arr, ok := raw.([]any); ok {
for _, t := range arr {
m := gconv.Map(t)
if m == nil {
continue
}
var tool gateway.ModelTool
tool.Id = gconv.String(m["id"])
tool.Type = gconv.String(m["type"])
if fn := gconv.Map(m["function"]); fn != nil {
tool.Function.Name = gconv.String(fn["name"])
tool.Function.Arguments = gconv.String(fn["arguments"])
}
toolCalls = append(toolCalls, tool)
}
}
return text, toolCalls
}
// emit 触发过程事件回调;未设置 OnEvent 时忽略(HTTP 同步调用路径)
func (a *ReActAgent) emit(ev ReActEvent) {
if a.OnEvent != nil {
a.OnEvent(ev)
}
}
// executeToolCall 执行单个工具调用,返回回灌给模型的工具结果消息。
// 未知工具由 tools.Default.Call 返回结构化 not_found;工具业务失败进 ToolResult.Code。
func (a *ReActAgent) executeToolCall(ctx context.Context, name, arguments string) string {
if arguments == "" {
return "工具参数为空"
}
var args map[string]any
if err := json.Unmarshal([]byte(arguments), &args); err != nil {
return fmt.Sprintf("参数解析失败: %v", err)
}
res, callErr := tools.Default.Call(ctx, name, args)
switch {
case callErr != nil:
msg := fmt.Sprintf("工具执行失败: %v", callErr)
return msg
case res.Code != 0:
return res.Message
default:
dataJSON, err := json.Marshal(res.Data)
if err != nil {
return fmt.Sprintf("工具结果序列化失败: %v", err)
}
return string(dataJSON)
}
}
// toolDescription 按工具名从 ToolList 中查找用途说明(推给前端展示用,不暴露工具名/参数)
func (a *ReActAgent) toolDescription(name string) string {
for _, t := range a.ToolList {
if t.Name == name {
return t.Description
}
}
return ""
}
// RawTools 将工具定义转为 OpenAI 原始数组,parameters 保持 JSON Schema 原样。
// businessParams 里传 tools 必须用它转换——直接传 []*tools.Tool 会被 json.Marshal 序列化 Func 字段而失败。
func (a *ReActAgent) rawTools(toolList []*tools.Tool) []any {
raw := make([]any, 0, len(toolList))
for _, t := range toolList {
params := t.Parameters
if params == nil {
params = map[string]any{}
}
raw = append(raw, map[string]any{
"type": "function",
"function": map[string]any{
"name": t.Name,
"description": t.Description,
"parameters": params,
},
})
}
return raw
}
+318 -1
View File
@@ -626,11 +626,15 @@ CREATE TABLE IF NOT EXISTS black_deacon_node_execution (
prompt_tokens INTEGER NOT NULL DEFAULT 0, -- 提示词token消耗
completion_tokens INTEGER NOT NULL DEFAULT 0, -- 补全token消耗
total_tokens INTEGER NOT NULL DEFAULT 0, -- 总token消耗
token_info JSONB DEFAULT '[]', -- 节点token明细(汇总节点聚合 total_tokens/total_fee 用)
status SMALLINT NOT NULL DEFAULT 1, -- 执行状态:1-运行中,2-成功,3-失败,4-暂停,5-等待执行
duration_ms BIGINT NOT NULL DEFAULT 0, -- 执行时长(毫秒)
error_message TEXT DEFAULT '' -- 错误信息
);
-- 存量库补列
ALTER TABLE black_deacon_node_execution ADD COLUMN IF NOT EXISTS token_info JSONB DEFAULT '[]';
-- 索引(高频查询)
CREATE INDEX IF NOT EXISTS idx_bne_tenant_id ON black_deacon_node_execution(tenant_id);
CREATE INDEX IF NOT EXISTS idx_bne_flow_execution_id ON black_deacon_node_execution(flow_execution_id);
@@ -658,7 +662,320 @@ COMMENT ON COLUMN black_deacon_node_execution.output_params_path IS '节点输
COMMENT ON COLUMN black_deacon_node_execution.prompt_tokens IS '提示词token消耗';
COMMENT ON COLUMN black_deacon_node_execution.completion_tokens IS '补全token消耗';
COMMENT ON COLUMN black_deacon_node_execution.total_tokens IS '总token消耗';
COMMENT ON COLUMN black_deacon_node_execution.token_info IS '节点token明细(汇总节点聚合 total_tokens/total_fee 用)';
COMMENT ON COLUMN black_deacon_node_execution.status IS '执行状态:1-运行中,2-成功,3-失败,4-暂停,5-等待执行';
COMMENT ON COLUMN black_deacon_node_execution.duration_ms IS '执行时长(毫秒)';
COMMENT ON COLUMN black_deacon_node_execution.error_message IS '错误信息';
--------------------pgsql创建black_deacon_node_execution表语句---------------------------
--------------------pgsql创建black_deacon_node_execution表语句---------------------------
-- ========== 会话记录 + 工作流/普通会话结果表(2026-08-13 ==========
--------------------pgsql创建black_deacon_session表语句---------------------------
-- 会话记录表
CREATE TABLE IF NOT EXISTS black_deacon_session (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY, -- 主键ID(非自增)
tenant_id BIGINT NOT NULL DEFAULT 0, -- 租户ID int8
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
session_id VARCHAR(64) NOT NULL, -- 会话ID(前端传入,雪花ID
session_name VARCHAR(128) NOT NULL DEFAULT '' -- 会话名称
);
-- 索引(高频查询)
CREATE UNIQUE INDEX IF NOT EXISTS uk_session_tenant_session_id ON black_deacon_session(tenant_id, session_id);
CREATE INDEX IF NOT EXISTS idx_session_tenant_id ON black_deacon_session(tenant_id);
CREATE INDEX IF NOT EXISTS idx_session_creator ON black_deacon_session(creator);
CREATE INDEX IF NOT EXISTS idx_session_deleted_at ON black_deacon_session(deleted_at);
-- 表和字段注释
COMMENT ON TABLE black_deacon_session IS '会话记录表';
COMMENT ON COLUMN black_deacon_session.id IS '主键ID(非自增)';
COMMENT ON COLUMN black_deacon_session.tenant_id IS '租户ID';
COMMENT ON COLUMN black_deacon_session.creator IS '创建人';
COMMENT ON COLUMN black_deacon_session.created_at IS '创建时间';
COMMENT ON COLUMN black_deacon_session.updater IS '更新人';
COMMENT ON COLUMN black_deacon_session.updated_at IS '更新时间';
COMMENT ON COLUMN black_deacon_session.deleted_at IS '删除时间(软删)';
COMMENT ON COLUMN black_deacon_session.session_id IS '会话ID(前端传入,雪花ID';
COMMENT ON COLUMN black_deacon_session.session_name IS '会话名称';
--------------------pgsql创建black_deacon_session表语句---------------------------
--------------------pgsql创建black_deacon_exec_chat表语句---------------------------
-- 普通会话执行记录表
CREATE TABLE IF NOT EXISTS black_deacon_exec_chat (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY,
tenant_id BIGINT NOT NULL DEFAULT 0,
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
session_id VARCHAR(64) NOT NULL DEFAULT '', -- 会话ID
duration BIGINT NOT NULL DEFAULT 0, -- 执行时长(秒)
request_params JSONB DEFAULT '{}', -- 请求参数
result_file_url VARCHAR(512) DEFAULT '', -- 结果文件路径(OSS
total_tokens INTEGER NOT NULL DEFAULT 0, -- 总token消耗
total_fee DOUBLE PRECISION NOT NULL DEFAULT 0, -- 总费用
error_message TEXT DEFAULT '', -- 错误信息(友好提示)
error TEXT DEFAULT '' -- 错误明细(原始错误)
);
-- 索引(高频查询)
CREATE INDEX IF NOT EXISTS idx_exec_chat_tenant_id ON black_deacon_exec_chat(tenant_id);
CREATE INDEX IF NOT EXISTS idx_exec_chat_session_id ON black_deacon_exec_chat(session_id);
CREATE INDEX IF NOT EXISTS idx_exec_chat_deleted_at ON black_deacon_exec_chat(deleted_at);
-- 表和字段注释
COMMENT ON TABLE black_deacon_exec_chat IS '普通会话执行记录表';
COMMENT ON COLUMN black_deacon_exec_chat.id IS '主键ID';
COMMENT ON COLUMN black_deacon_exec_chat.tenant_id IS '租户ID';
COMMENT ON COLUMN black_deacon_exec_chat.creator IS '创建人';
COMMENT ON COLUMN black_deacon_exec_chat.created_at IS '创建时间';
COMMENT ON COLUMN black_deacon_exec_chat.updater IS '更新人';
COMMENT ON COLUMN black_deacon_exec_chat.updated_at IS '更新时间';
COMMENT ON COLUMN black_deacon_exec_chat.deleted_at IS '删除时间(软删)';
COMMENT ON COLUMN black_deacon_exec_chat.session_id IS '会话ID';
COMMENT ON COLUMN black_deacon_exec_chat.duration IS '执行时长(秒)';
COMMENT ON COLUMN black_deacon_exec_chat.request_params IS '请求参数';
COMMENT ON COLUMN black_deacon_exec_chat.result_file_url IS '结果文件路径(OSS';
COMMENT ON COLUMN black_deacon_exec_chat.total_tokens IS '总token消耗';
COMMENT ON COLUMN black_deacon_exec_chat.total_fee IS '总费用';
COMMENT ON COLUMN black_deacon_exec_chat.error_message IS '错误信息(友好提示)';
COMMENT ON COLUMN black_deacon_exec_chat.error IS '错误明细(原始错误)';
-- 兼容已有库:错误信息拆分,error_message 存友好提示、error 存原始错误
ALTER TABLE black_deacon_exec_chat ADD COLUMN IF NOT EXISTS error TEXT DEFAULT '';
--------------------pgsql创建black_deacon_exec_chat表语句---------------------------
--------------------pgsql创建black_deacon_exec_workflow表语句---------------------------
-- 工作流执行记录表
CREATE TABLE IF NOT EXISTS black_deacon_exec_workflow (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY,
tenant_id BIGINT NOT NULL DEFAULT 0,
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
session_id VARCHAR(64) NOT NULL DEFAULT '', -- 会话ID
flow_id BIGINT NOT NULL DEFAULT 0, -- 工作流ID
duration BIGINT NOT NULL DEFAULT 0, -- 执行时长(秒)
request_params JSONB DEFAULT '{}', -- 请求参数
status SMALLINT NOT NULL DEFAULT 1, -- 状态:1-运行中,2-成功,3-失败
total_tokens INTEGER NOT NULL DEFAULT 0, -- 总token消耗
total_fee DOUBLE PRECISION NOT NULL DEFAULT 0, -- 总费用
error_message TEXT DEFAULT '', -- 错误信息(友好提示)
error TEXT DEFAULT '' -- 错误明细(原始错误)
);
-- 索引(高频查询)
CREATE INDEX IF NOT EXISTS idx_exec_workflow_tenant_id ON black_deacon_exec_workflow(tenant_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_session_id ON black_deacon_exec_workflow(session_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_flow_id ON black_deacon_exec_workflow(flow_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_status ON black_deacon_exec_workflow(status);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_deleted_at ON black_deacon_exec_workflow(deleted_at);
-- 表和字段注释
COMMENT ON TABLE black_deacon_exec_workflow IS '工作流执行记录表';
COMMENT ON COLUMN black_deacon_exec_workflow.id IS '主键ID';
COMMENT ON COLUMN black_deacon_exec_workflow.tenant_id IS '租户ID';
COMMENT ON COLUMN black_deacon_exec_workflow.creator IS '创建人';
COMMENT ON COLUMN black_deacon_exec_workflow.created_at IS '创建时间';
COMMENT ON COLUMN black_deacon_exec_workflow.updater IS '更新人';
COMMENT ON COLUMN black_deacon_exec_workflow.updated_at IS '更新时间';
COMMENT ON COLUMN black_deacon_exec_workflow.deleted_at IS '删除时间(软删)';
COMMENT ON COLUMN black_deacon_exec_workflow.session_id IS '会话ID';
COMMENT ON COLUMN black_deacon_exec_workflow.flow_id IS '工作流ID';
COMMENT ON COLUMN black_deacon_exec_workflow.duration IS '执行时长(秒)';
COMMENT ON COLUMN black_deacon_exec_workflow.request_params IS '请求参数';
COMMENT ON COLUMN black_deacon_exec_workflow.status IS '状态:1-运行中,2-成功,3-失败';
COMMENT ON COLUMN black_deacon_exec_workflow.total_tokens IS '总token消耗';
COMMENT ON COLUMN black_deacon_exec_workflow.total_fee IS '总费用';
COMMENT ON COLUMN black_deacon_exec_workflow.error_message IS '错误信息(友好提示)';
COMMENT ON COLUMN black_deacon_exec_workflow.error IS '错误明细(原始错误)';
-- 兼容已有库:错误信息拆分,error_message 存友好提示、error 存原始错误
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS error TEXT DEFAULT '';
--------------------pgsql创建black_deacon_exec_workflow表语句---------------------------
--------------------pgsql创建black_deacon_exec_workflow_result表语句---------------------------
-- 工作流执行结果表
CREATE TABLE IF NOT EXISTS black_deacon_exec_workflow_result (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY,
tenant_id BIGINT NOT NULL DEFAULT 0,
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
session_id VARCHAR(64) NOT NULL DEFAULT '', -- 会话ID
flow_id BIGINT NOT NULL DEFAULT 0, -- 工作流ID
exec_id BIGINT NOT NULL DEFAULT 0, -- 执行ID
result_file_url VARCHAR(512) DEFAULT '' -- 结果文件路径(OSS
);
-- 索引(高频查询)
CREATE INDEX IF NOT EXISTS idx_exec_workflow_result_tenant_id ON black_deacon_exec_workflow_result(tenant_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_result_session_id ON black_deacon_exec_workflow_result(session_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_result_exec_id ON black_deacon_exec_workflow_result(exec_id);
CREATE INDEX IF NOT EXISTS idx_exec_workflow_result_deleted_at ON black_deacon_exec_workflow_result(deleted_at);
-- 表和字段注释
COMMENT ON TABLE black_deacon_exec_workflow_result IS '工作流执行结果表';
COMMENT ON COLUMN black_deacon_exec_workflow_result.id IS '主键ID';
COMMENT ON COLUMN black_deacon_exec_workflow_result.tenant_id IS '租户ID';
COMMENT ON COLUMN black_deacon_exec_workflow_result.creator IS '创建人';
COMMENT ON COLUMN black_deacon_exec_workflow_result.created_at IS '创建时间';
COMMENT ON COLUMN black_deacon_exec_workflow_result.updater IS '更新人';
COMMENT ON COLUMN black_deacon_exec_workflow_result.updated_at IS '更新时间';
COMMENT ON COLUMN black_deacon_exec_workflow_result.deleted_at IS '删除时间(软删)';
COMMENT ON COLUMN black_deacon_exec_workflow_result.session_id IS '会话ID';
COMMENT ON COLUMN black_deacon_exec_workflow_result.flow_id IS '工作流ID';
COMMENT ON COLUMN black_deacon_exec_workflow_result.exec_id IS '执行ID';
COMMENT ON COLUMN black_deacon_exec_workflow_result.result_file_url IS '结果文件路径(OSS';
--------------------pgsql创建black_deacon_exec_workflow_result表语句---------------------------
--------------------pgsql创建black_deacon_flow_segment_result表语句---------------------------
-- 视频节点段级生成结果表(只存成功段,断点续跑复用;失败段不落库视为需重新生成)
CREATE TABLE IF NOT EXISTS black_deacon_flow_segment_result (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY,
tenant_id BIGINT NOT NULL DEFAULT 0,
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
execution_id BIGINT NOT NULL DEFAULT 0, -- 执行IDexec_workflowreExecute 复用同一条)
node_id VARCHAR(64) NOT NULL DEFAULT '', -- 视频生成节点ID
segment_index INT NOT NULL DEFAULT 0, -- 段序号
video_key VARCHAR(128) NOT NULL DEFAULT '', -- 视频输出字段key(重建输出记录保 key 一致,避免下游引用失配)
video_url VARCHAR(512) NOT NULL DEFAULT '' -- 已生成成功的视频地址
);
-- 唯一键 + 高频查询索引(ListByNode 命中 (execution_id, node_id) 前缀)
CREATE UNIQUE INDEX IF NOT EXISTS uk_segment_result_exec_node_idx ON black_deacon_flow_segment_result(execution_id, node_id, segment_index);
CREATE INDEX IF NOT EXISTS idx_segment_result_tenant_id ON black_deacon_flow_segment_result(tenant_id);
CREATE INDEX IF NOT EXISTS idx_segment_result_deleted_at ON black_deacon_flow_segment_result(deleted_at);
-- 表和字段注释
COMMENT ON TABLE black_deacon_flow_segment_result IS '视频节点段级生成结果表';
COMMENT ON COLUMN black_deacon_flow_segment_result.id IS '主键ID';
COMMENT ON COLUMN black_deacon_flow_segment_result.tenant_id IS '租户ID';
COMMENT ON COLUMN black_deacon_flow_segment_result.creator IS '创建人';
COMMENT ON COLUMN black_deacon_flow_segment_result.created_at IS '创建时间';
COMMENT ON COLUMN black_deacon_flow_segment_result.updater IS '更新人';
COMMENT ON COLUMN black_deacon_flow_segment_result.updated_at IS '更新时间';
COMMENT ON COLUMN black_deacon_flow_segment_result.deleted_at IS '删除时间';
COMMENT ON COLUMN black_deacon_flow_segment_result.execution_id IS '执行ID';
COMMENT ON COLUMN black_deacon_flow_segment_result.node_id IS '视频生成节点ID';
COMMENT ON COLUMN black_deacon_flow_segment_result.segment_index IS '段序号';
COMMENT ON COLUMN black_deacon_flow_segment_result.video_key IS '视频输出字段key';
COMMENT ON COLUMN black_deacon_flow_segment_result.video_url IS '已生成成功的视频地址';
--------------------pgsql创建black_deacon_flow_segment_result表语句---------------------------
--------------------工作流重试兜底:exec_workflow 新增重试/心跳列---------------------
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS retryable SMALLINT NOT NULL DEFAULT 0; -- 0=终局不重试(用户取消/计费门禁拦截),1=可重试(程序报错/关停中断/超时等)
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS retry_count INTEGER NOT NULL DEFAULT 0; -- 已重试次数
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS last_heartbeat BIGINT NOT NULL DEFAULT 0; -- 最后心跳(毫秒时间戳)
COMMENT ON COLUMN black_deacon_exec_workflow.retryable IS '是否可重试:0-用户取消,1-程序报错';
COMMENT ON COLUMN black_deacon_exec_workflow.retry_count IS '已重试次数';
COMMENT ON COLUMN black_deacon_exec_workflow.last_heartbeat IS '最后心跳时间(毫秒时间戳)';
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS charge_order_id BIGINT NOT NULL DEFAULT 0; -- 关联计费单IDshop-user-trade pricing0=未建单)
COMMENT ON COLUMN black_deacon_exec_workflow.charge_order_id IS '关联计费单IDshop-user-trade pricing0=未建单)';
--------------------工作流重试兜底:flow_async_task 统一异步任务表(Task 2 使用)---------------------
CREATE TABLE IF NOT EXISTS black_deacon_flow_async_task (
-- 基础字段(完全对齐项目规范)
id BIGINT PRIMARY KEY,
tenant_id BIGINT NOT NULL DEFAULT 0,
creator VARCHAR(64) NOT NULL,
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
updater VARCHAR(64) NOT NULL,
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
deleted_at timestamp(6),
-- 业务字段
execution_id BIGINT NOT NULL DEFAULT 0, -- 所属执行ID
node_id VARCHAR(64) NOT NULL DEFAULT '', -- 所属节点ID
segment_index INT NOT NULL DEFAULT -1, -- 段序号;非段调用为-1(哨兵,不用NULL)
model_id BIGINT NOT NULL DEFAULT 0, -- 模型ID
task_id BIGINT NOT NULL DEFAULT 0, -- model-gateway任务ID
msg_topic VARCHAR(255) NOT NULL DEFAULT '', -- 结果消息主题(恢复重订阅拿回结果)
state SMALLINT NOT NULL DEFAULT 0, -- 0=in-flight,1=done,2=failed
result JSONB DEFAULT '{}' -- 成功结果JSON(ModelCallRes)
);
CREATE UNIQUE INDEX IF NOT EXISTS uk_async_task_exec_node_seg ON black_deacon_flow_async_task(execution_id, node_id, segment_index);
CREATE INDEX IF NOT EXISTS idx_async_task_tenant_id ON black_deacon_flow_async_task(tenant_id);
CREATE INDEX IF NOT EXISTS idx_async_task_deleted_at ON black_deacon_flow_async_task(deleted_at);
--------------------崩溃恢复补全用户:exec_workflow.user_id(恢复续跑补 X-User-Info 过 model-gateway 最低余额门禁)---------------------
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS user_id BIGINT NOT NULL DEFAULT 0; -- 执行用户ID(数字;creator 仅 userName,恢复续跑须按此补全用户 Id)
COMMENT ON COLUMN black_deacon_exec_workflow.user_id IS '执行用户ID(数字,恢复续跑补 X-User-Info 用)';
--------------------终局回填业务实扣:exec_workflow.actual_amount(用户钱包实际扣除金额,元;区别于 total_fee=模型按次费用合计)---------------------
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS actual_amount NUMERIC(15,2) NOT NULL DEFAULT 0; -- 业务扣费(settle/cancel 结算实收;失败/未结算=0)
COMMENT ON COLUMN black_deacon_exec_workflow.actual_amount IS '业务扣费(用户钱包实际扣除金额,元;结算/取消回填实收,失败/未结算=0,区别于 total_fee=模型按次费用合计)';
--------------------三表键按逻辑运行(node_group_id)隔离:exec_workflow 补列回填 + async/segment 加列换键 + checkpoint 活行归组(2026-09-04,详见《工作流状态表键按逻辑运行隔离技术设计.md》)--------------------
-- 根因:flow_checkpoint 软删(execution_id 键)后同键 ON CONFLICT 重存不清 deleted_at → 续跑读不到断点从图头转圈。
-- 方案:exec_workflow 持久化 node_group_id(逻辑运行标识:续跑复用、forceNewRun 换新 uuid);
-- checkpoint_id / async / segment 唯一键一律纳入该组;同组重复只 Upsert 重置、软删即终态不复活、无物理删。
-- 幂等:全程 ADD COLUMN IF NOT EXISTS / UPDATE 仅改空值 / DROP+CREATE UNIQUE IF EXISTS。
-- 1) exec_workflow.node_group_id:仓库此前漏建该列 DDL(生产早期手工加列则此处 no-op)。
-- 历史行回填自身 id(=其遗留 checkpoint/async/segment 旧键):部署前中断/失败可续跑的 exec 无缝衔接到新组键续跑;
-- 已带真实 uuid 现役组的行(新代码产生的)不受影响。
ALTER TABLE black_deacon_exec_workflow ADD COLUMN IF NOT EXISTS node_group_id VARCHAR(64) NOT NULL DEFAULT '';
COMMENT ON COLUMN black_deacon_exec_workflow.node_group_id IS '节点组ID(逻辑运行标识:续跑复用,forceNewRun 换新 uuid';
UPDATE black_deacon_exec_workflow SET node_group_id = id::text WHERE node_group_id IS NULL OR node_group_id = '';
-- 2) async/segment 加列并按 exec 现役组回填。NULLIF 防空组并到 execution_id::text=旧"exec 作用域"读集,等价不丢不并)
ALTER TABLE black_deacon_flow_async_task ADD COLUMN IF NOT EXISTS node_group_id VARCHAR(64);
ALTER TABLE black_deacon_flow_segment_result ADD COLUMN IF NOT EXISTS node_group_id VARCHAR(64);
UPDATE black_deacon_flow_async_task a SET node_group_id = COALESCE(NULLIF(e.node_group_id,''), a.execution_id::text)
FROM black_deacon_exec_workflow e WHERE a.node_group_id IS NULL AND e.id = a.execution_id;
UPDATE black_deacon_flow_segment_result s SET node_group_id = COALESCE(NULLIF(e.node_group_id,''), s.execution_id::text)
FROM black_deacon_exec_workflow e WHERE s.node_group_id IS NULL AND e.id = s.execution_id;
-- exec 行缺失的孤儿兜底(理论无):按 execution_id 文本伪组,保证唯一性成立
UPDATE black_deacon_flow_async_task SET node_group_id = execution_id::text WHERE node_group_id IS NULL;
UPDATE black_deacon_flow_segment_result SET node_group_id = execution_id::text WHERE node_group_id IS NULL;
ALTER TABLE black_deacon_flow_async_task ALTER COLUMN node_group_id SET NOT NULL;
ALTER TABLE black_deacon_flow_segment_result ALTER COLUMN node_group_id SET NOT NULL;
-- 3) 换唯一键:旧 (execution_id,node_id,segment_index) → 新 (node_group_id,node_id,segment_index)
DROP INDEX IF EXISTS uk_async_task_exec_node_seg;
CREATE UNIQUE INDEX IF NOT EXISTS uk_async_task_group_node_seg ON black_deacon_flow_async_task(node_group_id, node_id, segment_index);
DROP INDEX IF EXISTS uk_segment_result_exec_node_idx;
CREATE UNIQUE INDEX IF NOT EXISTS uk_segment_result_group_node_idx ON black_deacon_flow_segment_result(node_group_id, node_id, segment_index);
-- 4) checkpoint 表零 DDL:把仍活的断点从"checkpoint_id=executionId(数值串)"归一到"checkpoint_id=exec 现役组"
-- 使新组键读到部署前中断/失败的断点。历史 exec 组=自身 id → 天然不变;仅 uuid 现役组发生重命名
-- (单 exec 至多一行活断点;组为 uuid 不与遗留数值串冲突;守卫排除空串/自同值防并桶)
UPDATE black_deacon_flow_checkpoint c SET checkpoint_id = e.node_group_id
FROM black_deacon_exec_workflow e
WHERE c.deleted_at IS NULL AND c.checkpoint_id = e.id::text
AND e.node_group_id IS NOT NULL AND e.node_group_id <> '' AND e.node_group_id <> e.id::text;
-- 5) 存量毒槽自愈:旧 forceNewRun 三清把"该 exec 仍可续跑"的 checkpoint 槽软删成了墓碑;uk_checkpoint_id 无条件唯一、
-- 墓碑仍占槽,Save 的 ON CONFLICT 只会再 upsert 进墓碑且 deleted_at 不清 → 续跑永远读不到断点 = "从头转圈"现场。
-- 软删即终态、不复活:被占槽不可再用,故给"可续跑(status 1/3)但当前槽无活断点"的 exec 重指全新 uuid 组,
-- 下次续跑写新活行、此后断点正常落库可断点续跑。代价:这批 exec 历史段/异步缓存(按旧 id 组)随之失联,
-- 属毒槽必然代价、只影响存量毒槽行;正常失败(槽上活断点在)的 exec 不动、断点续跑无缝衔接。
UPDATE black_deacon_exec_workflow e
SET node_group_id = gen_random_uuid()::text
WHERE e.status IN (1, 3)
AND NOT EXISTS (SELECT 1 FROM black_deacon_flow_checkpoint c
WHERE c.deleted_at IS NULL AND c.checkpoint_id = e.node_group_id);
+40
View File
@@ -0,0 +1,40 @@
package model
// 模型类型编码常量
const (
TypeInference = 100 // 推理模型
TypeImage = 200 // 图片模型
TypeAudio = 300 // 音频模型
TypeVector = 400 // 向量化模型
TypeOmni = 500 // 全模态模型
TypeVideo = 600 // 视频模型
// 图片子类型
ImageSubTextToImage = 201
ImageSubImageToImage = 202
ImageSubImageEdit = 203
ImageSubImageVariation = 204
ImageSubImageTextToImage = 205
// 音频子类型
AudioSubTextToSpeech = 301
AudioSubSpeechToText = 302
AudioSubSpeechToSpeech = 303
// 向量化子类型
VectorSubEmbedding = 401
VectorSubRerank = 402
// 全模态子类型
OmniSubTextImageAudio = 501
OmniSubVision = 502
// 视频子类型
VideoSubTextToVideo = 601
VideoSubImageToVideo = 602
VideoSubImageTextToVideo = 603
VideoSubVideoToVideo = 604
)
// ModelType 编码类型
type ModelType *int
+27
View File
@@ -0,0 +1,27 @@
package model
import "github.com/gogf/gf/v2/util/gconv"
var (
ResponseTypeSync = newResponseType(gconv.PtrInt8(1), "sync") // 同步
ResponseTypeAsync = newResponseType(gconv.PtrInt8(2), "async") // 异步
ResponseTypeStream = newResponseType(gconv.PtrInt8(3), "stream") // 流
)
type ResponseType *int8
type responseType struct {
code ResponseType
desc string
}
func (s responseType) Code() ResponseType {
return s.code
}
func (s responseType) Desc() string {
return s.desc
}
func newResponseType(code ResponseType, desc string) responseType {
return responseType{code: code, desc: desc}
}
+471 -97
View File
@@ -1,119 +1,493 @@
package node
// ======================== 【常量定义:所有中文文案放这里!】 ========================
// 分组名称
const (
NodeGroupNameComponent = "组件"
NodeGroupNameBase = "基础"
NodeGroupNameCustom = "自定义"
)
// 节点名称
const (
NodeNameTextModel = "生成文案"
NodeNameImageModel = "生成图片"
NodeNameVideoModel = "生成视频"
NodeNameAudioModel = "生成音频"
NodeNameBatchModel = "批量处理一起返回"
NodeNameDataConversionModel = "参数转换"
NodeNameModel = "模型"
NodeNameMerge = "结果合并"
NodeNameDataMerge = "结果汇集"
NodeNameJudge = "条件判断"
NodeNameLoop = "循环"
NodeNameForm = "表单"
NodeSubFlow = "子流程"
NodeNameHttp = "HTTP(S)接口"
NodeNameCustomNode = "自定义节点"
NodeNameSystemSum = "系统-结果汇总"
)
// 表单字段 Label
const (
FormLabelApiKey = "API Key"
FormLabelModel = "模型名称"
FormLabelCondition = "判断条件"
)
// ======================== 枚举类型 ========================
// NodeGroup 节点分组标识
type NodeGroup string
const (
NodeGroupComponent NodeGroup = "component"
NodeGroupBase NodeGroup = "base"
NodeGroupCustom NodeGroup = "custom"
)
// NodeType 节点类型标识
type NodeType string
// 分组常量定义
const (
// 组件
NodeTypeTextModel NodeType = "text_model"
NodeTypeImageModel NodeType = "image_model"
NodeTypeVideoModel NodeType = "video_model"
NodeTypeAudioModel NodeType = "audio_model"
NodeTypeBatchModel NodeType = "batch_model"
NodeTypeDataConversionModel NodeType = "data_conversion_model"
// 基础
NodeTypeModel NodeType = "model"
NodeTypeMerge NodeType = "merge"
NodeTypeDataMerge NodeType = "data_merge"
NodeTypeJudge NodeType = "judge"
NodeTypeForm NodeType = "form"
NodeTypeIntent NodeType = "intent"
NodeTypeSubFlow NodeType = "sub_flow"
NodeTypeHttp NodeType = "http"
// 自定义
NodeTypeCustomNode NodeType = "custom_node"
// 系统
NodeTypeSystemSum NodeType = "system_sum"
NodeGroupBase NodeGroup = "base"
NodeGroupSystem NodeGroup = "system"
)
// 节点类型常量定义
const (
ModelTypeText = 100
ModelTypeImage = 200
ModelTypeAudio = 300
ModelTypeModality = 500
ModelTypeVideo = 600
NodeTypeStart NodeType = "__start__"
NodeTypeModel NodeType = "model"
NodeTypeDataMerge NodeType = "data_merge"
NodeTypeForm NodeType = "form"
NodeTypeSubFlow NodeType = "sub_flow"
NodeTypeHttp NodeType = "http"
NodeTypeSystemSum NodeType = "system_sum"
NodeTypeScriptTranscribe NodeType = "script_transcribe"
)
// ======================== 结构定义 ========================
type NodeFormField struct {
Value any `json:"value"`
Field string `json:"field"`
Label string `json:"label"` // 从常量来
Type string `json:"type"`
Required bool `json:"required"`
Default any `json:"default,omitempty"`
Options []SelectOption `json:"options"`
Expand any `json:"expand"`
FieldConstraint any `json:"fieldConstraint"`
// NodeGroupMeta 节点分组元数据
type NodeGroupMeta struct {
Key NodeGroup `json:"key"` // 后端存储标识
Name string `json:"name"` // 前端展示中文名称
}
// NodeTypeMeta 节点元数据
type NodeTypeMeta struct {
Key NodeType `json:"key"` // 节点类型标识
Name string `json:"name"` // 节点中文名称
Group NodeGroup `json:"group"` // 所属分组
Sort int `json:"sort"` // UI排序字段
Desc string `json:"desc,omitempty"` // 可选:节点简介
PatchLayout bool `json:"patchLayout"`
IsMultiParameter bool `json:"isMultiParameter"`
BatchExecOption bool `json:"batchExecOption"`
PreToolOption []NodePresetField `json:"preToolOption"`
PostToolOption []NodePresetField `json:"postToolOption"`
IsSaveFileOption bool `json:"isSaveFileOption"`
FormConfigOption bool `json:"formConfigOption"`
ModelConfigOption bool `json:"modelConfigOption"`
SkillOption bool `json:"skillOption"`
PromptOption bool `json:"promptOption"`
NegativePromptOption bool `json:"negativePromptOption"`
PresetOption []NodePresetField `json:"presetOption"`
OutputField []NodeOutputField `json:"outputField"`
}
type NodeOutputField struct {
Field string `json:"field"`
Label string `json:"label"`
}
type NodePresetField struct {
Value string `json:"value"`
ValueSource []ValueSource `json:"valueSource"`
Field string `json:"field"`
Label string `json:"label"`
Type string `json:"type"`
IsFormField bool `json:"isFormField"`
Constraint FieldConstraint `json:"constraint"`
Required bool `json:"required"`
Options []SelectOption `json:"options"`
}
type ValueSource struct {
NodeId string `json:"nodeId"`
Field string `json:"field"`
Label string `json:"label"`
}
// FieldConstraint 字段约束
type FieldConstraint struct {
// 数字类型:int、float、double、string
Type string `json:"type" dc:"类型"`
Min any `json:"min" dc:"最小值"`
Max any `json:"max" dc:"最大值"`
}
type SelectOption struct {
Label string `json:"label"`
Value string `json:"value"`
Key string `json:"key"`
Value string `json:"value"`
Config []NodePresetField `json:"config"`
}
type ModelItem struct {
ModelName string `json:"modelName"`
ModelForm []NodeFormField `json:"modelForm"`
// NodeGroupTree 对外输出树形结构:分组 + 分组下所有节点
type NodeGroupTree struct {
Group NodeGroupMeta `json:"group"`
Nodes []NodeTypeMeta `json:"nodes"`
}
type NodeItem struct {
NodeId string `json:"nodeId"`
NodeCode NodeType `json:"nodeCode"`
ModelType int `json:"modelType"`
NodeName string `json:"nodeName"` // 从常量来
SkillOption bool `json:"skillOption"`
PromptOption bool `json:"promptOption"`
IsSaveFile bool `json:"isSaveFile"`
FormConfig []NodeFormField `json:"formConfig"`
ModelConfig []ModelItem `json:"modelConfig"`
// ===================== 元数据集中注册区(英文标识+中文文案统一维护) =====================
var NodeGroupMetaList = []NodeGroupMeta{
{
Key: NodeGroupBase,
Name: "基础",
},
}
type NodeGroupItem struct {
Group NodeGroup `json:"group"`
Label string `json:"label"` // 从常量来
Items []NodeItem `json:"items"`
var NodeTypeMetaList = []NodeTypeMeta{
{
Key: NodeTypeModel,
Name: "模型",
Group: NodeGroupBase,
Sort: 1,
Desc: "模型调用节点,可配置模型参数、模型配置、技能、提示语、结果汇集、结果保存、结果返回、结果展示等信息。",
PatchLayout: true,
IsMultiParameter: true,
BatchExecOption: true,
PreToolOption: []NodePresetField{
{
Field: "perTool",
Label: "前置方法",
Type: "select",
Required: false,
Options: []SelectOption{},
},
},
PostToolOption: []NodePresetField{
{
Field: "postTool",
Label: "后置方法",
Type: "select",
Required: false,
Options: []SelectOption{},
},
},
IsSaveFileOption: true,
FormConfigOption: false,
ModelConfigOption: true,
SkillOption: false,
PromptOption: true,
NegativePromptOption: false,
},
{
Key: NodeTypeDataMerge,
Name: "结果汇集",
Group: NodeGroupBase,
Sort: 2,
Desc: "结果汇集节点,可配置结果汇集方式、结果保存、结果返回、结果展示等信息。",
IsMultiParameter: false,
BatchExecOption: false,
IsSaveFileOption: false,
FormConfigOption: false,
ModelConfigOption: false,
SkillOption: false,
PromptOption: false,
NegativePromptOption: false,
},
{
Key: NodeTypeForm,
Name: "表单",
Group: NodeGroupBase,
Sort: 3,
Desc: "表单节点,可配置表单字段、表单配置、结果汇集、结果保存、结果返回、结果展示等信息。",
IsMultiParameter: false,
BatchExecOption: false,
IsSaveFileOption: false,
FormConfigOption: true,
ModelConfigOption: false,
SkillOption: false,
PromptOption: false,
NegativePromptOption: false,
},
{
Key: NodeTypeSubFlow,
Name: "子流程",
Group: NodeGroupBase,
Sort: 4,
PresetOption: []NodePresetField{
{
Field: "maxConcurrency",
Label: "生成次数",
Type: "input",
Value: "1",
IsFormField: true,
Constraint: FieldConstraint{
Type: "int",
Min: 1,
Max: 10,
},
Required: true,
},
},
},
{
Key: NodeTypeHttp,
Name: "HTTP(S)接口",
Group: NodeGroupBase,
Sort: 5,
Desc: "HTTP(S)接口节点,可配置HTTP(S)接口地址、请求方式、请求头、请求体、结果返回结构、结果返回方式、结果返回结构、结果返回方式等信息。",
BatchExecOption: false,
IsSaveFileOption: true,
FormConfigOption: false,
ModelConfigOption: false,
SkillOption: false,
PromptOption: false,
NegativePromptOption: false,
PresetOption: []NodePresetField{
{
Field: "method",
Label: "请求方式",
Type: "select",
Required: true,
Options: []SelectOption{
{Key: "GET", Value: "GET"},
{Key: "POST", Value: "POST"},
},
},
{
Field: "url",
Label: "请求地址",
Type: "input",
Constraint: FieldConstraint{
Type: "string",
Min: 1,
Max: -1,
},
Required: true,
},
{
Field: "headers",
Label: "请求头(支持Authorization鉴权)",
Type: "keyValue",
Required: false,
},
{
Field: "bodyType",
Label: "请求体类型",
Type: "select",
Required: true,
Options: []SelectOption{
{Key: "None", Value: "无"},
{Key: "JSON", Value: "JSON"},
},
},
{
Field: "body",
Label: "请求体内容",
Type: "schemaJson",
Required: false,
},
{
Field: "response",
Label: "结果返回结构",
Type: "schemaJson",
Required: true,
},
{
Field: "responseType",
Label: "结果返回方式",
Type: "select",
Required: true,
Options: []SelectOption{
{Key: "sync", Value: "同步返回"},
{Key: "callback", Value: "等候回调", Config: []NodePresetField{
{
Field: "callbackUrl",
Label: "回调地址(只需要填写字段名称)",
Type: "input",
Constraint: FieldConstraint{
Type: "string",
Min: 1,
Max: -1,
},
Required: true,
},
}},
{Key: "pull", Value: "主动拉取", Config: []NodePresetField{
{
Field: "method",
Label: "请求方式",
Type: "select",
Required: true,
Options: []SelectOption{
{Key: "GET", Value: "GET"},
{Key: "POST", Value: "POST"},
},
},
{
Field: "url",
Label: "请求地址",
Type: "input",
Constraint: FieldConstraint{
Type: "string",
Min: 1,
Max: -1,
},
Required: true,
},
{
Field: "headers",
Label: "请求头(支持Authorization鉴权)",
Type: "keyValue",
Required: false,
},
{
Field: "bodyType",
Label: "请求体类型",
Type: "select",
Required: true,
Options: []SelectOption{
{Key: "None", Value: "无"},
{Key: "JSON", Value: "JSON"},
},
},
{
Field: "body",
Label: "请求体内容",
Type: "schemaJson",
Required: true,
},
{
Field: "response",
Label: "结果返回结构",
Type: "schemaJson",
Required: true,
},
}},
},
},
},
},
{
Key: NodeTypeScriptTranscribe,
Name: "脚本转写",
Group: NodeGroupBase,
Sort: 6,
Desc: "脚本转写节点,把文案/视频分析结果通过大模型转写为结构化分镜脚本([]Shot),作为视频生成节点的 shots 输入。",
IsMultiParameter: false,
BatchExecOption: false,
IsSaveFileOption: false,
FormConfigOption: false,
ModelConfigOption: true,
SkillOption: false,
PromptOption: false,
NegativePromptOption: false,
PresetOption: []NodePresetField{
{
Field: "totalDuration",
Label: "视频总时长",
Type: "input",
Constraint: FieldConstraint{
Type: "int",
Min: 1,
Max: 300,
},
Required: true,
},
{
Field: "modelId",
Label: "视频模型",
Type: "select",
Required: true,
Options: []SelectOption{},
},
},
OutputField: []NodeOutputField{
{
Field: "prompt",
Label: "转写内容",
},
{
Field: "duration",
Label: "时长",
},
{
Field: "seed",
Label: "种子",
},
{
Field: "reference_urls",
Label: "参考链接",
},
},
},
{
Key: NodeTypeSystemSum,
Name: "工作流内置节点-保存结果",
Group: NodeGroupSystem,
Sort: 7,
},
}
// ===================== 缓存map & 工具函数 =====================
var (
nodeTypeMetaMap map[NodeType]NodeTypeMeta
nodeGroupMap map[NodeGroup]NodeGroupMeta
)
func init() {
// 初始化分组缓存
nodeGroupMap = make(map[NodeGroup]NodeGroupMeta, len(NodeGroupMetaList))
for _, meta := range NodeGroupMetaList {
nodeGroupMap[meta.Key] = meta
}
// 初始化节点缓存
nodeTypeMetaMap = make(map[NodeType]NodeTypeMeta, len(NodeTypeMetaList))
for _, meta := range NodeTypeMetaList {
nodeTypeMetaMap[meta.Key] = meta
}
}
// GetNodeTypeMeta 根据节点类型获取元信息
func GetNodeTypeMeta(typ NodeType) (NodeTypeMeta, bool) {
meta, ok := nodeTypeMetaMap[typ]
return meta, ok
}
// GetNodeTypeName 快速获取节点中文名称,不存在返回原始key
func GetNodeTypeName(typ NodeType) string {
meta, ok := nodeTypeMetaMap[typ]
if !ok {
return string(typ)
}
return meta.Name
}
// GetNodeGroupMeta 根据分组标识获取分组元信息
func GetNodeGroupMeta(g NodeGroup) (NodeGroupMeta, bool) {
meta, ok := nodeGroupMap[g]
return meta, ok
}
// GetNodeGroupName 快速获取分组中文名称
func GetNodeGroupName(g NodeGroup) string {
meta, ok := nodeGroupMap[g]
if !ok {
return string(g)
}
return meta.Name
}
// GetAllNodeTypeMeta 返回全部节点元数据副本
func GetAllNodeTypeMeta() []NodeTypeMeta {
list := make([]NodeTypeMeta, len(NodeTypeMetaList))
copy(list, NodeTypeMetaList)
return list
}
// GetNodeGroupTree 获取【分组+节点】树形列表,完整所有节点
func GetNodeGroupTree() []NodeGroupTree {
return getFilteredNodeTree(nil)
}
// GetFilterNodeTree 根据指定分组过滤节点树形结构
// groups: 需要展示的分组key列表,传入nil返回全部
func GetFilterNodeTree(groups []NodeGroup) []NodeGroupTree {
return getFilteredNodeTree(groups)
}
// getFilteredNodeTree 内部通用树形构建逻辑
func getFilteredNodeTree(filterGroups []NodeGroup) []NodeGroupTree {
// 构建分组过滤集合
filterSet := make(map[NodeGroup]bool)
for _, g := range filterGroups {
filterSet[g] = true
}
groupNodeMap := make(map[NodeGroup][]NodeTypeMeta)
for _, node := range NodeTypeMetaList {
// 有过滤条件 && 当前分组不在过滤列表,则跳过
if len(filterSet) > 0 && !filterSet[node.Group] {
continue
}
groupNodeMap[node.Group] = append(groupNodeMap[node.Group], node)
}
var tree []NodeGroupTree
// 按定义顺序组装分组
for _, gMeta := range NodeGroupMetaList {
// 如果开启过滤,跳过不在筛选列表中的分组
if len(filterSet) > 0 && !filterSet[gMeta.Key] {
continue
}
nodes := groupNodeMap[gMeta.Key]
if len(nodes) == 0 {
continue
}
tree = append(tree, NodeGroupTree{
Group: gMeta,
Nodes: nodes,
})
}
return tree
}
+3
View File
@@ -0,0 +1,3 @@
package public
const GmqMsgPluginsName = "gmq_model_msg"
+17 -11
View File
@@ -7,15 +7,21 @@ const (
// 数据库表名
const (
TableNameCreationInfo = "creation_info"
TableNameFlowExecution = "flow_execution"
TableNameFlowTemplate = "flow_template"
TableNameFlowUser = "flow_user"
TableNameSkillTemplate = "skill_template"
TableNameSkillUser = "skill_user"
TableNameFileTemp = "file_temp"
TableNameActivePull = "active_pull"
TableNameWorkflowInterrupt = "workflow_interrupt"
TableNameNodePrompt = "node_prompt"
TableNameNodeExecution = "node_execution"
TableNameCreationInfo = "creation_info"
TableNameFlowExecution = "flow_execution"
TableNameFlowTemplate = "flow_template"
TableNameFlowUser = "flow_user"
TableNameSkillTemplate = "skill_template"
TableNameSkillUser = "skill_user"
TableNameFileTemp = "file_temp"
TableNameActivePull = "active_pull"
TableNameFlowCheckpoint = "flow_checkpoint"
TableNameNodePrompt = "node_prompt"
TableNameNodeExecution = "node_execution"
TableNameSession = "session"
TableNameExecChat = "exec_chat"
TableNameExecWorkflow = "exec_workflow"
TableNameExecWorkflowResult = "exec_workflow_result"
TableNameFlowSegmentResult = "flow_segment_result"
TableNameFlowAsyncTask = "flow_async_task"
)
@@ -4,32 +4,19 @@ import (
flowDto "ai-agent/workflow/model/dto/flow"
flowService "ai-agent/workflow/service/flow"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
)
type flowExecution struct{}
var FlowExecution = new(flowExecution)
func (c *flowExecution) Execute(ctx context.Context, req *flowDto.ExecuteReq) (res *flowDto.ExecuteRes, err error) {
return flowService.FlowExecutionService.Execute(ctx, req)
}
func (c *flowExecution) ComposeCallBack(ctx context.Context, req *flowDto.ComposeCallbackReq) (res *beans.ResponseEmpty, err error) {
err = flowService.FlowExecutionService.ComposeCallback(ctx, req)
return
}
func (c *flowExecution) ModelCallback(ctx context.Context, req *flowDto.ModelCallbackReq) (res *beans.ResponseEmpty, err error) {
err = flowService.FlowExecutionService.ModelCallback(ctx, req)
return
}
func (c *flowExecution) VideoCallback(ctx context.Context, req *flowDto.VideoCallbackReq) (res *beans.ResponseEmpty, err error) {
err = flowService.FlowExecutionService.VideoCallback(ctx, req)
return
}
//func (c *flowExecution) Execute(ctx context.Context, req *flowDto.ExecuteReq) (res *flowDto.ExecuteRes, err error) {
// return flowService.FlowExecutionService.Execute(ctx, req)
//}
//
//func (c *flowExecution) ReExecute(ctx context.Context, req *flowDto.ReExecuteReq) (res *flowDto.ExecuteRes, err error) {
// return flowService.FlowExecutionService.ReExecute(ctx, req)
//}
func (c *flowExecution) Get(ctx context.Context, req *flowDto.GetFlowExecutionReq) (res *flowDto.VOFlowExecution, err error) {
return flowService.FlowExecutionService.Get(ctx, req)
@@ -18,9 +18,8 @@ func (c *flowUser) Create(ctx context.Context, req *flowDto.CreateFlowUserReq) (
return
}
func (c *flowUser) Update(ctx context.Context, req *flowDto.UpdateFlowUserReq) (res *beans.ResponseEmpty, err error) {
err = flowService.FlowUserService.Update(ctx, req)
return
func (c *flowUser) Update(ctx context.Context, req *flowDto.UpdateFlowUserReq) (res *flowDto.CreateFlowUserRes, err error) {
return flowService.FlowUserService.Update(ctx, req)
}
func (c *flowUser) Delete(ctx context.Context, req *flowDto.DeleteFlowUserReq) (res *beans.ResponseEmpty, err error) {
@@ -0,0 +1,61 @@
package session
import (
flowDto "ai-agent/workflow/model/dto/flow"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/service/flow"
sessionService "ai-agent/workflow/service/session"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/glog"
)
type session struct{}
var Session = new(session)
func (c *session) WsExecute(ctx context.Context, req *sessionDto.WebSocketConnectReq) (res *beans.ResponseEmpty, err error) {
r := g.RequestFromCtx(ctx)
// 首次连接仅升级 WebSocket,不区分普通对话/工作流;由后续消息 type 路由到对应处理器
err = flow.WsConnect(ctx, r, req)
if err != nil {
glog.Errorf(ctx, "ws connect failed: %v", err)
return nil, err
}
// 标记响应缓冲区,使 MiddlewareHandlerResponse 跳过
r.Response.Writeln("")
return
}
func (c *session) Get(ctx context.Context, req *sessionDto.GetSessionInfoReq) (res *sessionDto.GetSessionInfoRes, err error) {
return sessionService.SessionService.Get(ctx, req)
}
func (c *session) List(ctx context.Context, req *sessionDto.ListSessionReq) (res *sessionDto.ListSessionRes, err error) {
return sessionService.SessionService.List(ctx, req)
}
func (c *session) Delete(ctx context.Context, req *sessionDto.DeleteSessionReq) (res *beans.ResponseEmpty, err error) {
if err = sessionService.SessionService.Delete(ctx, req); err != nil {
return
}
return &beans.ResponseEmpty{}, nil
}
func (c *session) DeleteRecord(ctx context.Context, req *sessionDto.DeleteSessionRecordReq) (res *beans.ResponseEmpty, err error) {
if err = sessionService.SessionService.DeleteRecord(ctx, req); err != nil {
return
}
return &beans.ResponseEmpty{}, nil
}
func (c *session) ResultList(ctx context.Context, req *sessionDto.ListWorkflowResultReq) (res *flowDto.ListFlowExecutionTreeRes, err error) {
return sessionService.SessionService.ResultList(ctx, req)
}
func (c *session) ResultDelete(ctx context.Context, req *sessionDto.DeleteWorkflowResultReq) (res *beans.ResponseEmpty, err error) {
err = sessionService.SessionService.ResultDelete(ctx, req)
return
}
@@ -0,0 +1,15 @@
package tool
import (
toolDto "ai-agent/workflow/model/dto/tool"
toolService "ai-agent/workflow/service/tool"
"context"
)
type tool struct{}
var Tool = new(tool)
func (c *tool) List(ctx context.Context, req *toolDto.ToolListReq) (res *toolDto.ToolListRes, err error) {
return toolService.ToolService.List(ctx, req)
}
+104
View File
@@ -0,0 +1,104 @@
package flow
import (
"ai-agent/workflow/consts/public"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
)
const (
FlowAsyncStateInflight = 0 // 任务已提交,结果未取(执行中/结果已发布未取/失败)
FlowAsyncStateDone = 1 // 成功,结果已缓存
FlowAsyncStateFailed = 2 // 确定失败
FlowAsyncSegSentinel = -1 // 非段异步调用的段序号哨兵值
)
var FlowAsyncTaskDao = &flowAsyncTaskDao{}
type flowAsyncTaskDao struct{}
// 缓存唯一键是 (node_group_id, node_id, segment_index)node_group_id 是"逻辑运行(attempt)"标识,
// 全新/换参重跑换新组 = 换新键;续跑/恢复复用 exec 记录的组 = 读到同组活行。
// 删除只有软删且只作用于"终态组"(成功尾部 / 换参重跑废弃的旧组),此类组此后永不再写,
// 软删行不会被同键重存(组永不复活)——与 flow_segment_result 同一约定。
// execution_id 仅作溯源保留,不参与唯一键。
// Get 查询唯一键 (node_group_id, node_id, segment_index) 的记录
func (d *flowAsyncTaskDao) Get(ctx context.Context, nodeGroupId string, nodeId string, segIdx int) (res *entity.FlowAsyncTask, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameFlowAsyncTask).
Where(entity.FlowAsyncTaskCol.NodeGroupId, nodeGroupId).
Where(entity.FlowAsyncTaskCol.NodeId, nodeId).
Where(entity.FlowAsyncTaskCol.SegmentIndex, segIdx).
One()
if err != nil {
return nil, err
}
if r.IsEmpty() {
return nil, nil
}
err = r.Struct(&res)
return
}
// Upsert 提交时写入/更新 in-flight 行:唯一键冲突则更新 model_id/task_id/msg_topic/state(保留已完成结果不动)。
// 与 flow_segment_result 相同走 OnConflict().Save();两点相对初稿的调整:
// 1. Result 列是 JSONB,空串会被 PG 拒绝(invalid input syntax for type json),
// 提交时本无结果,统一写 '{}' 占位(与表列默认值一致)。
// 2. 必须用 OnDuplicate 限定冲突更新列——GoFrame Save 默认把 Data 里所有列写进
// ON CONFLICT DO UPDATE SET,若不限定会把已缓存的结果覆盖成 '{}',与"保留已完成结果"矛盾。
//
// 提交/重提统一走本函数重置同组活行(state→in-flight + 新 task_id/msg_topic):不再"删行重建"
// 活行从未被软删,OnConflict 更新即可复位,无墓碑冲突、无物理删除。
func (d *flowAsyncTaskDao) Upsert(ctx context.Context, nodeGroupId string, execId int64, nodeId string, segIdx int, modelId, taskId int64, msgTopic string) error {
rec := &entity.FlowAsyncTask{
NodeGroupId: nodeGroupId,
ExecutionId: execId,
NodeId: nodeId,
SegmentIndex: segIdx,
ModelId: modelId,
TaskId: taskId,
MsgTopic: msgTopic,
State: FlowAsyncStateInflight,
Result: "{}",
}
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowAsyncTask).
Data(rec).
OnConflict(entity.FlowAsyncTaskCol.NodeGroupId, entity.FlowAsyncTaskCol.NodeId, entity.FlowAsyncTaskCol.SegmentIndex).
OnDuplicate(entity.FlowAsyncTaskCol.ModelId, entity.FlowAsyncTaskCol.TaskId, entity.FlowAsyncTaskCol.MsgTopic, entity.FlowAsyncTaskCol.State).
Save()
return err
}
// UpdateByKey 按唯一键更新 state/resultOmitNil 丢弃 nil 字段,map 值非 nil 全写入;
// state=0 也能落库)。Result 列是 JSONB,空串无法写入,统一落 '{}' 表示无结果。
func (d *flowAsyncTaskDao) UpdateByKey(ctx context.Context, nodeGroupId string, nodeId string, segIdx int, state int, result string) error {
resultVal := result
if resultVal == "" {
resultVal = "{}"
}
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameFlowAsyncTask).
Where(entity.FlowAsyncTaskCol.NodeGroupId, nodeGroupId).
Where(entity.FlowAsyncTaskCol.NodeId, nodeId).
Where(entity.FlowAsyncTaskCol.SegmentIndex, segIdx).
Data(map[string]any{
entity.FlowAsyncTaskCol.State: state,
entity.FlowAsyncTaskCol.Result: resultVal,
}).
Update()
return err
}
// DeleteByGroup 软删指定逻辑运行(组)的异步任务缓存(gfdb Model.Delete 在 deletedAt 配置下退化为软删)。
// 仅允许对"终态组"调用:① 工作流执行成功后(BuildExecution 尾部);② 同一条 exec 换参重跑(forceNewRun
// 废弃的旧组(execute 重置成功后回收)。终态组此后永不再被读写 → 软删行不复活、不占同键写入冲突。
// 失败/取消/重试耗尽一律不删(保留组行供同参数续跑复用)。绝无物理删除。
func (d *flowAsyncTaskDao) DeleteByGroup(ctx context.Context, nodeGroupId string) error {
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowAsyncTask).
Where(entity.FlowAsyncTaskCol.NodeGroupId, nodeGroupId).
Delete()
return err
}
+60
View File
@@ -0,0 +1,60 @@
package flow
import (
"ai-agent/workflow/consts/public"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
)
var FlowCheckpointDao = &flowCheckpointDao{}
type flowCheckpointDao struct{}
// Upsert 插入或更新checkpoint数据(按 checkpoint_id 冲突则更新)
func (d *flowCheckpointDao) Upsert(ctx context.Context, checkpointId string, data string) error {
record := &entity.FlowCheckpoint{
CheckpointId: checkpointId,
Data: data,
}
// Save 在 PostgreSQL 中自动执行 INSERT ON CONFLICT DO UPDATE
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowCheckpoint).
Save(record)
return err
}
func (d *flowCheckpointDao) SaveOrUpdate(ctx context.Context, checkpointId string, data string) (err error) {
res := &entity.FlowCheckpoint{
CheckpointId: checkpointId,
Data: data,
}
_, err = gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameFlowCheckpoint).Data(res).OnConflict(entity.FlowCheckpointCol.CheckpointId).Save()
return err
}
// Get 根据 checkpoint_id 获取 checkpoint 数据
func (d *flowCheckpointDao) Get(ctx context.Context, checkpointId string) (res *entity.FlowCheckpoint, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowCheckpoint).
Where(entity.FlowCheckpointCol.CheckpointId, checkpointId).
One()
if err != nil {
return nil, err
}
if r == nil {
return nil, nil
}
err = r.Struct(&res)
return
}
// Delete 根据 checkpoint_id 删除checkpoint数据
func (d *flowCheckpointDao) Delete(ctx context.Context, checkpointId string) error {
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowCheckpoint).
Where(entity.FlowCheckpointCol.CheckpointId, checkpointId).
Delete()
return err
}
@@ -0,0 +1,67 @@
package flow
import (
"ai-agent/workflow/consts/public"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
)
var FlowSegmentResultDao = &flowSegmentResultDao{}
type flowSegmentResultDao struct{}
// 缓存唯一键是 (node_group_id, node_id, segment_index)node_group_id 是"逻辑运行(attempt)"标识,
// 全新/换参重跑换新组 = 换新键;续跑/恢复复用 exec 记录的组 = 读到同组活行。
// 删除只有软删且只作用于"终态组"(成功尾部 / 换参重跑废弃的旧组),此类组此后永不再写 →
// 软删行永不被同键重存(不复活)。实体嵌入 SQLBaseDO(含 deleted_at)gfdb Model.Delete() 即软删,
// 不再需要 raw Exec 物理删除。
// Save 段成功后落库:唯一键 (node_group_id, node_id, segment_index),冲突则更新视频引用
func (d *flowSegmentResultDao) Save(ctx context.Context, nodeGroupId string, execId int64, nodeId string, segmentIndex int, videoKey, videoURL string) error {
rec := &entity.FlowSegmentResult{
NodeGroupId: nodeGroupId,
ExecutionId: execId,
NodeId: nodeId,
SegmentIndex: segmentIndex,
VideoKey: videoKey,
VideoURL: videoURL,
}
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowSegmentResult).
Data(rec).
OnConflict(entity.FlowSegmentResultCol.NodeGroupId, entity.FlowSegmentResultCol.NodeId, entity.FlowSegmentResultCol.SegmentIndex).
Save()
return err
}
// ListByNode 返回该节点已成功段(段序号 → 视频引用);仅读当前逻辑运行(组)的活行
func (d *flowSegmentResultDao) ListByNode(ctx context.Context, nodeGroupId string, nodeId string) (map[int]entity.SegmentRef, error) {
var list []*entity.FlowSegmentResult
err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowSegmentResult).
Where(entity.FlowSegmentResultCol.NodeGroupId, nodeGroupId).
Where(entity.FlowSegmentResultCol.NodeId, nodeId).
Scan(&list)
if err != nil {
return nil, err
}
m := make(map[int]entity.SegmentRef, len(list))
for _, r := range list {
m[r.SegmentIndex] = entity.SegmentRef{Key: r.VideoKey, URL: r.VideoURL}
}
return m, nil
}
// DeleteByGroup 软删指定逻辑运行(组)的段结果(gfdb Model.Delete 在 deletedAt 配置下退化为软删)。
// 仅允许对"终态组"调用:① 工作流执行成功后(BuildExecution 尾部);② 同一条 exec 换参重跑(forceNewRun
// 废弃的旧组(execute 重置成功后回收)。终态组此后永不再被读写 → 软删行不复活。
// 失败/取消不删(保留组行供 reExecute 复用已成功段)。绝无物理删除。
func (d *flowSegmentResultDao) DeleteByGroup(ctx context.Context, nodeGroupId string) error {
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).
Model(ctx, public.TableNameFlowSegmentResult).
Where(entity.FlowSegmentResultCol.NodeGroupId, nodeGroupId).
Delete()
return err
}
+8
View File
@@ -86,6 +86,14 @@ func (d *nodeExecutionDao) Get(ctx context.Context, req *nodeDto.GetNodeExecutio
func (d *nodeExecutionDao) ListByFlowExecutionId(ctx context.Context, req *nodeDto.ListNodeExecutionByFlowReq, fields ...string) (res []*entity.NodeExecution, total int, err error) {
model := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameNodeExecution).NoTenantId(ctx).Fields(fields).OmitEmpty()
model.Where(entity.NodeExecutionCol.FlowExecutionId, req.FlowExecutionId)
model.Where(entity.NodeExecutionCol.NodeGroupId, req.NodeGroupId)
if req.CreatedAtFrom != nil {
// 结算按订单收敛:只聚合订单创建后产生的节点记录(重跑开新单,created_at 各自独立,
// 避免把已终局运行(已结算扣费)的用量计入本次订单)
model.WhereGTE(entity.NodeExecutionCol.CreatedAt, *req.CreatedAtFrom)
}
model.Where(entity.NodeExecutionCol.Status, req.Status)
model.Where(entity.NodeExecutionCol.NodeId, req.NodeId)
model.OrderAsc(entity.NodeExecutionCol.CreatedAt)
if req.Page != nil {
model.Page(int(req.Page.PageNum), int(req.Page.PageSize))
+87
View File
@@ -0,0 +1,87 @@
package session
import (
"ai-agent/workflow/consts/public"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/util/gconv"
)
var ExecChatDao = &execChatDao{}
type execChatDao struct{}
func (d *execChatDao) Insert(ctx context.Context, req *sessionDto.CreateExecChatReq) (id int64, err error) {
var s = new(entity.ExecChat)
if err = gconv.Struct(req, &s); err != nil {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).Insert(s)
if err != nil {
return
}
return r.LastInsertId()
}
func (d *execChatDao) Update(ctx context.Context, req *sessionDto.UpdateExecChatReq) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).OmitEmpty().Data(&req).Where(entity.ExecChatCol.Id, req.Id).Update()
if err != nil {
return
}
return r.RowsAffected()
}
// ClearError 清空对话执行记录的报错信息(重新执行成功后调用,OmitEmpty 的 Update 会跳过空串,需显式写空)
func (d *execChatDao) ClearError(ctx context.Context, id int64) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).
Where(entity.ExecChatCol.Id, id).
Data(map[string]any{
entity.ExecChatCol.ErrorMessage: "",
entity.ExecChatCol.Error: "",
}).
Update()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execChatDao) Delete(ctx context.Context, req *sessionDto.DeleteExecChatReq) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).Where(entity.ExecChatCol.Id, req.Id).Delete()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execChatDao) List(ctx context.Context, creator string, page *beans.Page) (res []*entity.ExecChat, total int, err error) {
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).
Where(entity.ExecChatCol.Creator, creator)
m.OrderDesc(entity.ExecChatCol.CreatedAt)
if page != nil {
m.Page(int(page.PageNum), int(page.PageSize))
}
r, total, err := m.AllAndCount(false)
if err != nil {
return
}
err = r.Structs(&res)
return
}
// ListBySession 查询会话下普通对话执行记录(按创建时间倒序)
func (d *execChatDao) ListBySession(ctx context.Context, sessionId string) (res []*entity.ExecChat, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecChat).
Where(entity.ExecChatCol.SessionId, sessionId).
OrderDesc(entity.ExecChatCol.CreatedAt).
All()
if err != nil {
return
}
err = r.Structs(&res)
return
}
+252
View File
@@ -0,0 +1,252 @@
package session
import (
flow "ai-agent/workflow/consts/flow"
"ai-agent/workflow/consts/public"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"context"
"fmt"
"time"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/util/gconv"
)
var ExecWorkflowDao = &execWorkflowDao{}
type execWorkflowDao struct{}
func (d *execWorkflowDao) Insert(ctx context.Context, req *sessionDto.CreateWorkflowReq) (id int64, err error) {
var s = new(entity.ExecWorkflow)
if err = gconv.Struct(req, &s); err != nil {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).Insert(s)
if err != nil {
return
}
return r.LastInsertId()
}
func (d *execWorkflowDao) Delete(ctx context.Context, req *sessionDto.DeleteExecWorkflowReq) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).Where(entity.ExecWorkflowCol.Id, req.Id).Delete()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execWorkflowDao) Update(ctx context.Context, req *sessionDto.UpdateWorkflowReq) (rows int64, err error) {
if req.Id <= 0 {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).OmitEmpty().Data(&req).Where(entity.ExecWorkflowCol.Id, req.Id).Update()
if err != nil {
return
}
return r.RowsAffected()
}
// UpdateMap 按列更新执行记录(map 更新不走 OmitEmpty,可显式写 0 值)。
// 供 recordWorkflow 终态一次原子落库(status/error_message/error/retryable/retry_count 同语句),
// 避免"先 UpdateRetry 再 Update"两步写部分生效导致 retryable 与状态不一致
func (d *execWorkflowDao) UpdateMap(ctx context.Context, id int64, data map[string]any) error {
if id <= 0 || len(data) == 0 {
return nil
}
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
Data(data).
Update()
return err
}
// ClearError 清空执行记录的报错信息(重新执行成功后调用,OmitEmpty 的 Update 会跳过空串,需显式写空)
func (d *execWorkflowDao) ClearError(ctx context.Context, id int64) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
Data(map[string]any{
entity.ExecWorkflowCol.ErrorMessage: "",
entity.ExecWorkflowCol.Error: "",
}).
Update()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execWorkflowDao) GetById(ctx context.Context, id int64) (res *entity.ExecWorkflow, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
One()
if err != nil {
return
}
err = r.Struct(&res)
return
}
func (d *execWorkflowDao) List(ctx context.Context, creator string, page *beans.Page) (res []*entity.ExecWorkflow, total int, err error) {
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Creator, creator)
m.OrderDesc(entity.ExecWorkflowCol.CreatedAt)
if page != nil {
m.Page(int(page.PageNum), int(page.PageSize))
}
r, total, err := m.AllAndCount(false)
if err != nil {
return
}
err = r.Structs(&res)
return
}
// GetLatestBySessionAndFlow 查询会话+工作流下最近一次执行记录(按创建时间倒序,无记录返回 nil)
func (d *execWorkflowDao) GetLatestBySessionAndFlow(ctx context.Context, sessionId string, flowId int64) (res *entity.ExecWorkflow, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.SessionId, sessionId).
Where(entity.ExecWorkflowCol.FlowId, flowId).
OrderDesc(entity.ExecWorkflowCol.CreatedAt).
Limit(1).
One()
if err != nil {
return
}
if r.IsEmpty() {
return nil, nil
}
err = r.Struct(&res)
return
}
// ListBySession 查询会话下工作流执行记录(按创建时间倒序)
func (d *execWorkflowDao) ListBySession(ctx context.Context, sessionId string) (res []*entity.ExecWorkflow, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.SessionId, sessionId).
OrderDesc(entity.ExecWorkflowCol.CreatedAt).
All()
if err != nil {
return
}
err = r.Structs(&res)
return
}
// ResetRunning 置为运行中并刷新心跳(execute 复用失败记录 / reExecute 共用;
// 用 map 更新避免 OmitEmpty 省略 0 值,同时写 status、last_heartbeat、node_group_id)。
// 条件更新防双跑:仅当记录当前状态仍属于 prevStatuses 之一时才重置(DB 行锁原子判定),
// 其余场景(已被恢复例程/并发触发抢先重置)返回 reset=false,调用方应放弃执行并返回 errExecAlreadyRunning
// 由持有方收敛状态,避免同一 exec 被两条路径并发 BuildExecution。
func (d *execWorkflowDao) ResetRunning(ctx context.Context, id int64, nodeGroupId string, prevStatuses ...int8) (reset bool, err error) {
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id)
if len(prevStatuses) > 0 {
statuses := make([]interface{}, 0, len(prevStatuses))
for _, s := range prevStatuses {
statuses = append(statuses, s)
}
m = m.WhereIn(entity.ExecWorkflowCol.Status, statuses)
}
r, err := m.Data(map[string]any{
entity.ExecWorkflowCol.Status: gconv.Int8(*flow.FlowExecutionStatusRunning.Code()),
entity.ExecWorkflowCol.LastHeartbeat: time.Now().UnixMilli(),
entity.ExecWorkflowCol.NodeGroupId: nodeGroupId,
}).Update()
if err != nil {
return false, err
}
rows, err := r.RowsAffected()
if err != nil {
return false, err
}
return rows > 0, nil
}
// ResetRunningIfRecoverable 恢复例程专用:条件重置为运行中,仅当记录仍处于可恢复状态时
// status=3 可重试失败,或 status=1 心跳陈旧的僵尸运行中)才重置(DB 行锁原子判定防双跑)。
// staleBeforeMsstatus=1 时的心跳陈旧阈值(毫秒),与 ListRecoverable/isRecoverable 判定一致。
// 返回 reset=false 表示状态已被其它路径(用户 reExecute/execute 或其它节点恢复)抢先重置,
// 本恢复例程应放弃续跑、不落终态,状态由持有方收敛。
func (d *execWorkflowDao) ResetRunningIfRecoverable(ctx context.Context, id int64, nodeGroupId string, staleBeforeMs int64) (reset bool, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
Where(fmt.Sprintf("(%s = ? OR (%s = ? AND %s < ?))",
entity.ExecWorkflowCol.Status,
entity.ExecWorkflowCol.Status,
entity.ExecWorkflowCol.LastHeartbeat),
gconv.Int8(*flow.FlowExecutionStatusFailed.Code()),
gconv.Int8(*flow.FlowExecutionStatusRunning.Code()),
staleBeforeMs).
Data(map[string]any{
entity.ExecWorkflowCol.Status: gconv.Int8(*flow.FlowExecutionStatusRunning.Code()),
entity.ExecWorkflowCol.LastHeartbeat: time.Now().UnixMilli(),
entity.ExecWorkflowCol.NodeGroupId: nodeGroupId,
}).
Update()
if err != nil {
return false, err
}
rows, err := r.RowsAffected()
if err != nil {
return false, err
}
return rows > 0, nil
}
// TouchHeartbeat 更新执行心跳(毫秒时间戳),供后台心跳 goroutine 每 30s 调用一次
func (d *execWorkflowDao) TouchHeartbeat(ctx context.Context, id int64) error {
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
Data(map[string]any{entity.ExecWorkflowCol.LastHeartbeat: time.Now().UnixMilli()}).
Update()
return err
}
// UpdateRetry 更新重试标记与已重试次数(map 更新,retryable=0 也需写入)
func (d *execWorkflowDao) UpdateRetry(ctx context.Context, id int64, retryable int, retryCount int) error {
_, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
Where(entity.ExecWorkflowCol.Id, id).
Data(map[string]any{
entity.ExecWorkflowCol.Retryable: retryable,
entity.ExecWorkflowCol.RetryCount: retryCount,
}).
Update()
return err
}
// GetByIdNoTenant 跨租户按 id 读取执行记录(不追加 tenant_id 过滤)。
// 供恢复例程抢锁后、尚未知晓租户时重读 exec 使用;用户路径请继续用租户隔离的 GetById。
func (d *execWorkflowDao) GetByIdNoTenant(ctx context.Context, id int64) (res *entity.ExecWorkflow, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
NoTenantId(ctx).
Where(entity.ExecWorkflowCol.Id, id).
One()
if err != nil {
return
}
err = r.Struct(&res)
return
}
// ListRecoverable 返回可恢复执行:僵尸运行中(status=1 且心跳陈旧)或可重试失败(status=3 且 retryable=1 且未耗尽)。
// 调用方是跨租户的恢复扫描(无 HTTP 用户),显式 NoTenantId 走系统级扫描,避免僵尸执行因租户过滤漏检。
// ctx 必须携带 OTel spanNoTenantId 依赖 traceID 作为 gcache 标记键),恢复扫描由 scanAndRecover 保证。
func (d *execWorkflowDao) ListRecoverable(ctx context.Context, now int64, staleBefore int64, maxRetry int) (res []*entity.ExecWorkflow, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflow).
NoTenantId(ctx).
Where(fmt.Sprintf("(%s = ? AND %s < ?) OR (%s = ? AND %s = 1 AND %s < ?)",
entity.ExecWorkflowCol.Status, entity.ExecWorkflowCol.LastHeartbeat,
entity.ExecWorkflowCol.Status, entity.ExecWorkflowCol.Retryable, entity.ExecWorkflowCol.RetryCount),
gconv.Int8(*flow.FlowExecutionStatusRunning.Code()), staleBefore,
gconv.Int8(*flow.FlowExecutionStatusFailed.Code()), maxRetry).
All()
if err != nil {
return
}
// 用 All+Structs 而非 ScanScan 生成的列清单会丢嵌入 SQLBaseDO 的 id 等基础列,恢复例程需要 id
err = r.Structs(&res)
return
}
@@ -0,0 +1,129 @@
package session
import (
"ai-agent/workflow/consts/public"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/util/gconv"
)
var ExecWorkflowResultDao = &execWorkflowResultDao{}
type execWorkflowResultDao struct{}
func (d *execWorkflowResultDao) Insert(ctx context.Context, req *sessionDto.CreateWorkflowResultReq) (id int64, err error) {
var s = new(entity.ExecWorkflowResult)
if err = gconv.Struct(req, &s); err != nil {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).Insert(s)
if err != nil {
return
}
return r.LastInsertId()
}
func (d *execWorkflowResultDao) BatchInsert(ctx context.Context, req []*sessionDto.CreateWorkflowResultReq) (rows int64, err error) {
var res []*entity.ExecWorkflowResult
if err = gconv.Structs(req, &res); err != nil {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).Data(res).Save()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execWorkflowResultDao) Delete(ctx context.Context, req *sessionDto.DeleteWorkflowResultReq) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).Where(entity.ExecWorkflowResultCol.Id, req.Id).Delete()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *execWorkflowResultDao) List(ctx context.Context, creator string, page *beans.Page) (res []*entity.ExecWorkflowResult, total int, err error) {
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).
Where(entity.ExecWorkflowResultCol.Creator, creator)
m.OrderDesc(entity.ExecWorkflowResultCol.CreatedAt)
if page != nil {
m.Page(int(page.PageNum), int(page.PageSize))
}
r, total, err := m.AllAndCount(false)
if err != nil {
return
}
err = r.Structs(&res)
return
}
// ListBySession 查询会话下工作流结果(按创建时间倒序)
func (d *execWorkflowResultDao) ListBySession(ctx context.Context, sessionId string) (res []*entity.ExecWorkflowResult, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).
Where(entity.ExecWorkflowResultCol.SessionId, sessionId).
OrderDesc(entity.ExecWorkflowResultCol.CreatedAt).
All()
if err != nil {
return
}
err = r.Structs(&res)
return
}
// ListByExecId 查询指定工作流执行记录下的结果文件路径
func (d *execWorkflowResultDao) ListByExecId(ctx context.Context, execId int64) (res []*entity.ExecWorkflowResult, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).
Where(entity.ExecWorkflowResultCol.ExecId, execId).
All()
if err != nil {
return
}
err = r.Structs(&res)
return
}
// ListDates 按创建人查询去重后的创建日期(倒序,支持分页;page 为 nil 返回全部日期)
func (d *execWorkflowResultDao) ListDates(ctx context.Context, creator string, page *beans.Page) (dates []string, err error) {
fieldAlias := "create_date"
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).
Fields("DATE("+entity.ExecWorkflowResultCol.CreatedAt+") AS "+fieldAlias).
Where(entity.ExecWorkflowResultCol.Creator, creator).
Group("DATE(" + entity.ExecWorkflowResultCol.CreatedAt + ")").
OrderDesc(fieldAlias)
if page != nil {
m.Page(int(page.PageNum), int(page.PageSize))
}
r, err := m.All()
if err != nil {
return nil, err
}
for _, rec := range r {
dates = append(dates, rec[fieldAlias].String())
}
return dates, nil
}
// ListByDates 按创建人查询指定创建日期(DATE(created_at) 命中)内的结果记录,按创建时间倒序
func (d *execWorkflowResultDao) ListByDates(ctx context.Context, creator string, dates []string) (res []*entity.ExecWorkflowResult, err error) {
if len(dates) == 0 {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameExecWorkflowResult).
Where(entity.ExecWorkflowResultCol.Creator, creator).
WhereIn("DATE(created_at)", dates).
OrderDesc(entity.ExecWorkflowResultCol.CreatedAt).
All()
if err != nil {
return
}
err = r.Structs(&res)
return
}
+62
View File
@@ -0,0 +1,62 @@
package session
import (
"ai-agent/workflow/consts/public"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/util/gconv"
)
var SessionDao = &sessionDao{}
type sessionDao struct{}
func (d *sessionDao) Insert(ctx context.Context, req *sessionDto.CreateSessionReq) (id int64, err error) {
var s = new(entity.Session)
if err = gconv.Struct(req, &s); err != nil {
return
}
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameSession).Insert(s)
if err != nil {
return
}
return r.LastInsertId()
}
func (d *sessionDao) GetById(ctx context.Context, id string) (res *entity.Session, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameSession).
Where(entity.SessionCol.SessionId, id).
One()
if err != nil {
return
}
err = r.Struct(&res)
return
}
func (d *sessionDao) Delete(ctx context.Context, req *sessionDto.DeleteSessionReq) (rows int64, err error) {
r, err := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameSession).Where(entity.SessionCol.SessionId, req.SessionId).Delete()
if err != nil {
return
}
return r.RowsAffected()
}
func (d *sessionDao) List(ctx context.Context, creator string, page *beans.Page) (res []*entity.Session, total int, err error) {
m := gfdb.DB(ctx, public.DbNameBlackDeacon).Model(ctx, public.TableNameSession).
Where(entity.SessionCol.Creator, creator)
m.OrderDesc(entity.SessionCol.CreatedAt)
if page != nil {
m.Page(int(page.PageNum), int(page.PageSize))
}
r, total, err := m.AllAndCount(false)
if err != nil {
return
}
err = r.Structs(&res)
return
}
+36 -123
View File
@@ -10,6 +10,14 @@ import (
"github.com/gogf/gf/v2/os/gtime"
)
// NodeExecutionState 记录图的节点执行进度,用于 checkpoint/resume 时追踪哪些节点已完成
// SavedInput 用于在中断前保存节点入参,续跑后从状态中恢复
type NodeExecutionState struct {
CompletedNodes []string `json:"completedNodes"` // 已成功执行的节点名称列表
ExecutionCount int `json:"executionCount"` // 总执行调用次数
SavedFlowInput *FlowExecutionInput `json:"savedFlowInput"` // 中断前保存的节点入参,续跑后读取
}
// NodeExecutionInput 节点执行入参(包含配置+表单架构)
type NodeExecutionInput struct {
Config *entity.FlowNode `json:"config"` // 节点配置
@@ -17,129 +25,25 @@ type NodeExecutionInput struct {
NodeExecutionId int64 `json:"nodeExecutionId"`
}
// FlowExecutionInput 工作流执行入参(全程不变)
type FlowExecutionInput struct {
NodeGroupId string `json:"nodeGroupId"`
ExecutionId int64 `json:"executionId"`
FlowId int64 `json:"flowId"`
ConfigMap map[string]*entity.FlowNode `json:"configMap"`
SessionId string `json:"sessionId" dc:"会话ID"`
ExecutedNodes []ExecutedNode `json:"executedNodes"` // 已执行节点列表,包含执行状态
ForceNewRun bool `json:"forceNewRun" dc:"是否全新执行(false=断点续跑,视频节点段级复用已成功段)"`
SubFlowScope string `json:"subFlowScope" dc:"子流程批量子执行缓存作用域(拼进 async/segment 缓存键隔离各份;顶层为空)"`
}
// ExecutedNode 已执行节点记录,包含节点ID和执行状态
type ExecutedNode struct {
NodeId string `json:"nodeId"`
Status node.NodeExecutionStatus `json:"status"` // 执行状态:成功/失败
}
// FlowExecutionInput 工作流执行入参(全程不变)
type FlowExecutionInput struct {
NodeGroupId string `json:"nodeGroupId"`
IsDialogue bool `json:"isDialogue"`
ExecutionId int64 `json:"executionId"`
ConfigMap map[string]*entity.FlowNode `json:"configMap"`
SessionId string `json:"sessionId" dc:"会话ID"`
Desc string `json:"desc"`
SkillName string `json:"skillName"`
FileUrl []string `json:"fileUrl"`
ExecutedNodes []ExecutedNode `json:"executedNodes"` // 已执行节点列表,包含执行状态
}
type GetIsChatModelRes struct {
Model struct {
ModelName string `json:"modelName"`
ResponseBody map[string]any `json:"responseBody"`
}
}
type GetModelInfoReq struct {
ModelName string `json:"modelName"`
}
type GetModelInfoRes struct {
Model struct {
LastFrame string `json:"lastFrame"`
ResponseTokenField string `json:"responseTokenField"`
ResponseMapping map[string]any `json:"responseMapping"`
ResponseBody string `json:"responseBody"`
//QueryConfig struct {
// ResponseType string `json:"responseType"`
// CallbackUrl string `json:"callbackUrl"`
// Method string `json:"method"`
// Url string `json:"url"`
// Headers map[string]any `json:"headers"`
// Body map[string]any `json:"body"`
// Response []map[string]any `json:"response"`
// ResponseBody string `json:"responseBody"`
// ResponseTokenField string `json:"responseTokenField"`
//} `json:"queryConfig"`
} `json:"model"`
}
type ComposeMessagesReq struct {
BuildType int `json:"buildType"`
ModelName string `json:"modelName"`
SkillName string `json:"skillName"`
CallbackUrl string `json:"callbackUrl"`
Form []map[string]any `json:"form"`
UserForm []map[string]any `json:"userForm"`
Consult []Consult `json:"consult"`
SessionId string `json:"sessionId" dc:"会话ID"`
NodeId string `json:"nodeId"`
Cause string `json:"cause"`
}
type Consult struct {
Type string `json:"type"`
Url string `json:"url"`
}
type ComposeMessagesRes struct {
TaskId string `json:"taskId"`
}
type VideoConcatReq struct {
VideoUrls []string `json:"video_urls"`
Method string `json:"method"`
Upload bool `json:"upload"`
CallbackUrl string `json:"callback_url"`
}
type VideoConcatRes struct {
TaskId string `json:"taskId"`
}
type ModelGatewayReq struct {
ModelName string `json:"modelName"`
ModelKey string `json:"modelKey"`
BizName string `json:"bizName"`
CallbackUrl string `json:"callbackUrl"`
InputRef string `json:"inputRef"`
RequestPayload map[string]any `json:"requestPayload"`
EpicycleId int64 `json:"epicycleId" dc:"轮次ID"`
}
type ModelGatewayRes struct {
TaskId string `json:"taskId"`
}
type ComposeCallbackReq struct {
g.Meta `path:"/composeCallBack" method:"post" tags:"提示词处理" summary:"提示词 回调" dc:"提示词 成功后 GET 回调:callbackUrl/{bizName}"`
TaskId string `json:"taskId"`
Status string `json:"status"`
Messages struct {
TotalRounds int `json:"total_rounds"` // 总轮数
Rounds []map[string]any `json:"rounds"` // 每轮详情(动态类型)
} `json:"messages,omitempty"`
EpicycleId int64 `json:"epicycleId"`
ErrorMsg string `json:"errorMsg,omitempty"`
}
type ModelCallbackReq struct {
g.Meta `path:"/modelCallback" method:"post" tags:"提示词处理" summary:"model-gateway 回调" dc:"model-gateway 成功后 GET 回调:callbackUrl/{bizName}"`
TaskId string `p:"task_id" json:"task_id" v:"required#task_id不能为空" dc:"网关任务ID"`
State int `p:"state" json:"state" dc:"网关任务状态"`
OssFile string `p:"oss_file" json:"oss_file" dc:"结果文件地址"`
FileType string `p:"file_type" json:"file_type" dc:"结果文件类型"`
ErrorMsg string `json:"error_msg"`
}
type VideoCallbackReq struct {
g.Meta `path:"/videoCallback" method:"post" tags:"视频处理" summary:"media 回调" dc:"media 成功后 GET 回调:callbackUrl/{bizName}"`
TaskId string `json:"taskId"`
FileURL string `json:"fileUrl"`
}
//=======================================================================================
//=============================================================================
@@ -175,11 +79,12 @@ type ExecuteReq struct {
g.Meta `path:"/execute" method:"post" tags:"任务管理" summary:"执行任务" dc:"执行任务"`
FlowId int64 `json:"flowId" dc:"用户流程ID"`
FlowContent *entity.FlowInfo `json:"flowContent" description:"流程内容"`
SessionId string `json:"sessionId" dc:"会话ID"`
SessionName string `json:"sessionName"`
FlowName string `json:"flowName"`
NodeGroupId string `json:"nodeGroupId"`
FlowContent *entity.FlowInfo `json:"flowContent" description:"流程内容"`
NodeInputParams []*entity.FlowNode `json:"nodeInputParams" description:"节点输入参数"`
SessionId string `json:"sessionId" dc:"会话ID"`
Desc string `json:"desc"`
SkillName string `json:"skillName"`
FileUrl []string `json:"fileUrl"`
@@ -190,6 +95,12 @@ type ExecuteRes struct {
Id int64 `json:"id,string" dc:"执行记录ID,用于查询执行状态和结果"`
}
type ReExecuteReq struct {
g.Meta `path:"/reExecute" method:"post" tags:"任务管理" summary:"重新执行任务" dc:"重新执行任务"`
ExecutionId int64 `json:"executionId"`
}
type CancelReq struct {
g.Meta `path:"/cancel" method:"post" tags:"任务管理" summary:"取消任务" dc:"取消任务"`
@@ -218,11 +129,14 @@ type CreateFlowExecutionRes struct {
type UpdateFlowExecutionReq struct {
Id int64 `json:"id" v:"required#ID不能为空"`
NodeGroupId string `json:"nodeGroupId"`
FlowContent *entity.FlowInfo `json:"flowContent" description:"流程内容"`
DurationMs int64 `json:"durationMs" description:"执行时长(毫秒)"`
Status flow.FlowExecutionStatus `json:"status" description:"状态:1-运行中,2-成功,3-失败"`
OutputParams []map[string]interface{} `json:"outputParams" description:"输出参数"`
ErrorMessage string `json:"errorMessage" description:"错误信息"`
TraceId string `json:"traceId" description:"跟踪ID"`
TotalTokens int `json:"totalTokens" description:"总token"`
TotalFee float64 `json:"totalFee" description:"总费用"`
}
type GetFlowExecutionReq struct {
@@ -265,10 +179,9 @@ type VOFlowExecution struct {
// ========== 核心:构建树状结构 ==========
// 定义树结构
type OutputItem struct {
Timestamp string `json:"timestamp" description:"时间戳key"`
Content string `json:"content" description:"内容值"`
Type string `json:"type" description:"类型"`
Label string `json:"label" description:"后缀+数字标号"`
Content string `json:"content" description:"内容值"`
Type string `json:"type" description:"类型"`
Label string `json:"label" description:"后缀+数字标号"`
}
type FlowNode struct {
FlowName string `json:"flowName" description:"流程名称"`
+2
View File
@@ -34,6 +34,7 @@ type UpdateFlowUserReq struct {
NodeInputParams []*entity.FlowNode `json:"nodeInputParams" description:"节点输入参数"`
AccessLevel flow.FlowUserAccessLevel `json:"accessLevel" description:"访问权限:1私有,2团队,3公开"`
SourceFlowTemplateId int64 `json:"sourceFlowTemplateId" description:"来源流程模板ID"`
SubFlows []UpdateFlowUserReq `json:"subFlows" description:"子流程"`
}
type DeleteFlowUserReq struct {
@@ -52,6 +53,7 @@ type ListFlowUserReq struct {
g.Meta `path:"/list" method:"get" tags:"用户流程管理" summary:"获取用户流程列表" dc:"分页查询用户流程列表,支持多条件筛选"`
Page *beans.Page `json:"page"`
IsOwn bool `json:"isOwn" dc:"是否是自己的流程"`
Creator string `json:"creator"`
Keyword string `json:"keyword" dc:"关键词搜索"`
}
@@ -7,6 +7,7 @@ import (
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/gtime"
)
// CreateNodeExecutionReq 创建节点执行记录请求
@@ -38,6 +39,7 @@ type UpdateNodeExecutionReq struct {
PromptTokens int `json:"promptTokens"`
CompletionTokens int `json:"completionTokens"`
TotalTokens int `json:"totalTokens"`
TokenInfo []map[string]any `json:"tokenInfo"`
Status node.NodeExecutionStatus `json:"status"`
DurationMs int64 `json:"durationMs"`
ErrorMessage string `json:"errorMessage"`
@@ -58,8 +60,12 @@ type GetNodeExecutionReq struct {
// ListNodeExecutionByFlowReq 查询流程下所有节点执行记录请求
type ListNodeExecutionByFlowReq struct {
g.Meta `path:"/listByFlow" method:"get" tags:"节点执行记录" summary:"查询流程节点执行列表" dc:"查询指定流程执行下的所有节点执行记录"`
Page *beans.Page `json:"page"`
FlowExecutionId int64 `json:"flowExecutionId" v:"required#流程执行ID不能为空"`
Page *beans.Page `json:"page"`
FlowExecutionId int64 `json:"flowExecutionId" v:"required#流程执行ID不能为空"`
NodeGroupId string `json:"nodeGroupId"`
NodeId string `json:"nodeId"`
Status node.NodeExecutionStatus `json:"status"`
CreatedAtFrom *gtime.Time `json:"createdAtFrom" dc:"结算按订单收敛:只返回创建时间>=该值的记录(重跑开新单按各自 created_at 隔离)"`
}
// NodeExecutionResp 节点执行记录响应
+1 -17
View File
@@ -13,21 +13,5 @@ type WorkflowNodeTreeReq struct {
}
type WorkflowNodeTreeRes struct {
Groups []node.NodeGroupItem `json:"groups"`
}
type TypeGroup struct {
TypeId int `json:"typeId"`
Type string `json:"type"`
Items []ModelItem `json:"items"`
}
type ModelItem struct {
ID int64 `json:"id"`
Name string `json:"name"`
Form []node.NodeFormField `json:"form"`
}
type ModelTypeResponse struct {
Type map[int]string `json:"type"` // key 自动解析为整数 100/200/300...
Groups []node.NodeGroupTree `json:"groups"`
}
@@ -0,0 +1,24 @@
package session
import (
"ai-agent/workflow/model/entity"
)
type CreateExecChatReq struct {
SessionId string `json:"sessionId" description:"所属会话ID"`
RequestParams entity.ExecChatRequestParams `json:"requestParams" description:"请求参数"`
}
type UpdateExecChatReq struct {
Id int64 `json:"id" v:"required#会话执行记录ID不能为空"`
Duration int64 `json:"duration" description:"执行时长(秒)"`
ResultFileUrl string `json:"resultFileUrl" description:"结果文件路径"`
TotalTokens int `json:"totalTokens" description:"总token消耗"`
TotalFee float64 `json:"totalFee" description:"总费用"`
ErrorMessage string `json:"errorMessage" description:"错误信息(友好提示)"`
Error string `json:"error" description:"错误明细(原始错误)"`
}
type DeleteExecChatReq struct {
Id []int64 `json:"id" v:"required#会话执行记录ID不能为空"`
}
@@ -0,0 +1,34 @@
package session
import (
"ai-agent/workflow/consts/flow"
"ai-agent/workflow/model/entity"
)
type CreateWorkflowReq struct {
UserId int64 `json:"userId" description:"执行用户ID(数字,恢复续跑补 X-User-Info 用)"`
SessionId string `json:"sessionId" description:"所属会话ID"`
FlowId int64 `json:"flowId" description:"工作流ID"`
NodeGroupId string `json:"nodeGroupId" description:"节点组ID"`
Status flow.FlowExecutionStatus `json:"status" description:"状态:1-运行中,2-成功,3-失败"`
RequestParams *entity.FlowInfo `json:"requestParams" description:"请求参数"`
ErrorMessage string `json:"errorMessage" description:"错误信息(友好提示)"`
Error string `json:"error" description:"错误明细(原始错误)"`
LastHeartbeat int64 `json:"lastHeartbeat" description:"最后心跳时间(毫秒时间戳)"`
}
type DeleteExecWorkflowReq struct {
Id []int64 `json:"id" v:"required#工作流执行记录ID不能为空"`
}
type UpdateWorkflowReq struct {
Id int64 `json:"id" v:"required#工作流执行记录ID不能为空"`
NodeGroupId string `json:"nodeGroupId" description:"节点组ID"`
Status flow.FlowExecutionStatus `json:"status" description:"状态:1-运行中,2-成功,3-失败"`
Duration int64 `json:"duration" description:"执行时长(秒)"`
TotalTokens int `json:"totalTokens" description:"总token消耗"`
TotalFee float64 `json:"totalFee" description:"总费用"`
ErrorMessage string `json:"errorMessage" description:"错误信息(友好提示)"`
Error string `json:"error" description:"错误明细(原始错误)"`
RequestParams *entity.FlowInfo `json:"requestParams" description:"请求参数"`
}
@@ -0,0 +1,8 @@
package session
type CreateWorkflowResultReq struct {
SessionId string `json:"sessionId" description:"所属会话ID"`
FlowId int64 `json:"flowId" description:"工作流ID"`
ExecId int64 `json:"execId" description:"执行ID"`
ResultFileUrl string `json:"resultFileUrl" description:"结果文件路径"`
}
+101
View File
@@ -0,0 +1,101 @@
package session
import (
"ai-agent/workflow/model/entity"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/gtime"
)
// WebSocketConnectReq WebSocket 连接请求(query string 参数)
type WebSocketConnectReq struct {
g.Meta `path:"/wsExecute" method:"get" tags:"工作流执行" summary:"WebSocket执行工作流" dc:"通过WebSocket连接实时执行工作流,支持进度推送和取消"`
SessionId string `p:"sessionId" dc:"会话ID"`
}
type WebSocketExecChatReq struct {
Id int64 `json:"id" dc:"id"`
ModelId int64 `json:"modelId" dc:"模型ID" v:"required#模型ID不能为空"`
Question string `json:"question" dc:"用户提问" v:"required#用户提问不能为空"`
SystemPrompt string `json:"systemPrompt" dc:"系统提示词"`
}
type WebSocketExecWorkflowReq struct {
FlowId int64 `json:"flowId" dc:"用户流程ID" v:"required#用户流程ID不能为空"`
FlowContent *entity.FlowInfo `json:"flowContent" dc:"用户流程内容" v:"required#用户流程内容不能为空"`
}
type CreateSessionReq struct {
SessionId string `json:"sessionId" dc:"会话ID"`
SessionName string `json:"sessionName" dc:"会话名称"`
}
type ListSessionReq struct {
g.Meta `path:"/list" method:"get" tags:"会话管理" summary:"会话列表" dc:"会话列表"`
PageNum int64 `json:"pageNum" dc:"页码,从1开始"`
PageSize int64 `json:"pageSize" dc:"每页数量"`
}
type ListSessionRes struct {
List []*VOSession `json:"list"`
Total int `json:"total"`
}
type VOSession struct {
SessionId string `json:"sessionId" dc:"会话ID"`
SessionName string `json:"sessionName" dc:"会话名称"`
CreatedAt *gtime.Time `json:"createdAt" dc:"创建时间"`
}
type DeleteSessionReq struct {
g.Meta `path:"/delete" method:"delete" tags:"会话管理" summary:"删除会话" dc:"删除会话"`
SessionId string `json:"sessionId" v:"required#会话ID不能为空"`
}
type DeleteSessionRecordReq struct {
g.Meta `path:"/deleteRecord" method:"post" tags:"会话管理" summary:"删除会话记录" dc:"删除会话记录"`
Ids []struct {
Id int64 `json:"id" v:"required#ID不能为空"`
Type string `json:"type" v:"required#类型不能为空"`
}
}
type GetSessionInfoReq struct {
g.Meta `path:"/get" method:"get" tags:"会话管理" summary:"会话内结果列表" dc:"会话内结果列表,工作流+普通对话混排按时间倒序"`
PageNum int64 `json:"pageNum" dc:"页码,从1开始"`
PageSize int64 `json:"pageSize" dc:"每页数量"`
SessionId string `json:"sessionId" v:"required#会话ID不能为空"`
}
type GetSessionInfoRes struct {
List []*VOSessionInfoResult `json:"list"`
Total int `json:"total"`
}
// VOSessionInfoResult 会话内单条结果(工作流/普通对话混排)
type VOSessionInfoResult struct {
Id int64 `json:"id,string" dc:"结果ID"`
Type string `json:"type" dc:"workflow-工作流结果,chat-普通对话结果"`
Status int `json:"status" dc:"1-运行中,2-成功,3-失败"`
FlowId int64 `json:"flowId,string" dc:"工作流IDchat类型为空)"`
RequestParams map[string]any `json:"requestParams" description:"请求参数"`
ResultFileUrl string `json:"resultFileUrl" description:"结果文件路径(供预览/下载)"`
ResultContent string `json:"resultContent" description:"结果文件内容(服务端已读取,前端直接展示)"`
TotalTokens int `json:"totalTokens" dc:"总token消耗"`
TotalFee float64 `json:"totalFee" dc:"模型扣费合计(各模型按次费用)"`
ActualAmount float64 `json:"actualAmount" dc:"业务扣费(用户钱包实际扣除,元)"`
ErrorMsg string `json:"errorMsg" dc:"错误信息(友好提示)"`
Error string `json:"error" dc:"错误明细(原始错误)"`
CreatedAt *gtime.Time `json:"createdAt" dc:"创建时间"`
}
type ListWorkflowResultReq struct {
g.Meta `path:"/resultList" method:"get" tags:"会话管理" summary:"工作流执行结果树" dc:"按创建人分页查询工作流执行结果,按天分组返回树结构(日期→流程→结果文件),pageSize=每页天数,不传返回全部"`
Page *beans.Page `json:"page"`
}
type DeleteWorkflowResultReq struct {
g.Meta `path:"/resultDelete" method:"delete" tags:"会话管理" summary:"删除工作流执行结果" dc:"删除工作流执行结果"`
Id int64 `json:"id" v:"required#ID不能为空"`
}
+35
View File
@@ -0,0 +1,35 @@
package tool
import (
"github.com/gogf/gf/v2/frame/g"
)
// ToolListReq 工具列表查询请求
type ToolListReq struct {
g.Meta `path:"/list" method:"get" tags:"工具管理" summary:"工具列表" dc:"查询已注册的模型工具列表"`
}
// ToolVO 工具对外信息
type ToolVO struct {
Name string `json:"name"`
Description string `json:"description"`
}
// ToolListRes 工具列表查询响应
type ToolListRes struct {
List []*ToolVO `json:"list"`
}
// ToolAgentReq 工具对话请求:前端用户提问,模型通过 function calling 使用全部已注册模型工具作答
type ToolAgentReq struct {
g.Meta `path:"/agent" method:"post" tags:"工具管理" summary:"工具对话" dc:"用户提问,模型通过 function calling 使用全部已注册模型工具作答"`
Question string `json:"question" dc:"用户提问"`
ModelId int64 `json:"modelId,string" dc:"模型ID"`
SessionId string `json:"sessionId" dc:"会话ID,透传给网关记账"`
SystemPrompt string `json:"systemPrompt" dc:"系统提示词,为空使用默认"`
}
// ToolAgentRes 工具对话响应
type ToolAgentRes struct {
Answer string `json:"answer"`
}
+44
View File
@@ -0,0 +1,44 @@
package entity
import "gitea.redpowerfuture.com/red-future/common/beans"
// ExecChat 执行会话
type ExecChat struct {
beans.SQLBaseDO `orm:",inherit"`
SessionId string `orm:"session_id" json:"sessionId" description:"会话ID"`
Duration int64 `orm:"duration" json:"duration" description:"执行时长(秒)"`
RequestParams ExecChatRequestParams `orm:"request_params" json:"requestParams" description:"请求参数"`
ResultFileUrl string `orm:"result_file_url" json:"resultFileUrl" description:"结果文件路径"`
TotalTokens int `orm:"total_tokens" json:"totalTokens" description:"总token消耗"`
TotalFee float64 `orm:"total_fee" json:"totalFee" description:"总费用"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息(友好提示)"`
Error string `orm:"error" json:"error" description:"错误明细(原始错误)"`
}
type ExecChatRequestParams struct {
Question string `json:"question"`
}
type execChatCol struct {
beans.SQLBaseCol
SessionId string
Duration string
RequestParams string
ResultFileUrl string
TotalTokens string
TotalFee string
ErrorMessage string
Error string
}
var ExecChatCol = execChatCol{
SQLBaseCol: beans.DefSQLBaseCol,
SessionId: "session_id",
Duration: "duration",
RequestParams: "request_params",
ResultFileUrl: "result_file_url",
TotalTokens: "total_tokens",
TotalFee: "total_fee",
ErrorMessage: "error_message",
Error: "error",
}
+68
View File
@@ -0,0 +1,68 @@
package entity
import (
"ai-agent/workflow/consts/flow"
"gitea.redpowerfuture.com/red-future/common/beans"
)
// ExecWorkflow 执行工作流
type ExecWorkflow struct {
beans.SQLBaseDO `orm:",inherit"`
UserId int64 `orm:"user_id" json:"userId" description:"执行用户ID(数字,恢复续跑补 X-User-Info 用;creator 是 userName"`
SessionId string `orm:"session_id" json:"sessionId" description:"会话ID"`
FlowId int64 `orm:"flow_id" json:"flowId" description:"工作流ID"`
NodeGroupId string `orm:"node_group_id" json:"nodeGroupId" description:"节点组ID"`
Duration int64 `orm:"duration" json:"duration" description:"执行时长(秒)"`
RequestParams *FlowInfo `orm:"request_params" json:"requestParams" description:"请求参数"`
Status flow.FlowExecutionStatus `orm:"status" json:"status" description:"状态:1-运行中,2-成功,3-失败"`
TotalTokens int `orm:"total_tokens" json:"totalTokens" description:"总token消耗"`
TotalFee float64 `orm:"total_fee" json:"totalFee" description:"总费用"`
ActualAmount float64 `orm:"actual_amount" json:"actualAmount" description:"业务扣费(用户钱包实际扣除金额,元;结算/取消回填实收,失败/未结算=0,区别于 total_fee=模型按次费用合计)"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息(友好提示)"`
Error string `orm:"error" json:"error" description:"错误明细(原始错误)"`
Retryable int `orm:"retryable" json:"retryable" description:"是否可重试:0-终局不重试(用户取消/计费门禁拦截),1-可重试(程序报错/关停中断/超时等)"`
RetryCount int `orm:"retry_count" json:"retryCount" description:"已重试次数"`
ChargeOrderId int64 `orm:"charge_order_id" json:"chargeOrderId" description:"关联计费单IDshop-user-trade pricing0=未建单)"`
LastHeartbeat int64 `orm:"last_heartbeat" json:"lastHeartbeat" description:"最后心跳时间(毫秒时间戳)"`
}
type execWorkflowCol struct {
beans.SQLBaseCol
UserId string
SessionId string
FlowId string
NodeGroupId string
Duration string
RequestParams string
Status string
TotalTokens string
TotalFee string
ActualAmount string
ErrorMessage string
Error string
Retryable string
RetryCount string
ChargeOrderId string
LastHeartbeat string
}
var ExecWorkflowCol = execWorkflowCol{
SQLBaseCol: beans.DefSQLBaseCol,
UserId: "user_id",
SessionId: "session_id",
FlowId: "flow_id",
NodeGroupId: "node_group_id",
Duration: "duration",
RequestParams: "request_params",
Status: "status",
TotalTokens: "total_tokens",
TotalFee: "total_fee",
ActualAmount: "actual_amount",
ErrorMessage: "error_message",
Error: "error",
Retryable: "retryable",
RetryCount: "retry_count",
ChargeOrderId: "charge_order_id",
LastHeartbeat: "last_heartbeat",
}
@@ -0,0 +1,28 @@
package entity
import "gitea.redpowerfuture.com/red-future/common/beans"
// ExecWorkflowResult 执行工作流结果
type ExecWorkflowResult struct {
beans.SQLBaseDO `orm:",inherit"`
SessionId string `orm:"session_id" json:"sessionId" description:"会话ID"`
FlowId int64 `orm:"flow_id" json:"flowId" description:"工作流ID"`
ExecId int64 `orm:"exec_id" json:"execId" description:"执行ID"`
ResultFileUrl string `orm:"result_file_url" json:"resultFileUrl" description:"结果文件路径"`
}
type execWorkflowResultCol struct {
beans.SQLBaseCol
SessionId string
FlowId string
ExecId string
ResultFileUrl string
}
var ExecWorkflowResultCol = execWorkflowResultCol{
SQLBaseCol: beans.DefSQLBaseCol,
SessionId: "session_id",
FlowId: "flow_id",
ExecId: "exec_id",
ResultFileUrl: "result_file_url",
}
+46
View File
@@ -0,0 +1,46 @@
package entity
import "gitea.redpowerfuture.com/red-future/common/beans"
// FlowAsyncTask 统一异步模型任务表:提交时写入(in-flight),结果回来更新;
// 崩溃恢复靠持久化的 msg_topic 重订阅 NATS 拿回结果,避免重复调用模型
type FlowAsyncTask struct {
beans.SQLBaseDO `orm:",inherit"`
// NodeGroupId 所属逻辑运行(attempt)标识:全新/换参重跑=新组,续跑/恢复/自动重试=复用 exec 记录的组。
// 缓存唯一键以 (node_group_id, node_id, segment_index) 隔离,杜绝换参重跑软删墓碑与重跑同键写入冲突。
NodeGroupId string `orm:"node_group_id" json:"nodeGroupId" description:"所属逻辑运行(node_group_id)标识"`
ExecutionId int64 `orm:"execution_id" json:"executionId" description:"所属执行ID(溯源,不参与唯一键)"`
NodeId string `orm:"node_id" json:"nodeId" description:"所属节点ID"`
SegmentIndex int `orm:"segment_index" json:"segmentIndex" description:"段序号;非段调用为-1"`
ModelId int64 `orm:"model_id" json:"modelId" description:"模型ID"`
TaskId int64 `orm:"task_id" json:"taskId" description:"model-gateway任务ID"`
MsgTopic string `orm:"msg_topic" json:"msgTopic" description:"结果消息主题"`
State int `orm:"state" json:"state" description:"0=in-flight,1=done,2=failed"`
Result string `orm:"result" json:"result" description:"成功结果JSON(ModelCallRes)"`
}
type flowAsyncTaskCol struct {
beans.SQLBaseCol
NodeGroupId string
ExecutionId string
NodeId string
SegmentIndex string
ModelId string
TaskId string
MsgTopic string
State string
Result string
}
var FlowAsyncTaskCol = flowAsyncTaskCol{
SQLBaseCol: beans.DefSQLBaseCol,
NodeGroupId: "node_group_id",
ExecutionId: "execution_id",
NodeId: "node_id",
SegmentIndex: "segment_index",
ModelId: "model_id",
TaskId: "task_id",
MsgTopic: "msg_topic",
State: "state",
Result: "result",
}
+25
View File
@@ -0,0 +1,25 @@
package entity
import (
"gitea.redpowerfuture.com/red-future/common/beans"
)
// FlowCheckpoint checkpoint数据实体,对应 workflow_interrupt 表
type FlowCheckpoint struct {
beans.SQLBaseDO `orm:",inherit"` // 嵌入基础字段:Id, TenantId, Creator, CreatedAt, Updater, UpdatedAt, DeletedAt
// 业务字段
CheckpointId string `orm:"checkpoint_id" json:"checkpointId" description:"Checkpoint ID(执行ID"`
Data string `orm:"data" json:"data" description:"Checkpoint序列化数据"`
}
type flowCheckpointCol struct {
beans.SQLBaseCol
CheckpointId string
Data string
}
var FlowCheckpointCol = flowCheckpointCol{
SQLBaseCol: beans.DefSQLBaseCol,
CheckpointId: "checkpoint_id",
Data: "data",
}
+3
View File
@@ -22,6 +22,7 @@ type FlowExecution struct {
TraceId string `orm:"trace_id" json:"traceId" description:"跟踪ID"`
SessionId string `orm:"session_id" json:"sessionId" description:"会话ID"`
TotalTokens int `orm:"total_tokens" json:"totalTokens" description:"总token消耗"`
TotalFee int `orm:"total_fee" json:"totalFee" description:"总费用"`
}
type flowExecutionCol struct {
@@ -39,6 +40,7 @@ type flowExecutionCol struct {
TraceId string
SessionId string
TotalTokens string
TotalFee string
}
var FlowExecutionCol = flowExecutionCol{
@@ -56,4 +58,5 @@ var FlowExecutionCol = flowExecutionCol{
TraceId: "trace_id",
SessionId: "session_id",
TotalTokens: "total_tokens",
TotalFee: "total_fee",
}
@@ -0,0 +1,43 @@
package entity
import "gitea.redpowerfuture.com/red-future/common/beans"
// FlowSegmentResult 视频节点段级生成结果(只存成功段),供断点续跑复用
type FlowSegmentResult struct {
beans.SQLBaseDO `orm:",inherit"` // 嵌入基础字段:Id, TenantId, Creator, CreatedAt, Updater, UpdatedAt, DeletedAt
// 业务字段
// NodeGroupId 所属逻辑运行(attempt)标识:段结果唯一键 (node_group_id, node_id, segment_index)
// 换参重跑换组即换键,绝不复用被软删墓碑的旧组行(软删即终态,不复活)
NodeGroupId string `orm:"node_group_id" json:"nodeGroupId" description:"所属逻辑运行(node_group_id)标识"`
ExecutionId int64 `orm:"execution_id" json:"executionId" description:"执行ID(溯源,不参与唯一键)"`
NodeId string `orm:"node_id" json:"nodeId" description:"视频生成节点ID"`
SegmentIndex int `orm:"segment_index" json:"segmentIndex" description:"段序号"`
VideoKey string `orm:"video_key" json:"videoKey" description:"视频输出字段key"`
VideoURL string `orm:"video_url" json:"videoURL" description:"已生成成功的视频地址"`
}
// SegmentRef 一段已成功生成的视频引用(Key 保持模型原输出字段名,重建输出记录保证下游引用一致)
type SegmentRef struct {
Key string
URL string
}
type flowSegmentResultCol struct {
beans.SQLBaseCol
NodeGroupId string
ExecutionId string
NodeId string
SegmentIndex string
VideoKey string
VideoURL string
}
var FlowSegmentResultCol = flowSegmentResultCol{
SQLBaseCol: beans.DefSQLBaseCol,
NodeGroupId: "node_group_id",
ExecutionId: "execution_id",
NodeId: "node_id",
SegmentIndex: "segment_index",
VideoKey: "video_key",
VideoURL: "video_url",
}
+47 -25
View File
@@ -12,42 +12,64 @@ type FlowInfo struct {
StartNodeId string `json:"startNodeId"`
Nodes []FlowNode `json:"nodes"`
Edges []FlowEdge `json:"edges"`
ChargeMode string `json:"chargeMode"` // 计费方式:per_item/per_second/per_token(缺省 per_item,对齐 shop-user-trade consts/pricing
}
type FlowNode struct {
Id string `json:"id"`
NodeCode node.NodeType `json:"nodeCode"`
Name string `json:"name"`
Config map[string]interface{} `json:"config"`
SkillName string `json:"skillName"`
PromptContent string `json:"promptContent"`
IsSaveFile bool `json:"isSaveFile"`
InputSource []FlowNodeInputSource `json:"inputSource"` // 前端指定:来源节点ID
SubConfig *SubFlowConfig `json:"subConfig"`
FormConfig []node.NodeFormField `json:"formConfig"`
ModelConfig node.ModelItem `json:"modelConfig"`
OutputConfig []node.NodeFormField `json:"outputConfig"`
OutputResult []node.NodeFormField `json:"outputResult" ds:"节点输出结果"`
Id string `json:"id"`
NodeCode node.NodeType `json:"nodeCode"`
Name string `json:"name"`
Desc string `json:"desc"`
Config map[string]interface{} `json:"config"`
IsBatchExec bool `json:"isBatchExec"`
PreTool string `json:"preTool"`
PostTool string `json:"postTool"`
IsSaveFile bool `json:"isSaveFile"`
SubConfig *SubFlowConfig `json:"subConfig"`
ModelConfig ModelItem `json:"modelConfig"`
OutputConfig []map[string]any `json:"outputConfig"`
Prompt string `json:"prompt"`
NegativePrompt string `json:"negativePrompt"`
PatchLayout bool `json:"patchLayout"`
Templates []map[string]any `json:"templates"`
//SkillName string `json:"skillName"`
//PromptContent string `json:"promptContent"`
//InputSource []FlowNodeInputSource `json:"inputSource"` // 前端指定:来源节点ID
//FormConfig []node.NodeFormField `json:"formConfig"`
OutputResult []map[string]any `json:"outputResult" ds:"节点输出结果"`
}
type FlowNodeInputSource struct {
NodeId string `json:"nodeId"`
QuoteOutput bool `json:"quoteOutput"`
Field []string `json:"field"`
FieldMap []FlowField `json:"fieldMap"`
type ModelItem struct {
ModelId int64 `json:"modelId,string"`
ModelName string `json:"modelName"`
ModelFormFields []map[string]any `json:"modelFormFields"`
ModelRequestParams map[string]any `json:"modelRequestParams"`
ModelRequestParamsPath []FlowModelParams `json:"modelRequestParamsPath"`
ModelResponseBodyMapping map[string]any `json:"modelResponseBodyMapping"`
}
type FlowField struct {
Key string `json:"key"`
Value string `json:"value"`
Desc string `json:"desc"`
type FlowModelParams struct {
Label string `json:"label"`
Path string `json:"path"`
Type string `json:"type"`
Required bool `json:"required"`
Value any `json:"value"`
ValueSource []ValueSource `json:"valueSource"`
RefsName string `json:"refsName"`
}
type ValueSource struct {
NodeId string `json:"nodeId"`
Field string `json:"field"`
Label string `json:"label"`
}
// SubFlowConfig 子流程节点配置
type SubFlowConfig struct {
FlowId int64 `json:"flowId"`
MaxConcurrency int `json:"maxConcurrency"` // 子流程并发数
InputSource []FlowNodeInputSource `json:"inputSource"` // 前端指定:来源节点ID
WorkflowId int64 `json:"workflowId,string"`
WorkflowName string `json:"workflowName"`
Fields []map[string]any `json:"fields"`
MaxConcurrency int `json:"maxConcurrency"` // 子流程并发数
}
type FlowEdge struct {
+3
View File
@@ -22,6 +22,7 @@ type NodeExecution struct {
PromptTokens int `orm:"prompt_tokens" json:"promptTokens" description:"提示词token消耗"`
CompletionTokens int `orm:"completion_tokens" json:"completionTokens" description:"补全token消耗"`
TotalTokens int `orm:"total_tokens" json:"totalTokens" description:"总token消耗"`
TokenInfo []map[string]interface{} `orm:"token_info" json:"tokenInfo" description:"token信息"`
Status node.NodeExecutionStatus `orm:"status" json:"status" description:"执行状态:1-运行中,2-成功,3-失败,4-暂停,5-等待执行"`
DurationMs int64 `orm:"duration_ms" json:"durationMs" description:"执行时长(毫秒)"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息"`
@@ -40,6 +41,7 @@ type nodeExecutionCol struct {
PromptTokens string
CompletionTokens string
TotalTokens string
TokenInfo string
Status string
DurationMs string
ErrorMessage string
@@ -58,6 +60,7 @@ var NodeExecutionCol = nodeExecutionCol{
PromptTokens: "prompt_tokens",
CompletionTokens: "completion_tokens",
TotalTokens: "total_tokens",
TokenInfo: "token_info",
Status: "status",
DurationMs: "duration_ms",
ErrorMessage: "error_message",
+22
View File
@@ -0,0 +1,22 @@
package entity
import "gitea.redpowerfuture.com/red-future/common/beans"
// Session 会话记录
type Session struct {
beans.SQLBaseDO `orm:",inherit"` // 嵌入基础字段:Id, TenantId, Creator, CreatedAt, Updater, UpdatedAt, DeletedAt
SessionId string `orm:"session_id" json:"sessionId" description:"会话ID"`
SessionName string `orm:"session_name" json:"sessionName" description:"会话名称"`
}
type sessionCol struct {
beans.SQLBaseCol
SessionId string
SessionName string
}
var SessionCol = sessionCol{
SQLBaseCol: beans.DefSQLBaseCol,
SessionId: "session_id",
SessionName: "session_name",
}
+2 -1
View File
@@ -9,6 +9,7 @@ import (
"fmt"
"sort"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/util/gconv"
)
@@ -66,7 +67,7 @@ func (s *creationInfoService) List(ctx context.Context, req *dto.ListCreationInf
res = &dto.ListCreationInfoRes{
Total: total,
}
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return
}
+12 -12
View File
@@ -9,8 +9,7 @@ import (
"net/http"
"strings"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/utils"
"gitea.redpowerfuture.com/red-future/common/oss"
"github.com/cloudwego/eino/schema"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/glog"
@@ -260,7 +259,7 @@ func GenerateImageLambda(ctx context.Context, input any) (any, error) {
items = append(items, s)
}
}
imgAddressPrefix, err := utils.GetFileAddressPrefix(ctx)
imgAddressPrefix, err := oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
@@ -393,17 +392,18 @@ func getImageBytesFromURL(url string) ([]byte, error) {
}
func Upload(ctx context.Context, req *dto.UploadFileBytesReq) (*dto.UploadFileBytesRes, error) {
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Header {
headers[k] = v[0]
}
}
res := &dto.UploadFileBytesRes{}
err := commonHttp.Post(ctx, "oss/file/uploadFileBytes", headers, res, req)
// 统一走 common/oss:旧实现 POST oss/file/uploadFileBytes 是坏链(oss 服务从未注册该路由,必然 404),
// common/oss 打真实 oss/file/uploadFile,且 multipart field=file、X-User-Info 注入与旧透传等价
res, err := oss.UploadFileBytes(ctx, req.FileName, req.FileBytes)
if err != nil {
glog.Error(ctx, err)
return nil, err
}
return res, nil
return &dto.UploadFileBytesRes{
FileURL: res.FileURL,
FileSize: res.FileSize,
FileName: res.FileName,
FileFormat: res.FileFormat,
FileAddressPrefix: res.FileAddressPrefix,
}, nil
}
+160
View File
@@ -0,0 +1,160 @@
package flow
import (
"context"
"encoding/json"
"errors"
"sync"
"time"
"github.com/gogf/gf/v2/frame/g"
"ai-agent/gateway"
"ai-agent/workflow/consts/model"
flowDao "ai-agent/workflow/dao/flow"
"ai-agent/workflow/model/entity"
)
// 全局等待任务回调的工具
var (
asyncMu sync.Mutex
asyncTasks = make(map[string]chan any)
)
// Wait 阻塞等待回调结果
// 调用后会一直卡住,直到 Notify 唤醒 或 超时/取消
func Wait(ctx context.Context, taskId string) (any, error) {
asyncMu.Lock()
ch := make(chan any, 1)
asyncTasks[taskId] = ch
asyncMu.Unlock()
defer close(ch)
for {
select {
case result := <-ch:
return result, nil
case <-ctx.Done():
asyncMu.Lock()
delete(asyncTasks, taskId)
asyncMu.Unlock()
return nil, ctx.Err()
}
}
}
// Notify 回调时调用,唤醒等待的任务
func Notify(taskId string, result any) {
asyncMu.Lock()
defer asyncMu.Unlock()
ch, exist := asyncTasks[taskId]
if !exist {
return
}
ch <- result
delete(asyncTasks, taskId)
}
// asyncRecoverWaitTimeout in-flight 行重订阅等结果的超时上限。
// 任务可能仍执行中(消息未发布)或提交即失败(不会发布消息);超时视为结果未知,清记录重提。
const asyncRecoverWaitTimeout = 10 * time.Minute
type asyncAction int
const (
asyncActionResubmit asyncAction = iota // 无记录 / failed / done 空结果 → 重新提交
asyncActionReuse // done 有结果 → 复用,不重调
asyncActionFinalize // in-flight → 重订阅 msgTopic 收尾
)
// asyncCallAction 异步任务缓存行动决策(纯函数,可单测)
func asyncCallAction(rec *entity.FlowAsyncTask) asyncAction {
if rec == nil {
return asyncActionResubmit
}
switch rec.State {
case flowDao.FlowAsyncStateDone:
if rec.Result != "" && rec.Result != "{}" {
return asyncActionReuse
}
return asyncActionResubmit
case flowDao.FlowAsyncStateFailed:
return asyncActionResubmit
default: // in-flight
return asyncActionFinalize
}
}
// AsyncModelCallWithRecovery 统一异步模型调用入口:
// 提交时把 task_id/msg_topic 落库 flow_async_task,崩溃后重订阅 msg_topic 拿回已完成结果复用,不重复调用。
// 同步模型直接走 gateway.ModelCallResult,不落库(无恢复语义)。
// 注意:本函数是节点内阻塞调用(WaitModelCallResult 等回调),不是独立并发触发方,
// 不参与 exec 并发仲裁(谁抢到执行权谁跑)——仲裁语义见《工作流执行并发仲裁设计.md》。
// nodeGroupId 是逻辑运行(attempt)标识:缓存唯一键 (node_group_id,node_id,segment_index) 由它隔离,
// 续跑/恢复复用同组即可命中上一 attempt 的 in-flight/done 行(节点级重提/复用按 asyncCallAction)。
func AsyncModelCallWithRecovery(ctx context.Context, nodeGroupId string, execId int64, nodeId string, segIdx int, modelId int64, responseType model.ResponseType, sessionId string, requestParams map[string]any, businessParams map[string]any) (*gateway.ModelCallRes, error) {
if responseType == nil || *responseType != *model.ResponseTypeAsync.Code() {
return gateway.ModelCallResult(ctx, modelId, responseType, sessionId, requestParams, businessParams)
}
rec, err := flowDao.FlowAsyncTaskDao.Get(ctx, nodeGroupId, nodeId, segIdx)
if err != nil {
return nil, err
}
switch asyncCallAction(rec) {
case asyncActionReuse:
return unmarshalModelCallRes(rec.Result)
case asyncActionFinalize:
// 重订阅 msgTopic 收尾:拿回已发布结果(消息在 JetStream 保留 7 天);
// 成功 → 落 done 复用;超时/订阅失败 → 落入下方重提(Upsert 重置同组活行,不删行)
waitCtx, cancel := context.WithTimeout(ctx, asyncRecoverWaitTimeout)
res, waitErr := gateway.WaitModelCallResult(waitCtx, rec.MsgTopic)
cancel()
if waitErr != nil {
if errors.Is(waitErr, context.DeadlineExceeded) || errors.Is(waitErr, context.Canceled) {
break // 落入下方重新提交
}
return nil, waitErr
}
_ = flowDao.FlowAsyncTaskDao.UpdateByKey(ctx, nodeGroupId, nodeId, segIdx, flowDao.FlowAsyncStateDone, marshalModelCallRes(res))
return res, nil
case asyncActionResubmit:
// 清残留语义改为下方 Upsert 重置:failed 或 done 空结果的行是活行(同组从未被软删),
// OnConflict 直接复位为 in-flight + 新 task_id/msg_topic,无需也不可删行重建
}
// 重新提交:先落库 in-flighttask_id/msg_topic),等待结果
res, msgTopic, err := gateway.SubmitModelCall(ctx, modelId, responseType, sessionId, requestParams, businessParams)
if err != nil {
return nil, err
}
if err := flowDao.FlowAsyncTaskDao.Upsert(ctx, nodeGroupId, execId, nodeId, segIdx, modelId, res.TaskId, msgTopic); err != nil {
return nil, err
}
waitRes, waitErr := gateway.WaitModelCallResult(ctx, msgTopic)
if waitErr != nil {
// 结果失败(model error/取消):落 failed,调用方(段重试/恢复)决定后续
_ = flowDao.FlowAsyncTaskDao.UpdateByKey(ctx, nodeGroupId, nodeId, segIdx, flowDao.FlowAsyncStateFailed, "")
return nil, waitErr
}
_ = flowDao.FlowAsyncTaskDao.UpdateByKey(ctx, nodeGroupId, nodeId, segIdx, flowDao.FlowAsyncStateDone, marshalModelCallRes(waitRes))
return waitRes, nil
}
func marshalModelCallRes(res *gateway.ModelCallRes) string {
b, err := json.Marshal(res)
if err != nil {
g.Log().Warningf(context.Background(), "序列化模型调用结果失败: %v", err)
return "{}"
}
return string(b)
}
func unmarshalModelCallRes(s string) (*gateway.ModelCallRes, error) {
res := new(gateway.ModelCallRes)
if err := json.Unmarshal([]byte(s), res); err != nil {
return nil, err
}
return res, nil
}
+443
View File
@@ -0,0 +1,443 @@
package flow
import (
"context"
"errors"
"fmt"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/os/gtime"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
nodeDao "ai-agent/workflow/dao/node"
sessionDao "ai-agent/workflow/dao/session"
flowDto "ai-agent/workflow/model/dto/flow"
nodeDto "ai-agent/workflow/model/dto/node"
"ai-agent/workflow/model/entity"
)
// ====================== 计费本地 DTOshop-user-trade pricing,独立 module 不可 importJSON 对齐) ======================
type pricingOpenOrderReq struct {
UserId int64 `json:"userId"`
SubjectType string `json:"subjectType"`
SubjectID string `json:"subjectId"`
ChargeMode string `json:"chargeMode"`
BizOrderNo string `json:"bizOrderNo"`
}
type pricingGetConfigReq struct {
SubjectType string `json:"subjectType"`
SubjectID string `json:"subjectId"`
}
type pricingGetConfigRes struct {
Enabled int `json:"enabled"`
}
type pricingChargeOrderInfo struct {
ID int64 `json:"id"`
Status int `json:"status"` // 1已建单 2已结算 3已失败
ActualAmount float64 `json:"actualAmount"` // 实收/实扣(元):settle/cancel 响应回填,回写 exec_workflow.actual_amount
CreatedAt string `json:"createdAt"`
ChargeMode string `json:"chargeMode"` // per_item / per_token
}
// pricingSettleReq Settle 与 Cancel 同形状(OrderId + Usage
type pricingSettleReq struct {
OrderId int64 `json:"orderId"`
Usage map[string]any `json:"usage"`
}
type pricingFailReq struct {
OrderId int64 `json:"orderId"`
Reason string `json:"reason"`
}
// 计价对象/模式常量(对齐 shop-user-trade consts/pricing
const (
pricingSubjectWorkflow = "workflow"
pricingChargeModePerItem = "per_item"
pricingChargeModePerSecond = "per_second"
pricingChargeModePerToken = "per_token"
pricingOrderStatusCreated = 1
)
// errBillingGateBlocked 计费门禁拦截(余额不足/钱包不可用/费率非法/未配置计价/用户缺失/per_second 无视频模型):执行终局失败,
// 不进进程内重试、不落 recoverableshouldRetry 与 handleExecute 分类均排除)。
var errBillingGateBlocked = errors.New("计费门禁拦截")
// pricingURL 组装 shop-user-trade 计价接口地址。
// 跨服务路由前缀 /pricing/controller/ 由 common http.RouteRegister 按 controller struct 名推导
// pricingController → pricing/controller),GoFrame doSetHandler 恒 prefix+uri 拼接。
func pricingURL(sub string) string {
return "shop-user-trade/pricing/controller/" + sub
}
// ====================== 建单(执行开始,不动钱) ======================
// openBillingOrder 工作流执行开始建计费单(不动钱)。
// 幂等键 bizOrderNo=wf:{execId};复用终态 execId 重跑(原单已结算/失败)→ 开新单 wf:{execId}:{uuid8}
// 新单 ID 落 exec_workflow.charge_order_id,结算据此定位。
// 门禁(余额>=min_balance、钱包须存在)失败 → errBillingGateBlocked,执行终局失败不重试。
// 计费是工作流执行的前置条件:用户缺失/计价未配置/预检失败/per_second 无视频模型 → 终局失败,不免费跑。
func openBillingOrder(ctx context.Context, execId int64, flowContent *entity.FlowInfo) error {
user, err := utils.GetUserInfo(ctx)
if err != nil || user == nil || user.Id == 0 {
return fmt.Errorf("%w: 取不到用户 %v", errBillingGateBlocked, err)
}
// config/get 预检 = 服务存活探针 + 启用开关:
// 预检失败(服务宕机/路由不通)→ 终局失败;
// 预检通过(服务确认在、计价已开)后再调 open_order,其失败即为业务错误(余额/钱包/费率)→ 阻塞,
// 确保 shop-user-trade 宕机时工作流执行也被阻断而非免费跑。
var cfg pricingGetConfigRes
if err = commonHttp.Get(ctx, pricingURL("config/get"), utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), &cfg,
"subjectType", pricingSubjectWorkflow, "subjectId", pricingSubjectWorkflow); err != nil {
return fmt.Errorf("%w: 计价配置查询失败 %v", errBillingGateBlocked, err)
}
if cfg.Enabled != 1 {
return fmt.Errorf("%w: 计价未启用", errBillingGateBlocked)
}
chargeMode := ""
if flowContent != nil && flowContent.ChargeMode != "" {
chargeMode = flowContent.ChargeMode
} else {
return fmt.Errorf("%w: 工作流执行未选择计费模式", errBillingGateBlocked)
}
// per_second 按秒计费以生成视频总时长为基础,工作流须含视频模型节点,否则配置非法 → 终局失败
if chargeMode == pricingChargeModePerSecond && !flowHasVideoModel(ctx, flowContent) {
return fmt.Errorf("%w: per_second 计费须工作流包含视频模型", errBillingGateBlocked)
}
info, err := openPricingOrder(ctx, int64(user.Id), chargeMode, fmt.Sprintf("wf:%d", execId))
if err != nil {
return err
}
orderId := info.ID
if info.Status != pricingOrderStatusCreated {
// 复用终态 execId:原单已结算/失败,开新单让本次运行独立计费
info, err = openPricingOrder(ctx, int64(user.Id), chargeMode,
fmt.Sprintf("wf:%d:%s", execId, uuid.NewString()[:8]))
if err != nil {
return err
}
orderId = info.ID
}
if err = sessionDao.ExecWorkflowDao.UpdateMap(ctx, execId, map[string]any{
entity.ExecWorkflowCol.ChargeOrderId: orderId,
}); err != nil {
// 结算时按 bizOrderNo=wf:{execId} 兜底定位,不阻塞
glog.Errorf(ctx, "工作流计费:记录 charge_order_id 失败 execId=%d: %v", execId, err)
}
return nil
}
// flowHasVideoModel 工作流是否包含视频模型节点(per_second 计费前置条件)。
// 遍历节点,按 modelId 去重后经 isVideoModel 查模型类型;sub_flow 节点按 subConfig.workflowId
// 递归展开其子工作流(与 SubFlowLambda 运行期解析一致),任一可达层级的视频模型即 true——
// 视频模型放在子工作流内同样满足 per_second 前置条件。
// flowContent/子工作流失联、取子工作流失败按该支无视频模型处理(fail-closedper_second 被拦截)。
// 递归以被展开的子流程 workflowId 去重防环、跨层共享 modelId 去重,避免重复取子流程/重复查模型类型。
func flowHasVideoModel(ctx context.Context, flowContent *entity.FlowInfo) bool {
if flowContent == nil {
return false
}
return hasVideoModelRecursive(ctx, flowContent,
make(map[int64]struct{}), make(map[int64]struct{}))
}
// hasVideoModelRecursive 递归扫描单层 flow 是否含视频模型节点。
// 先查节点自身模型 id,再展开 sub_flow 子工作流递归(sub_flow 节点自身无模型,ModelId 恒 0)。
// expandedSubFlows 记录已展开的子流程 workflowId(展开前先标记,防 A→B→A 无限递归并去重重复引用);
// seenModels 记录已查过的模型 id,跨层共享避免同一模型重复调 isVideoModel。
func hasVideoModelRecursive(ctx context.Context, flowContent *entity.FlowInfo,
expandedSubFlows, seenModels map[int64]struct{}) bool {
for i := range flowContent.Nodes {
n := &flowContent.Nodes[i]
// sub_flow 节点:取子工作流继续递归;取不到/无法解析按该支无视频模型,继续扫其余节点
if sc := n.SubConfig; sc != nil && sc.WorkflowId > 0 {
if _, ok := expandedSubFlows[sc.WorkflowId]; ok {
continue
}
expandedSubFlows[sc.WorkflowId] = struct{}{}
subRes, err := FlowUserService.Get(ctx, &flowDto.GetFlowUserReq{Id: sc.WorkflowId})
if err != nil || subRes == nil || subRes.FlowContent == nil {
continue
}
if hasVideoModelRecursive(ctx, subRes.FlowContent, expandedSubFlows, seenModels) {
return true
}
continue
}
// 普通模型节点:按 modelId 去重后查类型
modelId := n.ModelConfig.ModelId
if modelId <= 0 {
continue
}
if _, ok := seenModels[modelId]; ok {
continue
}
seenModels[modelId] = struct{}{}
if isVideoModel(ctx, modelId) {
return true
}
}
return false
}
// openPricingOrder 调 shop-user-trade 建单(幂等:同 subject+bizOrderNo 返回既有单)
func openPricingOrder(ctx context.Context, userId int64, chargeMode, bizOrderNo string) (*pricingChargeOrderInfo, error) {
info := new(pricingChargeOrderInfo)
err := commonHttp.Post(ctx, pricingURL("open_order"), utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), info, &pricingOpenOrderReq{
UserId: userId,
SubjectType: pricingSubjectWorkflow,
SubjectID: pricingSubjectWorkflow,
ChargeMode: chargeMode,
BizOrderNo: bizOrderNo,
})
if err != nil {
return nil, fmt.Errorf("%w: %v", errBillingGateBlocked, err)
}
if info.ID == 0 {
return nil, fmt.Errorf("%w: 未返回计费单ID", errBillingGateBlocked)
}
return info, nil
}
// ====================== 结算(终态) ======================
// settleBilling 工作流终态计费(recordWorkflow 汇聚全部执行路径后调用):
// 成功→Settle 实收;用户取消→Cancel 按已消耗实收;永久失败(retryable=0/重试耗尽)→Fail 不扣费;
// 可恢复失败→跳过(订单留 CREATED,恢复续跑后结算/失败)。
// 全部计费调用错误仅记日志,不拖垮工作流终态落库。
func settleBilling(ctx context.Context, execId int64, runErr error, retryable, retryCount *int) {
exec, err := sessionDao.ExecWorkflowDao.GetById(ctx, execId)
if err != nil || exec == nil {
glog.Errorf(ctx, "工作流计费:查询执行失败,跳过结算 execId=%d: %v", execId, err)
return
}
orderId := exec.ChargeOrderId
if orderId == 0 {
// 兜底:charge_order_id 未落库(UpdateMap 失败)时按 bizOrderNo=wf:{execId} 定位;查不到→跳过
info, e := getPricingOrder(ctx, "", fmt.Sprintf("wf:%d", execId))
if e != nil || info == nil {
return
}
orderId = info.ID
}
usage, full, err := workflowChargeUsage(ctx, exec, orderId, errors.Is(runErr, context.Canceled))
if err != nil {
glog.Errorf(ctx, "工作流计费:计算用量失败,跳过结算 execId=%d: %v", execId, err)
return
}
// 终局实扣金额:settle/cancel 由 shop 结算响应回填(元);fail / 可恢复=0(未扣费)
var actual float64
switch {
case runErr == nil:
actual, _ = callSettlePricing(ctx, orderId, usage, pricingURL("settle"))
case errors.Is(runErr, context.Canceled):
actual, _ = callSettlePricing(ctx, orderId, usage, pricingURL("cancel")) // Cancel 按已消耗实收
case retryable != nil && *retryable == 0:
callFailPricing(ctx, orderId, runErr.Error())
case retryCount != nil && *retryCount >= execMaxRetryCount:
callFailPricing(ctx, orderId, runErr.Error()) // 重试耗尽 → 永久失败
default:
// 可恢复失败(retryable=1 且未耗尽/关停中断):订单留 CREATED,恢复续跑后结算
}
// 终局回填 exec_workflow(成功/取消/失败/可恢复统一落库,前端与对账均看此行):
// 模型消耗(total_tokens/total_fee,本次运行节点 token_info 按订单窗口聚合)+ 业务实扣
// actual_amount=settle/cancel 实收金额,失败/可恢复未结算=0)。失败/取消路径 SummaryLambda 不跑,
// total 列此前恒为空,此处按同一窗口补齐(与结算口径一致,不混入上一运行残留);
// 可恢复失败也先落已消耗,续跑成功后同一订单收敛重算覆盖。错误仅记日志不拖垮终态落库。
if full != nil {
if err := sessionDao.ExecWorkflowDao.UpdateMap(ctx, execId, map[string]any{
entity.ExecWorkflowCol.TotalTokens: full.TotalTokens,
entity.ExecWorkflowCol.TotalFee: full.TotalFee,
entity.ExecWorkflowCol.ActualAmount: actual,
}); err != nil {
glog.Errorf(ctx, "exec_workflow 回填消耗/实扣失败 execId=%d: %v", execId, err)
}
}
}
// workflowChargeUsage 计算工作流结算用量并返回全量聚合(full,含 TotalTokens/TotalFee 供终局回填 exec 行):
// per_token → feeByModel(各模型按次已消耗费用,结算侧按此合计实收——每次模型调用在发生时已由
// shop /calc 计价,含「不足1分按1分」的按次兜底与调用时媒体/费率快照,不再按聚合 token 重算,
// 避免两笔 0.01 合并重算成 0.01);
// per_item(按条)→ durationItems = 逐条产出时长(每条视频节点执行记录一项,各自落档求和,
// 子流程 maxConcurrency=N → N 条;见根目录《工作流按条(per_item)结算多产出设计.md》);
// per_second → durationSec = 本次生成视频总时长。仅生成视频的工作流按条/秒计费,其余模型调用按 token 计费。
// 用户取消(forCancel=true)时 per_item/per_second 在时长之外补收已消耗 token:把非视频节点的按次费用
// 一并上报(视频节点消耗已由时长计价覆盖,排除防双计)——中途取消通常无视频产出,但文本/分析节点
// 可能已完成并消耗了 token,须按已消耗补收而非按 0 计。
func workflowChargeUsage(ctx context.Context, exec *entity.ExecWorkflow, orderId int64, forCancel bool) (usage map[string]any, full *nodeUsageAgg, err error) {
info, err := getPricingOrder(ctx, gconv.String(orderId), "")
if err != nil {
return nil, nil, err
}
// 按订单收敛:只聚合订单创建后产生的节点记录。每次重跑(上单已终态)开新单,created_at 各自独立,
// 隔离「上次已结算运行」与「本次运行」——否则重跑后取消会把上一次已扣费的 token 一起再扣。
// shop 返回 gtime.String() 无时区本地墙钟,与 node_execution.created_attimestamp without tz)同格式可比。
var createdAtFrom *gtime.Time
if info.CreatedAt != "" {
createdAtFrom = gtime.NewFromStr(info.CreatedAt)
if createdAtFrom == nil || createdAtFrom.IsZero() {
return nil, nil, fmt.Errorf("计费单创建时间解析失败: %s", info.CreatedAt)
}
}
full, nonVideo, err := workflowNodeUsage(ctx, exec, createdAtFrom)
if err != nil {
return nil, nil, err
}
switch {
case info.ChargeMode == pricingChargeModePerToken:
// per_token:按次已记录费用结算(含视频模型——per_token 无时长计价,视频模型按自身按次费用计收)
return tokenUsageMap(full), full, nil
case info.ChargeMode == pricingChargeModePerItem:
// per_item 按条:逐条产出时长各自落档(成功只报逐条;取消附非视频按次已消耗费用补收)
if forCancel {
usage := tokenUsageMap(nonVideo)
usage["durationItems"] = full.DurationItems
return usage, full, nil
}
return map[string]any{"durationItems": full.DurationItems}, full, nil
case forCancel:
// per_second 取消补收:总时长(通常 0)+ 非视频节点按次已消耗费用
// (nonVideo 排除视频节点:其消耗已由时长计价覆盖,避免双计)
usage := tokenUsageMap(nonVideo)
usage["durationSec"] = full.DurationSec
return usage, full, nil
default:
// per_second 正常结算按总时长秒价;不附 token/费用拆分
return map[string]any{"durationSec": full.DurationSec}, full, nil
}
}
// nodeUsageAgg 本次执行聚合出的结算用量:FeeByModel(按生效(系统)模型 id 的按次已记录费用合计,
// shop 实收依据)+ DurationSec(时长,供 per_item/per_second 用)。逐模型 token/媒体明细不上报——
// 每调用一条留在 node_execution.token_infomodel_id/total_tokens/prompt_tokens/completion_tokens/
// media_type/total_fee),订单层按需可从明细再聚合,不再冗余携带。
type nodeUsageAgg struct {
// FeeByModel 各模型窗口内已消耗费用合计 = 节点 token_info.total_fee 求和。total_fee 本身是
// 该节点内各次模型调用(每次经 shop /calc 计价,calculator.charge 内部已 ceilFen,「不足1分按1分」
// 按调用次生效)费用之和 → 此处合计即「按次已记录费用」。结算侧按此实收,不再按聚合 token 重算。
FeeByModel map[string]float64
// DurationSec 各视频节点生成视频总时长(per_second 计价依据、终局展示口径)。
// 注意:per_item 已改按 DurationItems 逐条计价,本字段不再作 per_item 结算入参
DurationSec float64
// DurationItems 本次执行产出的逐条视频时长(per_item 计价依据):一条视频节点执行记录计一项
// (一条 node_execution = 一条成片产出;记录内 token_info 各条 total_duration 合计 >0 计一条)。
// 与 DurationSec(求和口径)并存:per_second 用求和、per_item 用逐条各自落档。
DurationItems []float64
// TotalTokens / TotalFee 全量节点消耗合计(模型消耗 token / 模型按次费用),
// 终局回填 exec_workflow.total_tokens/total_fee(区别于钱包实扣 ActualAmount)。
TotalTokens int64
TotalFee float64
}
// workflowNodeUsage 聚合本次执行(FlowExecutionId + 订单创建时间下界)下各节点执行记录写入的用量。
// 返回两组聚合:
// - full:全部节点记录——per_token 结算(视频模型也按自身按次费用计收,无时长计价)与时长累计用;
// - nonVideo:排除 total_duration>0 的节点(视频节点生成时长,per_item/per_second 取消补收时其消耗
// 已由时长计价覆盖,不再按次补收,避免双计)。
//
// 聚合内容:feeByModel = 各模型 total_fee 求和(total_fee = 该节点内各次模型调用经 shop /calc 计价
// (已 ceilFen)的费用之和 → 按次已记录费用,shop 实收依据);durationSec = 各视频节点生成视频总时长
// (模型返回,total_duration)累加。逐模型 token/媒体明细留在 node_execution.token_info,订单层不上报。
//
// 收敛到当前运行:重跑复用同一 exec 记录与节点组(检查点恢复的 SavedFlowInput 携带旧 node_group_id
// exec_workflow 表也无该列持久化),node_group_id 无法区分运行;改按 created_at >= 订单创建时间过滤——
// 每次重跑开新单,各自 created_at 隔离本次运行消耗,避免把已结算的上一次运行用量一起计入。
func workflowNodeUsage(ctx context.Context, exec *entity.ExecWorkflow, createdAtFrom *gtime.Time) (full, nonVideo *nodeUsageAgg, err error) {
records, _, err := nodeDao.NodeExecutionDao.ListByFlowExecutionId(ctx, &nodeDto.ListNodeExecutionByFlowReq{
FlowExecutionId: exec.Id,
CreatedAtFrom: createdAtFrom,
})
if err != nil {
return nil, nil, err
}
full = &nodeUsageAgg{FeeByModel: make(map[string]float64)}
nonVideo = &nodeUsageAgg{FeeByModel: make(map[string]float64)}
for _, rec := range records {
// 记录级时长:一条 node_execution = 一条成片产出(token_info 内部多段已先加总后 concat,见 ModelLambda)。
// 该记录各 token_info 的 total_duration 合计 >0 → 计一条,供 per_item 逐条落档(子流程多产出=多条同 NodeId 记录)
var recDuration float64
for _, ti := range rec.TokenInfo {
addNodeUsageEntry(full, ti)
// total_duration>0 = 视频节点产出了生成时长(lambda 只有视频模型节点累加并落库该字段),
// 该节点消耗由时长计价覆盖,排除出 nonVideoper_item/per_second 取消补收按此计,防双计)
if d := gconv.Float64(ti["total_duration"]); d > 0 {
recDuration += d
} else {
addNodeUsageEntry(nonVideo, ti)
}
}
if recDuration > 0 {
full.DurationItems = append(full.DurationItems, recDuration)
}
}
return full, nonVideo, nil
}
// addNodeUsageEntry 把单条 token_info(节点一次模型调用或调用汇总)累加进聚合。
// gconv.String 兼容字符串与 JSONB 数字(float64)两种 model_id 写入,避免 (string) 断言丢弃条目。
func addNodeUsageEntry(agg *nodeUsageAgg, ti map[string]any) {
modelID := gconv.String(ti["model_id"])
agg.DurationSec += gconv.Float64(ti["total_duration"])
agg.TotalTokens += gconv.Int64(ti["total_tokens"])
agg.TotalFee += gconv.Float64(ti["total_fee"])
if modelID == "" {
return // 无模型 id(异常条目)不计费用(feeByModel 按模型键)
}
agg.FeeByModel[modelID] += gconv.Float64(ti["total_fee"])
}
// tokenUsageMap 按次费用口径组装结算用量(token 部分):仅 feeByModelshop 实收依据,
// per_token 无建单快照)。逐模型 token/媒体明细留在 node_execution.token_info,订单层不上报。
func tokenUsageMap(agg *nodeUsageAgg) map[string]any {
return map[string]any{
"feeByModel": agg.FeeByModel,
}
}
// getPricingOrder 查计费单:id>0 按ID,否则按 subjectType+subjectId+bizOrderNo
func getPricingOrder(ctx context.Context, id, bizOrderNo string) (*pricingChargeOrderInfo, error) {
info := new(pricingChargeOrderInfo)
var data []any
if id != "" {
data = []any{"id", id}
} else {
data = []any{"subjectType", pricingSubjectWorkflow, "subjectId", pricingSubjectWorkflow, "bizOrderNo", bizOrderNo}
}
err := commonHttp.Get(ctx, pricingURL("order"), utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), info, data...)
if err != nil {
return nil, err
}
if info.ID == 0 {
return nil, errors.New("计费单不存在")
}
return info, nil
}
// callSettlePricing Settle(成功)或 Cancel(用户取消按已消耗实收),返回实收金额(元)。
// shop settle/cancel 响应体是结算后的 ChargeOrderInfoactualAmount=本次实扣),幂等重复结算返回既有金额;
// 调用失败返回 0 并记日志(此时无法确知钱包是否已扣,actual_amount 置 0 保守,不臆造金额)。
func callSettlePricing(ctx context.Context, orderId int64, usage map[string]any, url string) (actual float64, err error) {
info := new(pricingChargeOrderInfo)
if err := commonHttp.Post(ctx, url, utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), info,
&pricingSettleReq{OrderId: orderId, Usage: usage}); err != nil {
glog.Errorf(ctx, "工作流计费:结算失败 orderId=%d: %v", orderId, err)
return 0, err
}
return info.ActualAmount, nil
}
// callFailPricing Fail(不扣费)
func callFailPricing(ctx context.Context, orderId int64, reason string) {
if err := commonHttp.Post(ctx, pricingURL("fail"), utils.HeadersFromCtx(ctx, utils.HeadersOptions{TokenFromQuery: true}), &pricingChargeOrderInfo{},
&pricingFailReq{OrderId: orderId, Reason: reason}); err != nil {
glog.Errorf(ctx, "工作流计费:失败处理失败 orderId=%d: %v", orderId, err)
}
}
+68
View File
@@ -0,0 +1,68 @@
package flow
import (
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"context"
"encoding/json"
"time"
flowDao "ai-agent/workflow/dao/flow"
"github.com/cloudwego/eino/compose"
"github.com/cloudwego/eino/schema"
)
// 注册 checkpoint 序列化类型
func init() {
// ========== 1. Eino 断点核心状态(根类型) ==========
schema.RegisterName[*flowDto.NodeExecutionState]("flow.NodeExecutionState")
schema.RegisterName[*flowDto.NodeExecutionInput]("flow.NodeExecutionInput")
schema.RegisterName[*flowDto.FlowExecutionInput]("flow.FlowExecutionInput")
// ========== 2. 原有第三方类型 ==========
schema.RegisterName[json.Number]("json.Number")
schema.RegisterName[time.Time]("time.Time")
schema.RegisterName[time.Duration]("time.Duration")
// ========== 3. flowDto 内部嵌套类型 ==========
schema.RegisterName[flowDto.ExecutedNode]("flow.ExecutedNode")
// ========== 4. entity 核心链路类型(递归自 *entity.FlowNode ==========
schema.RegisterName[*entity.FlowNode]("entity.FlowNode")
schema.RegisterName[node.NodeType]("node.NodeType")
schema.RegisterName[*entity.SubFlowConfig]("entity.SubFlowConfig")
//schema.RegisterName[node.NodeFormField]("node.NodeFormField")
schema.RegisterName[entity.ModelItem]("node.ModelItem")
// ========== 5. FlowInfo 相关(流程拓扑结构) ==========
schema.RegisterName[entity.FlowInfo]("entity.FlowInfo")
schema.RegisterName[entity.FlowEdge]("entity.FlowEdge")
}
// DbCheckPointStore 数据库存储实现
type DbCheckPointStore struct{}
func NewDbCheckPointStore() compose.CheckPointStore {
return &DbCheckPointStore{}
}
func (d *DbCheckPointStore) Get(ctx context.Context, id string) ([]byte, bool, error) {
// 去掉取消信号:断连/取消场景下 graph ctx 已取消,仅保留值,避免恢复时被阻断
record, err := flowDao.FlowCheckpointDao.Get(context.WithoutCancel(ctx), id)
if err != nil {
return nil, false, err
}
if record == nil || record.Data == "" {
return nil, false, nil
}
return []byte(record.Data), true, nil
}
func (d *DbCheckPointStore) Set(ctx context.Context, id string, val []byte) error {
// 关键:Eino 在节点失败(Interrupt)时用 graph ctx 写 checkpoint。WS 断连/用户终止会使
// 该 ctx 已取消,直接透传会因 "context canceled" 落库失败,断点丢失、续跑失效。
// 去掉取消信号只保留 ctx 值,保证中断时断点必达。
return flowDao.FlowCheckpointDao.SaveOrUpdate(context.WithoutCancel(ctx), id, string(val))
}
+241
View File
@@ -0,0 +1,241 @@
package flow
import (
"context"
"fmt"
"sync"
"sync/atomic"
wsCommon "gitea.redpowerfuture.com/red-future/common/websocket"
)
// ====================== 执行事件中枢(attach to running execution ======================
//
// execHub 是单次工作流执行的事件中枢:把节点进度(node_start/node_complete)与终态
// flow_complete/error)广播到所有订阅的 WS 连接,并暴露执行取消入口(CancelByUser)。
//
// 背景:恢复例程捞起的执行没有 WS 连接、进度被丢弃;用户执行中再点"执行"时,现有代码只发
// round_start "正在执行中" 就返回,收不到进度也无法取消。execHub 让任意时刻建立的连接都能
// 订阅到同一 session+flow 运行中执行的后续进度,并通过 workflow_cancel 停止它。
//
// 生命周期:执行方(handleExecute 的新执行/断点续跑、recoverExecution 的恢复)创建并接管
// SetCancel + MarkOwnedhub,终态落库后 Publish 终态消息并 Close(退订全部连接、注销)。
// 候选 hub 由 handleExecute 提前注册(registerHubIfAbsent),同键已有运行中执行则复用其 hub。
type execHub struct {
sessionId string
flowId int64
execId int64 // 已知后设置,仅用于日志
mu sync.Mutex
subs map[*wsCommon.WsConnection]struct{}
cancel context.CancelFunc // 取消执行(用户执行=execCancel / 恢复=topCancel
owned bool // 已被执行方接管(MarkOwned);未接管视为候选/占位
closed bool
closeOnce sync.Once
done chan struct{}
userCancelled atomic.Bool
}
func newExecHub(sessionId string, flowId int64) *execHub {
return &execHub{
sessionId: sessionId,
flowId: flowId,
subs: make(map[*wsCommon.WsConnection]struct{}),
done: make(chan struct{}),
}
}
// ====================== 注册表(按 sessionId+flowId ======================
var (
hubRegMu sync.Mutex
hubReg = make(map[string]*execHub)
)
func hubKey(sessionId string, flowId int64) string {
return fmt.Sprintf("%s\x00%d", sessionId, flowId)
}
// registerHubIfAbsent 注册 hub;同键已有则返回现有 hub(候选丢弃,返回 created=false)。
// 保证同一进程内同 session+flow 同时只有一个 hub 对象,后续连接统一订阅到它。
func registerHubIfAbsent(sessionId string, flowId int64, hub *execHub) (existing *execHub, created bool) {
key := hubKey(sessionId, flowId)
hubRegMu.Lock()
defer hubRegMu.Unlock()
if h, ok := hubReg[key]; ok {
return h, false
}
hubReg[key] = hub
return hub, true
}
// ====================== 订阅 / 发布 ======================
func (h *execHub) Subscribe(conn *wsCommon.WsConnection) {
h.mu.Lock()
defer h.mu.Unlock()
h.subs[conn] = struct{}{}
}
func (h *execHub) Unsubscribe(conn *wsCommon.WsConnection) {
h.mu.Lock()
defer h.mu.Unlock()
delete(h.subs, conn)
}
// Publish 广播消息到所有订阅连接(跳过已关闭连接;WriteJSON 自带写锁,并发安全)
func (h *execHub) Publish(msg *wsCommon.WsPushMsg) {
h.mu.Lock()
conns := make([]*wsCommon.WsConnection, 0, len(h.subs))
for c := range h.subs {
conns = append(conns, c)
}
h.mu.Unlock()
for _, c := range conns {
if c.IsClosed() {
h.Unsubscribe(c)
continue
}
_ = writeJSON(c, msg)
}
}
// ReportStart / ReportComplete 实现 ProgressReporter:节点进度广播
func (h *execHub) ReportStart(nodeId, nodeName string, nodeIndex, nodeCount int) {
h.Publish(&wsCommon.WsPushMsg{
Type: "node_start",
Message: fmt.Sprintf("开始执行(%d/%d): %s ", nodeIndex, nodeCount, nodeName),
Data: map[string]interface{}{
"nodeId": nodeId, "nodeName": nodeName,
"nodeIndex": nodeIndex, "nodeCount": nodeCount,
},
})
}
func (h *execHub) ReportComplete(nodeId, nodeName string, nodeIndex, nodeCount int) {
h.Publish(&wsCommon.WsPushMsg{
Type: "node_complete",
Message: fmt.Sprintf("执行完成(%d/%d): %s ", nodeIndex, nodeCount, nodeName),
Data: map[string]interface{}{
"nodeId": nodeId, "nodeName": nodeName,
"nodeIndex": nodeIndex, "nodeCount": nodeCount,
},
})
}
// ====================== 接管 / 取消 ======================
// SetCancel 预留取消函数(候选 hub 在尚未确定执行方时设置,供用户提前取消)
func (h *execHub) SetCancel(cancel context.CancelFunc) {
h.mu.Lock()
h.cancel = cancel
h.mu.Unlock()
}
// MarkOwned 标记执行方已接管本 hub(真正开始 BuildExecution 前调用)
func (h *execHub) MarkOwned() {
h.mu.Lock()
h.owned = true
h.mu.Unlock()
}
// TryOwn 原子接管:仅当未被持有才置 owned 并设 cancel,返回是否抢到所有权。
// 并发恢复(扫描/用户附着)用它在锁/DB 前置位,避免"检查 Owned→MarkOwned"竞态下
// 后到者覆盖先到者的 cancel(用户取消会取消错 ctx)。失败者只附着订阅、不重复拉起执行。
func (h *execHub) TryOwn(cancel context.CancelFunc) bool {
h.mu.Lock()
defer h.mu.Unlock()
if h.owned {
return false
}
h.owned = true
h.cancel = cancel
return true
}
func (h *execHub) Owned() bool {
h.mu.Lock()
defer h.mu.Unlock()
return h.owned
}
// CancelByUser 用户显式取消(workflow_cancel):置 userCancelled 标志并取消执行。
// 恢复例程据此把终态写成"用户已终止执行 / retryable=0"(永久取消,不再被扫描捞起)。
func (h *execHub) CancelByUser() {
h.userCancelled.Store(true)
h.mu.Lock()
c := h.cancel
h.mu.Unlock()
if c != nil {
c()
}
}
func (h *execHub) UserCancelled() bool {
return h.userCancelled.Load()
}
// Close 关闭 hub:注销注册、退订全部连接、清其 meta、close donesync.Once,可被
// 执行方与占用方重复调用)。仅当连接 meta 仍指向本 hub 时清空(指针比较防误清新订阅)。
func (h *execHub) Close() {
h.closeOnce.Do(func() {
key := hubKey(h.sessionId, h.flowId)
hubRegMu.Lock()
if hubReg[key] == h {
delete(hubReg, key)
}
hubRegMu.Unlock()
h.mu.Lock()
conns := make([]*wsCommon.WsConnection, 0, len(h.subs))
for c := range h.subs {
conns = append(conns, c)
}
h.closed = true
h.mu.Unlock()
for _, c := range conns {
if cur, ok := wsCommon.GetMetaT[*execHub](c, "execHub"); ok && cur == h {
c.SetMeta("execHub", nil)
c.SetMeta("execCancel", nil)
}
}
close(h.done)
})
}
// Done 返回 hub 关闭通知 channel(供订阅 watcher 等退出)
func (h *execHub) Done() <-chan struct{} {
return h.done
}
// getProgressHub 从 context 取 hubreporter 即 hub 本身)
func getProgressHub(ctx context.Context) *execHub {
if h, ok := GetProgressReporter(ctx).(*execHub); ok {
return h
}
return nil
}
// subscribeConnToHub 把连接挂到 hub 上并接入取消:
// conn meta execCancel = hub.CancelByUser → 现有 workflow_cancel 处理器(handleCancel)直接生效。
// 断开连接不触发取消(用户执行经 closeCtx→execCtx 既有链取消;恢复执行脱离连接,订阅者仅观察)。
// 注意:必须转成 context.CancelFunc 再存,否则 GetMetaT[context.CancelFunc] 的类型断言
// (动态类型须与命名类型完全一致)会因方法值类型为 func() 而失败,取消静默失效。
func subscribeConnToHub(conn *wsCommon.WsConnection, hub *execHub) {
hub.Subscribe(conn)
conn.SetMeta("execHub", hub)
conn.SetMeta("execCancel", context.CancelFunc(hub.CancelByUser))
}
// execAttach 恢复例程附着的用户连接(用户点击执行发现陈旧运行中记录时传入)。
// hub 为连接已订阅的候选 hub:恢复例程复用同一实例接管(registerHubIfAbsent 按 session+flow 去重,
// 全进程同一时刻至多一个 hub 实例),避免"占位被关→重建"竞态导致连接订阅/取消丢失。
type execAttach struct {
conn *wsCommon.WsConnection
sessionId string
flowId int64
hub *execHub // 可能为 nil(调用方无候选时恢复例程自行注册)
}
+159
View File
@@ -0,0 +1,159 @@
package flow
import (
"context"
"errors"
"sync"
"sync/atomic"
"time"
"ai-agent/workflow/consts/flow"
sessionDao "ai-agent/workflow/dao/session"
"ai-agent/workflow/model/entity"
"github.com/gogf/gf/v2/frame/g"
"github.com/google/uuid"
)
// 运行中执行跟踪:优雅关停时取消全部执行(含脱离连接的恢复执行)并等待落完终态再退出,
// 避免"程序停止 → exec 仍卡在 status=1"。
//
// 背景:WS 执行(handleExecute)的 execCtx 派生自连接 closeCtxClose() 取消连接即随之中止落终态;
// 但恢复执行(recoverExecution)的 ctx 经 context.WithoutCancel 脱离连接,程序关停时若不显式取消,
// 恢复中的 exec 不会落终态(仍 status=1),重启要等心跳陈旧 60s 才能再捞。这里登记所有运行中执行,
// SetShuttingDown 统一取消、main 等待全部落库后再退出。
var (
execRunMu sync.Mutex
execRuns = make(map[string]context.CancelFunc)
)
// trackExecRun 登记一次运行中执行,返回 finish 在落完终态(recordWorkflow 后)调用。
// 每次运行唯一 key(重试/断点续跑复用同一 execId 也不冲突)。
func trackExecRun(cancel context.CancelFunc) (finish func()) {
key := uuid.NewString()
execRunMu.Lock()
execRuns[key] = cancel
execRunMu.Unlock()
return func() {
execRunMu.Lock()
delete(execRuns, key)
execRunMu.Unlock()
}
}
// cancelAllExecRuns 优雅关停:取消所有运行中执行。恢复执行 ctx 经 WithoutCancel 脱离连接,
// 必须显式取消才能随 WS 执行一起落终态(status=3/retryable=1)。
func cancelAllExecRuns() {
execRunMu.Lock()
cancels := make([]context.CancelFunc, 0, len(execRuns))
for _, c := range execRuns {
cancels = append(cancels, c)
}
execRunMu.Unlock()
for _, c := range cancels {
c()
}
}
// WaitExecRunsDrain 等待所有运行中执行落完终态(限时),供 main 优雅关停收尾后退出进程。
// 关停后不再启动新执行(execute/reExecute/recoverExecution 顶部有 IsShuttingDown 守卫),
// 因此运行中集合只减不增,轮询安全。
func WaitExecRunsDrain(timeout time.Duration) {
deadline := time.Now().Add(timeout)
for {
execRunMu.Lock()
n := len(execRuns)
execRunMu.Unlock()
if n == 0 {
return
}
if time.Now().After(deadline) {
g.Log().Warningf(context.Background(), "优雅关停等待执行落库超时,剩余 %d 个执行", n)
return
}
time.Sleep(100 * time.Millisecond)
}
}
// errExecAlreadyRunning 用户触发时该执行已在运行(本节点/其它节点后台恢复),不新建执行
var errExecAlreadyRunning = errors.New("工作流正在执行中,不重复执行")
// errInterruptedByShutdown 程序优雅关停导致连接 ctx 取消时的错误标记(区别于用户主动取消)。
// 落库为 status=3 + retryable=1,下次启动恢复扫描捞起续跑。
var errInterruptedByShutdown = errors.New("程序关停中断")
// shuttingDown 优雅关停标记:程序收到退出信号后置位。
// 用于区分"程序关停导致的 WS 连接取消"与"用户主动取消",避免前者被误分类为不可重试。
var shuttingDown atomic.Bool
// SetShuttingDown 置位优雅关停标记并取消所有运行中执行(main 信号处理在 Close() 前调用)。
// 恢复执行 ctx 经 WithoutCancel 脱离连接,必须在此显式取消,否则程序关停时恢复中的 exec
// 不会落终态(仍 status=1);取消后 BuildExecution 随之中止、走恢复错误分支写 status=3/retryable=1。
func SetShuttingDown() {
shuttingDown.Store(true)
cancelAllExecRuns()
}
// IsShuttingDown 是否处于优雅关停
func IsShuttingDown() bool {
return shuttingDown.Load()
}
// shouldRetry 错误分类(retryable 终局语义与 retry_count 预算见《工作流执行并发仲裁设计.md》§5):
// 用户取消不重试;计费门禁拦截不重试;其余程序报错重试
func shouldRetry(err error) bool {
return err != nil && !errors.Is(err, context.Canceled) && !errors.Is(err, errExecAlreadyRunning) &&
!errors.Is(err, errBillingGateBlocked)
}
// isRecoverable 判定可恢复(恢复侧谓词,见《工作流执行并发仲裁设计.md》§2/§5):僵尸运行中(status=1 且心跳陈旧)或可重试失败(status=3 且 retryable=1 且未耗尽)。
// 与 ListRecoverable SQL 判定一致;last_heartbeat=0(老数据/默认)视为陈旧。
func isRecoverable(exec *entity.ExecWorkflow, nowMs int64) bool {
if exec == nil || exec.Status == nil {
return false
}
status := *exec.Status
switch status {
case *flow.FlowExecutionStatusRunning.Code():
return exec.LastHeartbeat < nowMs-int64(heartbeatStaleAfter/time.Millisecond)
case *flow.FlowExecutionStatusFailed.Code():
return exec.Retryable == 1 && exec.RetryCount < execMaxRetryCount
default:
return false
}
}
// startHeartbeat 后台心跳 goroutine:每 30s touch last_heartbeat,返回 stop 函数。
// 覆盖正常执行与恢复执行,崩溃前最后一次心跳即崩溃近似时间戳(心跳=在跑活体标记/租约语义见《工作流执行并发仲裁设计.md》§3)。
// onLeaseLost:心跳连续失败达到陈旧阈值(租约丢失)时回调——调用方应取消执行 ctx,
// 使心跳与执行同生共死,避免"心跳已过期但执行还活着"的窗口被其它节点恢复导致双跑。
func startHeartbeat(ctx context.Context, execId int64, onLeaseLost func()) func() {
stopCh := make(chan struct{})
done := make(chan struct{})
go func() {
defer close(done)
ticker := time.NewTicker(heartbeatInterval)
defer ticker.Stop()
var consecutiveFail int
for {
select {
case <-ticker.C:
if err := sessionDao.ExecWorkflowDao.TouchHeartbeat(ctx, execId); err != nil {
g.Log().Warningf(ctx, "心跳落库失败 execId=%d: %v", execId, err)
consecutiveFail++
if onLeaseLost != nil && consecutiveFail >= heartbeatMaxFail {
g.Log().Errorf(ctx, "心跳连续失败 %d 次,租约丢失,中止执行 execId=%d", consecutiveFail, execId)
onLeaseLost()
}
} else {
consecutiveFail = 0
}
case <-stopCh:
return
case <-ctx.Done():
return
}
}
}()
return func() { close(stopCh); <-done }
}
+46
View File
@@ -0,0 +1,46 @@
package flow
import (
"context"
wsCommon "gitea.redpowerfuture.com/red-future/common/websocket"
)
// ====================== 进度上报 ======================
type wsProgressCtxKey struct{}
// ProgressReporter 节点执行进度回调接口
type ProgressReporter interface {
ReportStart(nodeId, nodeName string, nodeIndex, nodeCount int)
ReportComplete(nodeId, nodeName string, nodeIndex, nodeCount int)
}
// GetProgressReporter 从context中获取进度上报器
func GetProgressReporter(ctx context.Context) ProgressReporter {
if reporter, ok := ctx.Value(wsProgressCtxKey{}).(ProgressReporter); ok {
return reporter
}
return nil
}
// 进度上报由 exec_hub.go 的 execHub 实现(单执行事件中枢,可多连接订阅);wsProgressCtxKey/ProgressReporter/GetProgressReporter 保留。
// handleCancel 取消工作流执行
func handleCancel(ctx context.Context, conn *wsCommon.WsConnection, _ interface{}) {
if cancel := getExecCancel(conn); cancel != nil {
cancel()
}
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "ack", Message: "已取消工作流执行"})
}
// ====================== 工具函数 ======================
func getExecCancel(conn *wsCommon.WsConnection) context.CancelFunc {
cancel, _ := wsCommon.GetMetaT[context.CancelFunc](conn, "execCancel")
return cancel
}
// writeJSON 业务层写入,委托 WsConnection.WriteJSON(共享 writeMu 写锁)
func writeJSON(conn *wsCommon.WsConnection, data interface{}) error {
return conn.WriteJSON(data)
}
+263
View File
@@ -0,0 +1,263 @@
package flow
import (
"context"
"errors"
"fmt"
"time"
"gitea.redpowerfuture.com/red-future/common/oss"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/util/gconv"
"ai-agent/gateway"
"ai-agent/workflow/consts/flow"
"ai-agent/workflow/consts/node"
nodeDao "ai-agent/workflow/dao/node"
sessionDao "ai-agent/workflow/dao/session"
flowDto "ai-agent/workflow/model/dto/flow"
nodeDto "ai-agent/workflow/model/dto/node"
"ai-agent/workflow/model/entity"
)
// ====================== 执行记录落库 ======================
// recordExecutionFailure 记录一次失败状态。有 execId 直接更新该记录;拿不到 execId
// executeOrResume 在创建记录后、返回前 panic,或查询/创建执行记录失败)时,
// 兜底按会话+工作流查最近一条仍处于"运行中"的记录标记为失败,避免前端已报错但记录卡在 Running。
// 最近记录已是成功/失败状态则不处理(可能是上一次执行的结果,不应误改)。
func recordExecutionFailure(ctx context.Context, sessionId string, flowId int64, execId int64, runErr error) {
// 兜底失败路径同样携带 retryable 分类(与 handleExecute 一致):
// 用户取消=0;其余(程序关停中断/程序报错)可重试=1,保证任意 status=3 写库都带分类
var retryable, retryCnt *int
if runErr != nil {
if errors.Is(runErr, context.Canceled) && !IsShuttingDown() {
retryable, retryCnt = intptr(0), intptr(0)
} else {
retryable, retryCnt = intptr(1), intptr(0)
}
}
if !g.IsEmpty(execId) {
recordWorkflow(ctx, execId, 0, runErr, retryable, retryCnt)
return
}
lastExec, err := sessionDao.ExecWorkflowDao.GetLatestBySessionAndFlow(ctx, sessionId, flowId)
if err != nil || lastExec == nil {
glog.Errorf(ctx, "兜底标记失败状态失败: sessionId=%s flowId=%d err=%v", sessionId, flowId, err)
return
}
if lastExec.Status == nil || *lastExec.Status != *flow.FlowExecutionStatusRunning.Code() {
return
}
recordWorkflow(ctx, lastExec.Id, 0, runErr, retryable, retryCnt)
}
// intptr 取 int 值指针(recordWorkflow 的 retryable/retryCount 参数:nil=不改动)
func intptr(v int) *int { return &v }
// recordWorkflow 把一次工作流执行写入 exec_workflow/exec_workflow_result:运行记录 + 输出文件结果。
// retryable/retryCount 非 nil 时随终态同语句原子落库,避免"先 UpdateRetry 再 Update"两步写部分生效
// 导致 retryable 与 status/error_message 不一致(关停中断场景曾出现 retryable=0 但 message=程序关停中断)
func recordWorkflow(ctx context.Context, id int64, duration time.Duration, runErr error, retryable, retryCount *int) {
// exec_workflow 状态沿用 1-运行中,2-成功,3-失败;前端结果卡片也只识别 1/2/3
// 4 会误显示为"运行中"),故取消同样记为失败,错误信息写"用户已终止执行"
// error_message 存友好提示,error 存原始错误明细
status := flow.FlowExecutionStatusSuccess
var errorMessage, errorDetail string
if runErr != nil {
status = flow.FlowExecutionStatusFailed
switch {
case errors.Is(runErr, context.Canceled):
errorMessage = errWorkflowTerminated
case errors.Is(runErr, errInterruptedByShutdown):
errorMessage = "程序关停中断"
default:
errorMessage = "工作流执行失败"
errorDetail = runErr.Error()
}
}
data := map[string]any{
entity.ExecWorkflowCol.Status: *status.Code(),
}
if d := int64(duration.Seconds()); d != 0 {
data[entity.ExecWorkflowCol.Duration] = d
}
if errorMessage != "" {
data[entity.ExecWorkflowCol.ErrorMessage] = errorMessage
}
if errorDetail != "" {
data[entity.ExecWorkflowCol.Error] = errorDetail
}
if retryable != nil {
data[entity.ExecWorkflowCol.Retryable] = *retryable
data[entity.ExecWorkflowCol.RetryCount] = *retryCount
}
if err := sessionDao.ExecWorkflowDao.UpdateMap(ctx, id, data); err != nil {
glog.Errorf(ctx, "exec_workflow 终态落库失败 execId=%d: %v", id, err)
return
}
// 执行成功:重新执行复用了同一条记录,需显式清空,避免上一次失败的报错残留
if runErr == nil {
if _, err := sessionDao.ExecWorkflowDao.ClearError(ctx, id); err != nil {
glog.Errorf(ctx, "exec_workflow 报错信息清空失败: %v", err)
}
}
// 工作流计费:终态结算(成功→Settle/用户取消→Cancel/永久失败→Fail/可恢复→跳过)。
// recordWorkflow 汇聚全部路径(WS/恢复/panic),此处一处接线全覆盖;计费错误仅记日志不拖垮落库
settleBilling(ctx, id, runErr, retryable, retryCount)
}
// workflowResultFileUrls 查询指定工作流执行保存的结果文件路径(带文件前缀,与 session/get 返回一致)
func workflowResultFileUrls(ctx context.Context, execId int64) []string {
results, err := sessionDao.ExecWorkflowResultDao.ListByExecId(ctx, execId)
if err != nil {
glog.Errorf(ctx, "查询工作流结果路径失败: %v", err)
return nil
}
prefix, _ := oss.GetFileAddressPrefix(ctx)
urls := make([]string, 0, len(results))
for _, r := range results {
if r.ResultFileUrl != "" {
urls = append(urls, prefix+r.ResultFileUrl)
}
}
return urls
}
// ====================== 节点执行记录 ======================
// BuildNodeExecutionInput 构建节点执行入参,包含中断恢复逻辑
func BuildNodeExecutionInput(ctx context.Context, input any, flowNode entity.FlowNode) (*flowDto.FlowExecutionInput, *flowDto.NodeExecutionInput, error) {
execInput := new(flowDto.FlowExecutionInput)
wasInterrupted, _, _ := compose.GetInterruptState[any](ctx)
if wasInterrupted {
if err := compose.ProcessState(ctx, func(_ context.Context, s *flowDto.NodeExecutionState) error {
execInput = s.SavedFlowInput
return nil
}); err != nil {
return nil, nil, fmt.Errorf("节点:%v 进程状态读取失败: %v", flowNode.Name, err)
}
// 兼容旧 checkpoint(无 SavedFlowInput 时,降级使用 input 参数)
if execInput == nil {
var ok bool
execInput, ok = input.(*flowDto.FlowExecutionInput)
if !ok {
return nil, nil, fmt.Errorf("节点:%v 进程状态为空,节点入参类型不匹配", flowNode.Name)
}
if g.IsEmpty(execInput) {
return nil, nil, fmt.Errorf("节点:%v 进程状态为空,节点入参参数为空", flowNode.Name)
}
}
// 续跑必定非全新执行:checkpoint 恢复的 SavedFlowInput 里 ForceNewRun 是上次(fresh)执行留下的 true,
// 不清则 ModelLambda 误走"清段重生成"而非复用已成功段
execInput.ForceNewRun = false
} else {
var ok bool
execInput, ok = input.(*flowDto.FlowExecutionInput)
if !ok {
return nil, nil, fmt.Errorf("节点:%v 入参类型不匹配", flowNode.Name)
}
if g.IsEmpty(execInput) {
return nil, nil, fmt.Errorf("节点:%v 入参参数为空", flowNode.Name)
}
}
configMap := execInput.ConfigMap
currentConfig := configMap[flowNode.Id]
if currentConfig == nil {
return nil, nil, fmt.Errorf("节点:%v 节点信息为空", flowNode.Name)
}
// 构建节点执行入参
realInput := &flowDto.NodeExecutionInput{
Config: currentConfig,
Global: execInput,
}
return execInput, realInput, nil
}
// HandleSuccessfulNodeExecution 处理节点执行成功的后续操作
func HandleSuccessfulNodeExecution(ctx context.Context, execInput *flowDto.FlowExecutionInput, realInput *flowDto.NodeExecutionInput, nodeExecutionId int64, flowNode entity.FlowNode, durationMs int64) error {
// 上传输出到OSS
ossResult, err := gateway.Upload(ctx, fmt.Sprintf("nodeInput:%v.txt", time.Now().UnixMilli()), gconv.Bytes(gconv.String(realInput)))
if err != nil {
return fmt.Errorf("节点:%v 上传OSS失败: %v", realInput.Config.Name, err)
}
// 更新执行记录为成功
if err := UpdateNodeExecutionRecord(ctx, nodeExecutionId, durationMs, node.NodeExecutionStatusSuccess.Code(), ossResult, ""); err != nil {
return fmt.Errorf("节点:%v 更新成功状态错误: %v", flowNode.Name, err)
}
// 记录成功到已执行列表
RecordExecutionResult(execInput, flowNode.Id, node.NodeExecutionStatusSuccess.Code())
return nil
}
// HandleFailedNodeExecution 处理节点执行失败的后续操作
func HandleFailedNodeExecution(ctx context.Context, execInput *flowDto.FlowExecutionInput, nodeExecutionId int64, flowNode entity.FlowNode, err error, durationMs int64) error {
// 保存状态用于续跑
if stateErr := compose.ProcessState(ctx, func(_ context.Context, s *flowDto.NodeExecutionState) error {
s.CompletedNodes = append(s.CompletedNodes, flowNode.Name)
s.SavedFlowInput = execInput
s.ExecutionCount++
return nil
}); stateErr != nil {
fmt.Printf("节点:%v 进程状态保存失败: %v", flowNode.Name, stateErr)
}
if !g.IsEmpty(nodeExecutionId) {
// 更新执行记录为失败
if updateErr := UpdateNodeExecutionRecord(ctx, nodeExecutionId, durationMs, node.NodeExecutionStatusFailed.Code(), "", err.Error()); updateErr != nil {
fmt.Printf("节点:%v 更新失败状态错误: %v", flowNode.Name, updateErr)
}
}
// 触发中断
return compose.Interrupt(ctx, map[string]string{
"node": flowNode.Name,
"error": err.Error(),
})
}
// RecordExecutionResult 将节点执行结果写入 Global.ExecutedNodes
func RecordExecutionResult(execInput *flowDto.FlowExecutionInput, nodeId string, status node.NodeExecutionStatus) {
execInput.ExecutedNodes = append(execInput.ExecutedNodes, flowDto.ExecutedNode{
NodeId: nodeId,
Status: status,
})
}
// UpdateNodeExecutionRecord 更新节点执行记录
func UpdateNodeExecutionRecord(ctx context.Context, nodeExecutionId int64, durationMs int64, status node.NodeExecutionStatus, outputParamsPath string, errMsg string) error {
if _, err := nodeDao.NodeExecutionDao.Update(ctx, &nodeDto.UpdateNodeExecutionReq{
Id: nodeExecutionId,
DurationMs: durationMs,
Status: status,
OutputParamsPath: outputParamsPath,
ErrorMessage: errMsg,
}); err != nil {
return err
}
return nil
}
// CreateNodeExecutionRecord 创建节点执行记录,返回记录ID
func CreateNodeExecutionRecord(ctx context.Context, execInput *flowDto.FlowExecutionInput, flowNode entity.FlowNode, inputOssUrl string) (int64, error) {
id, err := nodeDao.NodeExecutionDao.Insert(ctx, &nodeDto.CreateNodeExecutionReq{
FlowExecutionId: execInput.ExecutionId,
NodeId: flowNode.Id,
NodeName: flowNode.Name,
NodeGroupId: execInput.NodeGroupId,
InputParamsPath: inputOssUrl,
Status: node.NodeExecutionStatusRunning.Code(),
})
if err != nil {
return 0, fmt.Errorf("节点:%v 创建节点执行记录失败: %v", flowNode.Name, err)
}
return id, nil
}
+236
View File
@@ -0,0 +1,236 @@
package flow
import (
"context"
"errors"
"fmt"
"time"
sessionDao "ai-agent/workflow/dao/session"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/utils"
wsCommon "gitea.redpowerfuture.com/red-future/common/websocket"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/gtrace"
"github.com/google/uuid"
)
// 恢复相关常量(可参数化调整)
const (
heartbeatInterval = 30 * time.Second // 心跳 touch 间隔
heartbeatStaleAfter = 2 * heartbeatInterval // 心跳陈旧阈值(>2×间隔),小于此视为僵尸
recoverScanInterval = 30 * time.Second // 周期扫描间隔
recoverLockTTL = 15 * time.Minute // 恢复锁 TTL(覆盖长时间续跑)
recoverLockPhase = 2 * time.Minute // 抢锁/前置判定阶段总时限(Redis SET + 一次 DB 读,毫秒级)
recoverExecTimeout = 12 * time.Hour // 单次恢复执行最长时长;超时按可重试失败落库交下一轮扫描
heartbeatMaxFail = 2 // 心跳连续失败达到陈旧阈值(2×30s=60s)即租约丢失,中止执行
execMaxRetryCount = 2 // 整次执行最多自动重试 2 次(共 3 次尝试)
)
// StartRecoveryLoop 启动恢复扫描:先立即扫一次,再周期扫描。
// 多节点各自扫描,靠 Redis 锁对同一 exec 抢占去重(Redis 锁只管瞬时互斥、运行期靠 DB 心跳,见《工作流执行并发仲裁设计.md》§1/§2)
func StartRecoveryLoop(ctx context.Context) {
go func() {
scanAndRecover(ctx)
ticker := time.NewTicker(recoverScanInterval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
scanAndRecover(ctx)
case <-ctx.Done():
return
}
}
}()
}
// scanAndRecover 扫描可恢复执行并逐个异步恢复(不阻塞扫描循环)。
// 恢复运行在无 HTTP 用户的后台:ListRecoverable 走跨租户 NoTenantId,需要 ctx 携带 OTel span
// NoTenantId 以 traceID 为 gcache 标记键,无 span 时 getTraceID 返回空 → NoTenantId 返回 nil 会 panic
func scanAndRecover(ctx context.Context) {
// 优雅关停期间不再捞起:连接 ctx 刚被取消、exec 正在落终态,避免本进程内部用合成用户重跑
if IsShuttingDown() {
return
}
scanCtx, span := gtrace.NewSpan(ctx, "workflow.recover.scan")
defer span.End()
now := time.Now().UnixMilli()
rows, err := sessionDao.ExecWorkflowDao.ListRecoverable(scanCtx, now, now-int64(heartbeatStaleAfter/time.Millisecond), execMaxRetryCount)
if err != nil {
g.Log().Errorf(scanCtx, "扫描可恢复执行失败: %v", err)
return
}
for _, r := range rows {
// 扫描触发无附着连接(attach=nil):hub 在确认可恢复后按 exec 的 session+flow 建立
go recoverExecution(context.WithoutCancel(scanCtx), r.Id, nil)
}
}
// recoverExecution 统一恢复例程:抢锁 → 判定(isRecoverable)→ 条件重置抢权 → 置运行中续跑
// (触发源/抢权闸/输家纪律见《工作流执行并发仲裁设计.md》§1/§2/§4)。
// 两个触发源共用:启动/周期扫描(attach=nil)、executeOrResume 对僵尸行的用户触发(attach 携带连接)。
func recoverExecution(parentCtx context.Context, execId int64, attach *execAttach) {
// 优雅关停期间不再启动新的恢复执行(周期扫描已有守卫;用户 executeOrResume 触发路径这里兜底)
if IsShuttingDown() {
return
}
// 在函数最前面创建并登记本执行的 cancel:trackExecRun 绑定最终控制执行的 cancel,
// 提前登记保证优雅关停(SetShuttingDown→cancelAllExecRuns)快照必然覆盖到本执行(已登记),
// 或本执行在开始前置判定前自行发现关停标记放弃——不留"已置 status=1 却无执行"的无主记录。
topCtx, topCancel := context.WithCancel(context.WithoutCancel(parentCtx))
defer topCancel()
finish := trackExecRun(topCancel)
defer finish()
// 登记后复查关停标记:已置位说明 cancelAllExecRuns 快照早于本登记(未覆盖本执行),
// 此时本执行尚未走 ResetRunningIfRecoverable(不会留下 status=1 无主执行),直接放弃;
// 若置位发生在复查之后,本执行已被快照覆盖,随关停取消并在错误分类路径落终态(status=3/retryable=1
if IsShuttingDown() {
return
}
// 事件中枢(attach 路径):用户点击执行触发恢复时,复用连接已订阅的候选 hub(executeOrResume 传入),
// 让该连接订阅到后续节点进度与终态、并能通过 workflow_cancel 永久取消(hub.CancelByUser→topCancel)。
// 此处只订阅不 TryOwn:所有权由"抢到锁+重置权"的执行者确定;提前退出路径(锁输/不可恢复/重置失败)
// 不关 hub,保证最终执行者与连接指向同一 hub 实例,取消/进度不因竞态丢失。
var hub *execHub
if attach != nil {
if attach.hub != nil {
hub = attach.hub
} else {
hub, _ = registerHubIfAbsent(attach.sessionId, attach.flowId, newExecHub(attach.sessionId, attach.flowId))
}
subscribeConnToHub(attach.conn, hub)
}
lockKey := fmt.Sprintf("workflow:exec:recover:%d", execId)
// 抢锁/前置判定用独立短超时 ctx:该阶段只做 Redis SET + 一次 DB 读 + 一次条件重置,应毫秒级完成
lockCtx, lockCancel := context.WithTimeout(context.WithoutCancel(parentCtx), recoverLockPhase)
defer lockCancel()
// 恢复体整体包进 utils.WithLock(自动续期 + 单次尝试,替代本地对象形态锁 redis_lock.go):
// - 自动续期:15min TTL 覆盖整段执行。长执行原本就靠心跳陈旧 + 条件重置防双跑,
// 锁持满只是让其它节点提前「锁忙跳过」;节点崩溃续期停 → TTL 过期 → 其它节点照常捞起;
// - 单次尝试(retryTimes=1):原「被其它节点持有就跳过」语义,抢不到不等待。
ok, err := utils.WithLock(lockCtx, lockKey, int64(recoverLockTTL/time.Second), func(_ context.Context) error {
// 抢锁后重读:此刻租户未知(该 exec 可能属于任意租户),必须跨租户读
exec, err := sessionDao.ExecWorkflowDao.GetByIdNoTenant(lockCtx, execId)
if err != nil || exec == nil {
return nil
}
if !isRecoverable(exec, time.Now().UnixMilli()) {
return nil // 锁等待期间状态已变化(其它节点已恢复/用户取消/已耗尽)
}
// 事件中枢(scan 路径,无附着连接):此刻才知道 exec 的 session+flow,建 hub 供后续用户连接附着
//(所有权在重置成功后统一接管,见下)
if attach == nil {
hub, _ = registerHubIfAbsent(exec.SessionId, exec.FlowId, newExecHub(exec.SessionId, exec.FlowId))
}
// 置运行中并续跑(BuildExecution false → 断点续跑:checkpoint 有则从中继续,无则从头 + 段/memo 复用)。
// 心跳在条件重置成功后才启动:避免对未抢到重置权的 exec 空跑心跳
saveCtx := context.WithoutCancel(parentCtx)
// 恢复无 HTTP 用户,但图节点 lambda 的 INSERTnode_execution/flow_async_task/segment_result)走
// insertHook 硬性要求 user,且节点模型调用外发 model-gateway 带 X-User-Info 要过单次调用最低
// 余额门禁(须 user.Id>0)——用 exec 所属租户合成系统用户 ctx(Id 取创建时落库的 user_id
// creator 仅 userName 推不回数字 id;旧记录 user_id=0 时其恢复续跑会被该门禁拦截),保留 span。
userCtx := context.WithValue(saveCtx, "user", &beans.User{Id: uint64(exec.UserId), UserName: exec.Creator, TenantId: exec.TenantId})
// 恢复 = 同一逻辑运行继续:复用 exec 记录的组读其 checkpoint 续跑(不换组,否则读不到断点从头跑)。
// 仅旧记录无组时新造并随重置持久化;置运行中与图执行的组标识须一致。
nodeGroupId := exec.NodeGroupId
if nodeGroupId == "" {
nodeGroupId = uuid.NewString()
}
// 条件重置(原子防与用户断点续跑双跑):仅当仍可恢复(status=3 或 status=1 心跳陈旧)时才抢到重置权
staleBeforeMs := time.Now().UnixMilli() - int64(heartbeatStaleAfter/time.Millisecond)
reset, err := sessionDao.ExecWorkflowDao.ResetRunningIfRecoverable(userCtx, execId, nodeGroupId, staleBeforeMs)
if err != nil {
g.Log().Errorf(lockCtx, "恢复置运行中失败 execId=%d: %v", execId, err)
return nil
}
if !reset {
// 状态已被其它路径(用户 reExecute/execute 或其它节点恢复)抢先重置:放弃续跑,状态由持有方收敛,不落终态
g.Log().Infof(lockCtx, "execId=%d 已被其它路径抢先重置为运行中,跳过恢复", execId)
return nil
}
// 事件中枢所有权:此时已抢到锁+重置权,本 goroutine 是实际执行者,接管 hubTryOwn 原子置 owned+cancel)。
// 已有持有者(同 session+flow 另有执行在跑)则本恢复不持有 hub:锁与重置权已到手,放弃会留 status=1
// 无主,继续执行但进度不广播(恢复仍正常完成落终态)。
if hub != nil {
if !hub.TryOwn(topCancel) {
hub = nil
} else {
defer hub.Close()
// 恢复建立前用户已取消(占位期 workflow_cancel 只置标志、cancel 尚未设置):立即中止,
// 走下方 UserCancelled 分类写永久取消(retryable=0
if hub.UserCancelled() {
topCancel()
}
}
}
// 执行与心跳绑定同一 execCtx(带 recoverExecTimeout 上限):
// 心跳停止(超时/进程死/租约丢失)与 BuildExecution 中止必须同步,否则出现
// "心跳已过期但执行还活着" 的窗口,被其它节点扫描恢复导致双跑。
// 心跳连续失败达陈旧阈值时回调 execCancel 中止执行(心跳与执行同生共死)。
// 从函数顶部登记的 topCtx 派生执行 ctx:注入用户信息(供 insertHook 落库)+ 12h 执行超时上限。
// 关停时 cancelAllExecRuns 取消 topCancel → 本 ctx 随之取消,BuildExecution 中止后走下方错误分类落终态。
execCtx, execCancel := context.WithTimeout(context.WithValue(topCtx, "user", &beans.User{Id: uint64(exec.UserId), UserName: exec.Creator, TenantId: exec.TenantId}), recoverExecTimeout)
defer execCancel()
stop := startHeartbeat(execCtx, execId, execCancel)
defer stop()
// 图节点进度经 hub 广播(无 hub 时 getProgressHub 返回 nilreporter nil 安全)
progressCtx := execCtx
if hub != nil {
progressCtx = context.WithValue(execCtx, wsProgressCtxKey{}, hub)
}
err = BuildExecution(progressCtx, false, exec.FlowId, execId, nodeGroupId, exec.SessionId, exec.RequestParams)
if err != nil {
// 用户显式取消(附着连接 workflow_cancel):永久取消,retryable=0,恢复扫描不再捞起,
// 杜绝"取消→恢复→再取消"循环;产物(checkpoint/段/异步缓存)保留,落"用户已终止执行"。
// 与 WS 路径一致:失败/取消不清,统一由"成功尾部 / 下一次 forceNewRun 起跑前"清理
if hub != nil && hub.UserCancelled() {
retryable, retryCnt := 0, 0
recordWorkflow(userCtx, execId, 0, context.Canceled, &retryable, &retryCnt)
hub.Publish(&wsCommon.WsPushMsg{Type: "error", Message: errWorkflowTerminated})
return nil
}
// 续跑失败:恢复例程无用户,任意错误(含执行超时/租约丢失取消/模型/DB/网络/panic
// 一律 retryable=1 交下一轮扫描决定是否再恢复;重试耗尽才终局失败。
// 产物(checkpoint/段/异步缓存)保留不在此清:重试耗尽后用户仍可手动同参数续跑
// 复用已成功段/异步结果,换参数则由 forceNewRun 起跑前统一清理(见 BuildExecution
retryable, retryCnt := 1, exec.RetryCount+1
// 优雅关停导致的取消:换错误标记让 recordWorkflow 写"程序关停中断"(与 WS 路径一致),
// 仍 retryable=1 下次启动扫描捞起续跑;非关停的取消(租约丢失/执行超时)保留原错误
if errors.Is(err, context.Canceled) && IsShuttingDown() {
err = errInterruptedByShutdown
}
recordWorkflow(userCtx, execId, 0, err, &retryable, &retryCnt)
// 终态广播(附着连接;scan 路径无连接则无人接收)
if hub != nil {
hub.Publish(&wsCommon.WsPushMsg{Type: "error", Message: "工作流执行失败", Error: err.Error()})
}
return nil
}
// 续跑成功:recordWorkflow 落 status=2BuildExecution 已清理 checkpoint/segment_result/flow_async_task
recordWorkflow(userCtx, execId, 0, nil, nil, nil)
// 终态广播:把本次执行保存的结果文件路径一并推给前端
if hub != nil {
hub.Publish(&wsCommon.WsPushMsg{Type: "flow_complete", Message: "工作流执行完成", Data: map[string]interface{}{
"resultFileUrls": workflowResultFileUrls(userCtx, execId),
}})
}
return nil
})
if err != nil {
g.Log().Errorf(lockCtx, "恢复抢锁失败 execId=%d: %v", execId, err)
return
}
if !ok {
g.Log().Infof(lockCtx, "恢复锁被其它节点持有,跳过 execId=%d", execId)
return
}
}
+538
View File
@@ -0,0 +1,538 @@
package flow
import (
"ai-agent/workflow/consts/flow"
flowDao "ai-agent/workflow/dao/flow"
sessionDao "ai-agent/workflow/dao/session"
flowDto "ai-agent/workflow/model/dto/flow"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"gitea.redpowerfuture.com/red-future/common/utils"
wsCommon "gitea.redpowerfuture.com/red-future/common/websocket"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
)
// ====================== WebSocket 服务器 ======================
func init() {
// 工作流消息处理器注册在统一的 SessionWsService 上:
// 首次连接仅升级,连接后按消息 type 路由,不再建连时区分普通对话/工作流
SessionWsService.OnMessage("workflow", handleExecute)
SessionWsService.OnMessage("workflow_cancel", handleCancel)
}
// defaultSessionName 工作流执行但查不到流程名时,会话的兜底名称
const defaultSessionName = "工作流执行"
// errWorkflowTerminated 前端终止工作流执行时的错误标记(写入 exec_workflow.error_message
var errWorkflowTerminated = "用户已终止执行"
// ====================== 消息处理 ======================
// handleExecute 处理工作流执行(由 workerPool 异步调用,不阻塞读循环)
func handleExecute(ctx context.Context, conn *wsCommon.WsConnection, payload interface{}) {
execPayload := new(sessionDto.WebSocketExecWorkflowReq)
if err := gconv.Struct(payload, execPayload); err != nil {
glog.Errorf(ctx, "工作流执行参数解析失败: %v", err)
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "error", Message: "执行参数解析失败", Error: err.Error()})
return
}
execCtx, execCancel := context.WithCancel(ctx)
// 同流程运行中再点"执行" = 附着看进度(此处不做预取消):下方 registerHubIfAbsent 复用同
// session+flow 的运行中 hubexecuteOrResume 返回 errExecAlreadyRunning,本连接附着其进度/取消,
// 不新建执行。原 getExecCancel 无条件预取消会把运行中的同流程执行一并杀掉,与附着语义冲突,故弃用。
// 仅当本连接 meta execHub 指向的是另一流程(换流程执行)的运行中 hub 时终止它:
// 避免会话内串跑两个流程、旧流程在后台继续消耗计费(workflow_cancel 是显式取消入口)。
if prev, ok := wsCommon.GetMetaT[*execHub](conn, "execHub"); ok && prev != nil && prev.flowId != execPayload.FlowId {
prev.CancelByUser()
}
// 异步执行工作流(直接 goroutine,不依赖上游 workerPool 二次排队)
go func() {
// 事件中枢:同 session+flow 已有运行中执行(本进程)则复用其 hub,让本连接附着其进度;
// 否则新建注册为候选,由本执行(execute/reExecute)或恢复例程接管
hub, _ := registerHubIfAbsent(conn.SessionId, execPayload.FlowId, newExecHub(conn.SessionId, execPayload.FlowId))
if !hub.Owned() {
// 候选/未持有:预留取消为当前连接的 execCancel(新执行 MarkOwned 后生效);
// 已持有(同 session+flow 运行中执行)则保留持有者 cancel,避免覆盖导致取消错对象
hub.SetCancel(execCancel)
}
subscribeConnToHub(conn, hub)
progressCtx := context.WithValue(execCtx, wsProgressCtxKey{}, hub)
// 登记运行中执行:优雅关停(SetShuttingDown)统一取消本执行;finish 在落完终态后解除登记,
// main 的 WaitExecRunsDrain 据此等待 WS 执行落库后再退出(避免进程退出时记录仍卡 status=1)
finish := trackExecRun(execCancel)
defer finish()
// 落库用不带取消的 ctx(保留 request 值),保证前端终止/断连后记录仍能写入
saveCtx := context.WithoutCancel(ctx)
start := time.Now()
var execId int64
var execErr error
owner := false // 本 goroutine 是否为执行持有者(附着/触发恢复时不持有)
// 持有者收尾:落完终态广播后关闭 hub(退订全部连接/清 meta/注销)。
// 附着路径不置 owner,由运行中的持有方统一 Closepanic 路径在下方 recover defer 中置 owner 兜底。
// 先注册此 defer → 后注册的 panic defer 先执行(先广播再 Close
defer func() {
if owner {
hub.Close()
}
}()
defer func() {
if r := recover(); r != nil {
glog.Errorf(execCtx, "workflow panic: %v", r)
execErr = fmt.Errorf("工作流异常: %v", r)
// 只置错误与所有权,不做终态落库/广播:recover 后本 goroutine 从 panic 处(execId/execErr
// 赋值可能未完成)继续执行,下方常规错误路径恰好覆盖此场景且只走一遍——
// execId=0 → else-if 兜底 recordExecutionFailure(按会话+流程查最近 Running 标记失败);
// execId>0 → recordWorkflow;随后单次终态 Publish。若在此提前落库/广播,主路径会重复
// 一次(前端连收两条错误、兜底标记被调两遍)。
// panic 必然发生在本 goroutine 持有的执行内(附着/恢复路径不跑 BuildExecution,不会 panic
owner = true
}
}()
// 会话落库:前端 sessionId 对应会话已存在则复用,否则按流程名新建
flowName := defaultSessionName
if flowUser, e := flowDao.FlowUserDao.Get(saveCtx, &flowDto.GetFlowUserReq{Id: execPayload.FlowId}); e == nil && flowUser != nil && flowUser.FlowName != "" {
flowName = flowUser.FlowName
}
if e := ensureSession(saveCtx, conn.SessionId, flowName); e != nil {
glog.Errorf(saveCtx, "工作流会话创建失败: %v", e)
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "error", Message: "工作流会话创建失败", Error: fmt.Sprintf("%v", e)})
}
// flowContent 经 WS 的 gconv 解析不跑 v:"required" 校验,缺省时按 0 节点提示,不 panic
nodeCount := 0
if execPayload.FlowContent != nil {
nodeCount = len(execPayload.FlowContent.Nodes)
}
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "ack", Message: fmt.Sprintf("开始执行工作流(共 %d 个节点)", nodeCount)})
execId, execErr = executeOrResume(progressCtx, conn, execPayload)
if errors.Is(execErr, errExecAlreadyRunning) {
// 已附着到运行中执行 / 已触发恢复 / 其它节点在跑:
// 连接已订阅到 hub(运行中持有者广播进度与终态并统一 CloseexecuteOrResume 对占位 hub 已 Close)。
// 本 goroutine 不落终态、不 Closeowner=false),由持有方收敛。
return
}
owner = true
if !g.IsEmpty(execId) {
hub.execId = execId
}
// in-process 重试:程序报错且未耗尽 → retry_count++ 落库(保持 status=1,前端不闪失败)→ 续跑
retryCount := 0
for execErr != nil && execId > 0 && shouldRetry(execErr) && retryCount < execMaxRetryCount {
retryCount++
if err := sessionDao.ExecWorkflowDao.UpdateRetry(saveCtx, execId, 1, retryCount); err != nil {
glog.Errorf(saveCtx, "重试标记落库失败 execId=%d: %v", execId, err)
break
}
glog.Infof(saveCtx, "工作流执行失败,自动重试 %d/%dexecId=%d: %v", retryCount, execMaxRetryCount, execId, execErr)
// 进程内重试:exec 仍为本进程持有的 status=1 运行中,条件重置仅允许从运行中重置
_, execErr = reExecute(progressCtx, execId, *flow.FlowExecutionStatusRunning.Code())
if errors.Is(execErr, errExecAlreadyRunning) {
return // 状态已被其它路径抢占:不写终态,由对方收敛
}
}
// 错误分类:程序关停取消 → retryable=1(下次启动恢复续跑);
// 用户取消 → retryable=0 不重试;
// 程序报错(非取消)→ retryable=1retry_count 已记(重试循环内递增)
// retryable/retry_count 随终态 recordWorkflow 一次原子写入,不单独 UpdateRetry
var retryable, retryCnt *int
if execErr != nil {
switch {
case errors.Is(execErr, context.Canceled) && IsShuttingDown():
// 关停标记置位:连接 ctx 取消来自程序优雅关停,非用户取消,
// 换错误标记让 recordWorkflow 写"程序关停中断",并可重试以便下次启动恢复
retryable, retryCnt = intptr(1), intptr(retryCount)
execErr = errInterruptedByShutdown
case errors.Is(execErr, context.Canceled):
retryable, retryCnt = intptr(0), intptr(0)
case errors.Is(execErr, errBillingGateBlocked):
// 计费门禁拦截:余额不足/钱包不可用/费率非法,终局失败不重试不恢复
retryable, retryCnt = intptr(0), intptr(0)
default:
// 程序报错且重试耗尽 → exec 永久失败:async/segment/checkpoint 一律**保留**
// 供用户同参数再点续跑(reExecute)复用已产出、免重复调模型/免双扣;
// 换参数重跑走 execute→forceNewRunBuildExecution 起跑前统一清空,不误用旧残留
retryable, retryCnt = intptr(1), intptr(retryCount)
}
}
if !g.IsEmpty(execId) {
glog.Infof(saveCtx, "工作流执行完成,execId: %v", execId)
recordWorkflow(saveCtx, execId, time.Since(start), execErr, retryable, retryCnt)
} else if execErr != nil {
// 查询/创建执行记录失败(拿不到 execId)时,兜底把该会话+工作流最近一条"运行中"记录标记为失败
recordExecutionFailure(saveCtx, conn.SessionId, execPayload.FlowId, execId, execErr)
}
if execErr != nil {
// 终态广播(发起连接 + 附着订阅者)
hub.Publish(&wsCommon.WsPushMsg{Type: "error", Message: "工作流执行失败", Error: execErr.Error()})
return
}
// 成功:把本次执行保存的结果文件路径(exec_workflow_result)一并推给前端
hub.Publish(&wsCommon.WsPushMsg{
Type: "flow_complete",
Message: "工作流执行完成",
Data: map[string]interface{}{
"resultFileUrls": workflowResultFileUrls(saveCtx, execId),
},
})
}()
}
// executeOrResume 是"并发触发仲裁"的用户侧决策点:同一条 exec 的多个触发方
// (自动重试 / 手动续跑 / 恢复扫描 / 用户点击)同时发生时,谁真正跑由条件重置
// ResetRunning/ResetRunningIfRecoverableDB 行级原子)定夺,输家一律附着观察或
// 放弃(errExecAlreadyRunning),绝不双跑。点击落点分情形(status=1 心跳新鲜→附着 /
// 陈旧→触发恢复 / status=3 同参→手动续跑 / 其余→execute)见根目录
// 《工作流执行并发仲裁设计.md》§1/§4。
//
// executeOrResume 决策工作流执行方式:
// - 同会话+同工作流的最近一次执行失败,且本次传递参数与上次一致 → 断点续跑(reExecute,复用原执行记录,从失败断点继续)
// - 其余情况(上次成功 / 上次参数与本次不同 / 无历史记录 / 查询出错)→ 全新执行(execute)
func executeOrResume(ctx context.Context, conn *wsCommon.WsConnection, req *sessionDto.WebSocketExecWorkflowReq) (id int64, err error) {
lastExec, err := sessionDao.ExecWorkflowDao.GetLatestBySessionAndFlow(ctx, conn.SessionId, req.FlowId)
if err != nil {
glog.Errorf(ctx, "查询最近工作流执行记录失败: %v", err)
return 0, fmt.Errorf("查询最近工作流执行记录失败: %v", err)
}
if lastExec != nil {
if *lastExec.Status == *flow.FlowExecutionStatusRunning.Code() {
// status=1:可能正在跑(本节点或其它节点后台恢复)或僵尸遗留。
// 心跳新鲜 → 真在跑,不新建避免双跑;心跳陈旧 → 僵尸,触发后台恢复。
// 都不新建执行:恢复在后台完成,完成后状态自然收敛——输家不写终态、由持有方收敛
// (仲裁语义见《工作流执行并发仲裁设计.md》§1/§4)
nowMs := time.Now().UnixMilli()
hub := getProgressHub(ctx)
owned := hub != nil && hub.Owned()
if lastExec.LastHeartbeat < nowMs-int64(heartbeatStaleAfter/time.Millisecond) {
// 心跳陈旧 = 僵尸遗留:本进程恢复例程正在跑则附着其进度;否则拉起恢复并附着。
// 候选 hub 直接交给恢复例程复用同一实例(不 close):最终执行者与连接共用一 hub,
// 取消/进度不因"占位被关→重建"竞态丢失
if !owned {
go recoverExecution(context.WithoutCancel(ctx), lastExec.Id, &execAttach{conn: conn, sessionId: conn.SessionId, flowId: req.FlowId, hub: hub})
}
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "round_start", Message: "检测到未完成执行,正在恢复", Data: map[string]interface{}{"id": lastExec.Id}})
} else {
// 心跳新鲜 = 真在跑:owned → 连接已附着本进程运行中执行,直接订阅其进度;
// !owned → 其它节点在跑(无法跨节点附着)或本进程执行尚未接管(候选 hub 会被其接管)。
// 不 close 候选:避免在 execute()/恢复 TryOwn 前误关,导致执行者进度/取消无人接收
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "round_start", Message: "工作流正在执行中", Data: map[string]interface{}{"id": lastExec.Id}})
}
return lastExec.Id, errExecAlreadyRunning
}
if *lastExec.Status == *flow.FlowExecutionStatusFailed.Code() && flowContentEqual(lastExec.RequestParams, req.FlowContent) {
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "round_start", Message: "运行开始", Data: map[string]interface{}{
"id": lastExec.Id,
}})
glog.Infof(ctx, "工作流断点续跑,execId: %v", lastExec.Id)
return reExecute(ctx, lastExec.Id, *flow.FlowExecutionStatusFailed.Code())
}
glog.Infof(ctx, "工作流全新执行,lastExec: %v", lastExec)
return execute(ctx, conn, lastExec.Id, lastExec.Status, req)
}
glog.Infof(ctx, "工作流全新执行,无历史记录")
return execute(ctx, conn, 0, nil, req)
}
// flowContentEqual 判断两次工作流参数是否一致(JSON 序列化后字节比对。
// Go struct 按字段声明序序列化、map 键自动排序,同一内容结果确定,可用于参数等价判断)
func flowContentEqual(a, b *entity.FlowInfo) bool {
if a == nil || b == nil {
return a == b
}
ab, err1 := json.Marshal(a)
bb, err2 := json.Marshal(b)
if err1 != nil || err2 != nil {
return false
}
return bytes.Equal(ab, bb)
}
// execute 执行工作流(首次执行;同会话+同工作流最近一次执行为失败状态时复用该记录重新执行,不新建数据)
func execute(ctx context.Context, conn *wsCommon.WsConnection, execId int64, status flow.FlowExecutionStatus, req *sessionDto.WebSocketExecWorkflowReq) (id int64, err error) {
// 优雅关停期间不再启动新执行(避免关停后仍登记运行、落库被退出进程打断;走 Canceled 分类落终态)
if IsShuttingDown() {
return 0, context.Canceled
}
var nodeGroupId = uuid.NewString()
// 记录发起执行用户的数字 ID:崩溃恢复续跑无 WS/HTTP 用户,需按 exec.user_id 补全合成用户
// 的 Id,外发 model-gateway/modelCall 的 X-User-Info 才能过单次调用最低余额门禁
// creator 仅存 userName,推不回数字 id)。取不到用户不阻塞执行(openBillingOrder 后续会拦);
// user_id=0 仅影响此类记录自身的恢复续跑。
var execUserId int64
if u, e := utils.GetUserInfo(ctx); e == nil && u != nil {
execUserId = int64(u.Id)
}
// 全新执行与复用旧 ID 新建两条路径共用同一插入逻辑,收敛为 createExec
createExec := func() (int64, error) {
execId, err := sessionDao.ExecWorkflowDao.Insert(ctx, &sessionDto.CreateWorkflowReq{
UserId: execUserId,
SessionId: conn.SessionId,
FlowId: req.FlowId,
NodeGroupId: nodeGroupId,
Status: flow.FlowExecutionStatusRunning.Code(),
RequestParams: req.FlowContent,
LastHeartbeat: time.Now().UnixMilli(),
})
if err == nil && g.IsEmpty(execId) {
err = fmt.Errorf("创建执行记录返回空ID")
}
if err != nil {
glog.Errorf(ctx, "工作流执行记录创建失败: %v", err)
return 0, err
}
return execId, nil
}
// 复用失败记录重跑:仅当上次执行为失败状态(executeOrResume 传入的 lastExec.Status)时重置复用;
// 上次成功 / 无历史 → 一律新建执行记录(createExec)。
// FlowExecutionStatus 是 *int8 别名,Code() 返回包级指针,直接 == 是地址比较恒为 false,
// 需解引用按值比较,否则复用失败记录时不会重置为 Running、也不更新 RequestParams
if execId > 0 && status != nil && *status == *flow.FlowExecutionStatusFailed.Code() {
// 换参全新跑:先取将被废弃的旧逻辑运行(组)(ResetRunning 随即会把它覆盖成新组,须在重置前读)。
// 旧组此前属"失败可续跑"保留态,现用户改参数改走全新运行,旧组永不再续跑 → 重置成功后软删其
// checkpoint/段/异步残留回收(软删即终态,组不复活)。新组由下方 launchExecution 使用。
var oldGroup string
if prev, e := sessionDao.ExecWorkflowDao.GetById(ctx, execId); e == nil && prev != nil {
oldGroup = prev.NodeGroupId
}
var reset bool
reset, err = sessionDao.ExecWorkflowDao.ResetRunning(ctx, execId, nodeGroupId, *flow.FlowExecutionStatusFailed.Code())
if err != nil {
return
}
if !reset {
// 已被其它路径(恢复例程/并发触发)抢先重置为运行中:放弃本次执行,状态由持有方收敛
return execId, errExecAlreadyRunning
}
// 复用失败记录时参数可能已变:把新参数落库,供后续 reExecute/恢复例程按记录参数续跑
_, err = sessionDao.ExecWorkflowDao.Update(ctx, &sessionDto.UpdateWorkflowReq{Id: execId, RequestParams: req.FlowContent})
if err != nil {
return
}
// 已抢到重置权即本组唯一执行者,可安全回收旧组(无并发续跑方会读它)
if oldGroup != "" {
_ = flowDao.FlowCheckpointDao.Delete(ctx, oldGroup)
_ = flowDao.FlowAsyncTaskDao.DeleteByGroup(ctx, oldGroup)
_ = flowDao.FlowSegmentResultDao.DeleteByGroup(ctx, oldGroup)
}
} else {
execId, err = createExec()
if err != nil {
return
}
}
return launchExecution(ctx, conn, execId, req.FlowId, nodeGroupId, conn.SessionId, req.FlowContent, true)
}
// reExecute 重新执行工作流。
// prevStatus:调用方当前观察到的记录状态(失败=3 断点续跑;运行中=1 本进程自动重试),
// 传给 ResetRunning 做条件重置:仅当记录仍处于该状态时才重置(原子防双跑)。
// 状态已被其它路径抢先变更时返回 errExecAlreadyRunning,外层不写终态、由持有方收敛。
func reExecute(ctx context.Context, execWorkflowId int64, prevStatus int8) (id int64, err error) {
// 优雅关停期间不再启动新执行(返回 Canceled 让外层分类落 errInterruptedByShutdown 终态)
if IsShuttingDown() {
return 0, context.Canceled
}
flowInfo, err := sessionDao.ExecWorkflowDao.GetById(ctx, execWorkflowId)
if err != nil {
return
}
// 续跑 = 同一逻辑运行的延续:复用 exec 记录的组(读其 checkpoint/段/异步活行继续),不换组——
// 换组会读不到上一 attempt 写在该组下的断点而从图头重跑。仅旧记录无组时新造并随重置持久化。
nodeGroupId := flowInfo.NodeGroupId
if nodeGroupId == "" {
nodeGroupId = uuid.NewString()
}
reset, err := sessionDao.ExecWorkflowDao.ResetRunning(ctx, flowInfo.Id, nodeGroupId, prevStatus)
if err != nil {
return
}
if !reset {
// 状态已被其它路径(恢复例程/并发触发)抢先重置为运行中:放弃续跑
return flowInfo.Id, errExecAlreadyRunning
}
return launchExecution(ctx, nil, flowInfo.Id, flowInfo.FlowId, nodeGroupId, flowInfo.SessionId, flowInfo.RequestParams, false)
}
// launchExecution execute 与 reExecute 共用的启动尾部:计费建单 →(可选)推送 round_start →
// 心跳 → hub 接管 → BuildExecution。
// conn 非 nil(WS 路径)时在计费通过后推送 round_startreExecute 无连接传 nil 不推送。
// 返回语义与调用方原始约定一致:计费门禁失败返回 (execId, err)(终局失败),
// BuildExecution 失败返回 (execId, err)execId 已创建/复用,wrapper 凭其落终态并结算),
// 成功返回 (execId, nil)。
func launchExecution(ctx context.Context, conn *wsCommon.WsConnection, execId int64, flowId int64, nodeGroupId string, sessionId string, flowContent *entity.FlowInfo, forceNewRun bool) (id int64, err error) {
// 工作流计费:建计费单(门禁:余额>=min_balance,钱包须存在)。
// 业务错误(余额不足/钱包不可用/费率非法)→ errBillingGateBlocked 终局失败,不重试不恢复;
// 续跑复用 execId,原计费单仍 CREATED 则幂等沿用、原单已终态则开新单
if err := openBillingOrder(ctx, execId, flowContent); err != nil {
return execId, err
}
if conn != nil {
_ = writeJSON(conn, &wsCommon.WsPushMsg{Type: "round_start", Message: "运行开始", Data: map[string]interface{}{
"id": execId,
}})
}
// WS 路径心跳与 BuildExecution 已共享同一 ctx(连接取消/DB 故障同生共死),无需租约丢失回调
stop := startHeartbeat(ctx, execId, nil)
defer stop()
// 接管候选 hub:确认真正启动执行前标记 owned,供并发附着连接识别运行中持有者
if h := getProgressHub(ctx); h != nil && !h.Owned() {
h.MarkOwned()
}
if err = BuildExecution(ctx, forceNewRun, flowId, execId, nodeGroupId, sessionId, flowContent); err != nil {
// 报错也返回真实 execId(与本函数计费门禁失败 return execId, err 一致):execute/reExecute
// 已创建或复用了执行记录,wrapperhandleExecute)必须拿到它才能 recordWorkflow 落终态并结算。
// 原来丢 id 返回 (0, err) 会让 wrapper 退化为 recordExecutionFailure 兜底(按会话+流程查最近
// Running 记录);当图内已提前触发过 summary/记录状态不再是 Running 时兜底会静默跳过,
// 结算永不执行 → 计费单遗留 CREATED(取消/中断漏扣,2026-09-03 已修,见 lambda_summary.go)。
return execId, err
}
return execId, nil
}
func BuildExecution(ctx context.Context, forceNewRun bool, flowId, executionId int64, nodeGroupId string, sessionId string, flowContent *entity.FlowInfo) (err error) {
// =========================================================================
// 构建执行图
// =========================================================================
var nodeList []entity.FlowNode
var runGraph compose.Runnable[any, any]
nodeList, runGraph, err = BuildGraphFromFlowContent(ctx, flowContent)
if err != nil {
return fmt.Errorf("执行工作流失败: %v", err)
}
// =========================================================================
// 构建 ConfigMap
// =========================================================================
configMap := buildConfigMap(flowContent, nodeList)
// =========================================================================
// 构建全局执行入参
// =========================================================================
execInput := &flowDto.FlowExecutionInput{
NodeGroupId: nodeGroupId,
ExecutionId: executionId,
FlowId: flowId,
ConfigMap: configMap,
SessionId: sessionId,
ForceNewRun: forceNewRun,
}
// 全新执行/换参重跑(forceNewRun)无需起跑前清理:checkpoint/async/segment 均以 node_group_id
// (逻辑运行标识)为键,forceNewRun 换新组 = 全新键,天然不命中任何旧残留(也无软删墓碑可碰撞);
// 被废弃的旧组残留由 execute() 复用失败 exec 时在重置成功后软删回收(见 execute)。
// 运行中/可续跑组永不删除。
// 驱动循环:编译期已对每个业务节点注册 WithInterruptAfterNodesgraph_build.go),节点正常完成后
// Eino 自动暂停并落 checkpoint。这里识别出"纯进度暂停"后同 checkpoint id 立即续跑,直至图完整跑完
// (err==nil) 或遇到真正终态(节点失败 / 用户取消 / 非中断错误)。崩溃硬杀恢复 BuildExecution(false)
// 走同一循环:查得断点即从断点续跑,已完成的同步节点不再重跑/重复计费。
// 判别器:节点失败中断 RerunNodes 恒非空(HandleFailedNodeExecution→compose.Interrupt);ctx 取消由
// 下方 ctx.Err() 短路;异步模型调用是阻塞式(async.go),不产生空 RerunNodes 伪暂停。故 RerunNodes
// 为空 = 编译期纯进度暂停 → 续跑。详见《工作流节点断点续跑技术设计.md》。
// WithForceNewRun 只允许出现在全新跑首轮(此时起跑前三清也已只做一次);续跑/暂停轮绝不能带,
// 否则把断点续跑打成"忽略断点从图头重跑"。
first := forceNewRun
maxIter := len(flowContent.Nodes)*2 + 8 // 纯进度暂停每轮必推进 ≥1 节点, 正常轮数 ≤ 节点数+1; 超限即疑似死循环
iter := 0
for {
runOpts := []compose.Option{compose.WithCheckPointID(nodeGroupId)}
if first {
runOpts = append(runOpts, compose.WithForceNewRun())
first = false
}
_, err = runGraph.Invoke(ctx, execInput, runOpts...)
if err == nil {
break // 图完整跑完 → 下方成功尾部三清
}
// 图执行被 ctx 取消(WS 断连/用户终止):返回 context.Canceled 语义,让 recordWorkflow
// 记为"用户已终止执行"。此时 Eino 已把断点写入 checkpoint storeDbCheckPointStore 用
// WithoutCancel 落库),重新提交相同参数即可断点续跑。
if ctxErr := ctx.Err(); ctxErr != nil {
return fmt.Errorf("执行工作流失败: %w", ctxErr)
}
info, infoOk := compose.ExtractInterruptInfo(err)
if !infoOk {
return fmt.Errorf("执行工作流失败: %v", err)
}
if len(info.RerunNodes) == 0 {
// 纯进度暂停(编译期 after-node checkpoint 已落库)→ 同 id 续跑下一段
iter++
if iter > maxIter {
return fmt.Errorf("执行工作流失败: 断点续跑超限(%d 次), 疑似死循环", maxIter)
}
continue
}
// 节点失败中断 → 终态失败
var sb strings.Builder
var errNodeCount int
for _, item := range info.InterruptContexts {
if item.Info == nil {
continue
}
if g.NewVar(item.Info).IsMap() {
errNodeCount++
valMap := gconv.Map(item.Info)
fmt.Fprintf(&sb, "\n节点:%v, 失败原因:%v", valMap["node"], valMap["error"])
}
}
if sb.Len() > 0 {
err = fmt.Errorf("%v个节点,%v", errNodeCount, strings.TrimPrefix(sb.String(), "\n"))
}
return fmt.Errorf("执行工作流失败: %v", err)
}
// 执行成功:软删本逻辑运行(组)的 checkpoint/段/异步缓存(终态组,此后不再被读写,软删不复活)。
// 失败/取消不走到这里,组行保留供 reExecute / 恢复续跑复用;换参重跑由 execute 回收旧组。
_ = flowDao.FlowCheckpointDao.Delete(ctx, nodeGroupId)
_ = flowDao.FlowSegmentResultDao.DeleteByGroup(ctx, nodeGroupId)
_ = flowDao.FlowAsyncTaskDao.DeleteByGroup(ctx, nodeGroupId)
return
}
// SessionWsService 会话 WebSocket 服务器:普通对话与工作流共用一条连接,
// 首次连接仅升级,后续按消息 type 路由到对话/工作流处理器
// (对话处理器在 react_ws_exec.go 注册,工作流处理器在上方 init 注册)。
var SessionWsService = wsCommon.NewWsServer(
wsCommon.WithConnKeyPrefix("ws:session:"),
)
// WsConnect 控制器统一入口:升级 WebSocket(普通对话/工作流均由消息 type 区分,此处不区分)
func WsConnect(ctx context.Context, r *ghttp.Request, req *sessionDto.WebSocketConnectReq) error {
_, err := SessionWsService.Upgrade(ctx, r, req.SessionId)
return err
}
// ensureSession 解析前端 sessionId 并确保会话存在:命中已存在会话则复用其 id,否则按 name 新建。
// 普通对话(react_ws_exec.go)与工作流(exec_ws.go)共用。
func ensureSession(ctx context.Context, sessionId string, name string) error {
exist, err := sessionDao.SessionDao.GetById(ctx, sessionId)
if err != nil {
return err
}
if exist != nil {
return nil
}
if r := []rune(name); len(r) > 128 { // session_name VARCHAR(128)
name = string(r[:128])
}
_, err = sessionDao.SessionDao.Insert(ctx, &sessionDto.CreateSessionReq{SessionId: sessionId, SessionName: name})
return err
}
+8 -574
View File
@@ -1,32 +1,19 @@
package flow
import (
"ai-agent/workflow/consts/flow"
"ai-agent/workflow/consts/node"
fileDao "ai-agent/workflow/dao/file"
flowDao "ai-agent/workflow/dao/flow"
nodeDao "ai-agent/workflow/dao/node"
"ai-agent/workflow/model/dto"
fileDto "ai-agent/workflow/model/dto/file"
flowDto "ai-agent/workflow/model/dto/flow"
nodeDto "ai-agent/workflow/model/dto/node"
"ai-agent/workflow/model/entity"
"context"
"errors"
"fmt"
"sort"
"strconv"
"strings"
"sync"
"time"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/gtime"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
"go.opentelemetry.io/otel/trace"
)
var FlowExecutionService = &flowExecutionService{}
@@ -39,7 +26,7 @@ func (s *flowExecutionService) Get(ctx context.Context, req *flowDto.GetFlowExec
return nil, err
}
res = new(flowDto.VOFlowExecution)
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
@@ -58,43 +45,34 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
return nil, err
}
// ===================== 核心修复:只统计【有数据】的执行记录,空的直接跳过 =====================
executionNumber := make(map[int64]int) // executionId -> 倒序编号(最新=1
// 第一次遍历:只处理【有输出参数】的记录,统计并分配编号
var validList []*entity.FlowExecution // 只存有效(非空)记录
executionNumber := make(map[int64]int)
var validList []*entity.FlowExecution
for _, execution := range list {
if g.IsEmpty(execution.OutputParams) {
continue // 空数据直接过滤,不参与编号、不展示
continue
}
validList = append(validList, execution)
}
// 给有效记录分配【时间倒序编号】(最新=1)
totalValid := len(validList)
for idx, execution := range validList {
executionNumber[execution.Id] = totalValid - idx
}
// 2. 分组映射:日期 -> 流程节点
type flowWrap struct {
flowNode flowDto.FlowNode
createdAt *gtime.Time
}
dateMap := make(map[string]*[]flowWrap)
// 遍历【有效数据】构建结构
for _, execution := range validList {
createDate := execution.CreatedAt.Format("Y-m-d")
flowName := execution.FlowName
outputParams := execution.OutputParams
// 编号只算有效数据,不会把空的算进去
num := executionNumber[execution.Id]
displayFlowName := fmt.Sprintf("会话-%d(%s)", num, flowName)
// 3. 解析 outputParams
var tempItems []flowDto.OutputItem
for _, paramMap := range outputParams {
for tsKey, value := range paramMap {
@@ -102,32 +80,21 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
continue
}
tempItems = append(tempItems, flowDto.OutputItem{
Timestamp: tsKey,
Content: gconv.String(value),
Content: gconv.String(value),
})
}
}
// ===================== 修复1:如果解析后依然为空,直接跳过,不生成第二层节点 =====================
if len(tempItems) == 0 {
continue
}
// 时间戳正序
sort.Slice(tempItems, func(i, j int) bool {
t1, _ := strconv.ParseInt(tempItems[i].Timestamp, 10, 64)
t2, _ := strconv.ParseInt(tempItems[j].Timestamp, 10, 64)
return t1 < t2
})
// 标号:相同类型递增,不同重置
suffixCount := make(map[string]int)
for idx := range tempItems {
item := &tempItems[idx]
val := item.Content
suffix := "内容"
ext := ""
ext = GetFileTypeByPath(val)
ext := GetFileTypeByPath(val)
if ext == "image" {
suffix = "图片"
}
@@ -148,7 +115,6 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
item.Label = fmt.Sprintf("%s_%d", suffix, suffixCount[suffix])
}
// 组装节点
flowNode := flowDto.FlowNode{
FlowName: displayFlowName,
Id: execution.Id,
@@ -165,10 +131,8 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
})
}
// 6. 构建树 + 排序
var tree []flowDto.DateNode
for date, wraps := range dateMap {
// 第二层按创建时间倒序(最新在前)
sort.Slice(*wraps, func(i, j int) bool {
return (*wraps)[i].createdAt.After((*wraps)[j].createdAt)
})
@@ -178,7 +142,6 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
flowNodes = append(flowNodes, w.flowNode)
}
// ===================== 修复2:日期下没有流程,也过滤掉 =====================
if len(flowNodes) == 0 {
continue
}
@@ -189,38 +152,17 @@ func (s *flowExecutionService) List(ctx context.Context, req *flowDto.ListFlowEx
})
}
// 第一层日期倒序
sort.Slice(tree, func(i, j int) bool {
return tree[i].CreateDate > tree[j].CreateDate
})
imgPrefix, err := utils.GetFileAddressPrefix(ctx)
imgPrefix, err := oss.GetFileAddressPrefix(ctx)
return &flowDto.ListFlowExecutionTreeRes{
Tree: tree,
ImgAddressPrefix: imgPrefix,
}, nil
}
// ComposeCallback 提示词回调接口
func (s *flowExecutionService) ComposeCallback(ctx context.Context, req *flowDto.ComposeCallbackReq) (err error) {
Notify(req.TaskId, req)
return nil
}
// ModelCallback 模型回调接口
func (s *flowExecutionService) ModelCallback(ctx context.Context, req *flowDto.ModelCallbackReq) (err error) {
// 唤醒等待的任务
Notify(req.TaskId, req)
return nil
}
// VideoCallback 视频拼接回调接口
func (s *flowExecutionService) VideoCallback(ctx context.Context, req *flowDto.VideoCallbackReq) (err error) {
// 唤醒等待的任务
Notify(req.TaskId, req)
return nil
}
// HttpNodeCallback http节点回调接口
func (s *flowExecutionService) HttpNodeCallback(ctx context.Context) (err error) {
r := g.RequestFromCtx(ctx)
@@ -228,511 +170,3 @@ func (s *flowExecutionService) HttpNodeCallback(ctx context.Context) (err error)
Notify(taskId, r)
return nil
}
// ===================== 核心改造:替换为 sync.Map 存储取消上下文 =====================
var (
// cancelMap: traceID -> context.CancelFunc
cancelMap sync.Map
)
func (s *flowExecutionService) Cancel(ctx context.Context, req *flowDto.CancelReq) (err error) {
getRes, err := flowDao.FlowExecutionDao.Get(ctx, &flowDto.GetFlowExecutionReq{
SessionId: req.SessionId,
})
if err != nil {
return err
}
if g.IsEmpty(getRes) {
return fmt.Errorf("会话[%s] 不存在", req.SessionId)
}
// 从 sync.Map 获取取消函数
cancelVal, exist := cancelMap.Load(getRes.TraceId)
if !exist {
return fmt.Errorf("traceID[%s] 不存在或已执行完成", getRes.TraceId)
}
// 执行取消
cancel, ok := cancelVal.(context.CancelFunc)
if !ok {
return fmt.Errorf("traceID[%s] 对应的取消函数类型错误", getRes.TraceId)
}
cancel()
// 取消后清理(可选:也可以在流程结束时统一清理)
cancelMap.Delete(getRes.TraceId)
// 同步更新流程执行状态为已取消
_, err = flowDao.FlowExecutionDao.Update(ctx, &flowDto.UpdateFlowExecutionReq{
Id: getRes.Id,
Status: flow.FlowExecutionStatusCancel.Code(),
})
if err != nil {
return fmt.Errorf("更新取消状态失败: %v", err)
}
return nil
}
func (s *flowExecutionService) Execute(ctx context.Context, req *flowDto.ExecuteReq) (res *flowDto.ExecuteRes, err error) {
// ===================== 核心改造1:创建可取消的上下文 =====================
execCtx, cancel := context.WithCancel(ctx)
traceId := ""
defer func() {
// 流程结束(成功/失败)时清理 cancelMap
if traceId != "" {
cancelMap.Delete(traceId)
}
cancel()
}()
//getRes, err := FlowUserService.Get(ctx, &flowDto.GetFlowUserReq{
// Id: req.FlowId,
//})
//if err != nil {
// return nil, err
//}
nodeInputParams := ExtractFlowNodeFrom(req.FlowContent)
flowInfo, err := flowDao.FlowExecutionDao.Get(ctx, &flowDto.GetFlowExecutionReq{
SessionId: req.SessionId,
})
if err != nil {
return
}
var executionId int64
var isDialogue bool
var nodeGroupId = uuid.NewString()
if flowInfo == nil {
isDialogue = false
var r = new(flowDto.CreateFlowExecutionReq)
r.FlowUserId = req.FlowId
r.FlowName = req.FlowName
r.NodeGroupId = nodeGroupId
r.TriggerType = flow.FlowExecutionTriggerTypeManual.Code()
r.FlowContent = req.FlowContent
//r.NodeInputParams = nodeInputParams
r.SessionId = req.SessionId
r.Status = flow.FlowExecutionStatusRunning.Code()
span := trace.SpanFromContext(ctx)
if span != nil && span.SpanContext().HasTraceID() {
r.TraceId = span.SpanContext().TraceID().String()
traceId = r.TraceId
cancelMap.Store(traceId, cancel)
}
executionId, err = flowDao.FlowExecutionDao.Insert(ctx, r)
if err != nil {
return
}
} else {
isDialogue = true
executionId = flowInfo.Id
span := trace.SpanFromContext(ctx)
if span != nil && span.SpanContext().HasTraceID() {
traceId = span.SpanContext().TraceID().String()
cancelMap.Store(traceId, cancel)
}
executionReq := flowDto.UpdateFlowExecutionReq{
Id: executionId,
NodeGroupId: nodeGroupId,
Status: flow.FlowExecutionStatusRunning.Code(),
TraceId: traceId,
}
_, err = flowDao.FlowExecutionDao.Update(ctx, &executionReq)
if err != nil {
return
}
}
if !g.IsEmpty(req.FileUrl) {
createFileTempReq := make([]*fileDto.CreateFileTempReq, 0, len(req.FileUrl))
for _, fileUrl := range req.FileUrl {
var createReq = new(fileDto.CreateFileTempReq)
createReq.BusinessId = req.SessionId
createReq.FileUrl = fileUrl
createFileTempReq = append(createFileTempReq, createReq)
}
_, err = fileDao.FileTempDao.BatchInsert(ctx, createFileTempReq)
if err != nil {
return nil, err
}
}
if isDialogue && !g.IsEmpty(flowInfo) && !g.IsEmpty(req.ResultUrl) {
req.NodeGroupId = nodeGroupId
if strings.HasSuffix(gconv.String(req.ResultUrl), ".inc") {
err = TextModelSingleLambda(ctx, req, flowInfo)
return
} else if strings.HasSuffix(gconv.String(req.ResultUrl), ".png") {
err = ImgModelSingleLambda(ctx, req, flowInfo)
return
} else if strings.HasSuffix(gconv.String(req.ResultUrl), ".html") {
err = TextImgModelSingleLambda(ctx, req, flowInfo)
return
}
return nil, errors.New("文件格式不支持")
}
// =========================================================================
// ✅【第2步】构建执行图
// =========================================================================
var nodeList []entity.FlowNode
var runGraph compose.Runnable[any, any]
nodeList, runGraph, err = BuildGraphFromFlowContent(execCtx, req.FlowContent)
if err != nil {
executionReq := flowDto.UpdateFlowExecutionReq{
Id: executionId,
Status: flow.FlowExecutionStatusFailed.Code(),
ErrorMessage: err.Error(),
}
_, err1 := flowDao.FlowExecutionDao.Update(ctx, &executionReq)
if err1 != nil {
return
}
return nil, fmt.Errorf("执行工作流失败: %v", err)
}
// =========================================================================
// ✅【第3步】构建 ConfigMap
// =========================================================================
configMap := make(map[string]*entity.FlowNode)
for _, cfg := range nodeInputParams {
configMap[cfg.Id] = cfg
}
for _, i := range nodeList {
configMap[i.Id] = &i
}
// =========================================================================
// ✅【第4步】构建全局执行入参(现在 schemaMap 是有值的!)
// =========================================================================
execInput := &flowDto.FlowExecutionInput{
NodeGroupId: nodeGroupId,
IsDialogue: isDialogue,
ExecutionId: executionId,
ConfigMap: configMap,
SessionId: req.SessionId,
Desc: req.Desc,
SkillName: req.SkillName,
FileUrl: req.FileUrl,
}
// 执行工作流
_, err = runGraph.Invoke(execCtx, execInput)
if err != nil {
// 检测是否是取消导致的错误
if errors.Is(execCtx.Err(), context.Canceled) {
executionReq := flowDto.UpdateFlowExecutionReq{
Id: executionId,
Status: flow.FlowExecutionStatusCancel.Code(),
}
_, _ = flowDao.FlowExecutionDao.Update(ctx, &executionReq)
return nil, fmt.Errorf("工作流已被取消: %v", err)
}
executionReq := flowDto.UpdateFlowExecutionReq{
Id: executionId,
Status: flow.FlowExecutionStatusFailed.Code(),
ErrorMessage: err.Error(),
}
_, err1 := flowDao.FlowExecutionDao.Update(ctx, &executionReq)
if err1 != nil {
return
}
return nil, fmt.Errorf("执行工作流失败: %v", err)
}
return
}
func BuildGraph(ctx context.Context, flowContent *entity.FlowInfo) ([]entity.FlowNode, *compose.Graph[any, any]) {
// 注册自定义合并函数:处理 *flowDto.FlowExecutionInput 类型合并
// 由于 ConfigMap 是 map 引用类型,所有并行分支修改已经写入共享内存
// 直接返回第一个实例即可,所有修改都已经可见
compose.RegisterValuesMergeFunc(func(values []*flowDto.FlowExecutionInput) (*flowDto.FlowExecutionInput, error) {
if len(values) == 0 {
return nil, nil
}
// 返回第一个实例,ConfigMap 是指针,所有修改都已经写入共享数据结构
return values[0], nil
})
graph := compose.NewGraph[any, any]()
var nodeList []entity.FlowNode
nodeId := uuid.NewString()
originalEndNodes := findEndNodes(flowContent.StartNodeId, flowContent.Edges)
for i := range originalEndNodes {
sprintf := fmt.Sprintf("%v_%d", nodeId, i)
summaryNode := entity.FlowNode{
Id: sprintf,
NodeCode: node.NodeTypeSystemSum,
Name: node.NodeNameSystemSum,
InputSource: []entity.FlowNodeInputSource{}, // 后续自动聚合所有节点输出
FormConfig: nil,
ModelConfig: node.ModelItem{},
}
nodeList = append(nodeList, summaryNode)
flowContent.Nodes = append(flowContent.Nodes, summaryNode)
}
// 注册所有节点
nodeMap := make(map[string]entity.FlowNode)
for _, item := range flowContent.Nodes {
nodeMap[item.Id] = item
if item.NodeCode != node.NodeTypeJudge {
registerNodeToGraph(graph, item)
}
}
// 注册所有边
if flowContent.StartNodeId != "" {
_ = graph.AddEdge(compose.START, flowContent.StartNodeId)
}
for i, endID := range originalEndNodes {
sprintf := fmt.Sprintf("%v_%d", nodeId, i)
_ = graph.AddEdge(endID, sprintf)
_ = graph.AddEdge(sprintf, compose.END)
}
// 构建边关系
upstreamMap := make(map[string][]string)
edgeMap := make(map[string][]entity.FlowEdge)
for _, edge := range flowContent.Edges {
edgeMap[edge.From] = append(edgeMap[edge.From], edge)
upstreamMap[edge.To] = append(upstreamMap[edge.To], edge.From)
}
// 处理连线 & 分支
for fromNodeID, edges := range edgeMap {
fromNode := nodeMap[fromNodeID]
// 判断节点 → 分支处理
if fromNode.NodeCode == node.NodeTypeJudge {
branchMap := make(map[string]bool)
for _, e := range edges {
branchMap[e.To] = true
}
judgeLambda := func(ctx context.Context, input any) (string, error) {
execInput, ok := input.(*flowDto.FlowExecutionInput)
if !ok {
return "", fmt.Errorf("入参类型错误")
}
currentConfig := execInput.ConfigMap[fromNodeID]
if currentConfig == nil {
return "", fmt.Errorf("判断节点%s无配置", fromNodeID)
}
branchIdNameMap := make(map[string]string)
var branchIDs []string
for nodeID := range branchMap {
branchIDs = append(branchIDs, nodeID)
// 从configMap获取分支节点的名称
if branchNodeCfg, ok := execInput.ConfigMap[nodeID]; ok {
branchIdNameMap[nodeID] = branchNodeCfg.Name
} else {
branchIdNameMap[nodeID] = "未命名节点" // 兜底
}
}
// 把分支ID-名称映射塞进 ModelConfig,带给意图节点
m := make(map[string]interface{})
m["branch_ids"] = branchIDs
m["branch_id_name_map"] = branchIdNameMap // 传递ID-名称映射
currentConfig.Config = m
// 关键修改:构造 NodeExecutionInput 传入 JudgeLambda
nodeExecInput := &flowDto.NodeExecutionInput{
Config: currentConfig, // 当前判断节点配置
Global: execInput, // 全局执行入参
}
return JudgeLambda(ctx, nodeExecInput) // 传入 NodeExecutionInput 类型
}
_ = graph.AddBranch(upstreamMap[fromNodeID][0], compose.NewGraphBranch(judgeLambda, branchMap))
continue
}
// 普通节点连线
for _, e := range edges {
toNode := nodeMap[e.To]
if toNode.NodeCode == node.NodeTypeJudge {
continue
}
_ = graph.AddEdge(e.From, e.To)
}
}
return nodeList, graph
}
// BuildGraphFromFlowContent 根据前端保存的工作流JSON,自动构建执行图
func BuildGraphFromFlowContent(ctx context.Context, flowContent *entity.FlowInfo) ([]entity.FlowNode, compose.Runnable[any, any], error) {
nodeList, graph := BuildGraph(ctx, flowContent)
compile, err := graph.Compile(ctx, compose.WithGraphName("auto_build_workflow"))
return nodeList, compile, err
}
// -------------------------- 节点自动注册器(核心分发) --------------------------
func registerNodeToGraph(graph *compose.Graph[any, any], flowNode entity.FlowNode) {
// 通用包装:全程入参都是 *FlowExecutionInput
wrapLambda := func(lambda func(ctx context.Context, input any) (any, error)) func(ctx context.Context, input any) (any, error) {
return func(ctx context.Context, input any) (any, error) {
// ✅ 【关键】全程入参类型永远不变
execInput, ok := input.(*flowDto.FlowExecutionInput)
if !ok {
return nil, fmt.Errorf("入参必须是 *FlowExecutionInput, 实际是 %T", input)
}
configMap := execInput.ConfigMap
currentConfig := configMap[flowNode.Id]
if currentConfig == nil {
return nil, fmt.Errorf("节点%s无配置", flowNode.Id)
}
// 获取入参 - 适配切片类型:遍历所有来源节点
realInput := new(flowDto.NodeExecutionInput)
if len(flowNode.InputSource) > 0 { // 改为判断切片长度
// 遍历所有指定的来源节点,聚合输出结果
for _, inputSource := range flowNode.InputSource { // 遍历切片
if sourceConfig, ok := configMap[inputSource.NodeId]; ok {
currentConfig.OutputResult = append(currentConfig.OutputResult, sourceConfig.OutputResult...)
}
}
}
// ✅ 封装节点执行入参(配置+表单架构)
realInput = &flowDto.NodeExecutionInput{
Config: currentConfig,
Global: execInput, // ✅ 把【全部节点】的对象直接塞进来
}
// ✅ 插入节点执行记录,初始状态为运行中
startTime := time.Now()
// 上传OSS(每条独立上传)
ossResult, err := Upload(ctx, &dto.UploadFileBytesReq{
FileBytes: gconv.Bytes(gconv.String(realInput)),
FileName: fmt.Sprintf("nodeInput:%v.txt", time.Now().UnixMilli()),
})
if err != nil {
return nil, err
}
nodeExecutionId, err := nodeDao.NodeExecutionDao.Insert(ctx, &nodeDto.CreateNodeExecutionReq{
FlowExecutionId: execInput.ExecutionId,
NodeId: flowNode.Id,
NodeName: flowNode.Name,
NodeGroupId: execInput.NodeGroupId,
InputParamsPath: ossResult.FileURL,
Status: node.NodeExecutionStatusRunning.Code(),
})
if err != nil {
// 记录失败到已执行列表
execInput.ExecutedNodes = append(execInput.ExecutedNodes, flowDto.ExecutedNode{
NodeId: flowNode.Id,
Status: node.NodeExecutionStatusFailed.Code(),
})
return nil, err
}
realInput.NodeExecutionId = nodeExecutionId
// 执行节点
_, err = lambda(ctx, realInput)
durationMs := time.Since(startTime).Milliseconds()
updateReq := &nodeDto.UpdateNodeExecutionReq{
Id: nodeExecutionId,
DurationMs: durationMs,
}
if err != nil {
// 执行失败,更新状态
updateReq.Status = node.NodeExecutionStatusFailed.Code()
updateReq.ErrorMessage = err.Error()
_, _ = nodeDao.NodeExecutionDao.Update(ctx, updateReq)
// 记录失败到已执行列表
execInput.ExecutedNodes = append(execInput.ExecutedNodes, flowDto.ExecutedNode{
NodeId: flowNode.Id,
Status: node.NodeExecutionStatusFailed.Code(),
})
return nil, err
}
// 上传OSS(每条独立上传)
ossResult1, err := Upload(ctx, &dto.UploadFileBytesReq{
FileBytes: gconv.Bytes(gconv.String(realInput)),
FileName: fmt.Sprintf("nodeInput:%v.txt", time.Now().UnixMilli()),
})
if err != nil {
return nil, err
}
updateReq.OutputParamsPath = ossResult1.FileURL
// 执行成功,更新状态
updateReq.Status = node.NodeExecutionStatusSuccess.Code()
_, _ = nodeDao.NodeExecutionDao.Update(ctx, updateReq)
// 记录成功到已执行列表
execInput.ExecutedNodes = append(execInput.ExecutedNodes, flowDto.ExecutedNode{
NodeId: flowNode.Id,
Status: node.NodeExecutionStatusSuccess.Code(),
})
// ✅ 关键:返回整个 execInput,让下一个节点继续用!
return execInput, nil
}
}
switch flowNode.NodeCode {
case "__start__":
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(StartLambda)))
case node.NodeTypeSystemSum:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(SummaryLambda)))
case node.NodeTypeTextModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(TextModelLambda)))
case node.NodeTypeImageModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(ImageModelLambda)))
case node.NodeTypeVideoModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(VideoModelLambda)))
case node.NodeTypeAudioModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(AudioModelLambda)))
case node.NodeTypeBatchModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(BatchModelLambda)))
case node.NodeTypeDataConversionModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(DataConversionLambda)))
case node.NodeTypeCustomNode:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(CustomLambda)))
case node.NodeTypeForm:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(FormLambda)))
//case node.NodeTypeIntent:
// _ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(IntentLambda)))
case node.NodeTypeMerge:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(MergeLambda)))
case node.NodeTypeDataMerge:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(DataMergeLambda)), compose.WithGraphCompileOptions(compose.WithNodeTriggerMode(compose.AllPredecessor)))
case node.NodeTypeSubFlow:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(SubFlowLambda)))
case node.NodeTypeHttp:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(HttpLambda)))
}
}
func findEndNodes(startNodeId string, edges []entity.FlowEdge) []string {
nextMap := make(map[string][]string)
for _, e := range edges {
nextMap[e.From] = append(nextMap[e.From], e.To)
}
endNodeSet := make(map[string]struct{})
visited := make(map[string]struct{})
queue := []string{startNodeId}
for len(queue) > 0 {
node := queue[0]
queue = queue[1:]
if _, exist := visited[node]; exist {
continue
}
visited[node] = struct{}{}
nextList := nextMap[node]
if len(nextList) == 0 {
endNodeSet[node] = struct{}{}
continue
}
queue = append(queue, nextList...)
}
res := make([]string, 0, len(endNodeSet))
for k := range endNodeSet {
res = append(res, k)
}
return res
}
+87
View File
@@ -0,0 +1,87 @@
package flow
import (
"ai-agent/workflow/model/entity"
"net/url"
"path/filepath"
"strings"
)
// FindEndNodes 从指定起始节点开始,遍历图找到所有末端节点(没有出边的节点)
func FindEndNodes(startNodeId string, edges []entity.FlowEdge) []string {
nextMap := make(map[string][]string)
for _, e := range edges {
nextMap[e.From] = append(nextMap[e.From], e.To)
}
endNodeSet := make(map[string]struct{})
visited := make(map[string]struct{})
queue := []string{startNodeId}
for len(queue) > 0 {
node := queue[0]
queue = queue[1:]
if _, exist := visited[node]; exist {
continue
}
visited[node] = struct{}{}
nextList := nextMap[node]
if len(nextList) == 0 {
endNodeSet[node] = struct{}{}
continue
}
queue = append(queue, nextList...)
}
res := make([]string, 0, len(endNodeSet))
for k := range endNodeSet {
res = append(res, k)
}
return res
}
// ExtractFlowNodeFrom 从 FlowInfo 中提取节点列表(返回指针切片)
func ExtractFlowNodeFrom(flowContent *entity.FlowInfo) []*entity.FlowNode {
var flowNodes []*entity.FlowNode
for _, item := range flowContent.Nodes {
flowNodes = append(flowNodes, &item)
}
return flowNodes
}
// GetFileTypeByPath 根据文件路径/URL的后缀名判断文件类型
func GetFileTypeByPath(filePath string) string {
if filePath == "" {
return ""
}
// 解析 URL,获取真实路径(兼容 http 链接)
u, err := url.Parse(filePath)
if err == nil {
filePath = u.Path
}
// 获取后缀(小写)
ext := filepath.Ext(filePath)
ext = strings.ToLower(ext)
// 判断类型
switch ext {
case ".jpg", ".jpeg", ".png", ".gif", ".bmp", ".webp":
return "image"
case ".mp4", ".mov", ".avi", ".flv", ".wmv", ".mkv":
return "video"
case ".mp3", ".wav", ".m4a", ".flac", ".aac", ".ogg":
return "audio"
case ".txt", ".md", ".log", ".json", ".xml", ".inc":
return "text"
case ".html":
return "html"
case ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx":
return "document"
default:
return ""
}
}
+122 -68
View File
@@ -2,7 +2,7 @@ package flow
import (
"ai-agent/workflow/consts/flow"
"ai-agent/workflow/consts/node"
"ai-agent/workflow/consts/public"
flowDao "ai-agent/workflow/dao/flow"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
@@ -10,7 +10,9 @@ import (
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/database/gdb"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
@@ -40,31 +42,13 @@ func (s *flowUserService) Create(ctx context.Context, req *flowDto.CreateFlowUse
return &flowDto.CreateFlowUserRes{Id: id}, err
}
func (s *flowUserService) Update(ctx context.Context, req *flowDto.UpdateFlowUserReq) (err error) {
func (s *flowUserService) Update(ctx context.Context, req *flowDto.UpdateFlowUserReq) (res *flowDto.CreateFlowUserRes, err error) {
id := req.Id
admin, err := service.UtilService.IsAdmin(ctx)
if err != nil {
return
}
req.NodeInputParams = ExtractFlowNodeFrom(req.FlowContent)
get, err := flowDao.FlowTemplateDao.Get(ctx, &flowDto.GetFlowTemplateReq{
Id: req.Id,
})
if err != nil {
return err
}
if !g.IsEmpty(get) && !admin {
_, err = flowDao.FlowUserDao.Insert(ctx, &flowDto.CreateFlowUserReq{
FlowName: req.FlowName,
Description: req.Description,
FlowContent: req.FlowContent,
NodeInputParams: req.NodeInputParams,
SourceFlowTemplateId: get.Id,
})
if err != nil {
return
}
}
if admin {
_, err = flowDao.FlowTemplateDao.Update(ctx, &flowDto.UpdateFlowTemplateReq{
Id: req.Id,
@@ -75,37 +59,50 @@ func (s *flowUserService) Update(ctx context.Context, req *flowDto.UpdateFlowUse
Status: flow.FlowTemplateStatusEnable.Code(),
})
} else {
_, err = flowDao.FlowUserDao.Update(ctx, req)
}
return
}
var get *entity.FlowTemplate
get, err = flowDao.FlowTemplateDao.Get(ctx, &flowDto.GetFlowTemplateReq{
Id: req.Id,
})
if err != nil {
return nil, err
}
func ExtractFlowNodeFrom(flowContent *entity.FlowInfo) []*entity.FlowNode {
// 构建每个节点的上游节点映射
upstreamMap := make(map[string][]string)
for _, edge := range flowContent.Edges {
upstreamMap[edge.To] = append(upstreamMap[edge.To], edge.From)
}
// 同时更新 flowContent.Nodes 中的 DataMerge 节点
for i := range flowContent.Nodes {
n := &flowContent.Nodes[i]
// 对于 DataMerge 节点,自动根据边关系填充 InputSource
if n.NodeCode == node.NodeTypeDataMerge {
n.InputSource = nil
for _, fromId := range upstreamMap[n.Id] {
n.InputSource = append(n.InputSource, entity.FlowNodeInputSource{
NodeId: fromId,
if !g.IsEmpty(get) {
// 模版 → 用户流程拷贝(含递归子流程拷贝)整体放一个事务:任一拷贝失败则整体回滚,
// 避免主流程已落库而子流程缺失/引用错乱的脏数据。
// Transaction 会把 tx 注入回调 ctx,回调内用该 ctx 调 DAO 即自动进入事务。
txErr := gfdb.DB(ctx, public.DbNameBlackDeacon).Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
// 子流程引用重写:模版里的 sub_flow 节点指向子模版/子流程 id,
// 拷贝成用户自己的流程时要把子工作流也复制一份(子流程可能再嵌套子流程,递归),
// 并把主流程 SubConfig.WorkflowId 改写为新拷贝的 id
subIdMap, copyErr := copySubFlows(ctx, req.SubFlows)
if copyErr != nil {
return copyErr
}
rewriteSubFlowWorkflowIds(req.FlowContent, subIdMap)
// SubConfig 改写后重新提取节点参数,保证落库的 NodeInputParams 与 FlowContent 一致
req.NodeInputParams = ExtractFlowNodeFrom(req.FlowContent)
newId, insertErr := flowDao.FlowUserDao.Insert(ctx, &flowDto.CreateFlowUserReq{
FlowName: req.FlowName,
Description: req.Description,
FlowContent: req.FlowContent,
NodeInputParams: req.NodeInputParams,
SourceFlowTemplateId: get.Id,
})
if insertErr != nil {
return insertErr
}
id = newId
return nil
})
if txErr != nil {
return nil, txErr
}
} else {
_, err = flowDao.FlowUserDao.Update(ctx, req)
}
}
var flowNodes []*entity.FlowNode
for _, item := range flowContent.Nodes {
flowNodes = append(flowNodes, &item)
}
return flowNodes
return &flowDto.CreateFlowUserRes{Id: id}, err
}
func (s *flowUserService) Delete(ctx context.Context, req *flowDto.DeleteFlowUserReq) (err error) {
@@ -178,23 +175,29 @@ func (s *flowUserService) List(ctx context.Context, req *flowDto.ListFlowUserReq
}
return
}
res = &flowDto.ListFlowRes{
IsAdmin: admin,
}
if !req.IsOwn {
var t int
var l []*entity.FlowTemplate
l, t, err = flowDao.FlowTemplateDao.List(ctx, &flowDto.ListFlowTemplateReq{
Keyword: req.Keyword,
Page: req.Page,
})
if err != nil {
return
}
r := &flowDto.ListFlowTemplateRes{
Total: t,
}
err = gconv.Struct(l, &r.List)
if err != nil {
return
}
res.ListFlowTemplateRes = r
}
var t int
var l []*entity.FlowTemplate
l, t, err = flowDao.FlowTemplateDao.List(ctx, &flowDto.ListFlowTemplateReq{
Keyword: req.Keyword,
Page: req.Page,
})
if err != nil {
return
}
r := &flowDto.ListFlowTemplateRes{
Total: t,
}
err = gconv.Struct(l, &r.List)
if err != nil {
return
}
var user *beans.User
user, err = utils.GetUserInfo(ctx)
if err != nil {
@@ -212,11 +215,62 @@ func (s *flowUserService) List(ctx context.Context, req *flowDto.ListFlowUserReq
if err != nil {
return
}
res = &flowDto.ListFlowRes{
ListFlowUserRes: re,
ListFlowTemplateRes: r,
IsAdmin: admin,
}
res.ListFlowUserRes = re
return
}
// copySubFlows 递归拷贝子工作流(子流程可能再包含子流程),返回 旧id→新id 映射。
// 映射键是子工作流在来源(模版)中的 id,即父流程 sub_flow 节点 SubConfig.WorkflowId 指向的值;
// 拷贝时把子工作流落为当前用户自己的流程(SourceFlowTemplateId 记录来源)。
// 同一子流程被多个节点引用时只拷贝一份,避免产生孤儿副本。
func copySubFlows(ctx context.Context, subs []flowDto.UpdateFlowUserReq) (map[int64]int64, error) {
idMap := make(map[int64]int64)
for i := range subs {
sub := subs[i]
if _, done := idMap[sub.Id]; done {
continue
}
nestedMap, err := copySubFlows(ctx, sub.SubFlows)
if err != nil {
return nil, err
}
for k, v := range nestedMap {
idMap[k] = v
}
// 改写本子流程对更深层子流程的引用后,再落库为新的用户流程
rewriteSubFlowWorkflowIds(sub.FlowContent, idMap)
var nodeInputParams []*entity.FlowNode
if sub.FlowContent != nil {
nodeInputParams = ExtractFlowNodeFrom(sub.FlowContent)
}
newId, err := flowDao.FlowUserDao.Insert(ctx, &flowDto.CreateFlowUserReq{
FlowName: sub.FlowName,
Description: sub.Description,
FlowContent: sub.FlowContent,
NodeInputParams: nodeInputParams,
SourceFlowTemplateId: sub.Id,
})
if err != nil {
return nil, err
}
idMap[sub.Id] = newId
}
return idMap, nil
}
// rewriteSubFlowWorkflowIds 把 flowContent 中 sub_flow 节点的 WorkflowId 按 idMap 改写为新拷贝的流程 id
func rewriteSubFlowWorkflowIds(flowContent *entity.FlowInfo, idMap map[int64]int64) {
if flowContent == nil {
return
}
for i := range flowContent.Nodes {
n := &flowContent.Nodes[i]
if n.SubConfig == nil || n.SubConfig.WorkflowId == 0 {
continue
}
if newId, ok := idMap[n.SubConfig.WorkflowId]; ok {
n.SubConfig.WorkflowId = newId
}
}
}
+253
View File
@@ -0,0 +1,253 @@
package flow
import (
"ai-agent/gateway"
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"context"
"fmt"
"time"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
// init 注册自定义合并函数:处理 *flowDto.FlowExecutionInput 类型合并。
// 合并函数全局唯一且与图内容无关,放包级 init 注册一次(避免每次 BuildGraph 重注册全局状态)。
func init() {
compose.RegisterValuesMergeFunc(func(values []*flowDto.FlowExecutionInput) (*flowDto.FlowExecutionInput, error) {
if len(values) == 0 {
return nil, nil
}
// 首次运行所有并行分支共享同一个 ConfigMap 指针,直接返回 values[0] 即可。
// 但续跑(ReExecute)时各分支从 checkpoint 反序列化出独立的 ConfigMap 副本,
// 只返回 values[0] 会丢失其他分支写入的 OutputResult(用户实测:node-8 成功的结果
// 在汇合节点 node-7 变 null)。以第一个为基底,把其余分支中缺失的节点输出合并进来。
base := values[0]
for _, v := range values[1:] {
if v == nil {
continue
}
for nodeId, cfg := range v.ConfigMap {
if cfg == nil {
continue
}
baseCfg, ok := base.ConfigMap[nodeId]
if !ok || baseCfg == nil {
base.ConfigMap[nodeId] = cfg
continue
}
if len(baseCfg.OutputResult) == 0 && len(cfg.OutputResult) > 0 {
baseCfg.OutputResult = cfg.OutputResult
}
}
// 合并已执行节点列表(按 NodeId 去重),续跑时被恢复分支的进度不丢失
for _, en := range v.ExecutedNodes {
dup := false
for _, b := range base.ExecutedNodes {
if b.NodeId == en.NodeId {
dup = true
break
}
}
if !dup {
base.ExecutedNodes = append(base.ExecutedNodes, en)
}
}
}
return base, nil
})
}
// BuildGraph 根据 FlowInfo 构建完整的 Eino Graph 拓扑
func BuildGraph(ctx context.Context, flowContent *entity.FlowInfo) ([]entity.FlowNode, *compose.Graph[any, any]) {
graph := compose.NewGraph[any, any](
// 本地状态初始化
compose.WithGenLocalState(func(ctx context.Context) *flowDto.NodeExecutionState {
return &flowDto.NodeExecutionState{}
}),
)
// 注册所有节点
for _, item := range flowContent.Nodes {
registerNodeToGraph(graph, item)
}
// 注册开始节点
if flowContent.StartNodeId != "" {
_ = graph.AddEdge(compose.START, flowContent.StartNodeId)
}
var nodeList []entity.FlowNode
originalEndNodes := FindEndNodes(flowContent.StartNodeId, flowContent.Edges)
for _, endID := range originalEndNodes {
// 保存结果节点 ID 必须稳定:ReExecute 续跑重建图时复用同一 ID,
// 否则 checkpoint 里 ConfigMap 存的是上次的旧 ID,续跑时新图按新 ID 查不到配置,
// 报"节点信息为空"。一个末端节点对应一个保存结果节点,用 endID 派生即唯一且稳定。
summaryNodeId := fmt.Sprintf("%s_%s", node.NodeTypeSystemSum, endID)
summaryNode := entity.FlowNode{
Id: summaryNodeId,
NodeCode: node.NodeTypeSystemSum,
Name: node.GetNodeTypeName(node.NodeTypeSystemSum),
}
nodeList = append(nodeList, summaryNode)
flowContent.Nodes = append(flowContent.Nodes, summaryNode)
registerNodeToGraph(graph, summaryNode)
_ = graph.AddEdge(endID, summaryNodeId)
_ = graph.AddEdge(summaryNodeId, compose.END)
}
// 构建边关系
edgeMap := make(map[string][]entity.FlowEdge)
for _, edge := range flowContent.Edges {
edgeMap[edge.From] = append(edgeMap[edge.From], edge)
}
// 处理连线 & 分支
for _, edges := range edgeMap {
// 普通节点连线
for _, e := range edges {
_ = graph.AddEdge(e.From, e.To)
}
}
return nodeList, graph
}
// BuildGraphFromFlowContent 根据前端保存的工作流JSON,自动构建执行图并编译
func BuildGraphFromFlowContent(ctx context.Context, flowContent *entity.FlowInfo) ([]entity.FlowNode, compose.Runnable[any, any], error) {
nodeList, graph := BuildGraph(ctx, flowContent)
// BuildGraph 已把 summary(保存结果)节点追加进 flowContent.Nodes,此时是全部已注册节点的完整集合。
// 方案: 每个业务节点正常完成后自动暂停并落 checkpoint(编译期 WithInterruptAfterNodes),
// 崩溃恢复(BuildExecution(false)续跑)即跳过已完成的同步节点, 不再重跑/重复计费。
// 详见根目录《工作流节点断点续跑技术设计.md》。Start 型空载节点不为它落 cp; 只接 END 的节点
// Eino 不落暂停(无下游续跑点), 列了也无副作用。
interruptAfter := make([]string, 0, len(flowContent.Nodes))
for _, n := range flowContent.Nodes {
if n.NodeCode == node.NodeTypeStart {
continue
}
interruptAfter = append(interruptAfter, n.Id)
}
compile, err := graph.Compile(ctx,
compose.WithGraphName("auto_build_workflow"),
compose.WithCheckPointStore(NewDbCheckPointStore()),
compose.WithNodeTriggerMode(compose.AllPredecessor),
compose.WithInterruptAfterNodes(interruptAfter),
)
return nodeList, compile, err
}
// buildConfigMap 由 FlowInfo + 图节点列表构建 ConfigMap:先放流程配置节点,再放图中补充节点
// (保存结果节点等),供节点执行时按 nodeId 查配置。nodeList 取自 BuildGraph 返回值。
func buildConfigMap(flowContent *entity.FlowInfo, nodeList []entity.FlowNode) map[string]*entity.FlowNode {
configMap := make(map[string]*entity.FlowNode)
for _, cfg := range ExtractFlowNodeFrom(flowContent) {
configMap[cfg.Id] = cfg
}
for i := range nodeList {
configMap[nodeList[i].Id] = &nodeList[i]
}
return configMap
}
// registerNodeToGraph 将单个节点注册到图中(包含通用包装逻辑)
func registerNodeToGraph(graph *compose.Graph[any, any], flowNode entity.FlowNode) {
// 通用包装:全程入参都是 *FlowExecutionInput
wrapLambda := func(lambda func(ctx context.Context, input any) (any, error)) func(ctx context.Context, input any) (any, error) {
return func(ctx context.Context, input any) (any, error) {
startTime := time.Now()
// 构建节点执行入参(含中断恢复)
execInput, realInput, err := BuildNodeExecutionInput(ctx, input, flowNode)
if err != nil {
return nil, err
}
flowNodeDesc := flowNode.Desc
if g.IsEmpty(flowNodeDesc) {
flowNodeDesc = flowNode.Name
}
// 上报节点执行进度(WebSocket场景下推送进度给前端)
if reporter := GetProgressReporter(ctx); reporter != nil {
reporter.ReportStart(flowNode.Id, flowNodeDesc, nodeReportIndex(execInput, flowNode.Id, 1), len(execInput.ConfigMap))
}
// 上传入参到OSS
ossResult, err := gateway.Upload(ctx, fmt.Sprintf("nodeInput:%v.txt", time.Now().UnixMilli()), gconv.Bytes(gconv.String(realInput)))
if err != nil {
return nil, HandleFailedNodeExecution(ctx, execInput, 0, flowNode, err, 0)
}
// 创建节点执行记录
nodeExecutionId, err := CreateNodeExecutionRecord(ctx, execInput, flowNode, ossResult)
if err != nil {
return nil, HandleFailedNodeExecution(ctx, execInput, 0, flowNode, err, 0)
}
realInput.NodeExecutionId = nodeExecutionId
// 执行节点
_, err = lambda(ctx, realInput)
durationMs := time.Since(startTime).Milliseconds()
if err != nil {
// 执行失败处理
return nil, HandleFailedNodeExecution(ctx, execInput, nodeExecutionId, flowNode, err, durationMs)
}
// 执行成功处理
if err = HandleSuccessfulNodeExecution(ctx, execInput, realInput, nodeExecutionId, flowNode, durationMs); err != nil {
return nil, HandleFailedNodeExecution(ctx, execInput, nodeExecutionId, flowNode, err, durationMs)
}
// 上报节点执行进度(WebSocket场景下推送进度给前端)
if reporter := GetProgressReporter(ctx); reporter != nil {
reporter.ReportComplete(flowNode.Id, flowNodeDesc, nodeReportIndex(execInput, flowNode.Id, 0), len(execInput.ConfigMap))
}
// 返回整个 execInput,让下一个节点继续用
return execInput, nil
}
}
switch flowNode.NodeCode {
case node.NodeTypeStart:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(StartLambda)))
case node.NodeTypeSystemSum:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(SummaryLambda)))
case node.NodeTypeModel:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(ModelLambda)))
case node.NodeTypeForm:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(FormLambda)))
case node.NodeTypeDataMerge:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(DataMergeLambda)))
case node.NodeTypeSubFlow:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(SubFlowLambda)))
case node.NodeTypeHttp:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(HttpLambda)))
case node.NodeTypeScriptTranscribe:
_ = graph.AddLambdaNode(flowNode.Id, compose.InvokableLambda(wrapLambda(ScriptTranscribeLambda)))
}
}
// nodeReportIndex 计算节点在进度上报中的序号:节点已在已执行列表则取其位置,
// 否则按当前已执行数 + offset(start 上报时节点尚未入列 offset=1complete 后已入列命中 IndexOf
func nodeReportIndex(execInput *flowDto.FlowExecutionInput, nodeId string, offset int) int {
if idx := IndexOf(execInput.ExecutedNodes, nodeId); idx != -1 {
return idx
}
return len(execInput.ExecutedNodes) + offset
}
// IndexOf 返回元素第一次出现的下标,不存在返回 -1
func IndexOf(slice []flowDto.ExecutedNode, target string) int {
for i, v := range slice {
if v.NodeId == target {
return i + 1
}
}
return -1
}
+265
View File
@@ -0,0 +1,265 @@
package flow
import (
"ai-agent/gateway"
flowDao "ai-agent/workflow/dao/flow"
nodeDao "ai-agent/workflow/dao/node"
flowDto "ai-agent/workflow/model/dto/flow"
nodeDto "ai-agent/workflow/model/dto/node"
"ai-agent/workflow/model/entity"
"ai-agent/workflow/service/flow/processor/builtin/media"
"ai-agent/workflow/service/flow/values"
"context"
"fmt"
"sync"
"github.com/gogf/gf/v2/util/gconv"
)
// StartLambda 启动节点
func StartLambda(ctx context.Context, input any) (any, error) {
return input, nil
}
// FormLambda 表单调用节点
func FormLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
// 解析 valueSource 引用,填充表单节点输出配置(供下游引用)
for _, output := range nodeInput.Config.OutputConfig {
values.ProcessValueSourceRecursive(output, nodeInput.Global)
}
return nodeInput, nil
}
// cacheNodeId 缓存键节点 id:scope 非空(子流程批量子执行)时拼上作用域后缀,使各份子执行的
// async/segment 缓存行互相隔离(同一 exec 下多份内层节点 id 相同,不隔离会互相命中/覆盖 done 结果,
// 见《工作流子流程批量缓存隔离设计.md》);顶层 scope 为空 → 原样返回,行为不变。
// 仅用于缓存读写键,节点的 Config.Id / node_execution 记录 / 输出引用一律不受影响。
func cacheNodeId(global *flowDto.FlowExecutionInput, nodeId string) string {
if global == nil || global.SubFlowScope == "" {
return nodeId
}
return nodeId + global.SubFlowScope
}
// ModelLambda 模型调用节点
func ModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
modelParams, err := values.BuildModelRequestBody(nodeInput.Config.ModelConfig.ModelRequestParamsPath, nodeInput.Global)
if err != nil {
return nil, err
}
// async/segment 缓存键节点 id(子流程批量子执行带 scope,顶层即 Config.Id
cNodeId := cacheNodeId(nodeInput.Global, nodeInput.Config.Id)
// 2. 前置工具:决定模型调用入参(单次/多次)
// 入参统一为扁平模型请求体(BuildModelRequestBody 输出,key 为点分路径)。
// 分批处理器按默认上限拆分集合字段,其余前置工具(如 split_shots_pipeline)读取扁平参数。
preToolParams := modelParams
paramsList, err := invokePreTool(ctx, nodeInput.Config.PreTool, preToolParams)
if err != nil {
return nil, err
}
// 3. 逐批调用模型,汇总输出(保持请求顺序),累计 token/费用供节点记录落库
var outputRes []map[string]any
var totalTokens int64
var totalPrompt int64
var totalCompletion int64
var totalCost float64
var totalDuration int64
// 计价用生效模型 id:引用行由 model-gateway 解析为系统模型 idModelCallRes.ModelId,计价按系统模型);
// 未返回(model-gateway 旧版本)时回落节点配置的模型 id。media_type 供 per_token 命中媒体价。
effModelID := nodeInput.Config.ModelConfig.ModelId
var effMediaType string
if len(paramsList) > 1 {
// 段级续跑仅在"多段 + 视频模型"启用;非视频分段(批量文本等)走原逻辑零影响。
// 段身份 = 列表位置(0-based):paramsList 顺序即段序,concat 按列表顺序拼接;
// 位置互不重复且跨 reExecute 稳定(参数一致 → 段数/顺序不变)。不依赖 params 里的
// segment_index——真实链路(上游 split_shots_pipeline 转写 → 下游 split_segment 按
// __segment_fields 拆分,invokePreTool 剥离 __ 内部键)下 paramsList 只有模型参数。
// segVideo=false 时走既有非段级合并路径(全量生成、不落库、不复用),全新执行行为不变,
// 后续自动 concat 判断(独立的 isVideoModel 调用)仍正常执行。
segVideo := isVideoModel(ctx, nodeInput.Config.ModelConfig.ModelId)
// 续跑(!ForceNewRun)时读取该节点已成功段;全新执行不查(BuildExecution 已清旧段),saved 为 nil → 全量重生成
var saved map[int]entity.SegmentRef
if !nodeInput.Global.ForceNewRun && segVideo {
saved, err = flowDao.FlowSegmentResultDao.ListByNode(ctx, nodeInput.Global.NodeGroupId, cNodeId)
if err != nil {
return nil, err
}
}
idxList, needGen := planSegmentResume(paramsList, saved)
results := make([][]map[string]any, len(paramsList))
tokenRes := make([]*gateway.ModelCallRes, len(paramsList))
errs := make([]error, len(paramsList))
saveErrs := make([]error, len(paramsList))
isInference := make([]bool, len(paramsList))
var wg sync.WaitGroup
for i, params := range paramsList {
if !needGen[i] {
continue
}
wg.Add(1)
go func(i int, params map[string]any) {
defer wg.Done()
// 每段单次调用,不原地重试:段失败即走节点失败收口(HandleFailedNodeExecution → Interrupt),
// 下次 reExecute 由 planSegmentResume 复用已成功段、仅重生成失败段
results[i], tokenRes[i], isInference[i], errs[i] = ModelCallResultLambda(ctx, nodeInput.Global.NodeGroupId, nodeInput.Config.ModelConfig.ModelId, nodeInput.Global.SessionId, params, nodeInput.Config.Prompt, nodeInput.Global.ExecutionId, cNodeId, idxList[i])
// 每段成功立即落库:该段刚成功即持久化,其他段仍在跑时已成功段也不丢;
// 后续段失败或进程崩溃(panic/OOM/kill)时,已完成段已在库中,reExecute 可直接复用
if segVideo && errs[i] == nil {
for _, rec := range results[i] {
key := media.FindVideoKey(rec)
url := media.FindVideoURL(ctx, rec)
if key == "" || url == "" {
continue
}
if err := flowDao.FlowSegmentResultDao.Save(ctx, nodeInput.Global.NodeGroupId, nodeInput.Global.ExecutionId, cNodeId, idxList[i], key, url); err != nil {
saveErrs[i] = err
}
}
}
}(i, params)
}
wg.Wait()
// 仍有失败段或落库失败 → 节点失败(成功段已立即落库,供下次 reExecute 复用)
for i := range results {
if saveErrs[i] != nil {
return nil, saveErrs[i]
}
if needGen[i] && errs[i] != nil {
return nil, errs[i]
}
if needGen[i] && tokenRes[i] != nil {
totalTokens += tokenRes[i].TotalTokens
totalPrompt += tokenRes[i].PromptTokens
totalCompletion += tokenRes[i].CompletionTokens
totalCost += tokenRes[i].Cost
if isVideoModel(ctx, nodeInput.Config.ModelConfig.ModelId) {
totalDuration += tokenRes[i].Duration
}
if tokenRes[i].ModelId > 0 {
effModelID = tokenRes[i].ModelId
}
if effMediaType == "" {
effMediaType = tokenRes[i].MediaType
}
}
}
if segVideo {
// 复用段 + 新生段按段序号升序合并,concat 按列表顺序拼接 → 顺序保证
outputRes = mergeSegmentOutputs(idxList, needGen, results, saved)
} else {
if isInference[0] {
outputRes = mergeInferenceBatchResults(results)
} else {
for _, res := range results {
outputRes = append(outputRes, res...)
}
}
}
} else {
for _, params := range paramsList {
res, modelRes, _, err := ModelCallResultLambda(ctx, nodeInput.Global.NodeGroupId, nodeInput.Config.ModelConfig.ModelId, nodeInput.Global.SessionId, params, nodeInput.Config.Prompt, nodeInput.Global.ExecutionId, cNodeId, flowDao.FlowAsyncSegSentinel)
if err != nil {
return nil, err
}
if modelRes != nil {
totalTokens += modelRes.TotalTokens
totalPrompt += modelRes.PromptTokens
totalCompletion += modelRes.CompletionTokens
totalCost += modelRes.Cost
if isVideoModel(ctx, nodeInput.Config.ModelConfig.ModelId) {
totalDuration += modelRes.Duration
}
if modelRes.ModelId > 0 {
effModelID = modelRes.ModelId
}
if effMediaType == "" {
effMediaType = modelRes.MediaType
}
}
outputRes = append(outputRes, res...)
}
}
// 3.5 把本次节点消耗的 token/费用/生成视频时长写入节点执行记录,供汇总节点聚合到 exec_workflow
// model_id 供 per_token 结算按模型聚合 tokentotal_duration 供 per_item/per_second 按生成视频总时长计费;
// per_char 模型把输出字数映射到 completion_tokens 传输,随 token 拆分一并落库
if nodeInput.NodeExecutionId > 0 && (totalTokens > 0 || totalCost > 0 || totalDuration > 0) {
if _, err = nodeDao.NodeExecutionDao.Update(ctx, &nodeDto.UpdateNodeExecutionReq{
Id: nodeInput.NodeExecutionId,
TokenInfo: []map[string]any{{
// model_id 写字符串:token_info 为 JSONBint64 落库成 JSON 数字,读回是 float64
// billing 侧按 model 聚合时 (string) 断言会失败导致 per_token 永远记 0。
// 与 ModelItem.ModelId 的 json:"modelId,string" 约定一致,字符串精确回环(雪花id>2^53 无损)。
// effModelID 为解析后的系统模型 id(引用行),per_token 结算按此查价。
"model_id": gconv.String(effModelID),
"prompt_tokens": totalPrompt,
"completion_tokens": totalCompletion,
"media_type": effMediaType,
"total_tokens": totalTokens,
"total_fee": totalCost,
"total_duration": totalDuration,
}},
}); err != nil {
return nil, fmt.Errorf("节点:%v 写入token信息失败: %v", nodeInput.Config.Name, err)
}
}
// 4.5 视频模型节点返回多个视频时,自动调用视频合成工具(concat_videos)合并为单条;
// 已显式配置 concat_videos 后置工具时跳过,避免重复合并
if nodeInput.Config.PostTool != media.ProcessorName && len(outputRes) > 1 && isVideoModel(ctx, nodeInput.Config.ModelConfig.ModelId) {
outputRes, err = invokePostTool(ctx, media.ProcessorName, outputRes, map[string]any{"callback_url": "callback_url", "upload": true})
if err != nil {
return nil, err
}
} else {
// 4. 后置工具:加工模型输出(透传原始请求参数,供后置工具读取合并配置等)
outputRes, err = invokePostTool(ctx, nodeInput.Config.PostTool, outputRes, modelParams)
if err != nil {
return nil, err
}
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// mergeInferenceBatchResults 推理模型分批结果拼接为单条输出记录:
// 各批结果按批序对同名 key 的值做字符串拼接("拼到一个字段"),最终返回单条 {key:值} 记录。
// 非字符串值(如结构/数组字段)取最后一份,避免误拼接。
func mergeInferenceBatchResults(results [][]map[string]any) []map[string]any {
merged := make(map[string]any)
for _, res := range results {
for _, record := range res {
for key, val := range record {
prev, has := merged[key]
if !has {
merged[key] = val
continue
}
sPrev, pOK := prev.(string)
sVal, vOK := val.(string)
if pOK && vOK {
merged[key] = sPrev + "\n" + sVal
continue
}
merged[key] = val
}
}
}
return []map[string]any{merged}
}
+21
View File
@@ -0,0 +1,21 @@
package flow
import (
flowDto "ai-agent/workflow/model/dto/flow"
"context"
"fmt"
)
// HttpLambda 构建HTTP(S)接口
func HttpLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
outputRes, err := HttpCallResultLambda(ctx, nodeInput)
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
-812
View File
@@ -1,812 +0,0 @@
package flow
import (
"ai-agent/workflow/consts/flow"
"ai-agent/workflow/consts/node"
"ai-agent/workflow/consts/public"
fileDao "ai-agent/workflow/dao/file"
flowDao "ai-agent/workflow/dao/flow"
"ai-agent/workflow/model/dto"
fileDto "ai-agent/workflow/model/dto/file"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"context"
"fmt"
"strconv"
"strings"
"sync"
"time"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/cloudwego/eino-examples/compose/batch/batch"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/database/gdb"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
func StartLambda(ctx context.Context, input any) (any, error) {
return input, nil
}
func FormLambda(ctx context.Context, input any) (any, error) {
return input, nil
}
func SubFlowLambda(ctx context.Context, input any) (any, error) {
// 1. 类型断言(和其他节点保持一致的入参结构)
nodeExecInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("子流程节点入参类型错误,期望*flowDto.NodeExecutionInput,实际%T", input)
}
// 2. 解析子流程配置
subFlowConfig := nodeExecInput.Config.SubConfig
if subFlowConfig == nil {
return nil, fmt.Errorf("子流程节点缺少配置")
}
getRes, err := FlowUserService.Get(ctx, &flowDto.GetFlowUserReq{
Id: subFlowConfig.FlowId,
})
if err != nil {
return nil, err
}
// 3. 编译子流程Graph(复用现有 BuildGraphFromFlowContent 逻辑)
nodeList, subGraph := BuildGraph(ctx, getRes.FlowContent)
// 4. 构建子流程Workflow(绑定START/END,和示例对齐)
innerWorkflow := compose.NewWorkflow[*flowDto.FlowExecutionInput, *flowDto.FlowExecutionInput]()
// 挂载子图节点并绑定全局START
innerWorkflow.AddGraphNode("sub_flow_graph", subGraph).AddInput(compose.START)
// 绑定子图输出到全局END
innerWorkflow.End().AddInput("sub_flow_graph")
// 5. 构建BatchNode(批量执行子流程,复用示例逻辑)
batchNode := batch.NewBatchNode(&batch.NodeConfig[*flowDto.FlowExecutionInput, *flowDto.FlowExecutionInput]{
Name: fmt.Sprintf("sub_flow_batch_%s", nodeExecInput.Config.Id),
InnerTask: innerWorkflow,
MaxConcurrency: subFlowConfig.MaxConcurrency,
})
//skillName, from, userFrom := BuildParam(nodeExecInput)
//fmt.Printf("skillName: %s, from: %s, userFrom: %s\n", skillName, from, userFrom)
// 6. 提取批量输入(从全局入参中获取)
batchInputs := make([]*flowDto.FlowExecutionInput, 0)
nodeInputParams := ExtractFlowNodeFrom(getRes.FlowContent)
configMap := make(map[string]*entity.FlowNode)
for _, cfg := range nodeInputParams {
configMap[cfg.Id] = cfg
}
for _, i := range nodeList {
configMap[i.Id] = &i
}
// =========================================================================
// ✅【第4步】构建全局执行入参(现在 schemaMap 是有值的!)
// =========================================================================
execInput := &flowDto.FlowExecutionInput{
NodeGroupId: nodeExecInput.Global.NodeGroupId,
IsDialogue: nodeExecInput.Global.IsDialogue,
ExecutionId: nodeExecInput.Global.ExecutionId,
ConfigMap: configMap,
SessionId: nodeExecInput.Global.SessionId,
Desc: nodeExecInput.Global.Desc,
SkillName: nodeExecInput.Global.SkillName,
FileUrl: nodeExecInput.Global.FileUrl,
}
batchInputs = append(batchInputs, execInput)
// 7. 执行批量子流程
batchOutput, err := batchNode.Invoke(ctx, batchInputs)
if err != nil {
return nil, fmt.Errorf("执行子流程BatchNode失败: %v", err)
}
for idx, singleSubResult := range batchOutput {
fmt.Printf("【批量任务%d 最终消息条数】: %v\n", idx+1, singleSubResult)
}
// 8. 保存子流程执行结果到当前节点输出
//nodeExecInput.Config.OutputResult = append(nodeExecInput.Config.OutputResult, batchOutput)
return nodeExecInput, nil
}
// JudgeLambda 分支判断核心:读取IntentLambda的输出 → 返回目标节点ID做路由
func JudgeLambda(ctx context.Context, input any) (string, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return "", fmt.Errorf("入参类型错误,期望 *flowDto.NodeExecutionInput,实际 %T", input)
}
//inputMap, outputMap, modelMap := GetNodeContextContent(nodeInput.Global, nodeInput.Config)
//fmt.Printf("JudgeLambda路由:输入=%s\n", gjson.MustEncode(inputMap))
//fmt.Printf("JudgeLambda路由:输出=%s\n", gjson.MustEncode(outputMap))
//fmt.Printf("JudgeLambda路由:模型=%s\n", gjson.MustEncode(modelMap))
//configMap := gconv.Map(nodeInput.Config.Config)
//ids := gconv.Strings(configMap["branch_ids"])
//fmt.Printf("JudgeLambda路由:目标节点ID=%s\n", gconv.String(ids))
//
//m := map[string]bool{
// "80000a50-81e1-4c15-adae-aab6c0d781ad": true,
// "59a6ffa2-3252-4535-b6ed-d3e49cdf6c55": true,
//}
//
//return m, nil
// 1. 直接用你原来的方法(返回两个 map)
inputMap, outputMap, modelMap := GetNodeContextContent(nodeInput.Global, nodeInput.Config)
var outputResult []node.NodeFormField
outputResult = append(outputResult, inputMap...)
outputResult = append(outputResult, outputMap...)
//for _, valueAny := range inputMap {
// if field, ok := valueAny.(node.NodeFormField); ok {
// outputResult = append(outputResult, field)
// }
//}
//for _, valueAny := range outputMap {
// if field, ok := valueAny.(node.NodeFormField); ok {
// outputResult = append(outputResult, field)
// }
//}
for _, valueAny := range modelMap {
if field, ok := valueAny.(node.NodeFormField); ok {
outputResult = append(outputResult, field)
}
}
contextParts := ""
for _, v := range nodeInput.Config.FormConfig {
contextParts = fmt.Sprintf("%s,%s:%s", contextParts, v.Label, v.Value)
}
if !nodeInput.Global.IsDialogue {
for _, v := range outputResult {
contextParts = fmt.Sprintf("%s,%s:%s", contextParts, v.Label, v.Value)
}
}
if !g.IsEmpty(nodeInput.Global.Desc) {
contextParts = fmt.Sprintf("%s,%s:%s", contextParts, "描述", nodeInput.Global.Desc)
}
configMap := gconv.Map(nodeInput.Config.Config)
ids := gconv.Strings(configMap["branch_ids"])
branchIdNameMap := gconv.Map(configMap["branch_id_name_map"])
var branchIdNameLines []string
for _, id := range ids {
name := gconv.String(branchIdNameMap[id])
branchIdNameLines = append(branchIdNameLines, fmt.Sprintf("%s: %s", id, name))
}
getIsChatModel, err := GetIsChatModel(ctx)
if err != nil {
return "", err
}
composeResult, err := GetComposeResult(ctx, 2, getIsChatModel.Model.ModelName, "", "", []map[string]any{{"prompt": strings.Join(branchIdNameLines, "\n")}}, []map[string]any{{"prompt": contextParts}}, nodeInput.Global.FileUrl, nodeInput.Global.SessionId, nodeInput.Config.Id, "判断节点")
if err != nil {
return "", err
}
if g.IsEmpty(composeResult.TaskId) {
return "", fmt.Errorf("msg is empty")
}
content := ""
for key, _ := range getIsChatModel.Model.ResponseBody {
content = gconv.String(composeResult.Messages.Rounds[0][key])
}
fmt.Printf("JudgeLambda路由:目标节点ID=%s\n", gconv.String(content))
return content, nil
}
func BatchModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
reqMap := make([]map[string]any, 0)
for _, userItem := range userFrom {
m := gconv.Map(userItem)
for _, i := range nodeInput.Config.InputSource {
for _, f := range i.Field {
val := m[f]
if !g.IsEmpty(val) {
if g.NewVar(val).IsSlice() {
slice := gconv.SliceAny(val)
for _, item := range slice {
reqMap = append(reqMap, map[string]any{f: item})
}
} else {
reqMap = append(reqMap, map[string]any{f: val})
}
}
}
}
}
// 结果按索引存放,切片不同下标并发写无竞争,不用锁
res := make([][]node.NodeFormField, len(reqMap))
var wg sync.WaitGroup
subCtx, cancel := context.WithCancel(ctx)
defer cancel()
// 缓冲1错误通道,仅接收第一个错误
errCh := make(chan error, 1)
// 并发执行任务
for idx, item := range reqMap {
wg.Add(1)
go func(idx int, userItem map[string]any) {
defer wg.Done()
// 上下文已取消则直接退出
select {
case <-subCtx.Done():
return
default:
}
singleUserFrom := []map[string]any{userItem}
output, err := TextNode(subCtx, nodeInput, skillName, from, singleUserFrom)
if err != nil {
// 仅第一个错误写入通道
select {
case errCh <- err:
cancel() // 触发全局取消,其他协程快速退出
default:
}
return
}
res[idx] = output
}(idx, item)
}
// 任务全部结束后关闭错误通道
go func() {
wg.Wait()
close(errCh)
}()
// ========== 修正后的等待逻辑 ==========
var execErr error
select {
// 优先捕获业务错误
case execErr = <-errCh:
if execErr != nil {
// 收到真实业务错误,等待剩余协程收尾后返回
wg.Wait()
return nil, execErr
}
// execErr == nil 代表通道关闭、无任何错误,走到下方返回完整结果
case <-subCtx.Done():
// 上下文被取消,阻塞读完errCh,确认是否存在业务错误
execErr = <-errCh
}
// 拼接输出结果
var globalIndex int
var outputRes []node.NodeFormField
for _, items := range res {
for _, item := range items {
oldField := item.Field
if idx := strings.LastIndex(oldField, ":"); idx != -1 {
item.Field = oldField[:idx+1] + fmt.Sprint(globalIndex)
}
oldLabel := item.Label
if idx := strings.LastIndex(oldLabel, ":"); idx != -1 {
item.Label = oldLabel[:idx+1] + fmt.Sprint(globalIndex)
}
outputRes = append(outputRes, item)
}
globalIndex++
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// TextModelLambda 构建文案
func TextModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
outputRes, err := TextNode(ctx, nodeInput, skillName, from, userFrom)
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
//}
return nodeInput, nil
}
// ImageModelLambda 构建图片
func ImageModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
outputRes, err := ImgNode(ctx, nodeInput, skillName, from, userFrom)
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// AudioModelLambda 构建音频
func AudioModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
outputRes, err := AudioOptimizeNode(ctx, nodeInput, skillName, from, userFrom)
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// VideoModelLambda 构建视频
func VideoModelLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
res, err := VideoOptimizeNode(ctx, nodeInput, skillName, from, userFrom)
if err != nil {
return nil, err
}
videoURL := make([]string, 0)
for _, v := range res {
if strings.Contains(v.Field, "content") {
videoURL = append(videoURL, gconv.String(v.Value))
}
}
if g.IsEmpty(videoURL) {
return nil, fmt.Errorf("视频合成失败:模型生成视频失败")
}
waitRes, err := VideoConcat(ctx, videoURL)
if err != nil {
return nil, err
}
msg := new(flowDto.VideoCallbackReq)
if err = gconv.Struct(waitRes, msg); err != nil {
return nil, err
}
urlPrefix, err := utils.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
newS := strings.ReplaceAll(urlPrefix, g.Cfg().MustGet(ctx, "filePrefix").String(), g.Cfg().MustGet(ctx, "minioPrefix").String())
outputRes := make([]node.NodeFormField, 0)
if nodeInput.Config.IsSaveFile {
outputRes = append(outputRes, node.NodeFormField{
Field: fmt.Sprintf("video_oss_url:content:%d", 0),
Value: msg.FileURL,
Label: fmt.Sprintf("video_oss_url:content:%d", 0),
Type: "string",
})
}
outputRes = append(outputRes, node.NodeFormField{
Field: fmt.Sprintf("concat_video_url:content:%d", 0),
Value: newS + msg.FileURL,
Label: fmt.Sprintf("视频内容:content:%d", 0),
Type: "string",
})
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// HttpLambda 构建HTTP(S)接口
func HttpLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
outputRes := make([]node.NodeFormField, 0)
var err error
outputRes, err = HttpNode(ctx, nodeInput)
//if nodeInput.Config.Name == "生成视频" {
// outputRes, err = HttpNode(ctx, nodeInput)
//} else {
// a := []map[string]any{
// {
// "timeline": "0.0-2.1",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/1.mp4",
// },
// {
// "timeline": "2.1-4.5",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/2.mp4",
// },
// {
// "timeline": "4.5-12.2",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/3.mp4",
// },
// {
// "timeline": "12.2-13.6",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/4.mp4",
// },
// {
// "timeline": "13.6-17.7",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/5.mp4model-gateway",
// },
// {
// "timeline": "17.7-31.0",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/6.mp4",
// },
// {
// "timeline": "31.0-33.2",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/7.mp4",
// },
// {
// "timeline": "33.2-37.4",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/8.mp4",
// },
// {
// "timeline": "37.4-38.9",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/9.mp4",
// },
// {
// "timeline": "38.9-57.9",
// "url": "https://ark-auto-2127201628-cn-beijing-default.tos-cn-beijing.volces.com/%E8%A7%86%E9%A2%91/10.mp4",
// },
// }
// outputRes = append(outputRes, node.NodeFormField{
// Field: fmt.Sprintf("segments"),
// Value: a,
// Label: fmt.Sprintf("segments"),
// Type: "string",
// })
// outputRes = append(outputRes, node.NodeFormField{
// Field: fmt.Sprintf("audioUrl"),
// Value: "http://116.204.74.41:9000/tenantid-94/2026-06-11/9915351c-55b9-46d8-b783-3815126b.m4a",
// Label: fmt.Sprintf("audioUrl"),
// Type: "string",
// })
//}
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
// DataConversionLambda 构建数据转换
func DataConversionLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
skillName, from, userFrom := BuildParam(nodeInput)
outputRes, err := DataConversionNode(ctx, nodeInput, skillName, from, userFrom)
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = outputRes
return nodeInput, nil
}
func DataMergeLambda(ctx context.Context, input any) (res any, err error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("参数合并入参类型错误")
}
// var nodeIds []string
// for _, item := range nodeInput.Config.InputSource {
// nodeIds = append(nodeIds, item.NodeId)
// }
//
// // 检查是否所有输入节点都执行完成,并且检查是否有节点失败
// checkAllExecuted := func() (allExecuted bool, hasFailed bool, failedNode string) {
// executedCount := 0
// for _, executedNode := range nodeInput.Global.ExecutedNodes {
// // 检查是否是我们需要的输入节点,并且它失败了
// for _, targetId := range nodeIds {
// if executedNode.NodeId == targetId {
// if executedNode.Status == node.NodeExecutionStatusFailed.Code() {
// return false, true, targetId
// }
// executedCount++
// break
// }
// }
// }
// return executedCount == len(nodeIds), false, ""
// }
//
// // 初次检查
// allExecuted, hasFailed, failedNode := checkAllExecuted()
// if hasFailed {
// return nil, fmt.Errorf("输入节点[%s]执行失败", failedNode)
// }
//
// // 如果不是全部都已执行,阻塞等待直到全部完成、上下文取消或有节点失败
// if !allExecuted {
// // 轮询检查,每500ms检查一次,依赖ctx超时控制
// ticker := time.NewTicker(500 * time.Millisecond)
// defer ticker.Stop()
//
// for {
// select {
// case <-ctx.Done():
// // 如果上下文已经取消,说明已有节点报错,直接退出
// return nil, ctx.Err()
// case <-ticker.C:
// // 重新检查所有节点
// allExecuted, hasFailed, failedNode := checkAllExecuted()
// if hasFailed {
// // 有一个输入节点失败,直接退出
// return nil, fmt.Errorf("输入节点[%s]执行失败", failedNode)
// }
// if allExecuted {
// // 全部执行完成,退出循环继续执行
// goto allDone
// }
//
// // 再次检查上下文是否已经取消,如果已经取消则立即退出
// select {
// case <-ctx.Done():
// return nil, ctx.Err()
// default:
// }
// }
// }
// }
//allDone:
//
// // 最终检查:所有输入节点都成功了吗
// _, hasFailed, failedNode = checkAllExecuted()
// if hasFailed {
// // 有一个输入节点失败,直接退出
// return nil, fmt.Errorf("输入节点[%s]执行失败", failedNode)
// }
//
// // 构建已执行节点ID的map,方便合并时查找
// executedMap := make(map[string]*flowDto.ExecutedNode, len(nodeInput.Global.ExecutedNodes))
// for _, en := range nodeInput.Global.ExecutedNodes {
// executedMap[en.NodeId] = &en
// }
//
// // 合并所有输入源节点的输出结果
// for _, inputSource := range nodeInput.Config.InputSource {
// // 每次循环都检查上下文是否已取消,提前退出
// select {
// case <-ctx.Done():
// return nil, ctx.Err()
// default:
// }
// // 再次检查该节点是否失败
// if en, ok := executedMap[inputSource.NodeId]; ok && en.Status == node.NodeExecutionStatusFailed.Code() {
// return nil, fmt.Errorf("输入节点[%s]执行失败", inputSource.NodeId)
// }
// sourceNodeConfig := nodeInput.Global.ConfigMap[inputSource.NodeId]
// if sourceNodeConfig != nil && len(sourceNodeConfig.OutputResult) > 0 {
// nodeInput.Config.OutputResult = append(nodeInput.Config.OutputResult, sourceNodeConfig.OutputResult...)
// }
// }
return nodeInput, nil
}
func MergeLambda(ctx context.Context, input any) (res any, err error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("汇总节点入参类型错误")
}
// 1. 把所有节点输出拍平成 字段名->内容 的map
dataMap := make(map[string]node.NodeFormField)
_, outputMap, _ := GetNodeContextContent(nodeInput.Global, nodeInput.Config)
for _, field := range outputMap {
dataMap[field.Field] = field
}
// 2. 提取所有文案:text_content_0,1,2...
var contents []node.NodeFormField
for i := 0; ; i++ {
key := fmt.Sprintf("text_content:%d", i)
val, has := dataMap[key]
if !has || val.Value == "" {
break
}
contents = append(contents, val)
}
// 3. 提取所有图片:image_0,1,2...
var images []string
for i := 0; ; i++ {
key := fmt.Sprintf("img_url:%d", i)
val, has := dataMap[key]
if !has || val.Value == "" {
break
}
images = append(images, gconv.String(val.Value))
}
// 4. 🔥 核心算法:图片按顺序连续归属给每条文案
textImgMap := make(map[int][]string) // key:文案下标,value:图片列表
if len(contents) > 0 && len(images) > 0 {
imgIndex := 0 // 当前用到第几张图片
totalImg := len(images)
for i, item := range contents {
// 图片已分配完,直接退出
if imgIndex >= totalImg {
break
}
// 当前文案需要挂载的图片数量
needCount := gconv.Int(item.Expand)
if needCount <= 0 {
continue
}
var imgList []string
for imgc := 0; imgc < needCount; imgc++ {
// 关键:必须判断是否越界
if imgIndex >= totalImg {
break
}
imgList = append(imgList, images[imgIndex])
imgIndex++
}
// 有图片才存入 map
if len(imgList) > 0 {
textImgMap[i] = imgList
}
}
}
type Item struct {
Content string // 文案(可为空)
Images []string // 图片(可空、可多张)
}
// 🔥 把现有数据转换成通用 Item 列表(支持:纯文案、纯图片、图文任意组合)
var allItems []Item
url, err := utils.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
// 情况1:有文案 → 按文案条目生成 Item(每条文案+对应图片)
if len(contents) > 0 {
for i, val := range contents {
item := Item{
Content: url + gconv.String(val.Value), // 文案
Images: textImgMap[i], // 自动绑定该条目的图片(没有则为空切片)
}
allItems = append(allItems, item)
}
} else {
// 情况2:没有文案,只有图片 → 每张/每组图片生成独立 Item(纯图片条目)
if len(images) > 0 {
for _, img := range images {
allItems = append(allItems, Item{
Content: "",
Images: []string{img},
})
}
}
}
// 5. 生成多条独立HTML记录(通用方案:任意图文组合,每条独立生成+独立上传)
var outputRecords []node.NodeFormField
// 遍历所有【独立图文条目】 → 每条生成独立HTML、独立上传OSS、独立输出记录
for idx, item := range allItems {
// 生成单条HTML
htmlContent := BuildHtml(item.Content, item.Images)
outputRecords = append(outputRecords,
node.NodeFormField{
Field: fmt.Sprintf("item_html_%d", idx),
Value: htmlContent,
Label: fmt.Sprintf("条目%d HTML", idx+1),
Type: "textarea",
},
)
if nodeInput.Config.IsSaveFile {
// 上传OSS(每条独立上传)
fileName := fmt.Sprintf("item_%d_%d.html", idx, time.Now().UnixMilli())
ossResult, err := Upload(ctx, &dto.UploadFileBytesReq{
FileBytes: []byte(htmlContent),
FileName: fileName,
})
if err != nil {
return nil, err
}
outputRecords = append(outputRecords,
node.NodeFormField{
Field: fmt.Sprintf("item_html_url_%d", idx),
Value: ossResult.FileURL,
Label: fmt.Sprintf("条目%d 地址", idx+1),
Type: "text",
},
)
}
}
// 最终输出多条记录
nodeInput.Config.OutputResult = outputRecords
return nodeInput, nil
}
func SummaryLambda(ctx context.Context, input any) (any, error) {
execInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("汇总节点入参类型错误,实际是 %T", input)
}
// 聚合所有已执行节点的输出结果
var summaryResult []map[string]interface{}
for _, executedNode := range execInput.Global.ExecutedNodes {
nodeID := executedNode.NodeId
nodeConfig := execInput.Global.ConfigMap[nodeID]
if nodeConfig != nil && len(nodeConfig.OutputResult) > 0 {
for _, field := range nodeConfig.OutputResult {
if strings.Contains(field.Field, "http_file_url") || strings.Contains(field.Field, "audio_oss_url") || strings.Contains(field.Field, "video_oss_url") || strings.Contains(field.Field, "item_html_url") || strings.Contains(field.Field, "img_oss_url") || strings.Contains(field.Field, "text_url") {
// 生成 毫秒时间戳 作为 KEY
timeKey := strconv.FormatInt(time.Now().UnixMilli(), 10)
item := make(map[string]interface{})
item[timeKey] = field.Value
summaryResult = append(summaryResult, item)
}
}
}
}
// 把汇总结果存入当前节点的输出
g.Log().Info(ctx, fmt.Sprintf("结果汇总完成,汇总数据:%+v", summaryResult))
err := gfdb.DB(ctx, public.DbNameBlackDeacon).Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
flowInfo, err := flowDao.FlowExecutionDao.Get(ctx, &flowDto.GetFlowExecutionReq{
SessionId: execInput.Global.SessionId,
})
if err != nil {
return err
}
executionReq := flowDto.UpdateFlowExecutionReq{
Id: execInput.Global.ExecutionId,
Status: flow.FlowExecutionStatusSuccess.Code(),
OutputParams: summaryResult,
}
_, err = flowDao.FlowExecutionDao.Update(ctx, &executionReq)
if flowInfo != nil {
var url string
url, err = utils.GetFileAddressPrefix(ctx)
if err != nil {
return err
}
createFileTempReq := make([]*fileDto.CreateFileTempReq, 0, len(flowInfo.OutputParams))
for _, fileUrl := range flowInfo.OutputParams {
m := gconv.Map(fileUrl)
for _, v := range m {
var createReq = new(fileDto.CreateFileTempReq)
createReq.BusinessId = flowInfo.SessionId
createReq.FileUrl = url + gconv.String(v)
createFileTempReq = append(createFileTempReq, createReq)
}
}
if len(createFileTempReq) > 0 {
_, err = fileDao.FileTempDao.BatchInsert(ctx, createFileTempReq)
if err != nil {
return err
}
}
}
return nil
})
return execInput, err
}
// CustomLambda 构建自定义
func CustomLambda(ctx context.Context, input any) (any, error) {
fmt.Println("CustomLambda:", input)
return input, nil
}
File diff suppressed because it is too large Load Diff
-872
View File
@@ -1,872 +0,0 @@
package flow
import (
"ai-agent/workflow/consts/node"
nodeDao "ai-agent/workflow/dao/node"
"ai-agent/workflow/model/dto"
flowDto "ai-agent/workflow/model/dto/flow"
nodeDto "ai-agent/workflow/model/dto/node"
"ai-agent/workflow/model/entity"
"bytes"
"context"
"fmt"
"io"
"mime/multipart"
"net/http"
"net/url"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/errors/gerror"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
"github.com/tidwall/sjson"
)
// 全局等待任务回调的工具
var (
asyncMu sync.Mutex
asyncTasks = make(map[string]chan any)
)
// Wait 阻塞等待回调结果
// 调用后会一直卡住,直到 Notify 唤醒 或 超时/取消
func Wait(ctx context.Context, taskId string) (any, error) {
asyncMu.Lock()
ch := make(chan any, 1)
asyncTasks[taskId] = ch
asyncMu.Unlock()
defer close(ch)
for {
select {
case result := <-ch:
return result, nil
case <-ctx.Done():
asyncMu.Lock()
delete(asyncTasks, taskId)
asyncMu.Unlock()
return nil, ctx.Err()
}
}
}
// Notify 回调时调用,唤醒等待的任务
func Notify(taskId string, result any) {
asyncMu.Lock()
defer asyncMu.Unlock()
ch, exist := asyncTasks[taskId]
if !exist {
return
}
ch <- result
delete(asyncTasks, taskId)
}
func GetIsChatModel(ctx context.Context) (res *flowDto.GetIsChatModelRes, err error) {
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
res = new(flowDto.GetIsChatModelRes)
err = commonHttp.Get(ctx, "model-gateway/model/getIsChatModel", headers, res, nil)
return
}
func GetModelInfo(ctx context.Context, req *flowDto.GetModelInfoReq) (res *flowDto.GetModelInfoRes, err error) {
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
res = new(flowDto.GetModelInfoRes)
err = commonHttp.Get(ctx, "model-gateway/model/getModel", headers, res, req)
return
}
func GetComposeResult(ctx context.Context, buildType int, modelName, promptContent, skillName string, form []map[string]any, userForm []map[string]any, fileUrl []string, sessionId, nodeId string, cause string) (res *flowDto.ComposeCallbackReq, err error) {
if !g.IsEmpty(promptContent) {
userForm = append(userForm, map[string]any{
"prompt": promptContent,
})
}
var callbackUrl = utils.GetCallbackURL(ctx, "/flow/execution/composeCallBack")
var consult = make([]flowDto.Consult, 0)
var collectFileUrls func(val any) (fullyConsumed bool)
collectFileUrls = func(val any) (fullyConsumed bool) {
switch {
case g.NewVar(val).IsSlice():
slice := gconv.SliceAny(val)
allConsumed := false
for _, item := range slice {
if collectFileUrls(item) {
allConsumed = true
}
}
return allConsumed
case g.NewVar(val).IsMap():
m := gconv.Map(val)
allConsumed := false
for _, item := range m {
if collectFileUrls(item) {
allConsumed = true
}
}
return allConsumed
default:
s := gconv.String(val)
if s != "" {
getFileTypeByPath := GetFileTypeByPath(s)
if getFileTypeByPath != "" {
consult = append(consult, flowDto.Consult{
Type: getFileTypeByPath,
Url: s,
})
return true
}
}
return false
}
}
var newUserForm []map[string]any
for _, m := range userForm {
for k, v := range m {
if collectFileUrls(v) {
delete(m, k)
}
}
if len(m) > 0 {
newUserForm = append(newUserForm, m)
}
}
for _, v := range fileUrl {
getFileTypeByPath := GetFileTypeByPath(gconv.String(v))
if getFileTypeByPath != "" {
consult = append(consult, flowDto.Consult{
Type: getFileTypeByPath,
Url: gconv.String(v),
})
}
}
msgReq := flowDto.ComposeMessagesReq{
BuildType: buildType,
ModelName: modelName,
SkillName: skillName,
CallbackUrl: callbackUrl,
Cause: cause,
Form: form,
UserForm: newUserForm,
Consult: consult,
SessionId: sessionId,
NodeId: nodeId,
}
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
msgRes := new(flowDto.ComposeMessagesRes)
err = commonHttp.Post(ctx, "prompts-core/prompt/composeMessages", headers, msgRes, &msgReq)
if err != nil {
return
}
if g.IsEmpty(msgRes.TaskId) {
return nil, fmt.Errorf("msg is empty")
}
waitRes, err := Wait(ctx, msgRes.TaskId)
if err != nil {
return nil, err
}
msg := new(flowDto.ComposeCallbackReq)
if err = gconv.Struct(waitRes, msg); err != nil {
return nil, err
}
if !g.IsEmpty(msg.ErrorMsg) {
return nil, fmt.Errorf(msg.ErrorMsg)
}
return msg, nil
}
func CreateGatewayTask(ctx context.Context, epicycleId int64, model string, content map[string]any) (map[string]any, error) {
taskId, err := createGatewayTaskOnly(ctx, epicycleId, model, content)
if err != nil {
return nil, err
}
return waitGatewayResult(ctx, taskId)
}
// createGatewayTaskOnly creates a gateway task and returns the taskId only
// doesn't wait for completion
func createGatewayTaskOnly(ctx context.Context, epicycleId int64, model string, content map[string]any) (string, error) {
callbackUrl := utils.GetCallbackURL(ctx, "/flow/execution/modelCallback")
req := flowDto.ModelGatewayReq{
ModelName: model,
BizName: g.Cfg().MustGet(ctx, "server.name").String(),
CallbackUrl: callbackUrl,
RequestPayload: content,
EpicycleId: epicycleId,
}
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
res := new(flowDto.ModelGatewayRes)
err := commonHttp.Post(ctx, "model-gateway/task/createTask", headers, res, &req)
if err != nil {
return "", err
}
if g.IsEmpty(res.TaskId) {
return "", fmt.Errorf("创建模型任务失败,taskId为空")
}
return res.TaskId, nil
}
// waitGatewayResult waits for a created gateway task to complete and returns the result
func waitGatewayResult(ctx context.Context, taskId string) (map[string]any, error) {
waitRes, err := Wait(ctx, taskId)
if err != nil {
return nil, err
}
task := new(flowDto.ModelCallbackReq)
if err = gconv.Struct(waitRes, task); err != nil {
return nil, err
}
if task.State == 3 || !g.IsEmpty(task.ErrorMsg) {
return nil, fmt.Errorf("模型执行失败:%s", task.ErrorMsg)
}
if g.IsEmpty(task.OssFile) {
return nil, fmt.Errorf("模型返回结果为空")
}
// 获取远程文件内容
file, err := GetFileBytesFromURL(ctx, task.OssFile)
if err != nil {
return nil, err
}
return gconv.Map(file), nil
}
// updateTokenCount updates the token count in node execution
func updateTokenCount(ctx context.Context, nodeExecutionId int64, responseField string, result map[string]any) {
if responseField == "" {
return
}
_, _ = nodeDao.NodeExecutionDao.Update(ctx, &nodeDto.UpdateNodeExecutionReq{
Id: nodeExecutionId,
CompletionTokens: gconv.Int(result[responseField]),
TotalTokens: gconv.Int(result[responseField]),
})
}
func GetModelResult(ctx context.Context, sessionId string, nodeInput *flowDto.NodeExecutionInput, skillName string, form []map[string]any, userForm []map[string]any) (mapTaskResult []map[string]any, err error) {
buildType := 1
if nodeInput.Config.NodeCode == node.NodeTypeDataConversionModel {
buildType = 3
}
if !nodeInput.Global.IsDialogue {
sessionId = ""
}
composeResult, err := GetComposeResult(ctx, buildType, nodeInput.Config.ModelConfig.ModelName, nodeInput.Config.PromptContent, skillName, form, userForm, nodeInput.Global.FileUrl, sessionId, nodeInput.Config.Id, nodeInput.Config.Name)
if err != nil {
return nil, err
}
if composeResult.Status != "success" {
return nil, fmt.Errorf("模型提示词构建错误")
}
modelInfo, err := GetModelInfo(ctx, &flowDto.GetModelInfoReq{ModelName: nodeInput.Config.ModelConfig.ModelName})
if err != nil {
return nil, err
}
mapTaskResult = make([]map[string]any, len(composeResult.Messages.Rounds))
var taskResultMap map[string]any
needSequential := false
if buildType == 1 {
if needSequential {
for idx, item := range composeResult.Messages.Rounds {
if !g.IsEmpty(taskResultMap) {
var set string
set, err = sjson.Set(gconv.String(item), modelInfo.Model.LastFrame, gconv.String(taskResultMap[modelInfo.Model.ResponseBody]))
if err != nil {
return nil, err
}
item = gconv.Map(set)
}
var taskResult map[string]any
taskResult, err = CreateGatewayTask(ctx, composeResult.EpicycleId, nodeInput.Config.ModelConfig.ModelName, item)
if err != nil {
return nil, err
}
if g.IsEmpty(taskResult) {
return nil, fmt.Errorf("模型返回结果为空")
}
if nodeInput.Config.NodeCode == node.NodeTypeVideoModel {
ext := GetFileTypeByPath(gconv.String(taskResult[modelInfo.Model.ResponseBody]))
if ext == "image" {
taskResultMap = taskResult
} else {
taskResultMap = make(map[string]any)
}
} else {
taskResultMap = make(map[string]any)
}
mapTaskResult[idx] = taskResult
updateTokenCount(ctx, nodeInput.NodeExecutionId, modelInfo.Model.ResponseTokenField, taskResult)
}
} else {
taskIdList := make([]string, len(composeResult.Messages.Rounds))
for idx, item := range composeResult.Messages.Rounds {
taskId, err := createGatewayTaskOnly(ctx, composeResult.EpicycleId, nodeInput.Config.ModelConfig.ModelName, item)
if err != nil {
return nil, err
}
taskIdList[idx] = taskId
}
// 全局共享子上下文,实现一处报错全部终止
subCtx, globalCancel := context.WithCancel(ctx)
defer globalCancel() // 函数退出兜底释放
var wg sync.WaitGroup
errChan := make(chan error, len(taskIdList))
// 加互斥锁保护结果map
var mu sync.Mutex
for idx, taskId := range taskIdList {
wg.Add(1)
go func(idx int, taskId string) {
defer wg.Done()
taskResult, err := waitGatewayResult(subCtx, taskId)
if err != nil {
errChan <- err
globalCancel() // 全局取消,所有协程收到ctx取消信号快速退出
return
}
// 加锁写入map,解决并发竞态
mu.Lock()
mapTaskResult[idx] = taskResult
mu.Unlock()
updateTokenCount(ctx, nodeInput.NodeExecutionId, modelInfo.Model.ResponseTokenField, taskResult)
}(idx, taskId)
}
wg.Wait()
close(errChan)
// 收集全部错误,而非只读一条
var errs []error
for len(errChan) > 0 {
errs = append(errs, <-errChan)
}
if len(errs) > 0 {
// 返回第一个错误;如需汇总所有错误可拼接
return nil, errs[0]
}
}
} else {
for idx, item := range composeResult.Messages.Rounds {
mapTaskResult[idx] = item
updateTokenCount(ctx, nodeInput.NodeExecutionId, modelInfo.Model.ResponseTokenField, item)
}
}
return mapTaskResult, nil
}
func BuildNestedJson(body g.Map, mockConfigMap map[string]*entity.FlowNode) g.Map {
jsonStr := "{}"
for originKey, originItem := range body {
bodyItemMap := gconv.Map(originItem)
val := bodyItemMap["value"]
if v, ok := bodyItemMap["value"]; ok {
jsonStr, _ = sjson.Set(jsonStr, originKey, v)
}
// 判断 value 是不是引用结构(map)
if g.NewVar(val).IsMap() {
valMap := gconv.Map(val)
nodeId := gconv.String(valMap["nodeId"])
fieldName := gconv.String(valMap["field"])
if configValue, ok := mockConfigMap[nodeId]; ok {
if !g.IsEmpty(configValue.OutputResult) {
for _, v := range configValue.OutputResult {
if strings.Contains(v.Field, fieldName) {
if configValue.NodeCode == node.NodeTypeDataConversionModel {
switch {
case g.NewVar(v.Value).IsSlice() || g.NewVar(v.Value).IsMap():
// 核心:自动判断两种结构,精准赋值
vm := gconv.Map(v.Value)
// 先判断是否是 单个key包裹的对象(如 {"subtitle_style": {...}}
if len(vm) == 1 {
// 遍历取出唯一的 key 和 真实值
for innerKey, innerVal := range vm {
// 直接用 innerKeysubtitle_style)赋值
jsonStr, _ = sjson.Set(jsonStr, innerKey, innerVal)
}
} else {
// 直接是对象,用 originKey 赋值
jsonStr, _ = sjson.Set(jsonStr, originKey, v.Value)
}
default:
jsonStr, _ = sjson.Set(jsonStr, originKey, v.Value)
}
} else {
jsonStr, _ = sjson.Set(jsonStr, originKey, v.Value)
}
}
}
}
if !g.IsEmpty(configValue.FormConfig) {
for _, v := range configValue.FormConfig {
if v.Field == fieldName {
if v.Type == "uploadMultiple" {
if g.NewVar(v.FieldConstraint).IsMap() {
mapFieldConstraint := gconv.Map(v.FieldConstraint)
for key, value := range mapFieldConstraint {
if key == "maxFileCount" {
if gconv.Int(value) == 1 {
// 如果是单文件上传,则替换成字符串重新赋值给v.Value
if g.NewVar(v.Value).IsSlice() {
sliceVal := gconv.SliceAny(v.Value)
if len(sliceVal) > 0 {
v.Value = sliceVal[0]
}
}
}
}
}
}
}
jsonStr, _ = sjson.Set(jsonStr, originKey, v.Value)
}
}
}
}
}
}
return gconv.Map(jsonStr)
}
func VideoConcat(ctx context.Context, videoUrls []string) (r any, err error) {
var httpUrl = "media/video/concat/async"
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
var callbackUrl = utils.GetCallbackURL(ctx, "/flow/execution/videoCallback")
var newBody = flowDto.VideoConcatReq{
VideoUrls: videoUrls,
Method: "auto",
Upload: true,
CallbackUrl: callbackUrl,
}
res := new(flowDto.VideoConcatRes)
err = commonHttp.Post(ctx, httpUrl, headers, &res, newBody)
if err != nil {
return nil, err
}
return Wait(ctx, res.TaskId)
}
func GetFileBytesFromURL(ctx context.Context, fileUrl string) ([]byte, error) {
newS := strings.ReplaceAll(fileUrl, g.Cfg().MustGet(ctx, "filePrefix").String(), g.Cfg().MustGet(ctx, "minioPrefix").String())
// 使用 GoFrame 客户端(自带超时、追踪、日志等能力)
resp, err := g.Client().Get(ctx, newS)
if err != nil {
return nil, gerror.Wrapf(err, "failed to request url: %s", newS)
}
defer resp.Close()
// 校验状态码
if resp.StatusCode != http.StatusOK {
return nil, gerror.Newf("request failed with status code: %d, url: %s", resp.StatusCode, newS)
}
// 读取全部内容
allBytes, err := io.ReadAll(resp.Body)
if err != nil {
return nil, gerror.Wrapf(err, "failed to read response body, url: %s", fileUrl)
}
return allBytes, nil
}
func Upload(ctx context.Context, req *dto.UploadFileBytesReq) (*dto.UploadFileBytesRes, error) {
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("file", req.FileName)
if err != nil {
return nil, err
}
if _, err = part.Write(req.FileBytes); err != nil {
return nil, err
}
if err = writer.Close(); err != nil {
return nil, err
}
headers := make(map[string]string)
headers["Content-Type"] = writer.FormDataContentType()
if r := g.RequestFromCtx(ctx); r != nil {
if auth := r.Header.Get("Authorization"); auth != "" {
headers["Authorization"] = auth
}
}
// 发起上传请求
res := &dto.UploadFileBytesRes{}
httpUrl := "oss/file/uploadFile"
if err = commonHttp.Post(ctx, httpUrl, headers, res, body.Bytes()); err != nil {
return nil, err
}
g.Log().Infof(ctx, "[Upload] success url=%s size=%d", res.FileURL, res.FileSize)
return res, nil
}
func GetFileTypeByPath(filePath string) string {
if filePath == "" {
return ""
}
// 解析 URL,获取真实路径(兼容 http 链接)
u, err := url.Parse(filePath)
if err == nil {
filePath = u.Path
}
// 获取后缀(小写)
ext := filepath.Ext(filePath)
ext = strings.ToLower(ext)
// 判断类型
switch ext {
case ".jpg", ".jpeg", ".png", ".gif", ".bmp", ".webp":
return "image"
case ".mp4", ".mov", ".avi", ".flv", ".wmv", ".mkv":
return "video"
case ".mp3", ".wav", ".m4a", ".flac", ".aac", ".ogg":
return "audio"
case ".txt", ".md", ".log", ".json", ".xml", ".inc":
return "text"
case ".html":
return "html"
case ".pdf", ".doc", ".docx", ".xls", ".xlsx", ".ppt", ".pptx":
return "document"
default:
return ""
}
}
func BuildText(text string) string {
// 生成单条HTML
var htmlBuilder strings.Builder
htmlBuilder.WriteString(`
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: "Microsoft YaHei", "PingFang SC", Arial, sans-serif;
background: #f5f5f5;
color: #333;
line-height: 1.8;
padding: 20px;
}
.container {
max-width: 900px;
margin: 0 auto;
background: #fff;
border-radius: 12px;
box-shadow: 0 2px 12px rgba(0, 0, 0, 0.08);
overflow: hidden;
}
.item {
padding: 30px;
}
.image-group img {
width: 100%;
height: auto;
display: block;
margin-bottom: 6px;
border-radius: 8px;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.1);
}
.image-group img:last-child {
margin-bottom: 0;
}
.image-group {
margin-bottom: 25px;
}
.text {
padding: 0;
font-size: 15px;
line-height: 1.4;
color: #555;
}
.text h2 {
font-size: 28px;
font-weight: bold;
color: #1a1a1a;
margin-bottom: 15px;
line-height: 1.2;
}
.text h3 {
font-size: 20px;
font-weight: 600;
color: #2c3e50;
margin: 20px 0 12px;
padding-left: 12px;
border-left: 4px solid #409eff;
}
.text p {
margin-bottom: 12px;
text-align: justify;
}
.text strong {
color: #e74c3c;
font-weight: 600;
}
.text ul {
list-style: none;
padding: 0;
margin: 8px 0;
}
.text ul li {
padding: 10px 0 10px 30px;
position: relative;
line-height: 1.2;
}
.text ul li:before {
content: "●";
color: #409eff;
font-size: 12px;
position: absolute;
left: 12px;
top: 12px;
}
@media (max-width: 768px) {
body {
padding: 10px;
}
.text h2 {
font-size: 24px;
}
.text h3 {
font-size: 18px;
}
}
</style>
</head>
<body>
<div class="container">
<div class="item">
`)
// 🔥 写入文案前:删除 <p class="image-count">需要配图:X 张</p>
if text != "" {
// 写入清理后的文案
htmlBuilder.WriteString(fmt.Sprintf(`<div class="text">%s</div>`, ImageTagRegex(text)))
}
htmlBuilder.WriteString(`</div>
</div>
</body>
</html>`)
return htmlBuilder.String()
}
func BuildHtml(text string, images []string) string {
var htmlBuilder strings.Builder
htmlBuilder.WriteString(`<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: "Microsoft YaHei", sans-serif;
padding: 20px;
background-color: #f6f6f6;
line-height: 1.7;
font-size: 16px;
color: #333;
}
.container {
max-width: 750px;
margin: 0 auto;
background: #fff;
padding: 30px;
border-radius: 12px;
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.06);
}
</style>
</head>
<body>
<div class="container">
`)
// 写入图片(支持0张、1张、多张)
if len(images) > 0 {
htmlBuilder.WriteString(`<div class="image-group">`)
for _, imgUrl := range images {
htmlBuilder.WriteString(fmt.Sprintf(`<img src="%s" alt="图片"/>`, imgUrl))
}
htmlBuilder.WriteString(`</div>`)
}
htmlBuilder.WriteString(`
<div id="content">加载中...</div>
</div>
<script>
const incUrl = "` + text + `";
fetch(incUrl)
.then(res => {
if (!res.ok) throw new Error("加载失败");
return res.text();
})
.then(text => {
document.getElementById("content").innerHTML = text;
})
.catch(err => {
document.getElementById("content").innerHTML = "加载失败:" + err.message;
});
</script>
</body>
</html>`)
return htmlBuilder.String()
}
// ExtractImageCount 修复:支持单引号/双引号 + 换行 + 空格
func ExtractImageCount(content string) int {
// 🔥 关键:支持 class='image-count' (单引号)
re := regexp.MustCompile(`<p class=['"]image-count['"][^>]*>.*?(\d+).*?</p>`)
match := re.FindStringSubmatch(content)
if len(match) >= 2 {
num, err := strconv.Atoi(match[1])
if err == nil {
return num
}
}
return 0
}
func ImageTagRegex(html string) string {
// 🔥 修复:支持单引号、双引号、空格、换行,100% 删除 <p class='image-count'>
imageTagRegex := regexp.MustCompile(`<p class=['"]image-count['"][^>]*>[\s\S]*?</p>`)
return imageTagRegex.ReplaceAllString(html, "")
}
// StripHtmlTags 去掉所有HTML标签,保留换行和文本结构,并删除配图标记行
func StripHtmlTags(html string) string {
// 1. 替换块级标签为换行,保证排版
blockTags := regexp.MustCompile(`</?(div|p|h1|h2|h3|h4|h5|h6|li|ul|ol|br|tr|td|th)[^>]*>`)
text := blockTags.ReplaceAllString(html, "\n")
// 2. 去掉所有剩余的 HTML 标签
allTags := regexp.MustCompile(`<[^>]+>`)
text = allTags.ReplaceAllString(text, "")
// 4. 清理多余空行(多个换行只保留一个)
text = regexp.MustCompile(`\n\s*\n`).ReplaceAllString(text, "\n")
// 5. 只去掉首尾空白,中间换行保留
text = strings.TrimSpace(text)
return text
}
// SplitMultiContents 拆分模型返回的多条文案(基于HTML标签分隔)
func SplitMultiContents(htmlContent string) []string {
var contents []string
// 正则匹配<div class="content-item" id="content-{序号}">包裹的内容
re := regexp.MustCompile(`<div class="content-item" id="content-\d+">([\s\S]*?)</div>`)
matches := re.FindAllStringSubmatch(htmlContent, -1)
for _, match := range matches {
if len(match) > 1 {
// 清理空内容
trimmed := strings.TrimSpace(match[1])
if trimmed != "" {
contents = append(contents, trimmed)
}
}
}
// 兜底:如果没有匹配到结构化内容,按换行/分隔符拆分
if len(contents) == 0 {
contents = strings.Split(htmlContent, "===分隔符===") // 提示词中可新增此兜底规则
}
return contents
}
// GetAllImgSrcFromHtml 先把提取img src的工具方法放在外面
func GetAllImgSrcFromHtml(html string) []string {
var imgSrcList []string
re := regexp.MustCompile(`<img[^>]*src\s*=\s*["']([^"']+)["']`)
submatch := re.FindAllStringSubmatch(html, -1)
for _, match := range submatch {
if len(match) >= 2 {
imgSrcList = append(imgSrcList, match[1])
}
}
return imgSrcList
}
// ReplaceImgSrc 替换img src的方法
func ReplaceImgSrc(html string, oldSrc string, newSrc string) string {
// 精准替换:找到 <img xxx src="oldSrc" xxx>
re := regexp.MustCompile(`(<img[^>]*src\s*=\s*["'])` + regexp.QuoteMeta(oldSrc) + `(["'])`)
return re.ReplaceAllString(html, `${1}`+newSrc+`${2}`)
}
+113
View File
@@ -0,0 +1,113 @@
package flow
import (
"ai-agent/gateway"
"context"
"encoding/base64"
"fmt"
"strings"
"gitea.redpowerfuture.com/red-future/common/oss"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
)
// resolveSaveFileResult 解析结果值为可入库的 URL:
// - 已是 http(s) URL 或 MinIO 对象裸路径 → 直接返回
// - 非路径(base64 图片/文本)→ 上传 OSS 换取 URL
func resolveSaveFileResult(ctx context.Context, val any) (string, error) {
isPath, path, fileBytes, ext := resolveFileContent(val)
if isPath {
return path, nil
}
if ext == "" {
ext = ".png"
}
fileUrl, err := gateway.Upload(ctx, fmt.Sprintf("workflow_result_%s%s", uuid.NewString(), ext), fileBytes)
if err != nil {
return "", err
}
return fileUrl, nil
}
// resolveFileContent 判断结果值形态:
// - 已是 URL 路径(http/https 开头)→ 直接使用
// - data URIdata:<mime>;base64,<data>)→ 解码为字节,扩展名按 mime 推断
// - 纯 base64(可解码且长度足以认为是编码数据)→ 解码为字节,默认 .png
// - 其余(文本)→ 以 .inc 扩展名上传原文
func resolveFileContent(val any) (isPath bool, path string, fileBytes []byte, ext string) {
s := gconv.String(val)
if isFileURL(s) {
return true, s, nil, ""
}
// MinIO 对象裸路径(无 http 前缀,模型网关转存 OSS 后返回)
if oss.IsOSSPath(s) {
return true, s, nil, ""
}
// data URIdata:<mime>;base64,<payload>
if b, mime, ok := parseDataURI(s); ok {
return false, "", b, extOfMime(mime)
}
// 纯 base64:可解码且长度足够,视为编码后的文件内容
trimmed := strings.TrimSpace(s)
if len(trimmed) >= 64 {
if b, err := base64.StdEncoding.DecodeString(trimmed); err == nil && len(b) > 0 {
return false, "", b, ".png"
}
}
// 文本:以 .inc 存储
return false, "", []byte(s), ".inc"
}
// isFileURL 判断字符串是否已是对外可访问的 URL 路径(http/https 开头)
func isFileURL(s string) bool {
lower := strings.ToLower(s)
return strings.HasPrefix(lower, "http://") || strings.HasPrefix(lower, "https://")
}
// extOfMime 按 MIME 类型推断文件扩展名
func extOfMime(mime string) string {
switch strings.ToLower(strings.TrimSpace(mime)) {
case "image/png", "png":
return ".png"
case "image/jpeg", "image/jpg", "jpeg", "jpg":
return ".jpg"
case "image/webp":
return ".webp"
case "image/gif":
return ".gif"
case "audio/mpeg", "audio/mp3", "mp3":
return ".mp3"
case "audio/wav", "wav":
return ".wav"
case "video/mp4", "mp4":
return ".mp4"
case "application/json", "json":
return ".json"
default:
return ""
}
}
// parseDataURI 解析 data URIdata:<mime>;base64,<payload>,返回解码字节与 mime
func parseDataURI(s string) ([]byte, string, bool) {
const prefix = "data:"
if !strings.HasPrefix(s, prefix) {
return nil, "", false
}
rest := s[len(prefix):]
comma := strings.Index(rest, ",")
if comma < 0 {
return nil, "", false
}
mime := rest[:comma]
if semicolon := strings.Index(mime, ";"); semicolon >= 0 {
mime = mime[:semicolon]
}
payload := strings.TrimPrefix(rest[comma+1:], "base64,")
b, err := base64.StdEncoding.DecodeString(payload)
if err != nil {
return nil, "", false
}
return b, mime, true
}
@@ -0,0 +1,399 @@
package flow
import (
"ai-agent/workflow/consts/node"
"ai-agent/workflow/service/flow/processor"
"ai-agent/workflow/service/flow/processor/builtin/split_shots_pipeline"
"ai-agent/workflow/service/flow/values"
"context"
"encoding/json"
"fmt"
"strings"
"ai-agent/gateway"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/service/flow/processor/builtin/split_shots_pipeline/pipeline"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
// 脚本转写节点默认系统提示词(字段与 domain.Shot 的 JSON tag 对齐,结构化输出与 content 兜底两条路径一致)
const defaultScriptTranscribeSystemPrompt = `你是短剧分镜脚本师请根据提供的文案/视频分析结果把内容拆分为连续的分镜镜头脚本
每个镜头输出一个 JSON 对象字段固定为
- index镜头序号数字
- startTime开始时间格式 MM:SS
- endTime结束时间格式 MM:SS
- event事件描述/动作描写
- narration旁白/画外音没有则省略
- dialogue角色开口说的主台词没有则省略
- ambientSound环境音没有则省略
- cameraMovement运镜描述
- shotSize景别
- characters出演人物名列表字符串数组
- scene场景名
- props道具名列表字符串数组
时间码需前后衔接覆盖整个内容时长直接输出 JSON 数组不要输出其他文字`
// ScriptTranscribeLambda 脚本转写节点:
// 把节点输入(文案/视频分析结果,经 valueSource 解析)通过大模型转写为 []pipeline.Shot
// 再经 split_shots_pipeline 前置处理器拆成各段扁平请求参数列表([{"prompt","duration","seed",...},...]),
// 供下游视频生成节点逐段引用聚合。
func ScriptTranscribeLambda(ctx context.Context, input any) (any, error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
n := new([]node.NodePresetField)
err := gconv.Structs(nodeInput.Config.OutputConfig, n)
if err != nil {
return nil, err
}
var totalDuration int
var modelId int64
// 静音模式:清空镜头台词/旁白,生成视频不含口播、旁白、字幕与人物开口动作,默认开启
noSpeech := true
for _, item := range *n {
switch item.Field {
case "noSpeech":
if !g.IsEmpty(item.Value) {
noSpeech = gconv.Bool(item.Value)
}
case "totalDuration":
if !g.IsEmpty(item.Value) {
totalDuration = gconv.Int(item.Value)
} else {
if !g.IsEmpty(item.ValueSource) {
for _, k := range item.ValueSource {
nodeConfig := nodeInput.Global.ConfigMap[k.NodeId]
if nodeConfig != nil {
for _, output := range nodeConfig.OutputResult {
if !g.IsEmpty(output[k.Field]) {
totalDuration = gconv.Int(output[k.Field])
}
}
}
}
}
}
case "modelId":
modelId = gconv.Int64(item.Value)
}
}
modelParams, err := values.BuildModelRequestBody(nodeInput.Config.ModelConfig.ModelRequestParamsPath, nodeInput.Global)
if err != nil {
return nil, err
}
// 2. 构建系统提示词 + 用户输入
systemPrompt := nodeInput.Config.Prompt
if systemPrompt == "" {
systemPrompt = defaultScriptTranscribeSystemPrompt
}
// 参考素材名单注入:转写模型只从名单选名,保证镜头里的角色/场景/道具名与参考素材精确一致
// (名字绑定/类别推断都依赖名字对上)
refsName, refsItem := pipeline.ExtractRefs(nodeInput.Config.ModelConfig.ModelRequestParams)
if len(refsName) > 0 {
systemPrompt += "\n\n参考素材名单:" + strings.Join(refsName, "、") +
"\n约束:镜头里的 characters/scene/props 必须原样使用名单中的名字,不得改写、不得加修饰(如“主角小明”)、不得造新名;名单外的名字按原文输出。"
}
if totalDuration > 0 {
systemPrompt += fmt.Sprintf("\n\n视频总时长 %d 秒(MM:SS 为 %s):所有镜头的时间码需前后衔接并完整覆盖该总时长,最后一镜的 endTime 对齐到总时长。", totalDuration, formatSecondsToMMSS(totalDuration))
}
// 单镜头时长约束:按视频模型推导单段最大/最小时长注入转写提示词,从源头避免超长/超短镜头
//SplitOversized / GroupSegments 咬取补齐仍是机械兜底);推导失败仅降级跳过约束注入,不影响转写主流程。
if maxSeg, minSeg, err := split_shots_pipeline.SegmentBounds(ctx, modelId); err != nil {
g.Log().Warningf(ctx, "获取视频模型单段时长约束失败,跳过单镜时长约束注入: %v", err)
} else {
systemPrompt += shotDurationConstraintPrompt(maxSeg)
systemPrompt += shotMinDurationConstraintPrompt(minSeg, totalDuration)
}
// 静音模式硬约束:从转写源头杜绝对白/旁白/开口说话,后续清洗只做兜底
if noSpeech {
systemPrompt += noSpeechSystemPromptConstraint()
}
info, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: nodeInput.Config.ModelConfig.ModelId})
if err != nil {
return nil, err
}
params := map[string]any{
"system_prompt": systemPrompt,
}
// 结构化输出:chat 模型映射配置了 response_formatStructuredOutput)即走原生 json_schema 保证结构;
// 否则模型 content 直接出 JSON,靠容错解析兜底
val, ok := info.ModelManage.RequestBusinessFieldMapping["response_format"]
if ok {
if !g.IsEmpty(val) {
params["response_format"] = pipeline.ShotsStructuredFormat()
}
}
modelInfo, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: nodeInput.Config.ModelConfig.ModelId})
if err != nil {
return "", fmt.Errorf("获取模型配置失败: %w", err)
}
result, err := gateway.ModelCallResult(ctx, nodeInput.Config.ModelConfig.ModelId, modelInfo.ModelManage.ResponseType, nodeInput.Global.SessionId, modelParams, params)
if err != nil {
return nil, err
}
var content string
for _, v := range gconv.Map(result.Content) {
content += v.(string)
}
// 4. 解析镜头数组(兼容英文键结构化输出与中文键 content 兜底)
shots, err := unmarshalShots(content)
if err != nil {
return nil, err
}
// 静音模式:清空台词与旁白,保证写进视频 prompt 的只有事件/环境音/运镜/景别,
// 视频模型不会产生口播、旁白配音、字幕烧录,也不会把角色标记为开口优先做口型动画
if noSpeech {
for i := range shots {
shots[i].Dialogue = ""
shots[i].Narration = ""
// event 里的说话动词仍会经 事件:%s 块写进分段 prompt,导致视频模型生成口型/字幕,需确定性清洗
shots[i].Event = cleanSpeechVerbs(shots[i].Event)
}
}
// 5. 产出固定结构 {"shots": [...]}
var arr []any
if b, err := json.Marshal(shots); err == nil {
_ = json.Unmarshal(b, &arr)
}
args := split_shots_pipeline.SplitShotsInput{
ModelID: modelId,
Shots: shots,
TotalDuration: totalDuration,
FlatRefs: refsItem,
Seed: nodeInput.Global.ExecutionId % 1000000,
NegativePrompt: nodeInput.Config.NegativePrompt,
NoSpeech: noSpeech,
}
data, err := processor.Call(ctx, "split_shots_pipeline", gconv.Map(args))
if err != nil {
return nil, err
}
nodeInput.Config.OutputResult = gconv.Maps(data)
return nodeInput, nil
}
// unmarshalShots 解析镜头数组 JSON,容忍 markdown 代码围栏、json_schema 结构化输出的 {"shots":[...]} 包装,
// 以及模型按中文键输出(时间码/事件/台词旁白/景别/运镜/出演角色/场景/道具)的容错映射。
func unmarshalShots(s string) ([]pipeline.Shot, error) {
s = strings.TrimSpace(s)
if strings.HasPrefix(s, "```") {
s = strings.TrimPrefix(s, "```json")
s = strings.TrimPrefix(s, "```")
s = strings.TrimSuffix(s, "```")
s = strings.TrimSpace(s)
}
var raw []map[string]any
if err := json.Unmarshal([]byte(s), &raw); err == nil && len(raw) > 0 {
return parseShots(raw)
}
var wrapped struct {
Shots []map[string]any `json:"shots"`
}
if err := json.Unmarshal([]byte(s), &wrapped); err == nil && len(wrapped.Shots) > 0 {
return parseShots(wrapped.Shots)
}
return nil, fmt.Errorf("解析分镜脚本失败: %v", s)
}
// parseShots 把原始镜头对象数组归一为 domain.Shot,跳过没有内容字段的镜头。
func parseShots(raw []map[string]any) ([]pipeline.Shot, error) {
shots := make([]pipeline.Shot, 0, len(raw))
for i, m := range raw {
shot := shotFromMap(m)
if shot.Index == 0 {
shot.Index = i + 1
}
if isEmptyShot(shot) {
continue
}
shots = append(shots, shot)
}
if len(shots) == 0 {
return nil, fmt.Errorf("解析分镜脚本失败: 镜头内容为空")
}
return shots, nil
}
// isEmptyShot 镜头是否没有可用内容(仅有时间码/序号,或字段名对不上导致全空)。
func isEmptyShot(s pipeline.Shot) bool {
return s.Event == "" && s.Dialogue == "" && s.Narration == "" && s.AmbientSound == "" &&
s.CameraMovement == "" && s.ShotSize == "" && s.Scene == "" &&
len(s.Characters) == 0 && len(s.Props) == 0
}
// shotFromMap 把单个镜头对象映射为 domain.Shot,兼容英文键(结构化输出)与中文键(提示词兜底输出)。
func shotFromMap(m map[string]any) pipeline.Shot {
get := func(keys ...string) string {
for _, k := range keys {
switch v := m[k].(type) {
case string:
if t := strings.TrimSpace(v); t != "" {
return t
}
case float64:
return gconv.String(v)
}
}
return ""
}
getSlice := func(keys ...string) []string {
for _, k := range keys {
switch v := m[k].(type) {
case []any:
var out []string
for _, e := range v {
if t, ok := e.(string); ok && strings.TrimSpace(t) != "" {
out = append(out, strings.TrimSpace(t))
}
}
if len(out) > 0 {
return out
}
case string:
if out := splitList(v); len(out) > 0 {
return out
}
}
}
return nil
}
shot := pipeline.Shot{
Index: gconv.Int(m["index"]),
StartTime: get("startTime", "开始时间"),
EndTime: get("endTime", "结束时间"),
Event: get("event", "事件"),
Dialogue: get("dialogue", "台词"),
Narration: get("narration", "旁白"),
AmbientSound: get("ambientSound", "环境音"),
CameraMovement: get("cameraMovement", "运镜"),
ShotSize: get("shotSize", "景别"),
Scene: get("scene", "场景"),
Characters: getSlice("characters", "出演角色"),
Props: getSlice("props", "道具"),
}
if shot.StartTime == "" && shot.EndTime == "" {
shot.StartTime, shot.EndTime = splitTimeRange(get("时间码"))
}
if shot.Dialogue == "" && shot.Narration == "" {
shot.Dialogue, shot.Narration = splitDialogueNarration(get("台词/旁白"))
}
return shot
}
// formatSecondsToMMSS 秒转 MM:SS 时间码。
func formatSecondsToMMSS(sec int) string {
if sec < 0 {
sec = 0
}
return fmt.Sprintf("%02d:%02d", sec/60, sec%60)
}
// splitTimeRange 解析时间码 "MM:SS-MM:SS"(兼容 "—"/"~"/"到" 等分隔,或单个时间点)。
func splitTimeRange(s string) (start, end string) {
if s == "" {
return "", ""
}
normalized := strings.NewReplacer("—", "-", "", "-", "", "-", "~", "-", "到", "-", "至", "-").Replace(s)
parts := strings.Split(normalized, "-")
start = strings.TrimSpace(parts[0])
if len(parts) > 1 {
end = strings.TrimSpace(parts[1])
} else {
end = start
}
return start, end
}
// splitDialogueNarration 把"台词/旁白"合字段拆成 dialogue 与 narration
// 以"旁白"/"画外音"开头的内容归为旁白,其余视为角色开口的主台词。
func splitDialogueNarration(s string) (dialogue, narration string) {
s = strings.TrimSpace(s)
switch {
case strings.HasPrefix(s, "旁白"):
return "", strings.TrimSpace(strings.TrimLeft(strings.TrimPrefix(s, "旁白"), ":"))
case strings.HasPrefix(s, "画外音"):
return "", strings.TrimSpace(strings.TrimLeft(strings.TrimPrefix(s, "画外音"), ":"))
default:
return s, ""
}
}
// cleanSpeechVerbs 静音模式下清洗 event 中的说话动词:把常见说话/喊叫/对白表达替换为空串,
// 避免"事件:%s"块里残留的说话动词让视频模型生成口型/字幕。仅做机械兜底,硬约束在转写提示词。
// NewReplacer 按最长匹配替换,故先列含"说/喊"的非开口语义词做保护(no-op,如"说明""呐喊"),
// 再列开口表达;"叫"语义多变(呼叫/叫停/叫住),不做裸清洗以免误伤。
func cleanSpeechVerbs(s string) string {
if s == "" {
return ""
}
repl := strings.NewReplacer(
"说明", "说明", "解说", "解说", "据说", "据说", "传说", "传说", "小说", "小说",
"学说", "学说", "说法", "说法", "说服", "说服", "呐喊", "呐喊",
"开口说话", "", "开口说", "", "开口", "",
"说道:", "", "说道:", "", "说道", "",
"说着", "", "说话", "", "讲话", "", "台词", "", "对白", "",
"喊道:", "", "喊道:", "", "喊道", "", "大喊", "", "喊叫", "",
"叫道:", "", "叫道:", "", "叫道", "", "叫到", "", "叫喊", "",
"回答", "", "答道", "", "回应", "", "回话", "",
"问道:", "", "问道:", "", "问道", "",
"念叨", "", "嘟囔", "", "嘀咕", "", "自言自语", "",
"呼唤", "", "呼叫", "", "叫唤", "", "惊叫", "", "惨叫", "",
"说:", "", "说:", "", "说", "",
"喊:", "", "喊:", "", "喊", "",
)
return strings.TrimSpace(repl.Replace(s))
}
// splitList 按常见分隔符拆分人名/道具列表(兼容中英文顿号、逗号、分号、"和""及"等)。
func splitList(s string) []string {
repl := strings.NewReplacer("、", "|", "", "|", ",", "|", "", "|", ";", "|", "和", "|", "及", "|", "&", "|", "/", "|", " ", "|")
var out []string
for _, p := range strings.Split(repl.Replace(strings.TrimSpace(s)), "|") {
if t := strings.TrimSpace(p); t != "" {
out = append(out, t)
}
}
return out
}
// shotDurationConstraintPrompt 生成"单个镜头时长不超过 maxSeg 秒"的转写约束提示词片段;maxSeg<=0 返回空串。
func shotDurationConstraintPrompt(maxSeg int) string {
if maxSeg <= 0 {
return ""
}
return fmt.Sprintf("\n\n单个镜头时长不超过 %d 秒:每镜的 startTime 与 endTime 之差必须 ≤ %d 秒。", maxSeg, maxSeg)
}
// shotMinDurationConstraintPrompt 生成"单个镜头时长不少于 minSeg 秒"的转写约束提示词片段。
// minSeg<=0 返回空串;minSeg 超过视频总时长时也返回空串——此时与"末镜 endTime 对齐总时长"的约束
// 自相矛盾、模型无法满足,强行注入反而会让模型困惑(GroupSegments 对末段残余本就放行短于 min)。
func shotMinDurationConstraintPrompt(minSeg, totalDuration int) string {
if minSeg <= 0 {
return ""
}
if totalDuration > 0 && minSeg > totalDuration {
return ""
}
return fmt.Sprintf("\n\n单个镜头时长不少于 %d 秒:每镜的 startTime 与 endTime 之差必须 ≥ %d 秒。", minSeg, minSeg)
}
// noSpeechSystemPromptConstraint 静音模式的转写硬约束:要求模型从源头就不产出对白/旁白/说话动词,
// 后续 noSpeech 清洗(清空台词旁白 + cleanSpeechVerbs)只做机械兜底。
func noSpeechSystemPromptConstraint() string {
return "\n\n本片为静音模式,镜头里禁止任何声音类内容:\n" +
"- 所有镜头禁止出现台词、旁白、画外音,narration 与 dialogue 一律留空、不要输出;\n" +
"- 禁止角色开口说话,事件描述只能写无声的动作、表情、神态、场景变化,不要出现“说”“喊”“叫”“对白”“讲话”“开口”“问”“回答”“念叨”等说话类动词;\n" +
"- characters 只是出镜角色名,不代表开口说话;\n" +
"- 视频不包含口型动作与字幕,据此调整分镜描写。"
}
@@ -0,0 +1,53 @@
package flow
import (
"sort"
"ai-agent/workflow/model/entity"
)
// planSegmentResume 段级续跑决策:段身份取列表位置(0-based,paramsList 顺序即段序),
// 把各段映射到"是否需重新生成"。savedMap 为该节点已成功段(段序号 → {key,url});
// 段在表中缺失或地址为空则需重新生成。返回值与 paramsList 对齐。
// 全新执行(savedMap 为 nil/空)时全部需生成。
// 不用 params["segment_index"] 作为段身份:真实链路(上游 split_shots_pipeline 转写 →
// 下游 split_segment 按 __segment_fields 拆分,invokePreTool 剥离 __ 内部键)下
// paramsList 只有模型参数;列表位置互不重复、顺序即段序、参数一致时跨 reExecute 稳定。
func planSegmentResume(paramsList []map[string]any, savedMap map[int]entity.SegmentRef) (idxList []int, needGen []bool) {
idxList = make([]int, len(paramsList))
needGen = make([]bool, len(paramsList))
for i := range paramsList {
idxList[i] = i
ref, ok := savedMap[i]
needGen[i] = !ok || ref.URL == ""
}
return
}
// mergeSegmentOutputs 把复用段与新生段按段序号升序合并为 concat 输入列表(列表顺序即拼接顺序)。
// 复用段重建 {key:url} 记录(key 保持模型原输出字段,避免下游引用失配);新生段沿用模型原输出。
// 任一段既无复用又无生成结果(生成空)则跳过——与现有"空段贡献空"行为一致,最终由 concat 校验兜底。
func mergeSegmentOutputs(idxList []int, needGen []bool, newRes [][]map[string]any, savedMap map[int]entity.SegmentRef) []map[string]any {
type segOutput struct {
idx int
recs []map[string]any
}
out := make([]segOutput, 0, len(idxList))
for i, idx := range idxList {
if needGen[i] {
if len(newRes[i]) > 0 {
out = append(out, segOutput{idx: idx, recs: newRes[i]})
}
continue
}
if ref, ok := savedMap[idx]; ok && ref.URL != "" {
out = append(out, segOutput{idx: idx, recs: []map[string]any{{ref.Key: ref.URL}}})
}
}
sort.SliceStable(out, func(a, b int) bool { return out[a].idx < out[b].idx })
var merged []map[string]any
for _, o := range out {
merged = append(merged, o.recs...)
}
return merged
}
+225
View File
@@ -0,0 +1,225 @@
package flow
import (
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"ai-agent/workflow/service/flow/values"
"context"
"encoding/json"
"fmt"
"github.com/cloudwego/eino-examples/compose/batch/batch"
"github.com/cloudwego/eino/compose"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
func SubFlowLambda(ctx context.Context, input any) (any, error) {
// 1. 类型断言(和其他节点保持一致的入参结构)
nodeExecInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("子流程节点入参类型错误,期望*flowDto.NodeExecutionInput,实际%T", input)
}
// 2. 解析子流程配置
subFlowConfig := nodeExecInput.Config.SubConfig
if subFlowConfig == nil {
return nil, fmt.Errorf("子流程节点缺少配置")
}
getRes, err := FlowUserService.Get(ctx, &flowDto.GetFlowUserReq{
Id: subFlowConfig.WorkflowId,
})
if err != nil {
return nil, err
}
// 3. 引入参数解析:把首页表单值/上游引用值/静态默认值写入子流程开始节点 outputConfig。
// 须在 BuildGraph / ExtractFlowNodeFrom 之前执行,batchInputs 深拷贝的才是注入后的开始节点。
injectSubFlowFields(nodeExecInput.Global, getRes.FlowContent, subFlowConfig.Fields)
// 4. 并发数:从主流程开始节点 outputConfig 的 maxConcurrency 字段读取(前端把子流程节点生成次数表单字段聚合到主流程开始节点),读不到再用子流程节点配置兜底
maxConcurrency := mainFlowMaxConcurrency(nodeExecInput.Global, subFlowConfig.MaxConcurrency)
// 4. 编译子流程Graph(复用现有 BuildGraphFromFlowContent 逻辑)
nodeList, subGraph := BuildGraph(ctx, getRes.FlowContent)
// 4. 构建子流程Workflow(绑定START/END,和示例对齐)
innerWorkflow := compose.NewWorkflow[*flowDto.FlowExecutionInput, *flowDto.FlowExecutionInput]()
// 挂载子图节点并绑定全局START
innerWorkflow.AddGraphNode("sub_flow_graph", subGraph).AddInput(compose.START)
// 绑定子图输出到全局END
innerWorkflow.End().AddInput("sub_flow_graph")
// 生成次数(批量条数):maxConcurrency<=0 时按 1 次兜底
batchCount := maxConcurrency
if batchCount <= 0 {
batchCount = 1
}
// 5. 构建BatchNode(批量执行子流程,复用示例逻辑)
batchNode := batch.NewBatchNode(&batch.NodeConfig[*flowDto.FlowExecutionInput, *flowDto.FlowExecutionInput]{
Name: fmt.Sprintf("sub_flow_batch_%s", nodeExecInput.Config.Id),
InnerTask: innerWorkflow,
MaxConcurrency: batchCount,
})
// 6. 提取批量输入:按生成次数生成 N 份(每份独立克隆 ConfigMap,避免并发执行时节点输出写串)
configMap := buildConfigMap(getRes.FlowContent, nodeList)
batchInputs := make([]*flowDto.FlowExecutionInput, 0, batchCount)
for j := 0; j < batchCount; j++ {
// 缓存作用域:拼本(外层)子流程节点 Id + 份号 j,供 async/segment 缓存键隔离各份
// (同一 exec 下 N 份内层节点 id 相同,无作用域会互相命中/覆盖 done 结果,见《工作流子流程批量缓存隔离设计.md》)。
// 嵌套子流程继承父 scopeGlobal.SubFlowScope)递归叠加,保证跨 exec/嵌套路径唯一;
// 跨 launch 续跑按同序重放 batch → 各份 scope 稳定,仍能命中自己那份结果。
scope := nodeExecInput.Global.SubFlowScope + fmt.Sprintf("[%s#%d]", nodeExecInput.Config.Id, j)
batchInputs = append(batchInputs, &flowDto.FlowExecutionInput{
NodeGroupId: nodeExecInput.Global.NodeGroupId,
ExecutionId: nodeExecInput.Global.ExecutionId,
FlowId: nodeExecInput.Global.FlowId,
ConfigMap: cloneConfigMap(configMap),
SessionId: nodeExecInput.Global.SessionId,
SubFlowScope: scope,
})
}
// 7. 执行批量子流程
batchOutput, err := batchNode.Invoke(ctx, batchInputs)
if err != nil {
return nil, fmt.Errorf("执行子流程BatchNode失败: %v", err)
}
// 8. 展平每份子流程执行的节点输出,写回当前节点 OutputResult 供下游引用
var outputRes []map[string]any
for _, single := range batchOutput {
if single == nil {
continue
}
outputRes = append(outputRes, collectFlowNodeResults(single)...)
}
g.Log().Info(ctx, fmt.Sprintf("子流程执行完成,共 %d 次,输出 %d 条", batchCount, len(outputRes)))
nodeExecInput.Config.OutputResult = outputRes
return nodeExecInput, nil
}
// injectSubFlowFields 将子流程节点引入参数(subConfig.Fields)解析后写入子流程开始节点
// outputConfig,使子流程启动时能读到首页表单值/上游引用值/静态默认值。
// 每个字段的取值优先级:valueSource 引用解析成功 → field.value → field.defaultValue
// 匹配键为 field(前端约定以 field 为主,不兼容 path)。
func injectSubFlowFields(global *flowDto.FlowExecutionInput, subFlowContent *entity.FlowInfo, fields []map[string]any) {
if global == nil || subFlowContent == nil || len(fields) == 0 {
return
}
startNode := subFlowStartNode(subFlowContent)
if startNode == nil {
return
}
byField := make(map[string]map[string]any, len(startNode.OutputConfig))
for _, output := range startNode.OutputConfig {
byField[gconv.String(output["field"])] = output
}
for _, field := range fields {
entry := byField[gconv.String(field["field"])]
if entry == nil {
continue
}
value := field["value"]
if vs, has := field["valueSource"]; has && vs != nil {
if vsNodeId, vsField := firstValueSource(vs); vsNodeId != "" && vsField != "" {
if v, _, ok := values.ResolveValueSource(global, vsNodeId, vsField); ok {
value = v
}
}
}
if value == nil {
value = field["defaultValue"]
}
if value != nil {
entry["value"] = value
}
}
}
// firstValueSource 从 valueSource 提取第一个引用源 (nodeId, field)。
// 前端契约统一数组 [{nodeId, field}],旧 DSL 可能是单对象 {nodeId, field},两种形态都兼容;
// 子流程字段与引用源一一对应,只取第一个。
func firstValueSource(vs any) (nodeId, field string) {
if vs == nil {
return
}
switch v := vs.(type) {
case []any:
if len(v) > 0 {
return firstValueSource(v[0])
}
return
case []map[string]any:
if len(v) > 0 {
return firstValueSource(v[0])
}
return
}
m := gconv.Map(vs)
nodeId = gconv.String(m["nodeId"])
field = gconv.String(m["field"])
return
}
// subFlowStartNode 返回工作流开始节点
func subFlowStartNode(content *entity.FlowInfo) *entity.FlowNode {
if content == nil {
return nil
}
for i := range content.Nodes {
if content.Nodes[i].Id == content.StartNodeId {
return &content.Nodes[i]
}
}
return nil
}
// mainFlowMaxConcurrency 取子流程批量执行并发数:从主流程开始节点 outputConfig
// 的 maxConcurrency 字段读取(前端把子流程节点的生成次数表单字段聚合到主流程开始节点),
// 读不到再用子流程节点配置的兜底值。
func mainFlowMaxConcurrency(global *flowDto.FlowExecutionInput, fallback int) int {
if global == nil {
return fallback
}
for _, n := range global.ConfigMap {
if n == nil || n.NodeCode != node.NodeTypeStart {
continue
}
for _, output := range n.OutputConfig {
if gconv.String(output["field"]) != "maxConcurrency" {
continue
}
if v := gconv.Int(output["value"]); v > 0 {
return v
}
}
return fallback
}
return fallback
}
// cloneConfigMap 深拷贝 ConfigMap,保证各批次子流程并发执行时节点输出互不串扰。
// 浅拷贝会共享 *entity.FlowNode,并发写 OutputResult 产生竞态。
func cloneConfigMap(src map[string]*entity.FlowNode) map[string]*entity.FlowNode {
dst := make(map[string]*entity.FlowNode, len(src))
for k, v := range src {
data, err := json.Marshal(v)
if err != nil {
dst[k] = v
continue
}
n := new(entity.FlowNode)
if err = json.Unmarshal(data, n); err != nil {
dst[k] = v
continue
}
dst[k] = n
}
return dst
}
// collectFlowNodeResults 收集一次子流程执行中所有已执行节点的输出,展平成 {字段:值} 列表
func collectFlowNodeResults(execInput *flowDto.FlowExecutionInput) []map[string]any {
var res []map[string]any
for _, executed := range execInput.ExecutedNodes {
if nodeConfig := execInput.ConfigMap[executed.NodeId]; nodeConfig != nil {
res = append(res, nodeConfig.OutputResult...)
}
}
return res
}
+100
View File
@@ -0,0 +1,100 @@
package flow
import (
"ai-agent/workflow/consts/public"
sessionDao "ai-agent/workflow/dao/session"
flowDto "ai-agent/workflow/model/dto/flow"
sessionDto "ai-agent/workflow/model/dto/session"
"context"
"fmt"
"strings"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/database/gdb"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
func DataMergeLambda(ctx context.Context, input any) (res any, err error) {
nodeInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("参数合并入参类型错误")
}
return nodeInput, nil
}
func SummaryLambda(ctx context.Context, input any) (any, error) {
execInput, ok := input.(*flowDto.NodeExecutionInput)
if !ok {
return nil, fmt.Errorf("汇总节点入参类型错误,实际是 %T", input)
}
// 汇总节点只做本职:聚合本次执行已产出、需入库的文件结果(两层规则)并落 exec_workflow_result。
// 终态(exec_workflow.status / total_tokens / total_fee)与计费**禁止在图内写**——
// summary 会随子流程/多末端/断点续跑在整体还没跑完时提前执行,此处若把 exec 置成 status=2
// "成功",会骗过 wrapper 的失败兜底(recordExecutionFailure 只标记 Running 记录)→ 取消/中断后
// recordWorkflow/settleBilling 永不触发,计费单遗留 CREATED、已消耗 token 漏扣
//(线上 per_token/per_second 取消不扣费根因,2026-09-03)。终态只允许 BuildExecution 返回后的
// recordWorkflow 单点落库并结算(含 total_tokens/total_fee/actual_amount 回填),图内不再越权写 exec 行。
summaryResult := collectSaveFileResults(ctx, execInput.Global)
// 把汇总结果存入当前节点的输出
g.Log().Info(ctx, fmt.Sprintf("结果汇总完成,汇总数据:%+v", summaryResult))
if len(summaryResult) > 0 {
err := gfdb.DB(ctx, public.DbNameBlackDeacon).Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
_, err := sessionDao.ExecWorkflowResultDao.BatchInsert(ctx, summaryResult)
return err
})
if err != nil {
return nil, err
}
}
return execInput, nil
}
// collectSaveFileResults 按两层规则收集需入库的文件结果:
// 第一层:节点须开启"保存文件"IsSaveFile);
// 第二层:key 取自节点 OutputResult 的各字段,命中 ModelResponseBodyMapping 才入库;
// 原始响应体 key(respBody)恒入库(不要求映射声明);HTTP 节点产出以 http_file_url:{key}
// 标记的字段(IsSaveFile 时由 HttpCallResultLambda 生成)恒入库(无模型响应映射可查)。
// 结果值为 http(s) URL 或 MinIO 对象裸路径直接使用;非路径值(base64 图片/文本)先上传 OSS 换取 URL,
// 文本内容以 .inc 扩展名存储。
func collectSaveFileResults(ctx context.Context, execInput *flowDto.FlowExecutionInput) []*sessionDto.CreateWorkflowResultReq {
if execInput == nil {
return nil
}
var summaryResult []*sessionDto.CreateWorkflowResultReq
for _, executedNode := range execInput.ExecutedNodes {
nodeConfig := execInput.ConfigMap[executedNode.NodeId]
if nodeConfig == nil || len(nodeConfig.OutputResult) == 0 || !nodeConfig.IsSaveFile {
continue
}
// 第二层:key 取自节点 OutputResult 的各字段,
// 命中 ModelResponseBodyMapping 才入库;respBody 与 HTTP 节点 http_file_url:{key} 标记恒入库
saveKeys := nodeConfig.ModelConfig.ModelResponseBodyMapping
for _, respBody := range nodeConfig.OutputResult {
for key, val := range gconv.Map(respBody) {
isHTTPFile := strings.HasPrefix(key, "http_file_url:")
if !isHTTPFile {
if _, ok := saveKeys[key]; !ok && key != "respBody" {
continue
}
}
fileUrl, err := resolveSaveFileResult(ctx, val)
if err != nil {
g.Log().Warningf(ctx, "collectSaveFileResults 上传结果文件失败 key=%s err=%v", key, err)
continue
}
summaryResult = append(summaryResult, &sessionDto.CreateWorkflowResultReq{
SessionId: execInput.SessionId,
FlowId: execInput.FlowId,
ExecId: execInput.ExecutionId,
ResultFileUrl: fileUrl,
})
}
}
}
return summaryResult
}
+90
View File
@@ -0,0 +1,90 @@
package flow
import (
"ai-agent/gateway"
"ai-agent/workflow/consts/model"
"ai-agent/workflow/service/flow/processor"
"context"
"fmt"
"strings"
"github.com/gogf/gf/v2/frame/g"
)
// isVideoModel 判断模型是否为视频模型(模型类型 TypeVideo=600),用于视频节点多视频自动合成判断
func isVideoModel(ctx context.Context, modelId int64) bool {
modelInfo, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: modelId})
if err != nil {
g.Log().Warningf(ctx, "查询模型配置失败,跳过自动视频合成 modelId=%d err=%v", modelId, err)
return false
}
return modelInfo.ModelManage.ModelType != nil && *modelInfo.ModelManage.ModelType == model.TypeVideo
}
// invokePreTool 执行前置处理器,把模型请求参数转换为模型调用入参列表。
// 前置处理器契约:入参即模型请求参数本体;返回值:
// - map[string]any 一次模型调用,入参为返回值
// - []map[string]any 多次模型调用,逐个入参请求
// - nil 视为异常,节点失败(不允许静默跳过模型调用)
func invokePreTool(ctx context.Context, processorName string, modelParams map[string]any) (paramsList []map[string]any, err error) {
if processorName == "" {
return []map[string]any{stripInternalKeys(modelParams)}, nil
}
data, err := processor.Call(ctx, processorName, modelParams)
if err != nil {
return nil, fmt.Errorf("执行前置处理器[%s]失败: %v", processorName, err)
}
switch v := data.(type) {
case nil:
return nil, fmt.Errorf("前置处理器[%s]返回空", processorName)
case map[string]any:
return []map[string]any{stripInternalKeys(v)}, nil
case []map[string]any:
list := make([]map[string]any, 0, len(v))
for _, m := range v {
list = append(list, stripInternalKeys(m))
}
return list, nil
default:
return nil, fmt.Errorf("前置处理器[%s]返回类型不支持: %T", processorName, data)
}
}
// stripInternalKeys 剥离 __ 前缀的内部键(如 __segment_fields/__produced),
// 模型网关做参数严格校验(CheckParams strictUnknown)会拒绝未知字段,内部标记不得随请求体下发。
func stripInternalKeys(params map[string]any) map[string]any {
if params == nil {
return params
}
for k := range params {
if strings.HasPrefix(k, "__") {
delete(params, k)
}
}
return params
}
// invokePostTool 执行后置处理器,加工模型调用结果。
// 后置处理器契约:入参 {"output": 模型输出结果列表, "request": 原始模型请求参数}(列表须包成对象传入);返回值:
// - []map[string]any 替换模型输出
// - map[string]any 替换为单条输出
// - nil 保留原输出
func invokePostTool(ctx context.Context, processorName string, outputRes []map[string]any, requestParams map[string]any) ([]map[string]any, error) {
if processorName == "" {
return outputRes, nil
}
data, err := processor.Call(ctx, processorName, map[string]any{"output": outputRes, "request": requestParams})
if err != nil {
return nil, fmt.Errorf("执行后置处理器[%s]失败: %v", processorName, err)
}
switch v := data.(type) {
case nil:
return outputRes, nil
case []map[string]any:
return v, nil
case map[string]any:
return []map[string]any{v}, nil
default:
return nil, fmt.Errorf("后置处理器[%s]返回类型不支持: %T", processorName, data)
}
}
+259
View File
@@ -0,0 +1,259 @@
package flow
import (
"ai-agent/gateway"
"ai-agent/workflow/consts/model"
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/service/flow/values"
"context"
"fmt"
"strings"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
"github.com/gogf/gf/v2/util/gconv"
"github.com/google/uuid"
)
// ModelCallResultLambda 调用模型并返回输出内容列表,同时回传本次调用的 token/费用(*gateway.ModelCallRes
// 与是否推理模型(供 ModelLambda 决定分批结果是否拼接),供调用方(ModelLambda)累计写入节点执行记录
// token_info,最后由汇总节点聚合到 exec_workflow。
func ModelCallResultLambda(ctx context.Context, nodeGroupId string, modelId int64, sessionId string, modelRequestParams map[string]any, prompt string, execId int64, nodeId string, segIdx int) ([]map[string]any, *gateway.ModelCallRes, bool, error) {
modelInfo, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: modelId})
if err != nil {
return nil, nil, false, fmt.Errorf("获取模型配置失败: %w", err)
}
// model-gateway 对不存在的模型返回 modelManage=nullHTTP 仍 200),零值 struct 里 ResponseType 是 nil 指针,
// 直接传入 ModelCallResult 会在 *responseType 处 panic,这里提前报业务错误
if g.IsEmpty(modelInfo.ModelManage.Id) {
return nil, nil, false, fmt.Errorf("模型配置不存在: modelId=%d", modelId)
}
businessParams := make(map[string]any)
if !g.IsEmpty(prompt) {
if modelInfo.ModelManage.ModelType != nil && *modelInfo.ModelManage.ModelType == model.TypeVideo {
businessParams["user_prompt"] = prompt
} else if modelInfo.ModelManage.ModelType != nil && *modelInfo.ModelManage.ModelType == model.TypeInference {
businessParams["system_prompt"] = prompt
}
}
// 推理模型:分批调用结果需拼接为单个字段,模型类型仅网关配置携带,此处顺带判断
isInference := modelInfo.ModelManage.ModelType != nil && *modelInfo.ModelManage.ModelType == model.TypeInference
// 统一异步入口:提交落库 flow_async_task,崩溃恢复重订阅 msg_topic 拿回结果(同步模型直接调用,不落库)
responseParams, err := AsyncModelCallWithRecovery(ctx, nodeGroupId, execId, nodeId, segIdx, modelId, modelInfo.ModelManage.ResponseType, sessionId, modelRequestParams, businessParams)
if err != nil {
return nil, nil, false, err
}
if g.IsEmpty(responseParams) {
return nil, nil, false, fmt.Errorf("生成内容为空")
}
outputRes := make([]map[string]any, 0)
for key, val := range responseParams.Content {
outputRes = append(outputRes, map[string]any{
key: val,
})
}
return outputRes, responseParams, isInference, nil
}
func HttpCallResultLambda(ctx context.Context, nodeInput *flowDto.NodeExecutionInput) ([]map[string]any, error) {
var method, url, responseType, callbackUrl string
var headers map[string]string
var body map[string]any
var responseMapping map[string]any
n := new([]node.NodePresetField)
err := gconv.Structs(nodeInput.Config.OutputConfig, n)
if err != nil {
return nil, err
}
for _, item := range *n {
switch item.Field {
case "method":
method = gconv.String(item.Value)
case "url":
url = gconv.String(item.Value)
case "headers":
headers = gconv.MapStrStr(item.Value)
case "body":
body = gconv.Map(item.Value)
case "response":
// 先剥掉 {type, value/attrs} 包裹层,得到干净的输出结构模板
responseMapping = gconv.Map(values.UnwrapSchemaWrapper(gconv.Map(item.Value)))
case "responseType":
responseType = gconv.String(item.Value)
if responseType == "callback" {
callbackUrl = item.Options[0].Config[0].Value
}
}
}
if method == "" {
return nil, fmt.Errorf("method为空")
}
if url == "" {
return nil, fmt.Errorf("url为空")
}
if headers == nil {
headers = utils.HeadersFromCtx(ctx)
}
// 构建请求参数
values.ProcessValueSourceRecursive(body, nodeInput.Global)
// 递归剥掉 {type, value/attrs} 包裹层,只保留 key/value
wrapper := values.UnwrapSchemaWrapper(body)
newBody := gconv.Map(wrapper)
// body 值若为 MinIO 裸路径(模型网关转存 OSS 后返回,无 http 前缀),
// 补上前缀供目标 HTTP 服务直接下载文件
addFilePathPrefix(ctx, url, newBody)
// 1. 自己生成唯一 taskId(不用前端给)
taskId := "my_task_" + uuid.New().String() // 自己生成唯一ID
if responseType == "callback" {
newBody[callbackUrl] = utils.GetCallbackURL(ctx, "/httpNodeCallback?task_id="+taskId)
}
// ====================== 核心改动 ======================
// 1. 定义一个空map接收原始HTTP返回结果
var rawHttpResult map[string]any
// 2. 发送请求(不变)
if method == "GET" {
err = commonHttp.Get(ctx, url, headers, &rawHttpResult, newBody)
} else if method == "POST" {
err = commonHttp.Post(ctx, url, headers, &rawHttpResult, newBody)
} else if method == "PUT" {
err = commonHttp.Put(ctx, url, headers, &rawHttpResult, newBody)
} else if method == "DELETE" {
err = commonHttp.Delete(ctx, url, headers, &rawHttpResult, newBody)
} else {
return nil, fmt.Errorf("method 不支持")
}
if err != nil {
return nil, err
}
var e = ""
finalResult := make(map[string]any)
if responseType == "sync" {
httpResultJson := gconv.String(rawHttpResult)
// 按 responseMapping 定义的结构,从 http 返回结果中拷贝对应字段
finalResult = values.MapResultByTemplate(responseMapping, rawHttpResult)
e = httpResultJson
}
if responseType == "callback" {
var waitResult any
waitResult, err = Wait(ctx, taskId)
if err != nil {
return nil, err
}
request, ok := waitResult.(*ghttp.Request)
if !ok {
return nil, fmt.Errorf("入参类型错误")
}
bodyStr := request.GetBodyString()
// 按 responseMapping 定义的结构,从回调结果中拷贝对应字段
finalResult = values.MapResultByTemplate(responseMapping, gconv.Map(bodyStr))
e = bodyStr
}
if responseType == "pull" {
return nil, fmt.Errorf("pull 暂不支持")
}
if g.IsEmpty(finalResult) {
return nil, fmt.Errorf("http请求异常,返回结果为空:%v", e)
}
outputRes := make([]map[string]any, 0)
for i, item := range finalResult {
if nodeInput.Config.IsSaveFile {
outputRes = append(outputRes, map[string]any{
fmt.Sprintf("http_file_url:%v", i): item,
})
}
outputRes = append(outputRes, map[string]any{
fmt.Sprintf("%v", i): item,
})
}
return outputRes, nil
}
// addFilePathPrefix 递归把 body 中的 MinIO 裸路径(模型网关转存 OSS 后返回,无 http 前缀)补上文件前缀,
// 供目标 HTTP 服务直接下载文件;已是完整 URL 的值保持不变
func addFilePathPrefix(ctx context.Context, url string, body map[string]any) {
prefix, err := oss.GetFileAddressPrefix(ctx)
if err != nil {
g.Log().Warningf(ctx, "获取文件前缀失败,保持原路径: %v", err)
return
}
for k, v := range body {
if k == "templates" && g.IsEmpty(v) {
delete(body, k)
continue
}
body[k] = prependFilePathPrefix(prefix, v)
}
// template/template 模板接口要求 video_urls 为数组:标量值包装为单元素数组
if strings.Contains(url, "template/template") {
if v, ok := body["video_urls"]; ok {
body["video_urls"] = toVideoURLsArray(v)
}
if v, ok := body["subtitles"]; ok {
a := new([]flowDto.Sentence)
err = gconv.Structs(v, a)
v, err = BuildSubtitles(a)
body["subtitles"] = v
}
}
}
// toVideoURLsArray 把标量 video_urls 包装为数组;已是数组/切片则原样保留
func toVideoURLsArray(v any) any {
switch val := v.(type) {
case string:
if val == "" {
return []string{}
}
return []string{val}
case []string, []any:
return val
default:
return v
}
}
// prependFilePathPrefix 对单个值加前缀,递归处理嵌套 map/切片
func prependFilePathPrefix(prefix string, v any) any {
switch val := v.(type) {
case string:
if oss.IsOSSPath(val) {
return prefix + val
}
return val
case map[string]any:
for k, item := range val {
val[k] = prependFilePathPrefix(prefix, item)
}
return val
case []any:
for i, item := range val {
val[i] = prependFilePathPrefix(prefix, item)
}
return val
case []map[string]any:
for _, m := range val {
for k, item := range m {
m[k] = prependFilePathPrefix(prefix, item)
}
}
return val
default:
return val
}
}
@@ -0,0 +1,301 @@
package media
import (
"ai-agent/workflow/service/flow/processor"
"context"
"fmt"
"strings"
"time"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
// TaskKind 媒体任务类型(拼接/拼接+混音),决定提交与查询的接口路径
type TaskKind string
const (
TaskKindConcat TaskKind = "concat" // 纯拼接,无 BGM
TaskKindMerge TaskKind = "merge" // 拼接+混音,有 BGM
)
// MergeTask media 服务异步任务状态
type MergeTask struct {
TaskID string `json:"taskId"`
Status string `json:"status"` // pending/running/success/failed
FileURL string `json:"fileURL,omitempty"`
ErrorMessage string `json:"errorMessage,omitempty"`
DurationStr string `json:"durationStr,omitempty"`
}
type mergeSubmitReq struct {
VideoURLs []string `json:"video_urls"`
AudioURLs []string `json:"audio_urls,omitempty"`
Method string `json:"method,omitempty"`
Upload bool `json:"upload"`
CallbackURL string `json:"callback_url"`
}
type mergeSubmitRes struct {
TaskID string `json:"taskId"`
}
// ProcessorName 处理器注册名,供模型节点前置工具分发按名判定入参形态(扁平请求体)。
const ProcessorName = "concat_videos"
func init() {
processor.Register(ConcatVideosProcessor())
}
// ConcatVideosProcessor 合并视频
func ConcatVideosProcessor() *processor.Processor {
return &processor.Processor{
Name: ProcessorName,
Description: "合并视频",
IsShow: false,
Func: func(ctx context.Context, args map[string]any) (any, error) {
outputRes := parseOutputList(args)
segments, err := collectSegmentResults(ctx, outputRes)
if err != nil {
return nil, err
}
videoURLs := make([]string, 0, len(segments))
for _, seg := range segments {
videoURLs = append(videoURLs, seg.VideoURL)
}
reqParams := gconv.Map(args["request"])
bgmURLs := gconv.Strings(reqParams["bgm_urls"])
if len(bgmURLs) == 0 {
// 兼容串行结果把 bgm 带回 output 的情况
for _, m := range outputRes {
bgmURLs = append(bgmURLs, gconv.Strings(m["bgm_urls"])...)
}
}
upload := gconv.Bool(reqParams["upload"])
callback := gconv.String(reqParams["callback_url"])
merged, err := MergeSegments(ctx, videoURLs, bgmURLs, upload, callback, 30*time.Minute)
if err != nil {
return nil, err
}
// 合并结果沿用输入视频的 key 返回,保持 key 不变;
// 否则下游按原 key 引用(值来源/保存文件映射)会失配
retKey := "fileURL"
if len(outputRes) > 0 {
if k := findVideoKey(outputRes[0]); k != "" {
retKey = k
}
}
return map[string]any{
retKey: merged.FileURL,
}, nil
},
}
}
// parseOutputList 兼容不同反序列化形态的 output 列表
func parseOutputList(args map[string]any) []map[string]any {
switch v := args["output"].(type) {
case []map[string]any:
return v
case []any:
out := make([]map[string]any, 0, len(v))
for _, it := range v {
if m, ok := it.(map[string]any); ok {
out = append(out, m)
} else if m := gconv.Map(it); m != nil {
out = append(out, m)
}
}
return out
}
return nil
}
// MergeSegments 把分段视频按序合并:有 BGM 走拼接+混音(merge),否则纯拼接(concat)。
// 返回最终合并结果(FileURL)。
func MergeSegments(ctx context.Context, videoURLs, bgmURLs []string, upload bool, callbackURL string, timeout time.Duration) (*MergeTask, error) {
var kind TaskKind
var taskID string
var err error
if len(bgmURLs) > 0 {
kind = TaskKindMerge
taskID, err = SubmitMergeAsync(ctx, videoURLs, bgmURLs, upload, callbackURL)
} else {
kind = TaskKindConcat
taskID, err = SubmitConcatAsync(ctx, videoURLs, upload, callbackURL)
}
if err != nil {
return nil, err
}
return WaitMediaTask(ctx, kind, taskID, timeout)
}
// SubmitConcatAsync 提交纯拼接异步任务
func SubmitConcatAsync(ctx context.Context, videoURLs []string, upload bool, callbackURL string) (string, error) {
return submitMediaTask(ctx, TaskKindConcat, &mergeSubmitReq{
VideoURLs: videoURLs,
Method: "auto",
Upload: upload,
CallbackURL: callbackURL,
})
}
// SubmitMergeAsync 提交拼接+混音异步任务
func SubmitMergeAsync(ctx context.Context, videoURLs, audioURLs []string, upload bool, callbackURL string) (string, error) {
return submitMediaTask(ctx, TaskKindMerge, &mergeSubmitReq{
VideoURLs: videoURLs,
AudioURLs: audioURLs,
Upload: upload,
CallbackURL: callbackURL,
})
}
func submitMediaTask(ctx context.Context, kind TaskKind, req *mergeSubmitReq) (string, error) {
path := "media/video/" + string(kind) + "/async"
res := new(mergeSubmitRes)
if err := commonHttp.Post(ctx, path, utils.HeadersFromCtx(ctx), res, req); err != nil {
return "", fmt.Errorf("提交%s任务失败: %v", kind, err)
}
if res.TaskID == "" {
return "", fmt.Errorf("media 返回空 taskId")
}
return res.TaskID, nil
}
// WaitMediaTask 轮询媒体任务直到 success/failed,或超时。
func WaitMediaTask(ctx context.Context, kind TaskKind, taskID string, timeout time.Duration) (*MergeTask, error) {
deadline := time.Now().Add(timeout)
ticker := time.NewTicker(15 * time.Second)
defer ticker.Stop()
for {
t, err := getMediaTask(ctx, kind, taskID)
if err == nil {
switch t.Status {
case "success":
if t.FileURL == "" {
return nil, fmt.Errorf("%s任务成功但未返回文件URL", kind)
}
return t, nil
case "failed":
return nil, fmt.Errorf("%s任务失败: %s", kind, t.ErrorMessage)
}
}
if time.Now().After(deadline) {
return nil, fmt.Errorf("%s任务[%s]超时", kind, taskID)
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-ticker.C:
}
}
}
func getMediaTask(ctx context.Context, kind TaskKind, taskID string) (*MergeTask, error) {
path := "media/video/" + string(kind) + "/task/" + taskID
res := new(MergeTask)
if err := commonHttp.Get(ctx, path, utils.HeadersFromCtx(ctx), res); err != nil {
return nil, err
}
return res, nil
}
// segmentResult 一段视频生成的产出(原 ai-agent/video/plan.SegmentResultplan 包已并入处理器树)。
type segmentResult struct {
SegmentIndex int `json:"segment_index"`
VideoURL string `json:"video_url"`
Duration int `json:"duration"`
}
// collectSegmentResults 把模型节点/串行工具的产出([]map[string]any)收敛为有序的分段结果列表。
// 兼容两种形状:串行工具产出的 {segment_index, video_url, duration}
// 并行模型调用产出的 {<url字段>: url}(按列表顺序对应各段)。
func collectSegmentResults(ctx context.Context, outputRes []map[string]any) ([]segmentResult, error) {
if len(outputRes) == 0 {
return nil, fmt.Errorf("没有可合并的分段视频")
}
var segs []segmentResult
for i, m := range outputRes {
seg := segmentResult{
SegmentIndex: i,
Duration: gconv.Int(m["duration"]),
VideoURL: findVideoURL(ctx, m),
}
if idx := gconv.Int(m["segment_index"]); len(outputRes) > 1 && idx > 0 {
seg.SegmentIndex = idx
}
if seg.VideoURL == "" {
return nil, fmt.Errorf("第 %d 段未获取到视频URL", i)
}
segs = append(segs, seg)
}
return segs, nil
}
// findVideoURL 从模型返回参数中提取视频 URL:优先命中常见键,再兼容扁平点号键(content.attrs.video_url 等)任意含 url 的字段。
func findVideoURL(ctx context.Context, params map[string]any) string {
key := findVideoKey(params)
if key == "" {
return ""
}
return normalizeVideoURL(ctx, gconv.String(params[key]))
}
// findVideoKey 返回视频 URL 所在字段的 key(命中规则与 findVideoURL 一致),
// 供视频合并后以原 key 返回结果,避免下游按原 key 引用(值来源/保存文件映射)失配。
func findVideoKey(params map[string]any) string {
if params == nil {
return ""
}
for _, key := range []string{"video_url", "video_oss_url", "http_file_url", "file_url", "url"} {
if gconv.String(params[key]) != "" {
return key
}
}
// 兼容扁平点号键(如 content.attrs.video_url):优先命中含 video 的键,再兜底任意含 url 的键
var fallback string
for k, v := range params {
if !strings.Contains(strings.ToLower(k), "url") {
continue
}
if gconv.String(v) == "" {
continue
}
if strings.Contains(strings.ToLower(k), "video") {
return k
}
if fallback == "" {
fallback = k
}
}
return fallback
}
// normalizeVideoURL 统一视频地址:已是完整 http(s) 链接原样返回;相对路径(MinIO 对象路径)补上文件前缀,供 media 服务下载
func normalizeVideoURL(ctx context.Context, url string) string {
if url == "" || strings.HasPrefix(url, "http") {
return url
}
prefix, err := oss.GetFileAddressPrefix(ctx)
if err != nil {
g.Log().Warningf(ctx, "获取文件前缀失败,视频地址保持相对路径: %s err=%v", url, err)
return url
}
return prefix + url
}
// FindVideoKey 返回视频 URL 所在字段的 key(规则同 findVideoKey),供工作流段级续跑重建输出记录保持 key 一致
func FindVideoKey(params map[string]any) string {
return findVideoKey(params)
}
// FindVideoURL 从模型返回参数中提取视频 URL(规则同 findVideoURL),供段级续跑落库
func FindVideoURL(ctx context.Context, params map[string]any) string {
return findVideoURL(ctx, params)
}
@@ -0,0 +1,187 @@
// Package split_batch 工作流前置处理器:按默认上限(uploadTotalMaxCount 默认 15)拆分模型请求参数为多批。
// 入参为已构建好的扁平模型请求体(BuildModelRequestBody 输出,key 为点分路径,value 已解析填充),
// 集合字段按上限分批、元素对象取 url 为值;返回结构同入参(扁平 map 数组),未超量返回单份。
// 处理器实现自包含(算法随处理器走,不依赖业务包),通过 init 注册进 processor 注册表。
package split_batch
import (
"context"
"fmt"
"reflect"
"sort"
"github.com/gogf/gf/v2/util/gconv"
"ai-agent/workflow/service/flow/processor"
)
// ProcessorName 处理器注册名,供模型节点前置工具分发按名判定入参形态(扁平请求体)。
const ProcessorName = "split_batch_model_params"
// defaultMaxCount 每批最大元素数(constraint.uploadTotalMaxCount 当前默认值,后续动态传递)。
const defaultMaxCount = 15
func init() {
processor.Register(SplitBatchModelParamsProcessor())
}
// SplitBatchModelParamsProcessor 将模型请求参数按默认上限分批的前置处理器。
// 入参 args 即扁平模型请求体(valueSource 已解析、value 已填充)。
func SplitBatchModelParamsProcessor() *processor.Processor {
return &processor.Processor{
Name: ProcessorName,
Description: "按最大约束构建分批模型请求数据",
IsShow: true,
Func: func(ctx context.Context, args map[string]any) (any, error) {
if args == nil {
return nil, fmt.Errorf("缺少模型请求参数")
}
return SplitBatchModelParams(args), nil
},
}
}
// batchField 描述一个需要分批的扁平字段
type batchField struct {
key string // 扁平点分 key
items []any // 分批元素(元素为带 url 字段的对象时已取 url 为值)
}
// SplitBatchModelParams 把扁平模型请求体拆成多批,供分批请求模型使用。
// 处理流程:
// 1. 深拷贝入参,避免污染调用方数据;
// 2. 遍历扁平字段,值为集合(slice/array/map)的按元素数 / defaultMaxCount 向上取整,
// map 按 key 排序取 value 列表作为元素;元素为带 url 字段的对象时取 url 为值;
// 3. 总批数 = 各字段批数的最大值;每批 = 整份参数深拷贝 + 各分批字段替换为该批切片,
// 元素已耗尽的分批字段替换为空切片。
//
// 未超量时返回单份参数。调用方可遍历返回值逐个请求模型。
func SplitBatchModelParams(rawParams map[string]any) []map[string]any {
params, ok := deepCopyAny(rawParams).(map[string]any)
if !ok {
params = make(map[string]any)
}
fields := make([]batchField, 0)
batchCount := 1
for key, v := range params {
items, has := toItems(v)
if !has {
continue
}
n := (len(items) + defaultMaxCount - 1) / defaultMaxCount
if n > batchCount {
batchCount = n
}
if n > 1 {
fields = append(fields, batchField{key: key, items: items})
}
}
if batchCount <= 1 {
return []map[string]any{params}
}
batches := make([]map[string]any, 0, batchCount)
for i := 0; i < batchCount; i++ {
batch, _ := deepCopyAny(params).(map[string]any)
for _, f := range fields {
start := i * defaultMaxCount
if start >= len(f.items) {
batch[f.key] = []any{}
continue
}
end := start + defaultMaxCount
if end > len(f.items) {
end = len(f.items)
}
batch[f.key] = f.items[start:end]
}
batches = append(batches, batch)
}
return batches
}
// itemValue 取集合元素作为分批粒度时的值:元素为带 url 字段的对象时取 url"取url为值"),
// 其余元素原样保留。
func itemValue(e any) any {
if m := gconv.Map(e); m != nil {
if u, ok := m["url"]; ok && u != nil {
return u
}
}
return e
}
// toItems 把集合 value 转成元素列表:切片逐元素转换(对象取 url 为值);map 按 key 排序取 value
// 类型化切片([]string 等)经反射逐元素转换。
func toItems(v any) ([]any, bool) {
switch val := v.(type) {
case []any:
if len(val) == 0 {
return nil, false
}
items := make([]any, 0, len(val))
for _, e := range val {
items = append(items, itemValue(e))
}
return items, true
case map[string]any:
if len(val) == 0 {
return nil, false
}
keys := make([]string, 0, len(val))
for k := range val {
keys = append(keys, k)
}
sort.Strings(keys)
items := make([]any, 0, len(keys))
for _, k := range keys {
items = append(items, itemValue(val[k]))
}
return items, true
}
rv := reflect.ValueOf(v)
if rv.IsValid() && (rv.Kind() == reflect.Slice || rv.Kind() == reflect.Array) {
n := rv.Len()
if n == 0 {
return nil, false
}
items := make([]any, 0, n)
for i := 0; i < n; i++ {
items = append(items, itemValue(rv.Index(i).Interface()))
}
return items, true
}
return nil, false
}
// deepCopyAny 深拷贝 map[string]any / []any / 类型化切片嵌套结构,避免批次之间互相影响
func deepCopyAny(v any) any {
switch val := v.(type) {
case map[string]any:
res := make(map[string]any, len(val))
for k, child := range val {
res[k] = deepCopyAny(child)
}
return res
case []any:
res := make([]any, len(val))
for i, child := range val {
res[i] = deepCopyAny(child)
}
return res
}
rv := reflect.ValueOf(v)
if rv.IsValid() && (rv.Kind() == reflect.Slice || rv.Kind() == reflect.Array) {
n := rv.Len()
out := reflect.MakeSlice(rv.Type(), n, n)
for i := 0; i < n; i++ {
d := deepCopyAny(rv.Index(i).Interface())
if d != nil {
out.Index(i).Set(reflect.ValueOf(d))
}
}
return out.Interface()
}
return v
}
@@ -0,0 +1,161 @@
// Package split_segment 工作流前置处理器:把聚合了脚本转写各段参数的扁平请求体按段拆成多份,
// 供分段模型请求并行执行。
//
// 脚本转写节点(split_shots_pipeline 产出)的下游模型节点单源引用其字段时,
// BuildModelRequestBody 把这些字段的数组值收集进 __segment_fields(逗号分隔的扁平路径),
// 处理器按段序把每份参数拆成独立请求体,段与段之间互不影响(深拷贝)。
// 处理器自包含(算法随处理器走,不依赖业务包),通过 init 注册进 processor 注册表。
package split_segment
import (
"context"
"fmt"
"reflect"
"strings"
"ai-agent/workflow/service/flow/processor"
)
// ProcessorName 处理器注册名,供模型节点前置工具分发按名判定入参形态(扁平请求体 + __segment_fields 标记)。
const ProcessorName = "split_segment"
func init() {
processor.Register(SplitSegmentProcessor())
}
// SplitSegmentProcessor 按段拆分模型请求参数的前置处理器。
// 入参 args 即扁平模型请求体(BuildModelRequestBody 输出,含 __segment_fields 标记)。
func SplitSegmentProcessor() *processor.Processor {
return &processor.Processor{
Name: ProcessorName,
Description: "按段拆分脚本转写聚合的模型请求参数",
IsShow: true,
Func: func(ctx context.Context, args map[string]any) (any, error) {
if args == nil {
return nil, fmt.Errorf("缺少模型请求参数")
}
return SplitSegmentModelParams(args)
},
}
}
// segField 描述一个待拆分的分段字段
type segField struct {
path string // 扁平点分路径
items []any // 按段序排列的元素(元素为 nil 表示该段无此字段)
}
// SplitSegmentModelParams 把扁平模型请求体按段拆成多份,供分段请求模型使用。
// 处理流程:
// 1. 读取 __segment_fields(逗号分隔的扁平路径),无标记则返回单份参数;
// 2. 逐个解析分段字段的数组值,要求非空且各字段长度一致(不一致直接报错,避免按错位拆分);
// 3. 按段数深拷贝整份参数,各段覆盖其分段字段为该段元素;元素为 nil(该段无此字段)时删除该键,
// 避免把 null 传给模型网关。
//
// 返回的每份参数仍保留 __segment_fields(由 invokePreTool 统一剥离 __ 前缀内部键)。
func SplitSegmentModelParams(rawParams map[string]any) ([]map[string]any, error) {
paths := segmentFieldsFromArgs(rawParams)
if len(paths) == 0 {
return []map[string]any{rawParams}, nil
}
fields := make([]segField, 0, len(paths))
segmentCount := 0
for _, path := range paths {
v, has := rawParams[path]
if !has {
return nil, fmt.Errorf("分段字段[%s]缺失", path)
}
items, ok := asItems(v)
if !ok || len(items) == 0 {
return nil, fmt.Errorf("分段字段[%s]值不是数组或为空", path)
}
if segmentCount == 0 {
segmentCount = len(items)
} else if len(items) != segmentCount {
return nil, fmt.Errorf("分段字段长度不一致: %s=%d, 期望 %d", path, len(items), segmentCount)
}
fields = append(fields, segField{path: path, items: items})
}
batches := make([]map[string]any, 0, segmentCount)
for i := 0; i < segmentCount; i++ {
batch, _ := deepCopyAny(rawParams).(map[string]any)
for _, f := range fields {
if f.items[i] == nil {
delete(batch, f.path)
continue
}
batch[f.path] = f.items[i]
}
batches = append(batches, batch)
}
return batches, nil
}
// segmentFieldsFromArgs 解析 __segment_fields 标记为扁平路径列表,空值返回 nil。
func segmentFieldsFromArgs(args map[string]any) []string {
raw, ok := args["__segment_fields"].(string)
if !ok || strings.TrimSpace(raw) == "" {
return nil
}
var paths []string
for _, p := range strings.Split(raw, ",") {
if t := strings.TrimSpace(p); t != "" {
paths = append(paths, t)
}
}
return paths
}
// asItems 取分段字段值作为元素列表:[]any 直接返回(元素保持原样,nil 段位保留);
// 类型化切片([]string 等)经反射逐元素转 any。非切片返回 false。
func asItems(v any) ([]any, bool) {
if list, ok := v.([]any); ok {
return list, true
}
rv := reflect.ValueOf(v)
if rv.IsValid() && (rv.Kind() == reflect.Slice || rv.Kind() == reflect.Array) {
n := rv.Len()
if n == 0 {
return nil, true
}
items := make([]any, 0, n)
for i := 0; i < n; i++ {
items = append(items, rv.Index(i).Interface())
}
return items, true
}
return nil, false
}
// deepCopyAny 深拷贝 map[string]any / []any / 类型化切片嵌套结构,避免批次之间互相影响
func deepCopyAny(v any) any {
switch val := v.(type) {
case map[string]any:
res := make(map[string]any, len(val))
for k, child := range val {
res[k] = deepCopyAny(child)
}
return res
case []any:
res := make([]any, len(val))
for i, child := range val {
res[i] = deepCopyAny(child)
}
return res
}
rv := reflect.ValueOf(v)
if rv.IsValid() && (rv.Kind() == reflect.Slice || rv.Kind() == reflect.Array) {
n := rv.Len()
out := reflect.MakeSlice(rv.Type(), n, n)
for i := 0; i < n; i++ {
d := deepCopyAny(rv.Index(i).Interface())
if d != nil {
out.Index(i).Set(reflect.ValueOf(d))
}
}
return out.Interface()
}
return v
}
@@ -0,0 +1,100 @@
package pipeline
// Config 统一浮点/阈值配置(设计 §4)。所有阈值唯一出处,阶段函数内不出现硬编码字面量。
// 零值字段在 resolveConfig 统一由 DefaultConfig() 补齐。
type Config struct {
// 时长计算(秒,float 允许更精细的弹性分配)
CharPerSecond float64 // 语速(字/秒),默认 4
FastCharPerSecond float64 // 感叹/疑问多时的语速,默认 6
SlowCharPerSecond float64 // 低落语气时的语速,默认 3
MinDurBuffer float64 // 有声镜头时长缓冲(秒),默认 1
MinVisualDur float64 // 纯视觉镜头最小时长(秒),默认 1
VisualWeight float64 // 纯视觉镜头弹性权重,默认 2.0
SpokenWeight float64 // 有声镜头弹性权重,默认 0.5
// 切分/断句
SplitWindow int // 断句搜索窗口(rune),默认 60
ShortFragmentDur float64 // 短残片阈值(秒),默认 1;时间线为整数秒,残片 ≤ 该值(通常即 1s)走并入逻辑
BoundaryTolerance float64 // 语义切点容差(秒),<=0 时按 max(MaxSegmentDur*0.2, 2) 派生
MaxSplitIter int // 残片并入上限重切的迭代上限,默认 10;超限返回 ErrSegmentInfeasible
// 参考素材与截断
MaxRefs int // 单段参考素材上限,默认 5
MaxPromptChars int // prompt 截断长度(rune),<=0 不截断
MinPromptFloor int // prompt 截断保底长度(rune),默认 50
StrictInvariant bool // 时间线不变量校验:true 校验失败返回 ErrTimelineInvariantfalse 仅记录告警
}
// DefaultConfig 返回默认配置。BoundaryTolerance 依赖 MaxSegmentDur,由 resolveConfig 派生。
func DefaultConfig() Config {
return Config{
CharPerSecond: 4,
FastCharPerSecond: 6,
SlowCharPerSecond: 3,
MinDurBuffer: 1,
MinVisualDur: 1,
VisualWeight: 2.0,
SpokenWeight: 0.5,
SplitWindow: 60,
ShortFragmentDur: 1,
BoundaryTolerance: 0, // 派生子:max(MaxSegmentDur*0.2, 2)
MaxSplitIter: 10,
MaxRefs: 5,
MaxPromptChars: 0,
MinPromptFloor: 50,
}
}
// resolveConfig 用默认值补齐 cfg 的零值字段。maxSegmentDur 用于派生 BoundaryTolerance。
func resolveConfig(cfg Config, maxSegmentDur int) Config {
def := DefaultConfig()
if cfg.CharPerSecond <= 0 {
cfg.CharPerSecond = def.CharPerSecond
}
if cfg.FastCharPerSecond <= 0 {
cfg.FastCharPerSecond = def.FastCharPerSecond
}
if cfg.SlowCharPerSecond <= 0 {
cfg.SlowCharPerSecond = def.SlowCharPerSecond
}
if cfg.MinDurBuffer <= 0 {
cfg.MinDurBuffer = def.MinDurBuffer
}
if cfg.MinVisualDur <= 0 {
cfg.MinVisualDur = def.MinVisualDur
}
if cfg.VisualWeight <= 0 {
cfg.VisualWeight = def.VisualWeight
}
if cfg.SpokenWeight <= 0 {
cfg.SpokenWeight = def.SpokenWeight
}
if cfg.SplitWindow <= 0 {
cfg.SplitWindow = def.SplitWindow
}
if cfg.ShortFragmentDur <= 0 {
cfg.ShortFragmentDur = def.ShortFragmentDur
}
if cfg.MaxSplitIter <= 0 {
cfg.MaxSplitIter = def.MaxSplitIter
}
if cfg.MaxRefs <= 0 {
cfg.MaxRefs = def.MaxRefs
}
if cfg.MaxPromptChars <= 0 {
cfg.MaxPromptChars = def.MaxPromptChars
}
if cfg.MinPromptFloor <= 0 {
cfg.MinPromptFloor = def.MinPromptFloor
}
if cfg.BoundaryTolerance <= 0 {
tol := float64(maxSegmentDur) * 0.2
if tol < 2 {
tol = 2
}
cfg.BoundaryTolerance = tol
}
return cfg
}
@@ -0,0 +1,29 @@
package pipeline
import "fmt"
// 业务错误码(设计 §8.1)。上层可据 Code 差异化处理。
const (
ErrEmptyShots = "ERR_EMPTY_SHOTS" // 镜头为空
ErrInvalidInput = "ERR_INVALID_INPUT" // 参数非法(TotalDuration/Max/Min 等)
ErrSegmentInfeasible = "ERR_SEGMENT_INFEASIBLE" // 时长拆分不可行
ErrTimelineInvariant = "ERR_TIMELINE_INVARIANT" // 时间线不变量破坏
)
// PipelineError 带业务分类码的错误。Segment 关联段序号,-1 表示不特定于某段。
type PipelineError struct {
Code string
Message string
Segment int
}
func (e *PipelineError) Error() string {
if e.Segment >= 0 {
return fmt.Sprintf("[%s] 段%d: %s", e.Code, e.Segment, e.Message)
}
return fmt.Sprintf("[%s] %s", e.Code, e.Message)
}
func pipeErr(code, msg string) *PipelineError {
return &PipelineError{Code: code, Message: msg, Segment: -1}
}
@@ -0,0 +1,245 @@
// Package pipeline 分镜 → 时间线 → 模型语言 Pipeline(设计文档 docs/superpowers/specs/2026-08-11-shots-timeline-pipeline-design.md)。
//
// 纯函数包:只依赖 video/domain + 标准库,零 I/O、零 workflow 依赖。
// 入口 PlanSegments 编排 ①时间线构建 → ②prompt 构建;各阶段函数均可独立调用/单测。
package pipeline
import (
"strconv"
"strings"
)
// Input 外部直接传入的生成请求参数。
type Input struct {
Shots []Shot // 镜头脚本(调用方已归一为 domain.Shot
TotalDuration int // 目标总时长(秒),<=0 按镜头时间码累加兜底,仍<=0 默认 60
MaxSegmentDur int // 单段最大时长(秒),<=0 默认 15
MinSegmentDur int // 单段最小时长(秒),<=0 默认 5
Refs Refs // 参考素材(角色/场景/道具/产品,具名)
Seed int64 // 随机种子基数,各段 = Seed + 段序号
NegativePrompt string // 全局负面 prompt(单段可覆盖,见 §7.5)
NoSpeech bool // 静音模式:段级 prompt 追加静音硬约束(视频模型不产生口播/字幕/口型)
Cfg Config // 阈值/语速/容差等统一配置,零值取 DefaultConfig()
TokenCfg TokenConfig // 实体名替换 token 的生成配置
}
// TokenConfig token 前缀策略:按素材媒体类型(视频/图片/音频)分别配置。
type TokenConfig struct {
// 视频素材前缀模板,如 "video%d"。为空用默认前缀 "video" + 独立编号。
VideoTemplate string
// 图片素材前缀模板,如 "img%d"。为空用默认前缀 "image" + 独立编号。
ImageTemplate string
// 音频素材前缀模板,如 "audio%d"。为空用默认前缀 "audio" + 独立编号。
AudioTemplate string
}
// Refs 参考素材,与 domain/plan 的 Refs 概念一致(自包含定义,不依赖 plan)。
type Refs struct {
Characters []RefItem `json:"characters,omitempty"`
Scenes []RefItem `json:"scenes,omitempty"`
Props []RefItem `json:"props,omitempty"`
Products []RefItem `json:"products,omitempty"`
}
// RefItem 一个具名参考素材。Weight 选择权重(缺省按类别 演员4/场景3/道具2/产品1)。
type RefItem struct {
Name string `json:"name"`
URL string `json:"url"`
Weight int `json:"weight,omitempty"`
}
// Lookup 按类别和名称查找参考图 URL,未找到返回空串。
func (r Refs) Lookup(category, name string) string {
var list []RefItem
switch category {
case catCharacter:
list = r.Characters
case catScene:
list = r.Scenes
case catProp:
list = r.Props
case catProduct:
list = r.Products
}
for _, it := range list {
if it.Name == name {
return it.URL
}
}
return ""
}
// ExtractRefs 递归收集请求参数中散布的参考素材对象({refsName: 名称, value: URL},无固定数组 key)。
// 返回名称列表(去重保序,供转写提示词注入参考素材名单)与具名素材列表(RefItem{Name,URL})。
// 处理器解析与转写节点取素材复用同一入口。
func ExtractRefs(v any) (names []string, items []RefItem) {
seenName := map[string]bool{}
seenItem := map[string]bool{}
var walk func(any)
walk = func(v any) {
switch val := v.(type) {
case map[string]any:
if name := refString(val["refsName"]); name != "" {
if !seenName[name] {
seenName[name] = true
names = append(names, name)
}
url := refString(val["value"])
if url == "" {
url = refString(val["url"])
}
key := name + "\x00" + url
if !seenItem[key] {
seenItem[key] = true
items = append(items, RefItem{Name: name, URL: url})
}
}
for _, child := range val {
walk(child)
}
case []any:
for _, item := range val {
walk(item)
}
}
}
walk(v)
return names, items
}
// refString 从 JSON/gconv 值中取字符串,数字按原样转串(避免 URL 被科学计数法破坏)。
func refString(v any) string {
switch t := v.(type) {
case string:
return strings.TrimSpace(t)
case float64:
return strconv.FormatFloat(t, 'f', -1, 64)
}
return ""
}
// RefBinding token 与参考素材的显式绑定(设计 P0-4)。
type RefBinding struct {
Token string // "video1"(媒体类型前缀 + 独立编号)
Entity string // 原始实体名(归一前)
Category string // 演员/场景/道具/产品
URL string
}
// Segment 一个视频生成段:时间轴、段内镜头、prompt、参考素材。
type Segment struct {
Index int // 段序号(从 0 起)
StartSec int // 段在全局时间轴上的起点(秒)
Duration int // 段时长(秒)
Shots []Shot // 段内镜头(已回写对齐,不跨段)
Prompt string // 实体名→token 替换后的 prompt 文本
Refs []RefBinding // 参考素材显式绑定(token→URL),reference_urls/reference_labels 与 prompt 都由它生成
NegativePrompt string // 本段负面 prompt(默认继承 Input.NegativePrompt,可被段内镜头覆盖)
Seed int64
}
const (
catCharacter = "演员"
catScene = "场景"
catProp = "道具"
catProduct = "产品"
)
// NormalizeInput 补齐时长/阈值零值:总时长<=0 按镜头时间码累加兜底(仍<=0 默认 60),
// MaxSegmentDur<=0 默认 15、MinSegmentDur<=0 默认 5,且 min>max 时收敛 min=max。
func NormalizeInput(in Input) Input {
total := in.TotalDuration
if total <= 0 {
total = sumShotDurations(in.Shots)
}
if total <= 0 {
total = 60
}
maxSeg := in.MaxSegmentDur
if maxSeg <= 0 {
maxSeg = 15
}
minSeg := in.MinSegmentDur
if minSeg <= 0 {
minSeg = 5
}
if minSeg > maxSeg {
minSeg = maxSeg
}
in.TotalDuration = total
in.MaxSegmentDur = maxSeg
in.MinSegmentDur = minSeg
return in
}
// PlanSegments 编排①时间线构建 → ②prompt 构建(含参考素材绑定),产出各段。
// 适配层(workflow 前置处理器)产出 FLAT 请求参数、不产嵌套请求体,直接调用本函数。
func PlanSegments(in Input) ([]Segment, error) {
if len(in.Shots) == 0 {
return nil, pipeErr(ErrEmptyShots, "镜头为空")
}
segShots, segDurs, err := BuildTimeline(in)
if err != nil {
return nil, err
}
var allShots []Shot
for _, ss := range segShots {
allShots = append(allShots, ss...)
}
registry := NewTokenRegistry(allShots, in.Refs, in.TokenCfg)
segs := make([]Segment, 0, len(segShots))
startSec := 0
for i, ss := range segShots {
prompt, segRefs := BuildSegmentPrompt(ss, registry, in)
segs = append(segs, Segment{
Index: i,
StartSec: startSec,
Duration: segDurs[i],
Shots: ss,
Prompt: prompt,
Refs: segRefs,
NegativePrompt: buildSegmentNegativePrompt(ss, in.NegativePrompt),
Seed: in.Seed + int64(i),
})
startSec += segDurs[i]
}
return segs, nil
}
func sumShotDurations(shots []Shot) int {
total := 0
for _, sh := range shots {
total += parseSec(sh.EndTime) - parseSec(sh.StartTime)
}
return total
}
// buildSegmentNegativePrompt 按 §7.5 计算本段负面 prompt:
// 默认继承全局;段内镜头带负面时叠加;镜头负面以 "-" 前缀开头时仅用镜头自身的(抑制全局)。
func buildSegmentNegativePrompt(shots []Shot, global string) string {
var shotNps []string
useOnlyShot := false
for _, sh := range shots {
np := strings.TrimSpace(sh.NegativePrompt)
if np == "" {
continue
}
if strings.HasPrefix(np, "-") {
shotNps = append(shotNps, strings.TrimPrefix(np, "-"))
useOnlyShot = true
} else {
shotNps = append(shotNps, np)
}
}
if len(shotNps) == 0 {
return global
}
var parts []string
if !useOnlyShot && global != "" {
parts = append(parts, global)
}
parts = append(parts, shotNps...)
return strings.Join(parts, "\n")
}
@@ -0,0 +1,478 @@
package pipeline
import (
"fmt"
"sort"
"strconv"
"strings"
"unicode/utf8"
)
// ---------- 实体名归一(四审 P0-1----------
// 素材媒体类型(token 前缀与独立编号基准)。
const (
mediaVideo = "video"
mediaImage = "image"
mediaAudio = "audio"
)
// MediaTypeOf 按 URL 扩展名判定素材媒体类型;无法判定返回 ("", false)。
func MediaTypeOf(url string) (string, bool) {
u := url
if i := strings.IndexAny(u, "?#"); i >= 0 {
u = u[:i]
}
switch strings.ToLower(pathExt(u)) {
case ".mp4", ".mov", ".avi", ".webm", ".mkv":
return mediaVideo, true
case ".jpg", ".jpeg", ".png", ".gif", ".webp", ".bmp":
return mediaImage, true
case ".mp3", ".wav", ".ogg", ".aac", ".m4a", ".flac":
return mediaAudio, true
}
return "", false
}
// pathExt 提取路径最后一段的扩展名(含点),无则返回空串。
func pathExt(url string) string {
for i := len(url) - 1; i >= 0; i-- {
if url[i] == '/' {
break
}
if url[i] == '.' {
return url[i:]
}
}
return ""
}
// normalizeEntityName 归一清洗实体名,返回注册表唯一标准名:
// 去括号注释 → 统一全半角 → 去首尾标点 → 英文小写。
func normalizeEntityName(name string) string {
name = strings.TrimSpace(name)
name = toHalfWidth(name)
name = stripBrackets(name)
name = strings.Trim(name, "。,!?;:、()【】「」《》…,.!?;:\"'` \t\n\r")
return strings.ToLower(name)
}
// toHalfWidth 全角字母数字/符号转半角,全角空格转半角空格。
func toHalfWidth(s string) string {
var b strings.Builder
b.Grow(len(s))
for _, r := range s {
switch {
case r == ' ':
b.WriteRune(' ')
case r >= '' && r <= '':
b.WriteRune(r - 0xfee0)
default:
b.WriteRune(r)
}
}
return b.String()
}
// stripBrackets 剔除括号注释,如 "小红(女主)" → "小红"。
func stripBrackets(s string) string {
for {
next := stripBracketsOnce(s)
if next == s {
return s
}
s = next
}
}
var bracketPairs = []struct{ open, close rune }{
{'(', ')'}, {'', ''}, {'[', ']'}, {'【', '】'}, {'「', '」'}, {'《', '》'},
}
func stripBracketsOnce(s string) string {
rs := []rune(s)
for i := 0; i < len(rs); i++ {
for _, p := range bracketPairs {
if rs[i] != p.open {
continue
}
depth := 1
for j := i + 1; j < len(rs); j++ {
if rs[j] == p.open {
depth++
}
if rs[j] == p.close {
depth--
if depth == 0 {
out := make([]rune, 0, len(rs)-(j-i+1))
out = append(out, rs[:i]...)
out = append(out, rs[j+1:]...)
return string(out)
}
}
}
// 无匹配闭合 → 去掉从此处到末尾
return string(rs[:i])
}
}
return s
}
// ---------- TokenRegistry(全局稳定编号,设计 §6.1 + P2-4----------
type entity struct {
Category string // 演员/场景/道具/产品
Canonical string // 归一化标准名(注册表唯一 key 的一部分)
Aliases []string // 原始异形体(供文本替换匹配)
URL string
Weight int
MediaType string // 素材媒体类型:video/image/audio(注册时按 URL 判定)
order int // 全局首次出现顺序
}
func entKey(ent *entity) string {
return ent.Category + "/" + ent.Canonical
}
// TokenRegistry 一次执行内的全局 token 注册表:同一实体在所有段使用同一 token。
type TokenRegistry struct {
entities map[string]*entity // key = 类别/标准名
order []*entity // 注册顺序(token 编号基准)
tokenByKey map[string]string // key → token
aliasToToken map[string]string // 原始异形体 → token
tc TokenConfig
refs Refs
}
// NewTokenRegistry 遍历全部镜头(+ refs 产品)建立注册表。无 URL 的实体不进注册表(§6.3 保留原名)。
func NewTokenRegistry(allShots []Shot, refs Refs, tc TokenConfig) *TokenRegistry {
reg := &TokenRegistry{
entities: map[string]*entity{},
tokenByKey: map[string]string{},
aliasToToken: map[string]string{},
tc: tc,
refs: refs,
}
for _, sh := range allShots {
for _, c := range sh.Characters {
reg.register(catCharacter, c)
}
if sh.Scene != "" {
reg.register(catScene, sh.Scene)
}
for _, p := range sh.Props {
reg.register(catProp, p)
}
}
// 分析替换业务:产品图常驻注册表
for _, p := range refs.Products {
reg.register(catProduct, p.Name)
}
return reg
}
func (r *TokenRegistry) register(category, raw string) {
raw = strings.TrimSpace(raw)
if raw == "" {
return
}
canonical := normalizeEntityName(raw)
if canonical == "" {
return
}
key := category + "/" + canonical
if ent, ok := r.entities[key]; ok {
r.recordAlias(ent, raw)
return
}
url := r.refs.Lookup(category, raw)
if url == "" {
url = r.refs.Lookup(category, canonical)
}
if url == "" {
return
}
media, ok := MediaTypeOf(url)
if !ok {
return // 无法判定媒体类型:不进注册表,保留原名
}
weight := r.weightOf(category, raw)
if weight <= 0 {
weight = r.weightOf(category, canonical)
}
ent := &entity{
Category: category,
Canonical: canonical,
URL: url,
Weight: weight,
MediaType: media,
order: len(r.order),
}
r.entities[key] = ent
r.tokenByKey[key] = r.nextToken(ent)
r.order = append(r.order, ent)
r.recordAlias(ent, raw)
}
func (r *TokenRegistry) recordAlias(ent *entity, raw string) {
for _, a := range ent.Aliases {
if a == raw {
return
}
}
ent.Aliases = append(ent.Aliases, raw)
r.aliasToToken[raw] = r.tokenByKey[entKey(ent)]
}
func (r *TokenRegistry) nextToken(ent *entity) string {
tpl, base := tokenBase(r.tc, ent.MediaType)
n := 1
for _, e := range r.order {
if e.MediaType == ent.MediaType {
n++
}
}
if tpl != "" {
if strings.Contains(tpl, "%d") {
return fmt.Sprintf(tpl, n)
}
return tpl + strconv.Itoa(n)
}
return fmt.Sprintf("%s%d", base, n)
}
// tokenBase 返回该媒体类型的前置模板(可为空)与默认前缀。
func tokenBase(tc TokenConfig, media string) (tpl, base string) {
switch media {
case mediaVideo:
return tc.VideoTemplate, mediaVideo
case mediaImage:
return tc.ImageTemplate, mediaImage
case mediaAudio:
return tc.AudioTemplate, mediaAudio
}
return "", media
}
func (r *TokenRegistry) entityFor(category, name string) *entity {
canonical := normalizeEntityName(name)
if canonical == "" {
return nil
}
return r.entities[category+"/"+canonical]
}
func (r *TokenRegistry) weightOf(category, name string) int {
if it, ok := r.refs.findItem(category, name); ok && it.Weight > 0 {
return it.Weight
}
switch category {
case catCharacter:
return 4
case catScene:
return 3
case catProp:
return 2
case catProduct:
return 1
}
return 0
}
func (r Refs) findItem(category, name string) (RefItem, bool) {
var list []RefItem
switch category {
case catCharacter:
list = r.Characters
case catScene:
list = r.Scenes
case catProp:
list = r.Props
case catProduct:
list = r.Products
}
for _, it := range list {
if it.Name == name {
return it, true
}
}
return RefItem{}, false
}
// ---------- BuildSegmentPrompt(设计 §6 + P0-4/P2-3/P3-1----------
// BuildSegmentPrompt 构建单段 prompt 与参考素材显式绑定。
// 加权筛选(开口角色必选 + 权重降序)选取 ≤ MaxRefs 个实体;prompt 用选中的 token 替换。
func BuildSegmentPrompt(segShots []Shot, reg *TokenRegistry, in Input) (string, []RefBinding) {
cfg := resolveConfig(in.Cfg, in.MaxSegmentDur)
type cand struct {
ent *entity
appOrder int
}
var cands []cand
seen := map[string]bool{}
speaking := map[string]bool{}
app := 0
add := func(category, name string) {
ent := reg.entityFor(category, name)
if ent == nil {
return
}
key := entKey(ent)
if seen[key] {
return
}
seen[key] = true
cands = append(cands, cand{ent: ent, appOrder: app})
app++
}
for _, sh := range segShots {
for _, c := range sh.Characters {
add(catCharacter, c)
}
if sh.Scene != "" {
add(catScene, sh.Scene)
}
for _, p := range sh.Props {
add(catProp, p)
}
if sh.Dialogue != "" || sh.Narration != "" {
for _, c := range sh.Characters {
if ent := reg.entityFor(catCharacter, c); ent != nil {
speaking[entKey(ent)] = true
}
}
}
}
// 产品图常驻候选
for _, e := range reg.order {
if e.Category == catProduct {
key := entKey(e)
if !seen[key] {
seen[key] = true
cands = append(cands, cand{ent: e, appOrder: app})
app++
}
}
}
// 硬优先级:开口角色;其余按权重降序、同权重按出现顺序
var speakCands, otherCands []cand
for _, c := range cands {
if speaking[entKey(c.ent)] {
speakCands = append(speakCands, c)
} else {
otherCands = append(otherCands, c)
}
}
sort.SliceStable(otherCands, func(i, j int) bool {
if otherCands[i].ent.Weight != otherCands[j].ent.Weight {
return otherCands[i].ent.Weight > otherCands[j].ent.Weight
}
return otherCands[i].appOrder < otherCands[j].appOrder
})
var chosen []*entity
for _, c := range speakCands {
if len(chosen) >= cfg.MaxRefs {
break
}
chosen = append(chosen, c.ent)
}
for _, c := range otherCands {
if len(chosen) >= cfg.MaxRefs {
break
}
chosen = append(chosen, c.ent)
}
// 显式绑定:token→URL 一一对应
segRefs := make([]RefBinding, 0, len(chosen))
for _, e := range chosen {
segRefs = append(segRefs, RefBinding{
Token: reg.tokenByKey[entKey(e)],
Entity: e.Aliases[0],
Category: e.Category,
URL: e.URL,
})
}
// 文本替换:按别名长度降序,命中任一异形体即替换为该 token
prompt := ShotsToPromptText(segShots)
type repl struct {
alias string
token string
}
var repls []repl
for _, e := range chosen {
token := reg.tokenByKey[entKey(e)]
for _, a := range e.Aliases {
repls = append(repls, repl{alias: a, token: token})
}
}
sort.SliceStable(repls, func(i, j int) bool {
return utf8.RuneCountInString(repls[i].alias) > utf8.RuneCountInString(repls[j].alias)
})
for _, r := range repls {
prompt = strings.ReplaceAll(prompt, r.alias, r.token)
}
prompt = truncatePrompt(prompt, cfg)
// 静音模式段级硬约束:放在 truncate 之后追加,避免被截断丢弃;
// 明确告知视频模型本段是无声画面,从 prompt 层面杜绝口播/字幕/口型
if in.NoSpeech {
prompt += "\n\n静音模式:本段为无声画面,禁止人物开口说话、禁止出现字幕与口型动作,只保留纯画面动作、表情、神态与场景变化。"
}
return prompt, segRefs
}
// truncatePrompt 语义保护截断(设计 §6.4 / P3-1):
// 先裁环境音/运镜/景别块,再从尾部最近句子边界截断,保底 MinPromptFloor 字符。
func truncatePrompt(p string, cfg Config) string {
max := cfg.MaxPromptChars
if max <= 0 {
return p
}
if utf8.RuneCountInString(p) <= max {
return p
}
var kept []string
for _, ln := range strings.Split(p, "\n") {
if strings.HasPrefix(ln, "环境音:") || strings.HasPrefix(ln, "运镜:") || strings.HasPrefix(ln, "景别:") {
continue
}
kept = append(kept, ln)
}
p2 := strings.Join(kept, "\n")
if utf8.RuneCountInString(p2) <= max {
return p2
}
rs := []rune(p2)
floor := cfg.MinPromptFloor
if floor <= 0 {
floor = 50
}
if floor >= len(rs) {
floor = len(rs) - 1
}
cut := max
if cut > len(rs) {
cut = len(rs)
}
for i := cut; i > floor; i-- {
if isSentenceEnd(rs[i-1]) {
cut = i
break
}
}
out := strings.TrimRight(string(rs[:cut]), " \t\n")
if out == "" {
out = "…"
} else {
out += "…"
}
return out
}
@@ -0,0 +1,134 @@
// 镜头模型(原 video/domain.Shot,随处理器自包含并入本包):脚本与视频模型之间的统一中间契约。
package pipeline
import (
"encoding/json"
"fmt"
"strings"
)
// Shot 单个镜头。文生视频 / 分析替换两种模式产出的脚本都归一为该结构,
// 后续拆段、prompt 构建、参考素材筛选都以它为准。
type Shot struct {
Index int `json:"index"`
StartTime string `json:"startTime"` // "MM:SS"
EndTime string `json:"endTime"` // "MM:SS"
Event string `json:"event"` // 事件描述/动作描写
Narration string `json:"narration,omitempty"` // 旁白/画外音
Dialogue string `json:"dialogue,omitempty"` // 主台词
AmbientSound string `json:"ambientSound,omitempty"` // 环境音
CameraMovement string `json:"cameraMovement"` // 运镜描述
ShotSize string `json:"shotSize"` // 景别
Characters []string `json:"characters"` // 出演人物名列表
Scene string `json:"scene"` // 场景名
Props []string `json:"props"` // 道具名列表
NegativePrompt string `json:"negativePrompt,omitempty"` // 本镜头负面 prompt(可选,见 video/pipeline §7.5
}
// ShotsToText 将镜头数组转回纯文本格式(带【镜头X】标题),供需要完整脚本文本的场景使用。
func ShotsToText(shots []Shot) string {
var b strings.Builder
if len(shots) == 0 {
return ""
}
for _, s := range shots {
fmt.Fprintf(&b, "【镜头%d】(%s-%s\n", s.Index, s.StartTime, s.EndTime)
writeShotFields(&b, s)
}
return strings.TrimRight(b.String(), "\n")
}
// ShotsToPromptText 将镜头数组转为纯文本格式(不含【镜头X】标题),用于构建视频模型请求的 prompt。
func ShotsToPromptText(shots []Shot) string {
var b strings.Builder
if len(shots) == 0 {
return ""
}
for _, s := range shots {
fmt.Fprintf(&b, "%s-%s\n", s.StartTime, s.EndTime)
writeShotFields(&b, s)
}
return strings.TrimRight(b.String(), "\n")
}
func writeShotFields(b *strings.Builder, s Shot) {
if s.Event != "" {
fmt.Fprintf(b, "事件:%s\n", s.Event)
}
if s.Dialogue != "" {
fmt.Fprintf(b, "台词:%s(角色开口说)\n", s.Dialogue)
}
if s.Narration != "" {
fmt.Fprintf(b, "画外音:%s(角色不开口,仅播放旁白配音)\n", s.Narration)
}
if s.AmbientSound != "" {
fmt.Fprintf(b, "环境音:%s\n", s.AmbientSound)
}
if s.ShotSize != "" {
fmt.Fprintf(b, "景别:%s\n", s.ShotSize)
}
if s.CameraMovement != "" {
fmt.Fprintf(b, "运镜:%s\n", s.CameraMovement)
}
b.WriteString("\n")
}
// ShotsJSONSchema 返回 []Shot 的 JSON Schema,供"脚本转写"节点作为模型 function 定义的 InputSchema
// 让模型以 function calling 形式产出固定结构的镜头数组。
func ShotsJSONSchema() map[string]any {
item := map[string]any{
"type": "object",
"properties": map[string]any{
"index": map[string]any{"type": "integer"},
"startTime": map[string]any{"type": "string", "description": "开始时间,格式 MM:SS"},
"endTime": map[string]any{"type": "string", "description": "结束时间,格式 MM:SS"},
"event": map[string]any{"type": "string", "description": "事件描述/动作描写"},
"narration": map[string]any{"type": "string", "description": "旁白/画外音"},
"dialogue": map[string]any{"type": "string", "description": "主台词"},
"ambientSound": map[string]any{"type": "string", "description": "环境音"},
"cameraMovement": map[string]any{"type": "string", "description": "运镜描述"},
"shotSize": map[string]any{"type": "string", "description": "景别"},
"characters": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "description": "出演人物名列表"},
"scene": map[string]any{"type": "string", "description": "场景名"},
"props": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "description": "道具名列表"},
},
"required": []string{"index", "startTime", "endTime", "event", "cameraMovement", "shotSize", "characters", "scene"},
}
return map[string]any{
"type": "array",
"items": item,
}
}
// ShotsStructuredFormat 返回 []Shot 的原生结构化输出(response_format)值:
// OpenAI 兼容 json_schema 模式,object 包装 {shots:[...]}。模型是否支持由 chat 模型
// RequestBusinessFieldMapping 是否配置 response_format 决定(见 gateway.ModelConfig.StructuredOutput)。
func ShotsStructuredFormat() map[string]any {
return map[string]any{
"type": "json_schema",
"json_schema": map[string]any{
"name": "shots",
"schema": map[string]any{
"type": "object",
"properties": map[string]any{"shots": ShotsJSONSchema()},
"required": []string{"shots"},
},
},
}
}
// IsShotsJSON 判断 script 内容是否为 JSON 镜头数组
func IsShotsJSON(script string) bool {
if len(script) == 0 {
return false
}
trimmed := strings.TrimSpace(script)
if !strings.HasPrefix(trimmed, "[") {
return false
}
var shots []Shot
if err := json.Unmarshal([]byte(trimmed), &shots); err != nil {
return false
}
return len(shots) > 0
}
@@ -0,0 +1,706 @@
package pipeline
import (
"fmt"
"math"
"strconv"
"strings"
"unicode/utf8"
)
// ---------- 时间工具 ----------
func parseSec(t string) int {
parts := strings.Split(t, ":")
if len(parts) != 2 {
return 0
}
m, err1 := strconv.Atoi(parts[0])
s, err2 := strconv.Atoi(parts[1])
if err1 != nil || err2 != nil {
return 0
}
return m*60 + s
}
func mmss(total int) string {
return fmt.Sprintf("%02d:%02d", total/60, total%60)
}
func shotDur(sh Shot) int {
return parseSec(sh.EndTime) - parseSec(sh.StartTime)
}
func absInt(v int) int {
if v < 0 {
return -v
}
return v
}
// ---------- 断句(SplitOversized 与 GroupSegments 共用的语义原语,设计 P0-2/四审 P0-1----------
func isSentenceEnd(r rune) bool {
switch r {
case '。', '', '', '', '\n', '!', '?', '.':
return true
}
return false
}
// sentenceEnds 返回 text 中所有句子结束位置(rune 下标,end-exclusive)。
func sentenceEnds(text string) []int {
var out []int
rs := []rune(text)
for i, r := range rs {
if isSentenceEnd(r) {
out = append(out, i+1)
}
}
return out
}
// findBoundary 在 rune 序列中寻找离 pos 最近(窗口内)的句末断点,返回 end-exclusive 下标。
func findBoundary(rs []rune, pos int, forwardFirst bool, window int) int {
if len(rs) == 0 || pos <= 0 {
return 0
}
if pos >= len(rs) {
return len(rs)
}
if window <= 0 {
window = 60
}
if forwardFirst {
end := pos + window
if end > len(rs) {
end = len(rs)
}
for i := pos; i < end; i++ {
if isSentenceEnd(rs[i]) {
return i + 1
}
}
start := pos - window
if start < 0 {
start = 0
}
for i := pos - 1; i >= start; i-- {
if isSentenceEnd(rs[i]) {
return i + 1
}
}
} else {
start := pos - window
if start < 0 {
start = 0
}
for i := pos - 1; i >= start; i-- {
if isSentenceEnd(rs[i]) {
return i + 1
}
}
end := pos + window
if end > len(rs) {
end = len(rs)
}
for i := pos; i < end; i++ {
if isSentenceEnd(rs[i]) {
return i + 1
}
}
}
return pos
}
// ---------- ① RebuildDurations(尊重 AI 时间码,设计 §5.1 修订 + P1-1 + 四审 P0-3----------
// RebuildDurations 优先保留 AI 的 startTime/endTime:时间码可用且可容纳时按相对时长整体缩放对齐到
// TotalDurationnormalizeAIDurations,整数化 + 均衡修正,构造即 Σ==total);否则回退按文本量重建
// rebuildFromText,旧算法保留)。AI 时间码是模型对镜头节奏的真实意图,文本量重建只作兜底。
func RebuildDurations(in Input) ([]Shot, error) {
shots := append([]Shot(nil), in.Shots...)
if len(shots) == 0 {
return nil, pipeErr(ErrEmptyShots, "镜头为空")
}
total := in.TotalDuration
if total <= 0 {
total = sumShotDurations(shots)
}
if total <= 0 {
total = 60
}
if aiDurationsUsable(shots) && total >= len(shots) {
return normalizeAIDurations(shots, total)
}
return rebuildFromText(shots, total, resolveConfig(in.Cfg, in.MaxSegmentDur))
}
// aiDurationsUsable AI 时间码是否值得保留:所有镜头时间码非空且时长之和 > 0。
// 单镜零/负时长在 normalizeAIDurations 中防御性钳到 1(设计决策:零时长镜头不应出现)。
func aiDurationsUsable(shots []Shot) bool {
sum := 0
for _, sh := range shots {
if strings.TrimSpace(sh.StartTime) == "" || strings.TrimSpace(sh.EndTime) == "" {
return false
}
sum += parseSec(sh.EndTime) - parseSec(sh.StartTime)
}
return sum > 0
}
// normalizeAIDurations 保留 AI 镜头相对时长,整体缩放对齐到目标总时长:
// 各镜时长 d[i] 按 total/Σd 等比缩放并四舍五入整数化,再做整体均衡修正(±1s 逐段抹平)保证
// Σscaled == total 构造即成立;随后按累积偏移重排连续时间码,末镜 endTime 精确对齐 total。
func normalizeAIDurations(shots []Shot, total int) ([]Shot, error) {
n := len(shots)
if n == 0 {
return nil, pipeErr(ErrEmptyShots, "镜头为空")
}
if total <= 0 {
return shots, nil
}
raw := make([]int, n)
rawSum := 0
for i, sh := range shots {
d := parseSec(sh.EndTime) - parseSec(sh.StartTime)
if d < 1 {
d = 1
}
raw[i] = d
rawSum += d
}
if rawSum <= 0 {
return nil, pipeErr(ErrInvalidInput, "AI 时间码时长之和为 0")
}
scaled := make([]int, n)
sum := 0
for i := 0; i < n; i++ {
s := int(math.Round(float64(raw[i]) * float64(total) / float64(rawSum)))
if s < 1 {
s = 1
}
scaled[i] = s
sum += s
}
// 均衡修正:把缩放舍入偏差逐段抹平,保证 Σscaled == total(调用方保证 total >= n 可行)
balance := total - sum
for i := n - 1; i >= 0 && balance != 0; i-- {
if balance > 0 {
scaled[i]++
balance--
} else if scaled[i] > 1 {
scaled[i]--
balance++
}
}
current := 0
for i := 0; i < n; i++ {
shots[i].StartTime = mmss(current)
current += scaled[i]
shots[i].EndTime = mmss(current)
}
if n > 0 {
shots[n-1].EndTime = mmss(total)
}
return shots, nil
}
// rebuildFromText 丢弃 AI 时间码、按文本量重建镜头时长(旧 RebuildDurations 算法):
// 有文字镜头按语速算最小时长,纯视觉镜头以 AI 时长意图为基准,盈余按弹性权重分配,
// 超出目标先压视觉再等比压有声最后截尾;整数化 + 整体均衡修正保证 Σ == total 构造即恒等。
func rebuildFromText(shots []Shot, total int, cfg Config) ([]Shot, error) {
n := len(shots)
base := make([]int, n)
weights := make([]float64, n)
totalBase := 0
for i, sh := range shots {
textLen := utf8.RuneCountInString(sh.Dialogue) + utf8.RuneCountInString(sh.Narration)
aiDur := parseSec(sh.EndTime) - parseSec(sh.StartTime)
if aiDur <= 0 {
aiDur = 1
}
if textLen == 0 {
d := aiDur
if float64(d) < cfg.MinVisualDur {
d = int(cfg.MinVisualDur)
}
if d < 1 {
d = 1
}
base[i] = d
weights[i] = cfg.VisualWeight
} else {
cps := inferCharPerSecond(sh.Dialogue+" "+sh.Narration, cfg)
speaking := math.Ceil(float64(textLen) / cps)
minDur := speaking + cfg.MinDurBuffer
if minDur < 2 {
minDur = 2
}
base[i] = int(minDur)
weights[i] = cfg.SpokenWeight
}
totalBase += base[i]
}
if totalBase <= total {
// 盈余按弹性权重分配(float64),整数化后整体均衡修正
surplus := total - totalBase
var totalWeight float64
for _, w := range weights {
totalWeight += w
}
extra := make([]int, n)
sumExtra := 0
if totalWeight > 0 {
for i := 0; i < n; i++ {
e := int(math.Floor(float64(surplus) * weights[i] / totalWeight))
extra[i] = e
sumExtra += e
}
}
// 均衡修正:把舍入偏差逐段抹平,保证 sum(extra) == surplus
balance := surplus - sumExtra
for i := n - 1; i >= 0 && balance != 0; i-- {
if balance > 0 {
extra[i]++
balance--
} else if extra[i] > 0 {
extra[i]--
balance++
}
}
current := 0
for i := 0; i < n; i++ {
shots[i].StartTime = mmss(current)
current += base[i] + extra[i]
shots[i].EndTime = mmss(current)
}
if n > 0 {
shots[n-1].EndTime = mmss(total)
}
} else {
// 总最小时长超出目标 → 压缩
overshoot := totalBase - total
// 1) 压缩纯视觉镜头
for i := 0; i < n && overshoot > 0; i++ {
if weights[i] >= cfg.VisualWeight && base[i] > 1 {
maxReduce := base[i] - 1
reduce := overshoot
if reduce > maxReduce {
reduce = maxReduce
}
base[i] -= reduce
overshoot -= reduce
}
}
if overshoot > 0 {
// 2) 重建有声镜头底线,等比压缩
reBase := make([]int, n)
newTotal := 0
for i, sh := range shots {
if weights[i] >= cfg.VisualWeight {
reBase[i] = base[i]
} else {
textLen := utf8.RuneCountInString(sh.Dialogue) + utf8.RuneCountInString(sh.Narration)
cps := inferCharPerSecond(sh.Dialogue+" "+sh.Narration, cfg)
speaking := math.Ceil(float64(textLen) / cps)
minDur := speaking + cfg.MinDurBuffer
if minDur < 2 {
minDur = 2
}
reBase[i] = int(minDur)
}
newTotal += reBase[i]
}
if newTotal > total {
ratio := float64(total) / float64(newTotal)
current := 0
keep := n
for i := 0; i < n; i++ {
d := int(float64(reBase[i]) * ratio)
if d < 1 {
d = 1
}
if current+d > total {
d = total - current
}
if d < 1 {
d = 1
}
shots[i].StartTime = mmss(current)
current += d
shots[i].EndTime = mmss(current)
if current >= total {
keep = i + 1
break
}
}
if keep < n {
shots = shots[:keep]
}
if len(shots) > 0 && current < total {
shots[len(shots)-1].EndTime = mmss(total)
}
} else {
current := 0
for i := 0; i < n; i++ {
shots[i].StartTime = mmss(current)
current += reBase[i]
shots[i].EndTime = mmss(current)
}
if current < total && n > 0 {
shots[n-1].EndTime = mmss(total)
}
}
} else {
// 仅压缩视觉镜头就够了
current := 0
for i := 0; i < n; i++ {
shots[i].StartTime = mmss(current)
current += base[i]
shots[i].EndTime = mmss(current)
}
if current < total && n > 0 {
shots[n-1].EndTime = mmss(total)
}
}
}
for i := range shots {
shots[i].Index = i + 1
}
return shots, nil
}
// inferCharPerSecond 根据文本情绪推断语速:感叹/疑问多→快,低落→慢。
func inferCharPerSecond(text string, cfg Config) float64 {
runes := utf8.RuneCountInString(text)
if runes == 0 {
return cfg.CharPerSecond
}
excl := strings.Count(text, "") + strings.Count(text, "!")
ques := strings.Count(text, "") + strings.Count(text, "?")
emotion := excl + ques
if emotion >= 2 && emotion*100/runes >= 8 {
return cfg.FastCharPerSecond
}
if strings.Contains(text, "…") || strings.Contains(text, "唉") {
return cfg.SlowCharPerSecond
}
return cfg.CharPerSecond
}
// ---------- ② SplitOversized(设计 §5.2 + 四审 P0-4----------
// SplitOversized 超 MaxSegmentDur 的镜头按句末断句切子镜头,避免"说话说一半"。
// 每个子镜头按「基础起点 + 子段序号×Max」重算全局 StartTime/EndTime(末段延伸到原终点),
// 保证子镜头全局时间轴连续、无重叠、无缝隙。
func SplitOversized(shots []Shot, in Input) ([]Shot, error) {
maxDur := in.MaxSegmentDur
if maxDur <= 0 || len(shots) == 0 {
return shots, nil
}
cfg := resolveConfig(in.Cfg, maxDur)
window := cfg.SplitWindow
fragMin := int(cfg.ShortFragmentDur)
if fragMin < 1 {
fragMin = 1
}
var result []Shot
for _, sh := range shots {
dur := parseSec(sh.EndTime) - parseSec(sh.StartTime)
if dur <= maxDur {
result = append(result, sh)
continue
}
parts := (dur + maxDur - 1) / maxDur
// 最后一片段 < 短残片阈值时减少拆分段数
remainder := dur - (parts-1)*maxDur
if remainder < fragMin && parts > 1 {
parts--
}
dRunes := []rune(sh.Dialogue)
nRunes := []rune(sh.Narration)
baseStart := parseSec(sh.StartTime)
dPos, nPos := 0, 0
for p := 0; p < parts; p++ {
sub := sh
pStart := baseStart + p*maxDur
pEnd := pStart + maxDur
if p > 0 {
sub.StartTime = mmss(pStart)
}
if p == parts-1 {
pEnd = baseStart + dur
}
sub.EndTime = mmss(pEnd)
if p < parts-1 {
dTarget := len(dRunes) * (p + 1) / parts
if dTarget > dPos && dTarget <= len(dRunes) {
dEnd := findBoundary(dRunes, dTarget, true, window)
if dEnd <= dPos {
dEnd = dPos + 1
}
if dEnd > len(dRunes) {
dEnd = len(dRunes)
}
sub.Dialogue = string(dRunes[dPos:dEnd])
dPos = dEnd
}
nTarget := len(nRunes) * (p + 1) / parts
if nTarget > nPos && nTarget <= len(nRunes) {
nEnd := findBoundary(nRunes, nTarget, true, window)
if nEnd <= nPos {
nEnd = nPos + 1
}
if nEnd > len(nRunes) {
nEnd = len(nRunes)
}
sub.Narration = string(nRunes[nPos:nEnd])
nPos = nEnd
}
} else {
sub.Dialogue = string(dRunes[dPos:])
sub.Narration = string(nRunes[nPos:])
}
result = append(result, sub)
}
}
for i := range result {
result[i].Index = i + 1
}
return result, nil
}
// ---------- ③ GroupSegments2026-08-12 修订,取代 CalcSegmentDurations/AlignSegments----------
// GroupSegments 按镜头边界贪心分组:段内镜头时长之和不超过 MaxSegmentDurmin 为硬约束
// (段不足 MinSegmentDur 时从下一镜头头部按句末断点咬取补齐,模型不能容忍更短的段)。
// 镜头时间码已在 RebuildDurations 归一为连续覆盖 [0,total),分组只做切片、不改写时间码;
// 末段为残余可短于 min(无后续镜头可咬)。前置条件:单镜时长 ≤ MaxSegmentDur(由 SplitOversized 保证)。
func GroupSegments(shots []Shot, in Input) ([][]Shot, []int, error) {
if len(shots) == 0 {
return nil, nil, pipeErr(ErrEmptyShots, "镜头为空")
}
cfg := resolveConfig(in.Cfg, in.MaxSegmentDur)
maxSeg := in.MaxSegmentDur
if maxSeg <= 0 {
maxSeg = 15
}
minSeg := in.MinSegmentDur
remaining := append([]Shot(nil), shots...)
var segs [][]Shot
var durs []int
for len(remaining) > 0 {
var seg []Shot
segDur := 0
// 贪心拉入整镜头,直到加入下一镜会超过 max(首镜总是可入;单镜 ≤ max 由 SplitOversized 保证)
for len(remaining) > 0 {
d := shotDur(remaining[0])
if segDur > 0 && segDur+d > maxSeg {
break
}
seg = append(seg, remaining[0])
segDur += d
remaining = remaining[1:]
}
// min 硬约束:段不足 min 且还有镜头时,从下一镜头头部咬取补齐
if minSeg > 0 && segDur < minSeg && len(remaining) > 0 {
sh := remaining[0]
s := parseSec(sh.StartTime)
e := parseSec(sh.EndTime)
need := minSeg - segDur
// segDur+d > maxSeg ≥ minSeg ⇒ need < shotDur(sh),必走咬取分支
if need > 0 && need < e-s {
cut := s + need
cutRune := -1
if c, r, ok := sentenceCut(sh, cut, cfg); ok && c > s && c < e {
cut, cutRune = c, r
}
// 吸附后钳制:min≈max 且句末断点在 need 点之后时,句末吸附会把段推过 max
//(例 max=10,min=9,segDur=8,断点在 +3s → 段 11s>10)。钳回 s+(maxSeg-segDur) 保 max
//(≥ min 恒成立,因 segDur < minSeg ≤ maxSeg),并回退纯时间切,避免 rune 与钳制后时间码错位。
if maxSeg > segDur && cut > s+(maxSeg-segDur) {
cut = s + (maxSeg - segDur)
cutRune = -1
}
if cut <= s {
cut = s + 1
}
if cut >= e {
cut = e - 1
}
s1, s2 := splitShotAt(sh, cut, cutRune)
seg = append(seg, s1)
segDur += parseSec(s1.EndTime) - parseSec(s1.StartTime)
remaining[0] = s2
} else if need > 0 {
// 防御:整镜补入(理论不可达,见上)
seg = append(seg, sh)
segDur += shotDur(sh)
remaining = remaining[1:]
}
}
segs = append(segs, seg)
durs = append(durs, segDur)
}
return segs, durs, nil
}
// sentenceCut 在镜头内寻找离 target 最近、且位移 ≤ BoundaryTolerance 的句末断点,
// 返回断点时间(秒)与断点 rune 下标(在 "台词\n旁白" 合并串中的位置)。找不到返回 (target,-1,false)。
func sentenceCut(sh Shot, target int, cfg Config) (int, int, bool) {
s := parseSec(sh.StartTime)
e := parseSec(sh.EndTime)
dur := e - s
if dur <= 0 {
return target, -1, false
}
text := sh.Dialogue + "\n" + sh.Narration
rs := []rune(text)
if len(rs) == 0 {
return target, -1, false
}
tol := int(cfg.BoundaryTolerance)
if tol < 0 {
tol = 0
}
best := target
bestRune := -1
bestDist := tol + 1
found := false
for _, p := range sentenceEnds(text) {
t := s + int(math.Round(float64(p)*float64(dur)/float64(len(rs))))
if t < s+1 || t > e-1 {
continue
}
dist := absInt(t - target)
if dist < bestDist {
bestDist = dist
best = t
bestRune = p
found = true
}
}
if !found {
return target, -1, false
}
return best, bestRune, true
}
// splitShotAt 把镜头按时间 cut 切成两个子镜头;cutRune >= 0 时按合并文本 rune 位置切分文本。
func splitShotAt(sh Shot, cut, cutRune int) (Shot, Shot) {
s1, s2 := sh, sh
s1.EndTime = mmss(cut)
s2.StartTime = mmss(cut)
if cutRune >= 0 {
d := []rune(sh.Dialogue)
n := []rune(sh.Narration)
switch {
case cutRune <= len(d):
s1.Dialogue = string(d[:cutRune])
s2.Dialogue = string(d[cutRune:])
s1.Narration = ""
s2.Narration = string(n)
case cutRune <= len(d)+1+len(n):
s1.Dialogue = string(d)
nsplit := cutRune - len(d) - 1
if nsplit < 0 {
nsplit = 0
}
if nsplit > len(n) {
nsplit = len(n)
}
s1.Narration = string(n[:nsplit])
s2.Dialogue = ""
s2.Narration = string(n[nsplit:])
default:
s1.Dialogue = string(d)
s1.Narration = string(n)
s2.Dialogue = ""
s2.Narration = ""
}
}
return s1, s2
}
// BuildTimeline 编排 RebuildDurations → SplitOversized → GroupSegments,收尾做 assertTimeline 不变量校验。
func BuildTimeline(in Input) ([][]Shot, []int, error) {
in = NormalizeInput(in)
if len(in.Shots) == 0 {
return nil, nil, pipeErr(ErrEmptyShots, "镜头为空")
}
shots, err := RebuildDurations(in)
if err != nil {
return nil, nil, err
}
shots, err = SplitOversized(shots, in)
if err != nil {
return nil, nil, err
}
segs, durs, err := GroupSegments(shots, in)
if err != nil {
return nil, nil, err
}
if err := assertTimeline(segs, durs, in.TotalDuration, in.MinSegmentDur, in.MaxSegmentDur, in.Cfg.StrictInvariant); err != nil {
return nil, nil, err
}
return segs, durs, nil
}
// ---------- assertTimeline(设计 §5.5 + P0-1----------
// assertTimeline 校验时间线不变量:
// 段时长总和精确等于 total;每个镜头只属于一个段;段内镜头时间码落在段窗口内且连续无重叠;
// 每段时长 ∈ [min, max]strict=false 时该条仅告警,结构性不变量始终强制)。
func assertTimeline(segShots [][]Shot, segDurs []int, total, minSeg, maxSeg int, strict bool) error {
sum := 0
for _, d := range segDurs {
sum += d
}
if sum != total {
return pipeErr(ErrTimelineInvariant, fmt.Sprintf("段时长总和 %d != TotalDuration %d", sum, total))
}
start := 0
for i, seg := range segShots {
dur := segDurs[i]
if strict && (dur < minSeg || dur > maxSeg) {
return pipeErr(ErrTimelineInvariant, fmt.Sprintf("段%d 时长 %d 不在 [%d,%d] 内", i, dur, minSeg, maxSeg))
}
if len(seg) == 0 {
start += dur
continue
}
prev := start
for _, sh := range seg {
s := parseSec(sh.StartTime)
e := parseSec(sh.EndTime)
if s != prev || e < s {
return pipeErr(ErrTimelineInvariant, fmt.Sprintf("段%d 镜头时间不连续/重叠(镜头%d %s-%s", i, sh.Index, sh.StartTime, sh.EndTime))
}
if s < start || e > start+dur {
return pipeErr(ErrTimelineInvariant, fmt.Sprintf("段%d 镜头%d 时间码超出段窗口", i, sh.Index))
}
prev = e
}
if prev != start+dur {
return pipeErr(ErrTimelineInvariant, fmt.Sprintf("段%d 镜头未覆盖段窗口 [%d,%d)", i, start, start+dur))
}
start += dur
}
return nil
}
@@ -0,0 +1,202 @@
// Package split_shots_pipeline 工作流前置处理器:用 pipeline 时间线算法拆段,产出各段模型请求参数。
// 与旧 split_shots 并存:节点 preTool 指向本处理器即灰度启用新算法(设计 §8.2/§10,开关即"用哪个 preTool")。
//
// 处理器自包含:纯拆段算法随处理器走(pipeline 子包,零外部依赖);唯一 I/O 是按 model_id 查模型网关
// 推导单段时长约束(模型配置是唯一事实来源)。产出与旧链路一致的 FLAT 请求参数——model-gateway 按
// requestBodyMapping/requestBusinessFieldMapping 对 flat 业务参数做字段映射,故不产嵌套请求体(会二次嵌套)。
package split_shots_pipeline
import (
"context"
"fmt"
"sort"
"strconv"
"strings"
"ai-agent/gateway"
"ai-agent/workflow/service/flow/processor"
"ai-agent/workflow/service/flow/processor/builtin/split_shots_pipeline/pipeline"
"github.com/gogf/gf/v2/util/gconv"
)
func init() {
processor.Register(SplitShotsPipelineProcessor())
}
// SplitShotsInput 视频生成模型节点的请求参数(ModelRequestParams)结构,处理器入参即它。
// 只含拆段与透传字段:serial/session_id/callback_url/bgm_urls 不参与拆段(串并行由节点 IsBatchExec、
// session 由 Global.SessionId 提供、合并字段由后置处理器读原始 request),故不入结构。
// 参考素材是散布在请求参数中的 {refsName,value} 对象,parseRefs 递归提取,类别由 categorizeRefs 按名字推断。
type SplitShotsInput struct {
ModelID int64 `json:"model_id"` // 视频模型 ID(走网关 modelCall
Shots []pipeline.Shot `json:"shots"` // 已归一化的镜头脚本
TotalDuration int `json:"total_duration"` // 目标总时长(秒)
FlatRefs []pipeline.RefItem `json:"flat_refs"` // 参考素材(平铺形态,类别由 categorizeRefs 推断)
Seed int64 `json:"seed"` // 随机种子基数,各段 = baseSeed + 段序号
NegativePrompt string `json:"negative_prompt,omitempty"`
NoSpeech bool `json:"no_speech,omitempty"` // 静音模式:透传 pipeline,段级 prompt 追加静音硬约束
}
// SplitShotsPipelineProcessor 新拆段前置处理器。入参 args 即模型请求参数(SplitShotsInput 形状),
// 按 model_id 查模型网关推导单段时长约束 → pipeline 时间线算法拆段 → 输出每段 FLAT 模型请求参数。
func SplitShotsPipelineProcessor() *processor.Processor {
return &processor.Processor{
Name: "split_shots_pipeline",
Description: "按 pipeline 时间线算法拆段,产出各段模型请求参数(与 split_shots 并存,灰度用)",
IsShow: false,
Func: func(ctx context.Context, args map[string]any) (any, error) {
input, err := parseSplitShotsInput(args)
if err != nil {
return nil, err
}
refs := categorizeRefs(input.Shots, input.FlatRefs)
maxSeg, minSeg, err := SegmentBounds(ctx, input.ModelID)
if err != nil {
return nil, err
}
segs, err := pipeline.PlanSegments(pipeline.NormalizeInput(pipeline.Input{
Shots: input.Shots,
TotalDuration: input.TotalDuration,
MaxSegmentDur: maxSeg,
MinSegmentDur: minSeg,
Refs: refs,
Seed: input.Seed,
NegativePrompt: input.NegativePrompt,
NoSpeech: input.NoSpeech,
TokenCfg: pipeline.TokenConfig{},
}))
if err != nil {
return nil, err
}
list := make([]map[string]any, 0, len(segs))
for _, seg := range segs {
list = append(list, segmentParamsMap(input, seg))
}
return list, nil
},
}
}
func parseSplitShotsInput(args map[string]any) (*SplitShotsInput, error) {
if args == nil {
return nil, fmt.Errorf("缺少视频生成请求参数")
}
input := new(SplitShotsInput)
if err := gconv.Struct(args, input); err != nil {
return nil, fmt.Errorf("解析视频生成请求参数失败: %v", err)
}
if input.ModelID <= 0 {
return nil, fmt.Errorf("缺少 model_id")
}
if len(input.Shots) == 0 {
return nil, fmt.Errorf("缺少 shots 镜头脚本")
}
return input, nil
}
// categorizeRefs 按名字在镜头中出现的字段推断平铺参考素材的类别(角色/场景/道具),
// 都命不中归产品(产品常驻注册表,见 pipeline.NewTokenRegistry)。类别只决定 token 前缀与权重,不改变 URL 绑定。
func categorizeRefs(shots []pipeline.Shot, items []pipeline.RefItem) pipeline.Refs {
var refs pipeline.Refs
charNames := map[string]bool{}
sceneNames := map[string]bool{}
propNames := map[string]bool{}
for _, sh := range shots {
for _, c := range sh.Characters {
charNames[strings.TrimSpace(c)] = true
}
if sh.Scene != "" {
sceneNames[strings.TrimSpace(sh.Scene)] = true
}
for _, p := range sh.Props {
propNames[strings.TrimSpace(p)] = true
}
}
for _, it := range items {
name := strings.TrimSpace(it.Name)
switch {
case charNames[name]:
refs.Characters = append(refs.Characters, it)
case sceneNames[name]:
refs.Scenes = append(refs.Scenes, it)
case propNames[name]:
refs.Props = append(refs.Props, it)
default:
refs.Products = append(refs.Products, it)
}
}
return refs
}
// SegmentBounds 查模型网关推导单段时长约束:兜底 max/min = 10/4 秒;模型配置的
// MaxDuration/MinDuration 字段(>0)可覆盖兜底值,min 收敛到 ≤ max。
// 导出供脚本转写节点注入"单镜时长 ≤ max"约束(源头预防超长镜头)。
func SegmentBounds(ctx context.Context, modelID int64) (maxSeg, minSeg int, err error) {
maxSeg, minSeg = 10, 4
info, err := gateway.GetModelInfoById(ctx, &gateway.GetModelInfoByIdReq{ModelId: modelID})
if err != nil {
return 0, 0, fmt.Errorf("获取模型信息失败: %v", err)
}
if info.ModelManage.MaxDuration > 0 {
maxSeg = info.ModelManage.MaxDuration
}
if info.ModelManage.MinDuration > 0 {
minSeg = info.ModelManage.MinDuration
}
if minSeg > maxSeg {
minSeg = maxSeg
}
return maxSeg, minSeg, nil
}
// segmentParamsMap 把一段 pipeline 产出序列化为 FLAT 模型请求参数(对齐旧 plan.SegmentParamsToMap 语义)。
// reference_urls 按 token 编号排序,保证媒体数组顺序与 prompt 中的 token 编号一致(位置识别模型兼容)。
func segmentParamsMap(input *SplitShotsInput, seg pipeline.Segment) map[string]any {
m := map[string]any{
"segment_index": seg.Index,
"prompt": seg.Prompt,
"duration": seg.Duration,
"seed": seg.Seed,
}
if seg.NegativePrompt != "" {
m["negative_prompt"] = seg.NegativePrompt
}
if refs := sortedRefs(seg.Refs); len(refs) > 0 {
urls := make([]string, 0, len(refs))
labels := make(map[string]string, len(refs))
for _, rb := range refs {
if rb.URL == "" {
continue
}
urls = append(urls, rb.URL)
labels[rb.Entity] = rb.Token
}
if len(urls) > 0 {
m["reference_urls"] = urls
m["reference_labels"] = labels
}
}
return m
}
// sortedRefs 按 token 编号(video3→3、image10→10)升序稳定排序,
// 让 reference_urls 顺序尽量贴近 prompt 中 token 的编号顺序。
func sortedRefs(refs []pipeline.RefBinding) []pipeline.RefBinding {
out := append([]pipeline.RefBinding(nil), refs...)
sort.SliceStable(out, func(i, j int) bool {
return tokenNum(out[i].Token) < tokenNum(out[j].Token)
})
return out
}
func tokenNum(tok string) int {
i := 0
for i < len(tok) && (tok[i] < '0' || tok[i] > '9') {
i++
}
n, _ := strconv.Atoi(tok[i:])
return n
}
@@ -0,0 +1,74 @@
// Package processor 工作流节点的前置/后置处理器注册表。
//
// 与模型工具(common/tools)区分:处理器是绑定固定业务场景的处理函数,按名注册与分发,
// 供节点配置(preTool/postTool)引用,不由模型 function calling 调用。
package processor
import (
"context"
"fmt"
"sort"
)
// Processor 工作流前置/后置处理函数。按名注册与分发,供节点配置引用。
type Processor struct {
Name string
Description string
IsShow bool
Func func(ctx context.Context, args map[string]any) (any, error)
}
// registry 处理器注册表
var registry = make(map[string]*Processor)
// Register 注册处理器,同名覆盖
func Register(list ...*Processor) {
for _, p := range list {
if p == nil || p.Name == "" {
continue
}
registry[p.Name] = p
}
}
// Call 按名调用处理器。未知处理器返回错误。
func Call(ctx context.Context, name string, args map[string]any) (any, error) {
if err := ctx.Err(); err != nil {
return nil, err
}
p := registry[name]
if p == nil || p.Func == nil {
return nil, fmt.Errorf("处理器[%s]不存在或未实现", name)
}
return p.Func(ctx, args)
}
func List(ctx context.Context) ([]*Processor, error) {
list := make([]*Processor, 0, len(registry))
for _, t := range registry {
list = append(list, t)
}
sort.Slice(list, func(i, j int) bool { return list[i].Name < list[j].Name })
return list, nil
}
// ProducedKey 前置处理器已完成模型调用、直接产出最终结果时,在返回列表的每个 map 上打的标记键。
// 模型调用节点识别到该标记后跳过模型调用循环,把前置处理器的结果直接交给后置处理器。
// 用于"串行视频生成"这类无法用并发循环表达的前置处理器。
const ProducedKey = "__produced"
// IsProduced 判断前置处理器返回列表是否全部带已产出标记(列表中每个 map 的标记值都须为 true)
func IsProduced(list []map[string]any) bool {
if len(list) == 0 {
return false
}
for _, m := range list {
if m == nil {
return false
}
if v, ok := m[ProducedKey].(bool); !ok || !v {
return false
}
}
return true
}
+232
View File
@@ -0,0 +1,232 @@
package flow
import (
"ai-agent/gateway"
sessionDao "ai-agent/workflow/dao/session"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
"context"
"errors"
"fmt"
"time"
"ai-agent/tools/runner"
"gitea.redpowerfuture.com/red-future/common/tools"
wsCommon "gitea.redpowerfuture.com/red-future/common/websocket"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/util/gconv"
)
func init() {
// 普通对话消息处理器(会话服务器 SessionWsService 在 exec_ws.go 定义)
SessionWsService.OnMessage("agent", handleToolAgent)
SessionWsService.OnMessage("agent_cancel", handleToolAgentCancel)
}
// 工具对话默认系统提示词
const defaultAgentSystemPrompt = "你是一个智能助手,可以调用工具完成任务。请根据任务需要选择合适的工具,参考工具返回结果,最终给出完整回答。"
// 工具对话 ReAct 最大循环步数
const defaultAgentMaxStep = 15
// handleToolAgent 处理工具对话消息:解析 payload 后异步运行 ReAct 循环,逐步推送过程事件。
// 首条消息惰性建会话(复用已存在会话),跑完后把问答/token 写入 exec_chat 落库。
func handleToolAgent(ctx context.Context, conn *wsCommon.WsConnection, payload interface{}) {
var p sessionDto.WebSocketExecChatReq
if err := gconv.Struct(payload, &p); err != nil {
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "参数解析失败", Error: err.Error()})
return
}
if p.Question == "" {
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "内容不能为空", Error: "提问内容不能为空"})
return
}
saveCtx := context.WithoutCancel(ctx)
id := p.Id
if g.IsEmpty(id) {
// 会话落库:前端临时 sessionId 对应已存在会话则复用,否则新建
err := ensureSession(saveCtx, conn.SessionId, p.Question)
if err != nil {
glog.Errorf(ctx, "会话创建失败: %v", err)
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "会话创建失败", Error: err.Error()})
return
}
// 问答落库:
chatId, err := sessionDao.ExecChatDao.Insert(ctx, &sessionDto.CreateExecChatReq{
SessionId: conn.SessionId,
RequestParams: entity.ExecChatRequestParams{Question: p.Question},
})
if err != nil {
glog.Errorf(ctx, "问答创建失败: %v", err)
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "问答创建失败", Error: err.Error()})
return
}
id = chatId
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventRoundStart, Id: chatId})
}
modelTools, err := tools.Default.List(ctx)
if err != nil {
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "工具列表获取失败", Error: err.Error()})
errChat := recordChat(saveCtx, id, "", "工具列表获取失败", err, 0, 0, 0)
if errChat != nil {
glog.Errorf(ctx, "普通对话落库失败: %v", errChat)
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "对话落库失败", Error: errChat.Error()})
}
return
}
systemPrompt := p.SystemPrompt
if systemPrompt == "" {
systemPrompt = defaultAgentSystemPrompt
}
// 支持前端终止:agent 上下文可取消;落库用不带取消的 ctx(保留 request 值),保证终止后 token 仍能记录
agentCtx, agentCancel := context.WithCancel(ctx)
if oldCancel := getToolCancel(conn); oldCancel != nil {
oldCancel()
}
conn.SetMeta("toolCancel", agentCancel)
defer conn.SetMeta("toolCancel", nil)
defer agentCancel()
agent := runner.NewReActAgent(p.ModelId, conn.SessionId, modelTools, systemPrompt, defaultAgentMaxStep)
agent.OnEvent = func(ev runner.ReActEvent) {
pushAgentEvent(conn, ev)
}
start := time.Now()
answer, runErr := agent.Run(agentCtx, p.Question)
duration := int64(time.Since(start).Seconds())
// 前端终止:结果/错误不推前端,仅把已产生的 token 正常落库(友好提示记「用户已终止对话」,不记原始错误)
var errMsg string
terminated := runErr != nil && errors.Is(runErr, context.Canceled)
if terminated {
errMsg = errChatTerminated.Error()
runErr = nil
} else if runErr != nil {
errMsg = "对话运行失败"
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "对话运行失败", Error: runErr.Error()})
}
err = recordChat(saveCtx, id, answer, errMsg, runErr, agent.TotalTokens, agent.TotalCost, duration)
if err != nil {
glog.Errorf(ctx, "普通对话落库失败: %v", err)
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventError, Message: "对话落库失败", Error: err.Error()})
}
pushAgentEvent(conn, runner.ReActEvent{Type: runner.ReActEventAnswer, Answer: answer})
}
// handleToolAgentCancel 终止正在运行的对话(前端停止按钮发送 agent_cancel
func handleToolAgentCancel(ctx context.Context, conn *wsCommon.WsConnection, _ interface{}) {
if cancel := getToolCancel(conn); cancel != nil {
cancel()
}
_ = conn.WriteJSON(&wsCommon.WsPushMsg{Type: "ack", Message: "已终止对话"})
}
// getToolCancel 获取当前 agent 运行的取消函数
func getToolCancel(conn *wsCommon.WsConnection) context.CancelFunc {
cancel, _ := wsCommon.GetMetaT[context.CancelFunc](conn, "toolCancel")
return cancel
}
// errChatTerminated 前端终止对话的错误标记(写入 exec_chat.error_message
var errChatTerminated = errors.New("用户已终止对话")
// recordChat 把一次普通对话写入 exec_chat:答案传 OSS 存 result_file_url
// 友好提示写 error_message,原始错误写 error,token 与费用(模型网关返回的累计 cost)落库
func recordChat(ctx context.Context, id int64, answer string, msg string, runErr error, totalTokens int64, totalCost float64, duration int64) error {
var resultFileUrl string
if runErr == nil && answer != "" {
url, uploadErr := gateway.Upload(ctx, fmt.Sprintf("chat_%v_%d.txt", id, time.Now().UnixMilli()), []byte(answer))
if uploadErr != nil {
glog.Errorf(ctx, "普通对话答案上传OSS失败: %v", uploadErr)
} else {
resultFileUrl = url
}
}
var errorDetail string
if runErr != nil {
errorDetail = runErr.Error()
}
_, err := sessionDao.ExecChatDao.Update(ctx, &sessionDto.UpdateExecChatReq{
Id: id,
Duration: duration,
ResultFileUrl: resultFileUrl,
TotalTokens: int(totalTokens),
TotalFee: totalCost,
ErrorMessage: msg,
Error: errorDetail,
})
if err != nil {
return err
}
// 执行成功:重新执行复用了同一条记录,OmitEmpty 的 Update 会跳过空 error_message/error
// 需显式清空,避免上一次失败的报错残留
if runErr == nil {
if _, err := sessionDao.ExecChatDao.ClearError(ctx, id); err != nil {
glog.Errorf(ctx, "exec_chat 报错信息清空失败: %v", err)
}
}
return nil
}
// pushAgentEvent 把 ReAct 过程事件转为 WS 推送消息
func pushAgentEvent(conn *wsCommon.WsConnection, ev runner.ReActEvent) {
if conn.IsClosed() {
return
}
switch ev.Type {
case runner.ReActEventRoundStart:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "运行开始",
Data: map[string]interface{}{"recordId": ev.Id},
})
case runner.ReActEventModelCall:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "模型思考中",
Data: map[string]interface{}{"step": ev.Step, "maxStep": ev.MaxStep},
})
case runner.ReActEventToolCall:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "调用工具",
Data: map[string]interface{}{"description": ev.Description},
})
case runner.ReActEventToolResult:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "工具返回",
Data: map[string]interface{}{"description": ev.Description},
})
case runner.ReActEventAnswerChunk:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "思考中",
Data: map[string]interface{}{"delta": ev.Delta},
})
case runner.ReActEventReasoningChunk:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "思考中",
Data: map[string]interface{}{"delta": ev.Delta},
})
case runner.ReActEventAnswer:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{
Type: string(ev.Type),
Message: "作答完成",
Data: map[string]interface{}{"answer": ev.Answer},
})
case runner.ReActEventError:
_ = conn.WriteJSON(&wsCommon.WsPushMsg{Type: string(ev.Type), Message: ev.Message, Error: ev.Error})
}
}
+166
View File
@@ -0,0 +1,166 @@
package flow
import (
flowDto "ai-agent/workflow/model/dto/flow"
"regexp"
"strings"
"unicode/utf8"
)
// punctRe 切分/剥离用的中文标点(含顿号、)
var punctRe = regexp.MustCompile(`[,。;!?、]`)
// BuildSubtitles 核心工具:单个sentence生成多条subtitle
func BuildSubtitles(sents *[]flowDto.Sentence) ([]flowDto.Subtitle, error) {
var subtitles []flowDto.Subtitle
for _, sent := range *sents {
// 1. 先按标点把文本拆成多个片段
segList := splitTextByPunct(sent.Text)
if len(segList) == 0 {
continue
}
// 去标点后得到纯净片段(纯空白/纯标点片段跳过)
var cleans []string
for _, seg := range segList {
c := strings.TrimSpace(cleanPunct(seg))
if c != "" {
cleans = append(cleans, c)
}
}
if len(cleans) == 0 || len(sent.Words) == 0 {
continue
}
// 2. 词级文本与句子文本一致时,按词精确对齐取首尾词时间(最准)
if spans, ok := alignAllSegments(sent.Words, cleans); ok {
for i, span := range spans {
subtitles = append(subtitles, flowDto.Subtitle{
Start: sent.Words[span[0]].StartTime,
End: sent.Words[span[1]].EndTime,
Text: cleans[i],
})
}
continue
}
// 3. ASR 词级转写与句子文本不一致时(如 血→谑、数字写法不一),
// 整句回退为按片段字符占比分配时间,避免整句被吞成一条字幕
segWords := allocWordsByProportion(sent.Words, cleans)
for i, ws := range segWords {
if len(ws) == 0 {
continue
}
subtitles = append(subtitles, flowDto.Subtitle{
Start: ws[0].StartTime,
End: ws[len(ws)-1].EndTime,
Text: cleans[i],
})
}
}
return subtitles, nil
}
// splitTextByPunct 按中文标点分割句子,同时保留标点在分段内
// 例如:"这个叫高血压调理方,注意是根源调理不是临时缓解,"
// 会变成:["这个叫高血压调理方,", "注意是根源调理不是临时缓解,"]
func splitTextByPunct(raw string) []string {
// 匹配中文标点并保留在文本中,按标点位置切分
indexes := punctRe.FindAllStringIndex(raw, -1)
if len(indexes) == 0 {
return []string{raw}
}
var res []string
prev := 0
for _, idx := range indexes {
end := idx[1] // 标点的结束位置
seg := raw[prev:end]
res = append(res, seg)
prev = end
}
// 处理最后一段没有标点的文本
if prev < len(raw) {
res = append(res, raw[prev:])
}
return res
}
// cleanPunct 去掉中文标点,得到纯净文本
func cleanPunct(raw string) string {
return punctRe.ReplaceAllString(raw, "")
}
// alignAllSegments 按顺序把各纯净片段与词级文本逐字符对齐(允许个别字符不一致)。
// 全部片段对齐成功且词被完整覆盖时返回各片段对应的词区间,否则 ok=false,
// 由调用方回退到时间占比分配。
func alignAllSegments(words []flowDto.Word, cleans []string) ([][2]int, bool) {
spans := make([][2]int, len(cleans))
wordIdx := 0
for i, seg := range cleans {
start := wordIdx
segRunes := []rune(seg)
s := 0
for wordIdx < len(words) && s < len(segRunes) {
for _, r := range []rune(words[wordIdx].Word) {
if s < len(segRunes) && r == segRunes[s] {
s++
}
}
wordIdx++
}
// 片段文本没被完整匹配,或该片段没吃到任何词 → 无法精确对齐
if s < len(segRunes) || start == wordIdx {
return nil, false
}
spans[i] = [2]int{start, wordIdx - 1}
}
// 有剩余词未被任何片段覆盖,说明对齐失败,避免吞掉剩余时间
if wordIdx < len(words) {
return nil, false
}
return spans, true
}
// allocWordsByProportion 按纯净片段字符占比把整句时间区间切成段,再按时间中点把
// 每个 word 归属到所属片段(对词级转写与句子文本不一致的情况兜底)。
func allocWordsByProportion(words []flowDto.Word, cleans []string) [][]flowDto.Word {
runes := make([]int, len(cleans))
totalChars := 0
for i, c := range cleans {
runes[i] = utf8.RuneCountInString(c)
totalChars += runes[i]
}
sentStart := words[0].StartTime
sentEnd := words[len(words)-1].EndTime
duration := sentEnd - sentStart
if duration < 0 {
duration = 0
}
bounds := make([]float64, len(cleans)+1)
bounds[0] = sentStart
accum := 0.0
for i := range cleans {
if totalChars > 0 {
accum += float64(runes[i]) / float64(totalChars)
}
bounds[i+1] = sentStart + accum*duration
}
segWords := make([][]flowDto.Word, len(cleans))
for _, w := range words {
mid := (w.StartTime + w.EndTime) / 2
idx := 0
for b := 0; b < len(bounds)-1; b++ {
if mid >= bounds[b+1] {
idx = b + 1
}
}
segWords[idx] = append(segWords[idx], w)
}
return segWords
}
@@ -0,0 +1,140 @@
package values
import (
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"context"
"strings"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/util/gconv"
)
// BuildModelRequestBody 从参数定义 + 全局执行上下文构建最终嵌套 JSON 请求体。
func BuildModelRequestBody(params []entity.FlowModelParams, globalParams *flowDto.FlowExecutionInput) (map[string]interface{}, error) {
// 1. 解析引用、过滤空值
resolved := parseMap(params, globalParams)
// 2. 转扁平路径映射
flat := toFlatMap(resolved)
// 3. 引用了脚本转写节点的字段打内部标记 __segment_fields(逗号分隔的扁平路径),
// 供前置处理器 split_segment 按段拆批;__ 前缀内部键由 invokePreTool 统一剥离,不传给模型网关
if seg := segmentFields(resolved, globalParams); len(seg) > 0 {
flat["__segment_fields"] = strings.Join(seg, ",")
}
return flat, nil
}
// segmentFields 收集引用了脚本转写节点的字段扁平路径(分段字段)。
// 脚本转写节点按段产出一份扁平参数列表,下游模型节点单源引用其字段时,
// 值按段序聚合成数组(见 ResolveValueSource 的 scriptTranscribe 分支),需随批拆分。
func segmentFields(resolved []entity.FlowModelParams, globalParams *flowDto.FlowExecutionInput) []string {
if globalParams == nil || globalParams.ConfigMap == nil {
return nil
}
var fields []string
for _, p := range resolved {
if g.IsEmpty(p.Path) || len(p.ValueSource) != 1 {
continue
}
src := p.ValueSource[0]
if nodeConfig := globalParams.ConfigMap[src.NodeId]; nodeConfig != nil &&
nodeConfig.NodeCode == node.NodeTypeScriptTranscribe {
fields = append(fields, flatPath(p.Path))
}
}
return fields
}
// parseMap 解析模型请求参数
func parseMap(data []entity.FlowModelParams, globalParams *flowDto.FlowExecutionInput) []entity.FlowModelParams {
newData := make([]entity.FlowModelParams, 0, len(data))
for _, item := range data {
var d entity.FlowModelParams
d.Path = item.Path
d.Type = item.Type
// 无引用源:直接取静态值
if g.IsEmpty(item.ValueSource) {
if isParamEmpty(item.Value) {
continue
}
d = item
newData = append(newData, d)
continue
}
// 有引用源:单源解析取非空值(模型解析引用其他节点的数组值不做类型化转换,整体传给模型);
// 多源把各源解析出的值拼成 "label: value"(无 label 只拼值),逗号分隔
var value, refsName any
if len(item.ValueSource) > 1 {
text, rn, ok := joinValueSources(globalParams, item.ValueSource, isParamEmpty, nil)
if !ok {
parseMapLogFail(item)
continue
}
value, refsName = text, rn
} else {
src := item.ValueSource[0]
var ok bool
value, refsName, ok = ResolveValueSource(globalParams, src.NodeId, src.Field)
if !ok || isParamEmpty(value) {
parseMapLogFail(item)
continue
}
}
d.Value = value
d.RefsName = gconv.String(refsName)
d.ValueSource = item.ValueSource
newData = append(newData, d)
}
return newData
}
// parseMapLogFail 引用解析失败留日志(单源/多源共用,不静默,便于排查引用丢失)
func parseMapLogFail(item entity.FlowModelParams) {
glog.Debugf(context.Background(), "resolve value source failed, nodeId=%+v path=%s", item.ValueSource, item.Path)
}
// toFlatMap 将解析后的参数列表转为 sjson 可用的扁平路径映射。
// key 为 Pathvalue 为参数值;保留 RefsName 供上层追踪引用来源。
func toFlatMap(params []entity.FlowModelParams) map[string]interface{} {
m := make(map[string]interface{}, len(params))
for _, p := range params {
if g.IsEmpty(p.Path) {
continue
}
m[flatPath(p.Path)] = p.Value
}
return m
}
// flatPath 把数组下标路径转扁平点分路径:a[0].b → a.0.b。
func flatPath(path string) string {
return arrayIndexPath.ReplaceAllString(path, `.$1`)
}
// isParamEmpty 判断参数值是否为"空"。
// 仅 nil、空字符串、空切片/映射视为空;0、false 等零值是合法值,保留。
func isParamEmpty(v interface{}) bool {
if v == nil {
return true
}
switch val := v.(type) {
case string:
return val == ""
case []byte:
return len(val) == 0
case []interface{}:
return len(val) == 0
case map[string]interface{}:
return len(val) == 0
default:
// 数字、布尔、结构体等一律视为非空
return false
}
}
@@ -0,0 +1,254 @@
package values
import (
"ai-agent/workflow/consts/node"
flowDto "ai-agent/workflow/model/dto/flow"
"ai-agent/workflow/model/entity"
"encoding/json"
"regexp"
"strconv"
"strings"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
"github.com/tidwall/gjson"
)
var (
// 匹配 [数字]
regNumIndex = regexp.MustCompile(`\[\d+\]`)
// 匹配 .attrs
regAttrs = regexp.MustCompile(`\.attrs`)
// 匹配带捕获组的数组下标,转扁平点分路径用
arrayIndexPath = regexp.MustCompile(`\[(\d+)\]`)
)
// CleanFieldPath 清理字段路径:移除 .attrs、数字下标转为 .#(gjson 数组通配符)
// 示例:usage.attrs.total_tokens → usage.total_tokens
// 示例:choices.attrs[0].attrs.message.attrs.content → choices.#.message.content
func CleanFieldPath(path string) string {
index := CleanFieldPathReplaceNumIndex(path)
attrs := CleanFieldPathRemoveAttrs(index)
return attrs
}
func CleanFieldPathReplaceNumIndex(path string) string {
// 1. 替换 [数字] 为 [*]
s := regNumIndex.ReplaceAllString(path, `.#`)
return s
}
func CleanFieldPathRemoveAttrs(path string) string {
// 2. 移除所有 .attrs
s := regAttrs.ReplaceAllString(path, "")
return s
}
// ProcessValueSourceRecursive 递归遍历map,同级同时存在value和valueSource则把value设置为"AA"
func ProcessValueSourceRecursive(rawParams map[string]interface{}, globalParams *flowDto.FlowExecutionInput) {
walkMap(rawParams, globalParams)
}
// ResolveValueSource 解析 valueSource {nodeId, field} 引用的实际值。
// 返回 (value, refsName, ok)ok=false 表示引用节点不存在或引用值仍为空。
// - 开始/表单节点:OutputConfig 平铺条目按 field == 引用字段匹配(前端约定以 field 为主,
// 不兼容 path),直接读 entry 的 value / refsName
// - scriptTranscribe 节点:OutputResult 是各段扁平请求参数,按段序收集字段为数组(段位留 nil)
// - 其他节点:读 OutputResult 中引用字段 field 路径对应的值
func ResolveValueSource(global *flowDto.FlowExecutionInput, nodeId, field string) (value any, refsName any, ok bool) {
if global == nil || global.ConfigMap == nil {
return nil, nil, false
}
nodeConfig := global.ConfigMap[nodeId]
if nodeConfig == nil {
return nil, nil, false
}
switch nodeConfig.NodeCode {
case node.NodeTypeStart, node.NodeTypeForm:
for _, output := range nodeConfig.OutputConfig {
if gconv.String(output["field"]) != field {
continue
}
if !g.IsEmpty(output["value"]) {
return output["value"], output["refsName"], true
}
}
case node.NodeTypeScriptTranscribe:
// 脚本转写节点 OutputResult 是各段扁平请求参数(split_shots_pipeline 产出,key 为字面量
// prompt/duration/seed 等),按段序读取 output[field] 收集为数组,供分段模型节点整体引用。
// 每段都占一位(字段缺失/为空留 nil),保证数组与段序对齐,供 split_segment 按段取值。
var list []any
for _, output := range nodeConfig.OutputResult {
list = append(list, output[field])
}
for _, v := range list {
if !g.IsEmpty(v) {
return list, "", true
}
}
default:
// templates 是模型节点在前端配置的静态输出模板,不在 OutputResult 中,需单独取
if field == "templates" {
if !g.IsEmpty(nodeConfig.Templates) {
return nodeConfig.Templates, "", true
}
return nil, nil, false
}
for _, output := range nodeConfig.OutputResult {
// 模型节点输出记录是单 key 的字面量扁平 key(如 "choices.attrs[0].attrs.delta.attrs.content"),
// gjson 会把 . 和 [0] 当结构路径解析,无法命中字面量 key,故先按字面量 key 直接取值;
// 未命中再回退 gjson 路径查询(兼容真正嵌套的输出结构)。
if v, has := output[field]; has {
value = v
} else {
value = gjson.Get(gconv.String(output), field).Value()
}
if !g.IsEmpty(value) {
return value, gjson.Get(gconv.String(output), CleanFieldPath("refsName")).Value(), true
}
}
}
return nil, nil, false
}
// walkMap 递归处理map/数组
func walkMap(data interface{}, globalParams *flowDto.FlowExecutionInput) {
switch v := data.(type) {
case map[string]interface{}:
// 有 valueSource:解析引用节点值
if valueSource, hasSource := v["valueSource"]; hasSource {
sources := new([]entity.ValueSource)
gconv.Structs(valueSource, sources)
// 多个引用源:把各源解析出的值拼成 "label: value"(无 label 只拼值),逗号分隔
if len(*sources) > 1 {
text, refsName, ok := joinValueSources(globalParams, *sources, schemaValueEmpty,
func(src entity.ValueSource, value any) any {
// 引用非模型节点(开始/表单/HTTP/脚本转写等)时,值按当前字段声明的 type 做类型化转换;
// 模型节点值由模型网关处理,复制时不需要转换
if !isModelSourceNode(globalParams, src.NodeId) {
return assignBySchemaType(v, value)
}
return value
})
if ok {
v["value"] = text
if !g.IsEmpty(refsName) {
v["refsName"] = refsName
}
return
}
} else if len(*sources) == 1 {
// 单个引用源:保持旧行为,值按原样赋值(非模型节点按声明 type 转换)
src := (*sources)[0]
value, refsName, ok := ResolveValueSource(globalParams, src.NodeId, src.Field)
if ok && !isModelSourceNode(globalParams, src.NodeId) {
value = assignBySchemaType(v, value)
}
if ok && !schemaValueEmpty(value) {
v["value"] = value
if !g.IsEmpty(refsName) {
v["refsName"] = refsName
}
return
}
}
}
// 统一兜底:无 valueSource(或解析失败/值为空)时,value 为空或 0 则取 defaultValue
if defaultValue, hasDefault := v["defaultValue"]; hasDefault && isEmptyForFallback(v["value"]) && !schemaValueEmpty(defaultValue) {
v["value"] = defaultValue
}
// 递归遍历所有子元素
for _, child := range v {
walkMap(child, globalParams)
}
case []interface{}:
// 数组遍历
for _, item := range v {
walkMap(item, globalParams)
}
}
}
// isEmptyForFallback 兜底场景判空:除 schemaValueEmpty 规则外,数字 0 也视为未填写,
// 便于配置了 defaultValue 的字段在值为 0 时用默认值兜底。
// 覆盖 json.Numbergconv 反序列化数字的运行时类型)与字符串 "0"/"0.0"。
func isEmptyForFallback(v interface{}) bool {
if schemaValueEmpty(v) {
return true
}
switch val := v.(type) {
case float32:
return val == 0
case float64:
return val == 0
case int:
return val == 0
case int8:
return val == 0
case int16:
return val == 0
case int32:
return val == 0
case int64:
return val == 0
case uint:
return val == 0
case uint8:
return val == 0
case uint16:
return val == 0
case uint32:
return val == 0
case uint64:
return val == 0
case json.Number:
if f, err := val.Float64(); err == nil {
return f == 0
}
case string:
if f, err := strconv.ParseFloat(val, 64); err == nil {
return f == 0
}
}
return false
}
// isModelSourceNode 判断引用源节点是否为模型节点(值由模型网关处理,复制时不转换)
func isModelSourceNode(global *flowDto.FlowExecutionInput, nodeId string) bool {
if global == nil || global.ConfigMap == nil {
return false
}
nodeConfig := global.ConfigMap[nodeId]
return nodeConfig != nil && nodeConfig.NodeCode == node.NodeTypeModel
}
// joinValueSources 拼接多个 valueSource 的解析值为 "label: value"(无 label 只拼值),逗号分隔。
// 返回拼接文本与第一个非空 refsName;所有源都为空/解析失败时 ok=false。
// isEmpty 为各场景的空值判断(walkMap 用 schemaValueEmpty,模型请求解析用 isParamEmpty);
// transform 对每个非空解析值做转换(walkMap 按声明 type 类型化、模型源不转换;模型请求场景传 nil)。
func joinValueSources(global *flowDto.FlowExecutionInput, sources []entity.ValueSource,
isEmpty func(any) bool, transform func(src entity.ValueSource, value any) any) (text string, refsName any, ok bool) {
var parts []string
for _, src := range sources {
value, rn, ok := ResolveValueSource(global, src.NodeId, src.Field)
if !ok || isEmpty(value) {
continue
}
if transform != nil {
value = transform(src, value)
}
s := toPlainString(value)
if src.Label != "" {
s = src.Label + ": " + s
}
parts = append(parts, s)
if !g.IsEmpty(rn) && g.IsEmpty(refsName) {
refsName = rn
}
}
if len(parts) == 0 {
return "", nil, false
}
return strings.Join(parts, ", "), refsName, true
}
@@ -0,0 +1,227 @@
package values
import (
"encoding/json"
"reflect"
"strconv"
"strings"
"github.com/gogf/gf/v2/util/gconv"
)
// UnwrapSchemaWrapper 递归剥掉 json-schema-editor 输出的 {type, value/attrs} 包裹层,
// 只保留干净的 key/value 嵌套结构。
// 示例:
//
// {"a": {"type":"string","value":"hi"}} → {"a": "hi"}
// {"b": {"type":"object","attrs":{"c":1}}} → {"b": {"c": 1}}
// {"arr": {"type":"array","attrs":[{"type":"number","value":1}]}} → {"arr": [1]}
func UnwrapSchemaWrapper(v any) any {
switch val := v.(type) {
case map[string]any:
// 识别包裹节点:{type: "<jsonType>", value/attrs: <实际值>, ...}
if t, ok := val["type"].(string); ok && isSchemaEditorType(t) {
dataKey := "value"
if t == "object" || t == "array" {
dataKey = "attrs"
}
if raw, has := val[dataKey]; has {
return UnwrapSchemaWrapper(raw)
}
}
res := make(map[string]any, len(val))
for k, child := range val {
res[k] = UnwrapSchemaWrapper(child)
}
return res
case []any:
res := make([]any, len(val))
for i, item := range val {
res[i] = UnwrapSchemaWrapper(item)
}
return res
default:
return val
}
}
// isSchemaEditorType 是否为 json-schema-editor 的 6 种类型标识
func isSchemaEditorType(t string) bool {
switch t {
case "string", "number", "boolean", "null", "object", "array":
return true
}
return false
}
// MapResultByTemplate 按 template 定义的结构,从 source 中拷贝对应字段的值。
// 只保留 template 里出现的字段:对象字段按同名字段递归拷贝,数组字段按模板元素结构逐元素过滤,标量字段直接拷贝 source 的值。
func MapResultByTemplate(template map[string]any, source map[string]any) map[string]any {
result := make(map[string]any, len(template))
for key, tmplVal := range template {
srcVal, ok := source[key]
if !ok {
continue
}
if tmplMap, isMap := tmplVal.(map[string]any); isMap {
if srcMap, isMap := srcVal.(map[string]any); isMap {
result[key] = MapResultByTemplate(tmplMap, srcMap)
}
continue
}
if tmplArr, isArr := tmplVal.([]any); isArr {
result[key] = mapTemplateArray(tmplArr, srcVal)
continue
}
result[key] = srcVal
}
return result
}
// mapTemplateArray 按模板数组的元素结构映射 source 数组:
// 模板首元素为对象时,逐元素按 MapResultByTemplate 过滤只保留模板字段;
// 模板数组为空或首元素非对象(无法确定元素结构)时,原样拷贝 source 数组。
func mapTemplateArray(tmplArr []any, srcVal any) any {
srcList, ok := srcVal.([]any)
if !ok || len(tmplArr) == 0 {
return srcVal
}
elemTmpl, ok := tmplArr[0].(map[string]any)
if !ok {
return srcVal
}
result := make([]any, 0, len(srcList))
for _, srcElem := range srcList {
if srcMap, isMap := srcElem.(map[string]any); isMap {
result = append(result, MapResultByTemplate(elemTmpl, srcMap))
} else {
result = append(result, srcElem)
}
}
return result
}
// assignBySchemaType 按当前字段声明的 schema 类型把值类型化:
// string 遇数组/对象转 JSON 字符串;number/boolean 解析字符串;object/array 解析 JSON 字符串;其余原样返回
func assignBySchemaType(node map[string]interface{}, value any) any {
t, _ := node["type"].(string)
return assignByType(t, value)
}
// assignByType 按字段声明的 type 把值类型化;walkMap 的 schema 节点与 parseMap 的模型参数共用
func assignByType(t string, value any) any {
switch t {
case "string":
return toSchemaString(value)
case "number":
return toSchemaNumber(value)
case "boolean":
return toSchemaBool(value)
case "object", "array":
return toSchemaStruct(value)
default:
return value
}
}
// toSchemaString 转 string:字符串原样,数组元素拼成字符串(单元素取元素本身,多元素逗号连接),对象序列化为 JSON 字符串
func toSchemaString(v any) any {
switch val := v.(type) {
case []interface{}:
parts := make([]string, 0, len(val))
for _, item := range val {
parts = append(parts, toPlainString(item))
}
return strings.Join(parts, ",")
case map[string]interface{}:
if b, err := json.Marshal(val); err == nil {
return string(b)
}
}
return v
}
// toPlainString 把数组元素转成不带括号的纯字符串:
// 数组([]any / 类型化切片)逐元素取纯字符串,单元素取元素本身,多元素逗号连接;
// 对象序列化为 JSON 字符串;其余原样字符串化。
func toPlainString(v any) string {
if s, ok := v.(string); ok {
return s
}
switch val := v.(type) {
case []interface{}:
parts := make([]string, 0, len(val))
for _, item := range val {
parts = append(parts, toPlainString(item))
}
return strings.Join(parts, ",")
case map[string]interface{}:
if b, err := json.Marshal(val); err == nil {
return string(b)
}
}
rv := reflect.ValueOf(v)
if rv.IsValid() && (rv.Kind() == reflect.Slice || rv.Kind() == reflect.Array) {
parts := make([]string, 0, rv.Len())
for i := 0; i < rv.Len(); i++ {
parts = append(parts, toPlainString(rv.Index(i).Interface()))
}
return strings.Join(parts, ",")
}
if b, err := json.Marshal(v); err == nil {
return string(b)
}
return gconv.String(v)
}
// toSchemaNumber 转 number:数字原样,字符串尝试解析为 float64,失败原样返回
func toSchemaNumber(v any) any {
if s, ok := v.(string); ok {
if f, err := strconv.ParseFloat(s, 64); err == nil {
return f
}
}
return v
}
// toSchemaBool 转 boolean:布尔原样,字符串尝试解析为 bool,失败原样返回
func toSchemaBool(v any) any {
if s, ok := v.(string); ok {
if b, err := strconv.ParseBool(s); err == nil {
return b
}
}
return v
}
// toSchemaStruct 转 object/array:合法 JSON 字符串解析为结构化数据,否则原样返回
func toSchemaStruct(v any) any {
s, ok := v.(string)
if !ok {
return v
}
if !json.Valid([]byte(s)) {
return v
}
var out any
if err := json.Unmarshal([]byte(s), &out); err != nil {
return v
}
return out
}
// schemaValueEmpty 值是否为空;0/false 视为有效值不剔除
func schemaValueEmpty(v interface{}) bool {
switch val := v.(type) {
case nil:
return true
case string:
return val == ""
case []interface{}:
return len(val) == 0
case map[string]interface{}:
return len(val) == 0
default:
return false
}
}
+106 -364
View File
@@ -1,14 +1,15 @@
package node
import (
"context"
"strconv"
"ai-agent/gateway"
"ai-agent/workflow/consts/model"
"ai-agent/workflow/consts/node"
nodeDto "ai-agent/workflow/model/dto/node"
"context"
"fmt"
"ai-agent/workflow/service/flow/processor"
"gitea.redpowerfuture.com/red-future/common/beans"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"github.com/gogf/gf/v2/encoding/gjson"
"github.com/gogf/gf/v2/frame/g"
)
@@ -16,370 +17,111 @@ var NodeLibraryService = &nodeLibraryService{}
type nodeLibraryService struct{}
func GetModelType(ctx context.Context) (mainTypeMap map[int]string, err error) {
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
res := new(nodeDto.ModelTypeResponse)
err = commonHttp.Get(ctx, "model-gateway/model/listType", headers, res, nil)
// 通用过滤:只保留 能被 100 整除的主类型(100/200/300...
mainTypeMap = make(map[int]string)
for typ, name := range res.Type {
if typ%100 == 0 {
mainTypeMap[typ] = name
}
}
return
}
func (s *nodeLibraryService) GetNodeLibrary(ctx context.Context, req *nodeDto.WorkflowNodeTreeReq) (*nodeDto.WorkflowNodeTreeRes, error) {
WorkflowNodeGroups := []node.NodeGroupItem{
{
Group: node.NodeGroupComponent,
Label: node.NodeGroupNameComponent,
Items: []node.NodeItem{
{
NodeCode: node.NodeTypeTextModel,
NodeName: node.NodeNameTextModel,
ModelType: node.ModelTypeText,
SkillOption: false,
PromptOption: true,
IsSaveFile: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeImageModel,
NodeName: node.NodeNameImageModel,
ModelType: node.ModelTypeImage,
SkillOption: false,
PromptOption: true,
IsSaveFile: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeVideoModel,
NodeName: node.NodeNameVideoModel,
ModelType: node.ModelTypeVideo,
SkillOption: false,
PromptOption: true,
IsSaveFile: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeAudioModel,
NodeName: node.NodeNameAudioModel,
ModelType: node.ModelTypeAudio,
SkillOption: false,
PromptOption: true,
IsSaveFile: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeBatchModel,
NodeName: node.NodeNameBatchModel,
ModelType: node.ModelTypeText,
SkillOption: false,
PromptOption: true,
IsSaveFile: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
},
},
{
Group: node.NodeGroupBase,
Label: node.NodeGroupNameBase,
Items: []node.NodeItem{
{
NodeCode: node.NodeTypeSubFlow,
NodeName: node.NodeSubFlow,
SkillOption: false,
PromptOption: false,
IsSaveFile: false,
FormConfig: []node.NodeFormField{
{Field: "maxConcurrency", Label: "最大并发数", Type: "input", Required: true},
},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeDataConversionModel,
NodeName: node.NodeNameDataConversionModel,
ModelType: node.ModelTypeText,
SkillOption: false,
PromptOption: true,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeMerge,
NodeName: node.NodeNameMerge,
SkillOption: false,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeDataMerge,
NodeName: node.NodeNameDataMerge,
SkillOption: false,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeJudge,
NodeName: node.NodeNameJudge,
SkillOption: false,
FormConfig: []node.NodeFormField{
{Field: "condition", Label: node.FormLabelCondition, Type: "input", Required: true},
},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeForm,
NodeName: node.NodeNameForm,
SkillOption: false,
FormConfig: []node.NodeFormField{},
ModelConfig: []node.ModelItem{},
},
{
NodeCode: node.NodeTypeHttp,
NodeName: node.NodeNameHttp,
SkillOption: false,
IsSaveFile: true,
FormConfig: []node.NodeFormField{
{
Field: "method",
Label: "请求方式",
Type: "select",
Required: true,
Options: []node.SelectOption{
{Label: "GET", Value: "GET"},
{Label: "POST", Value: "POST"},
{Label: "PUT", Value: "PUT"},
{Label: "DELETE", Value: "DELETE"},
},
},
{
Field: "url",
Label: "请求地址",
Type: "input",
Required: true,
},
{
Field: "headers",
Label: "请求头(支持Authorization鉴权)",
Type: "keyValue",
Required: false,
},
{
Field: "bodyType",
Label: "请求体类型",
Type: "select",
Required: true,
Options: []node.SelectOption{
{Label: "无", Value: "None"},
{Label: "JSON", Value: "JSON"},
//{Label: "表单", Value: "FormUrlEncoded"},
//{Label: "文件上传", Value: "FormData"},
//{Label: "原生文本", Value: "Raw"},
},
},
{
Field: "body",
Label: "请求体内容",
Type: "keyValue",
Required: false,
},
{
Field: "response",
Label: "结果返回结构",
Type: "keyValue",
Required: false,
},
{
Field: "responseType",
Label: "结果返回方式",
Type: "select",
Required: true,
Options: []node.SelectOption{
{Label: "同步返回", Value: "sync"},
{Label: "等候回调", Value: "callback"},
{Label: "主动拉取", Value: "pull"},
},
Expand: []node.NodeFormField{
{
Field: "method",
Label: "请求方式",
Type: "select",
Required: true,
Options: []node.SelectOption{
{Label: "GET", Value: "GET"},
{Label: "POST", Value: "POST"},
{Label: "PUT", Value: "PUT"},
{Label: "DELETE", Value: "DELETE"},
},
},
{
Field: "url",
Label: "请求地址",
Type: "input",
Required: true,
},
{
Field: "headers",
Label: "请求头(支持Authorization鉴权)",
Type: "keyValue",
Required: false,
},
{
Field: "bodyType",
Label: "请求体类型",
Type: "select",
Required: true,
Options: []node.SelectOption{
{Label: "无", Value: "None"},
{Label: "JSON", Value: "JSON"},
//{Label: "表单", Value: "FormUrlEncoded"},
//{Label: "文件上传", Value: "FormData"},
//{Label: "原生文本", Value: "Raw"},
},
},
{
Field: "body",
Label: "请求体内容",
Type: "keyValue",
Required: false,
},
{
Field: "response",
Label: "结果返回结构",
Type: "keyValue",
Required: false,
},
{
Field: "timeout",
Label: "超时时间(秒)",
Type: "inputNumber",
Required: false,
Default: 30,
},
{
Field: "insecureSkipVerify",
Label: "跳过HTTPS证书校验",
Type: "switch",
Required: false,
Default: false,
},
},
},
{
Field: "callbackUrl",
Label: "回调地址(只需要填写字段名称)",
Type: "input",
Required: false,
Default: "",
},
{
Field: "timeout",
Label: "超时时间(秒)",
Type: "inputNumber",
Required: false,
Default: 30,
},
{
Field: "insecureSkipVerify",
Label: "跳过HTTPS证书校验",
Type: "switch",
Required: false,
Default: false,
},
},
ModelConfig: []node.ModelItem{},
},
//{
// NodeCode: node.NodeTypeModel,
// NodeName: node.NodeNameModel,
// SkillOption: true,
// FormConfig: []node.NodeFormField{},
// ModelConfig: []node.ModelItem{},
//},
},
},
//{
// Group: node.NodeGroupCustom,
// Label: node.NodeGroupNameCustom,
// Items: []node.NodeItem{
// {
// NodeCode: node.NodeTypeCustomNode,
// NodeName: node.NodeNameCustomNode,
// SkillOption: true,
// FormConfig: []node.NodeFormField{
// {Field: "nodeName", Label: node.FormLabelApiKey, Type: "input", Required: true},
// {Field: "nodeType", Label: node.FormLabelModel, Type: "input", Required: true},
// },
// ModelConfig: []node.ModelItem{},
// },
// },
//},
tree := node.GetFilterNodeTree([]node.NodeGroup{node.NodeGroupBase})
if opts, err := videoModelOptions(ctx); err != nil {
g.Log().Warningf(ctx, "加载视频模型选项失败,节点库降级返回: %v", err)
} else {
applyVideoModelOptions(tree, opts)
}
tree := &nodeDto.WorkflowNodeTreeRes{
Groups: WorkflowNodeGroups,
// 前置/后置方法下拉填充处理器注册表数据
if opts, err := processorOptions(ctx); err != nil {
g.Log().Warningf(ctx, "加载工作流前后置处理器选项失败,节点库降级返回: %v", err)
} else {
applyProcessorOptions(tree, opts)
}
// 3. 遍历分组,根据 typeId=1 给【文本模型节点】追加固定表单
for gIdx := range tree.Groups {
group := &tree.Groups[gIdx]
// 遍历分组下的每个节点
for itemIdx := range group.Items {
item := &group.Items[itemIdx]
if item.NodeCode == node.NodeTypeTextModel ||
item.NodeCode == node.NodeTypeImageModel ||
item.NodeCode == node.NodeTypeVideoModel ||
item.NodeCode == node.NodeTypeAudioModel ||
item.NodeCode == node.NodeTypeBatchModel ||
item.NodeCode == node.NodeTypeDataConversionModel {
item.ModelConfig = append(item.ModelConfig, node.ModelItem{
ModelName: "自定义",
})
}
}
}
return tree, nil
return &nodeDto.WorkflowNodeTreeRes{
Groups: tree,
}, nil
}
// SetUserInfo 设置用户信息
func (s *nodeLibraryService) SetUserInfo(ctx context.Context, creator string, tenantId uint64) (headers map[string]string, err error) {
// 创建完整的用户信息
userInfo := &beans.User{
UserName: creator,
TenantId: tenantId,
}
ctx = context.WithValue(ctx, "user", *userInfo)
// 提取并保存请求头(在连接升级前)
headers = make(map[string]string)
// 提取其他headers
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
// 将完整用户信息序列化为JSON,放到X-User-Info请求头
userInfoJson, err := gjson.Encode(userInfo)
// processorOptions 从处理器注册表构建前置/后置方法下拉选项:key=处理器名,value=描述(无描述回落名称)。
func processorOptions(ctx context.Context) ([]node.SelectOption, error) {
list, err := processor.List(ctx)
if err != nil {
return nil, fmt.Errorf("用户信息序列化失败: %w", err)
return nil, err
}
opts := make([]node.SelectOption, 0, len(list))
for _, p := range list {
if p == nil || p.Name == "" || !p.IsShow {
continue
}
value := p.Description
if value == "" {
value = p.Name
}
opts = append(opts, node.SelectOption{
Key: p.Name,
Value: value,
})
}
return opts, nil
}
// applyProcessorOptions 把处理器选项填充进模型节点的 preTool/postTool 下拉。
// 深拷贝 PreToolOption/PostToolOption 后再改,避免改写全局 NodeTypeMetaList。
func applyProcessorOptions(tree []node.NodeGroupTree, opts []node.SelectOption) {
for gi := range tree {
for ni := range tree[gi].Nodes {
n := &tree[gi].Nodes[ni]
if n.Key != node.NodeTypeModel {
continue
}
n.PreToolOption = withProcessorOptions(n.PreToolOption, opts)
n.PostToolOption = withProcessorOptions(n.PostToolOption, opts)
return
}
}
}
// withProcessorOptions 返回 preTool/postTool 字段列表的深拷贝,并把 select 类型字段的 Options 替换为处理器选项。
func withProcessorOptions(fields []node.NodePresetField, opts []node.SelectOption) []node.NodePresetField {
out := append([]node.NodePresetField(nil), fields...)
for i := range out {
if out[i].Type == "select" {
out[i].Options = opts
}
}
return out
}
// videoModelOptions 按视频模型类型(600)查模型网关,转成 modelId 下拉选项:key=模型IDvalue=模型名称。
func videoModelOptions(ctx context.Context) ([]node.SelectOption, error) {
res, err := gateway.ListModelManage(ctx, &gateway.ListModelManageReq{ModelType: model.TypeVideo})
if err != nil {
return nil, err
}
opts := make([]node.SelectOption, 0, len(res.List))
for _, item := range res.List {
if item == nil || item.Id <= 0 || item.ModelName == "" {
continue
}
opts = append(opts, node.SelectOption{
Key: strconv.FormatInt(item.Id, 10),
Value: item.ModelName,
})
}
return opts, nil
}
// applyVideoModelOptions 把 script_transcribe 节点的 modelId 预设下拉替换为视频模型选项。
// 深拷贝 PresetOption 后再改,避免改写全局 NodeTypeMetaList。
func applyVideoModelOptions(tree []node.NodeGroupTree, opts []node.SelectOption) {
for gi := range tree {
for ni := range tree[gi].Nodes {
n := &tree[gi].Nodes[ni]
if n.Key != node.NodeTypeScriptTranscribe {
continue
}
presets := append([]node.NodePresetField(nil), n.PresetOption...)
for pi := range presets {
if presets[pi].Field == "modelId" {
presets[pi].Options = opts
break
}
}
n.PresetOption = presets
return
}
}
headers["X-User-Info"] = string(userInfoJson)
return
}
+345
View File
@@ -0,0 +1,345 @@
package session
import (
"ai-agent/gateway"
"ai-agent/workflow/consts/flow"
flowDao "ai-agent/workflow/dao/flow"
sessionDao "ai-agent/workflow/dao/session"
flowDto "ai-agent/workflow/model/dto/flow"
sessionDto "ai-agent/workflow/model/dto/session"
"ai-agent/workflow/model/entity"
flowService "ai-agent/workflow/service/flow"
"context"
"fmt"
"sort"
"strings"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/os/glog"
"github.com/gogf/gf/v2/util/gconv"
)
var SessionService = &sessionService{}
type sessionService struct{}
// 结果状态(与 workflow_session_result.status / VOSessionInfoResult.Status 一致)
const (
resultStatusSuccess = 2
resultStatusFailed = 3
resultStatusCancel = 4
)
func (s *sessionService) List(ctx context.Context, req *sessionDto.ListSessionReq) (res *sessionDto.ListSessionRes, err error) {
user, err := utils.GetUserInfo(ctx)
if err != nil {
return
}
var page *beans.Page
if req.PageSize > 0 {
page = &beans.Page{PageNum: req.PageNum, PageSize: req.PageSize}
}
list, total, err := sessionDao.SessionDao.List(ctx, user.UserName, page)
if err != nil {
return
}
res = &sessionDto.ListSessionRes{Total: total}
for _, item := range list {
res.List = append(res.List, &sessionDto.VOSession{
SessionId: item.SessionId,
SessionName: item.SessionName,
CreatedAt: item.CreatedAt,
})
}
return
}
func (s *sessionService) Delete(ctx context.Context, req *sessionDto.DeleteSessionReq) (err error) {
_, err = sessionDao.SessionDao.Delete(ctx, req)
return
}
func (s *sessionService) DeleteRecord(ctx context.Context, req *sessionDto.DeleteSessionRecordReq) (err error) {
var chatIds, wfIds []int64
for _, item := range req.Ids {
if item.Type == "chat" {
chatIds = append(chatIds, item.Id)
} else {
wfIds = append(wfIds, item.Id)
}
}
if len(chatIds) > 0 {
if _, e := sessionDao.ExecChatDao.Delete(ctx, &sessionDto.DeleteExecChatReq{Id: chatIds}); e != nil {
return e
}
}
if len(wfIds) > 0 {
if _, e := sessionDao.ExecWorkflowDao.Delete(ctx, &sessionDto.DeleteExecWorkflowReq{Id: wfIds}); e != nil {
return e
}
}
return
}
// Get 会话内结果:普通对话 + 工作流执行混排,按创建时间倒序,分页
func (s *sessionService) Get(ctx context.Context, req *sessionDto.GetSessionInfoReq) (res *sessionDto.GetSessionInfoRes, err error) {
chatList, err := sessionDao.ExecChatDao.ListBySession(ctx, req.SessionId)
if err != nil {
return
}
wfList, err := sessionDao.ExecWorkflowDao.ListBySession(ctx, req.SessionId)
if err != nil {
return
}
wfResultList, err := sessionDao.ExecWorkflowResultDao.ListBySession(ctx, req.SessionId)
if err != nil {
return
}
prefix, _ := oss.GetFileAddressPrefix(ctx)
// 工作流结果按 exec_id 分组,合并到对应执行记录的结果文件URL
resultByExec := make(map[int64][]string)
for _, wr := range wfResultList {
if wr.ResultFileUrl != "" {
resultByExec[wr.ExecId] = append(resultByExec[wr.ExecId], prefix+wr.ResultFileUrl)
}
}
res = &sessionDto.GetSessionInfoRes{}
for _, c := range chatList {
c.ResultFileUrl = prefix + c.ResultFileUrl
res.List = append(res.List, chatExecVO(c))
}
for _, w := range wfList {
res.List = append(res.List, workflowExecVO(w, strings.Join(resultByExec[w.Id], ",")))
}
sort.Slice(res.List, func(i, j int) bool {
ci, cj := res.List[i].CreatedAt, res.List[j].CreatedAt
if ci == nil {
return false
}
if cj == nil {
return true
}
return ci.After(cj)
})
res.Total = len(res.List)
if req.PageSize > 0 {
start := int((req.PageNum - 1) * req.PageSize)
if start < 0 {
start = 0
}
if start >= res.Total {
res.List = nil
return
}
end := start + int(req.PageSize)
if end > res.Total {
end = res.Total
}
res.List = res.List[start:end]
}
// 读取结果文件内容(仅 .txt)放入 ResultContent(仅当前页),供前端直接展示;路径仍保留在 ResultFileUrl
for _, vo := range res.List {
vo.ResultContent = readResultFileContent(ctx, vo.ResultFileUrl)
}
return
}
// readResultFileContent 读取结果 txt 文件内容(支持逗号分隔的多个 URL),仅 .txt 文件被读取,多个内容用换行连接
func readResultFileContent(ctx context.Context, fileUrl string) string {
if fileUrl == "" {
return ""
}
var parts []string
for _, u := range strings.Split(fileUrl, ",") {
u = strings.TrimSpace(u)
if u == "" || !strings.HasSuffix(strings.ToLower(u), ".txt") {
continue
}
fileBytes, err := gateway.GetFileBytesFromURL(ctx, u)
if err != nil {
glog.Warningf(ctx, "读取结果 txt 文件失败: %v", err)
continue
}
parts = append(parts, string(fileBytes))
}
return strings.Join(parts, "\n")
}
func chatExecVO(c *entity.ExecChat) *sessionDto.VOSessionInfoResult {
status := resultStatusSuccess
if c.ErrorMessage != "" {
status = resultStatusFailed
}
return &sessionDto.VOSessionInfoResult{
Id: c.Id,
Type: "chat",
Status: status,
RequestParams: map[string]any{"question": c.RequestParams.Question},
ResultFileUrl: c.ResultFileUrl,
TotalTokens: c.TotalTokens,
TotalFee: c.TotalFee,
ErrorMsg: c.ErrorMessage,
Error: c.Error,
CreatedAt: c.CreatedAt,
}
}
func workflowExecVO(w *entity.ExecWorkflow, resultFileUrl string) *sessionDto.VOSessionInfoResult {
status := 1
// FlowExecutionStatus 是 *int8 别名,Code() 返回包级指针,直接 == 是地址比较恒为 false,
// 需解引用按值比较,否则所有执行记录在前端都会误显示为"运行中"
if w.Status != nil {
if *w.Status == *flow.FlowExecutionStatusFailed.Code() {
status = resultStatusFailed
} else if *w.Status == *flow.FlowExecutionStatusSuccess.Code() {
status = resultStatusSuccess
} else if *w.Status == *flow.FlowExecutionStatusCancel.Code() {
status = resultStatusCancel
}
}
return &sessionDto.VOSessionInfoResult{
Id: w.Id,
Type: "workflow",
Status: status,
FlowId: w.FlowId,
RequestParams: gconv.Map(w.RequestParams),
ResultFileUrl: resultFileUrl,
TotalTokens: w.TotalTokens,
TotalFee: w.TotalFee,
ActualAmount: w.ActualAmount,
ErrorMsg: w.ErrorMessage,
Error: w.Error,
CreatedAt: w.CreatedAt,
}
}
// ResultList 工作流执行结果树:按创建人分页查询工作流结果记录(exec_workflow_result),
// 返回按天分组的树结构(日期→流程→结果文件)。
// 只依赖结果表,不关联 exec_workflow 执行记录——即使执行记录被删除/清理,产出文件仍可查看。
// 分页单位为"天":每页返回 pageSize 个完整日期,同一天内的流程与文件不会被拆到不同页;pageSize 为 0 时返回全部。
func (s *sessionService) ResultList(ctx context.Context, req *sessionDto.ListWorkflowResultReq) (res *flowDto.ListFlowExecutionTreeRes, err error) {
user, err := utils.GetUserInfo(ctx)
if err != nil {
return
}
dates, err := sessionDao.ExecWorkflowResultDao.ListDates(ctx, user.UserName, req.Page)
if err != nil {
return
}
res = &flowDto.ListFlowExecutionTreeRes{}
res.ImgAddressPrefix, _ = oss.GetFileAddressPrefix(ctx)
if len(dates) == 0 {
return
}
// 仅查结果表:同一执行的多个结果按 exec_id 归为一个流程节点
results, err := sessionDao.ExecWorkflowResultDao.ListByDates(ctx, user.UserName, dates)
if err != nil {
return
}
// 按 exec_id 分组(ListByDates 按创建时间倒序,同执行内结果最后反转回正序,保持旧列表输出顺序)
type resultGroup struct {
ExecId int64
SessionId string
FlowId int64
Date string
Results []*entity.ExecWorkflowResult
}
groupByExec := make(map[int64]*resultGroup)
var groups []*resultGroup
flowIdSet := make(map[int64]struct{})
for _, r := range results {
g := groupByExec[r.ExecId]
if g == nil {
g = &resultGroup{ExecId: r.ExecId, SessionId: r.SessionId, FlowId: r.FlowId}
groupByExec[r.ExecId] = g
groups = append(groups, g)
if r.CreatedAt != nil {
g.Date = r.CreatedAt.Format("Y-m-d")
}
}
g.Results = append(g.Results, r)
flowIdSet[r.FlowId] = struct{}{}
}
for _, g := range groups {
for i, j := 0, len(g.Results)-1; i < j; i, j = i+1, j-1 {
g.Results[i], g.Results[j] = g.Results[j], g.Results[i]
}
}
flowNameMap := make(map[int64]string)
for fid := range flowIdSet {
if fu, e := flowDao.FlowUserDao.Get(ctx, &flowDto.GetFlowUserReq{Id: fid}); e == nil && fu != nil && fu.FlowName != "" {
flowNameMap[fid] = fu.FlowName
}
}
// 按日期分组构建树(flow 顺序即各组首次出现的倒序,与旧行为一致)
flowsByDate := make(map[string][]flowDto.FlowNode)
for _, g := range groups {
flowName := flowNameMap[g.FlowId]
if flowName == "" {
flowName = "工作流"
}
var items []flowDto.OutputItem
suffixCount := make(map[string]int)
for _, rf := range g.Results {
if rf.ResultFileUrl == "" {
continue
}
content := rf.ResultFileUrl
ext := flowService.GetFileTypeByPath(content)
suffix := outputItemSuffix(ext)
suffixCount[suffix]++
items = append(items, flowDto.OutputItem{
Content: content,
Type: ext,
Label: fmt.Sprintf("%s_%d", suffix, suffixCount[suffix]),
})
}
if len(items) == 0 {
continue
}
flowsByDate[g.Date] = append(flowsByDate[g.Date], flowDto.FlowNode{
FlowName: flowName,
Id: g.ExecId,
SessionId: g.SessionId,
Items: items,
})
}
for _, d := range dates {
if fs := flowsByDate[d]; len(fs) > 0 {
res.Tree = append(res.Tree, flowDto.DateNode{CreateDate: d, Flows: fs})
}
}
return
}
// outputItemSuffix 按文件类型映射结果项的中文标签前缀(与 flow 侧旧逻辑保持一致)
func outputItemSuffix(ext string) string {
switch ext {
case "image":
return "图片"
case "video":
return "视频"
case "audio":
return "音频"
case "text":
return "文案"
case "html":
return "HTML"
default:
return "内容"
}
}
func (s *sessionService) ResultDelete(ctx context.Context, req *sessionDto.DeleteWorkflowResultReq) (err error) {
_, err = sessionDao.ExecWorkflowResultDao.Delete(ctx, req)
return
}
+5 -4
View File
@@ -11,6 +11,7 @@ import (
"strings"
"gitea.redpowerfuture.com/red-future/common/beans"
"gitea.redpowerfuture.com/red-future/common/oss"
"gitea.redpowerfuture.com/red-future/common/utils"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
@@ -149,7 +150,7 @@ func (s *skillUserService) Get(ctx context.Context, req *skillDto.GetSkillUserRe
return nil, err
}
res = &skillDto.SkillUserVO{}
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
@@ -169,7 +170,7 @@ func (s *skillUserService) Get(ctx context.Context, req *skillDto.GetSkillUserRe
return nil, err
}
res = &skillDto.SkillUserVO{}
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
@@ -250,7 +251,7 @@ func (s *skillUserService) GetUserOrTemplate(ctx context.Context, req *skillDto.
}
if !g.IsEmpty(list) {
res = &skillDto.SkillUserVO{}
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
@@ -273,7 +274,7 @@ func (s *skillUserService) GetUserOrTemplate(ctx context.Context, req *skillDto.
return nil, err
}
res = &skillDto.SkillUserVO{}
res.ImgAddressPrefix, err = utils.GetFileAddressPrefix(ctx)
res.ImgAddressPrefix, err = oss.GetFileAddressPrefix(ctx)
if err != nil {
return nil, err
}
+26
View File
@@ -0,0 +1,26 @@
package tool
import (
toolDto "ai-agent/workflow/model/dto/tool"
"ai-agent/workflow/service/flow/processor"
"context"
)
var ToolService = &toolService{}
type toolService struct{}
func (s *toolService) List(ctx context.Context, req *toolDto.ToolListReq) (*toolDto.ToolListRes, error) {
list, err := processor.List(ctx)
if err != nil {
return nil, err
}
voList := make([]*toolDto.ToolVO, 0, len(list))
for _, t := range list {
voList = append(voList, &toolDto.ToolVO{
Name: t.Name,
Description: t.Description,
})
}
return &toolDto.ToolListRes{List: voList}, nil
}
+2 -10
View File
@@ -4,7 +4,7 @@ import (
"context"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"github.com/gogf/gf/v2/frame/g"
"gitea.redpowerfuture.com/red-future/common/utils"
)
var UtilService = &utilService{}
@@ -13,16 +13,8 @@ type utilService struct{}
// IsAdmin 调用admin-go服务检查是否是管理员
func (s *utilService) IsAdmin(ctx context.Context) (res bool, err error) {
headers := make(map[string]string)
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Request.Header {
if len(v) > 0 {
headers[k] = v[0]
}
}
}
var r = make(map[string]bool)
if err = commonHttp.Get(ctx, "admin-go/api/v1/system/user/checkIsSuperAdmin", headers, &r); err != nil {
if err = commonHttp.Get(ctx, "admin-go/api/v1/system/user/checkIsSuperAdmin", utils.HeadersFromCtx(ctx), &r); err != nil {
return false, err
}
return r["isSuperAdmin"], err