Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d7cf0a9380 | ||
|
|
9fd71027a4 | ||
|
|
6a48edb742 | ||
|
|
d0a8d6889c | ||
|
|
d3a8043337 | ||
|
|
1ce1913437 | ||
|
|
071ebe2adc | ||
|
|
cc23697201 | ||
|
|
4d95ff75d2 | ||
|
|
a332aa438b | ||
|
|
d18016e698 | ||
|
|
7e849b1c0f | ||
|
|
84421b4ee5 | ||
|
|
62003c25a0 | ||
|
|
2edbe54fdc | ||
|
|
35e955a617 | ||
|
|
bdd5f93e2c | ||
|
|
326a8acac9 | ||
|
|
3ca8f93778 | ||
|
|
8ef6a180f7 | ||
|
|
29d3f7ffb2 | ||
|
|
2bc9d76e18 | ||
|
|
c71823abdb | ||
|
|
b00e5d34f4 | ||
|
|
85dd4e8e84 | ||
|
|
3feb912d86 | ||
|
|
63092be679 | ||
|
|
e11155b604 | ||
|
|
c22a38da9a | ||
|
|
aee243d4eb | ||
|
|
16f2967569 | ||
|
|
73f630636d | ||
|
|
eaa2942957 | ||
|
|
525b391f09 | ||
|
|
c33f231136 | ||
|
|
47b38f221c | ||
|
|
3ccce74465 | ||
|
|
610481effd | ||
|
|
39b61f1867 | ||
|
|
4cc44bf57c | ||
|
|
dc06d1bb9a | ||
|
|
ecaaa5bdbc | ||
|
|
b21d7a8dbf | ||
|
|
fddaf36f48 |
@@ -0,0 +1 @@
|
||||
.git
|
||||
@@ -1,206 +1,437 @@
|
||||
# model-asynch(模型异步中间件)[2026.5.12前,暂时弃置]
|
||||
# Model Gateway — 智能模型网关
|
||||
|
||||
一个独立的异步中间件服务:按模型配置路由调用不同模型服务,统一生成 `task_id`,后台异步执行,结果上传 OSS,并提供查询/批量领取/自动重试/自动清理能力,便于业务方“拿走结果并转移”。
|
||||
|
||||
> 分支约定:`dev` 为开发分支;`main`(或 master)为线上主分支。
|
||||
统一的 AI 模型网关服务,负责多模态(文本、图像、音频、向量、视频、全模态)模型调用的路由、执行与管理。基于 Go (GoFrame v2) 构建,使用 PostgreSQL、Redis 和 Consul 服务发现。
|
||||
|
||||
---
|
||||
|
||||
## 1. 核心功能
|
||||
## 架构概览
|
||||
|
||||
### 1.1 模型配置(asynch_models)
|
||||
- 增删改查模型服务配置(`model_name` 唯一标识)
|
||||
- 支持配置:
|
||||
- 请求地址:`base_url + route`
|
||||
- 请求方式:`http_method`(GET/POST)
|
||||
- 请求头:`head_msg`(以请求头注入,支持多个 header)
|
||||
- 超时:`timeout_seconds`
|
||||
- 并发:`max_concurrency`(按租户+模型的 Redis 分布式信号量限流)
|
||||
- 重试:`retry_times`(失败后最多再重试 N 次)
|
||||
- 保留:`auto_clean_seconds`(任务被业务领取到 `state=4` 后的保留秒数,到期清理)
|
||||
```
|
||||
客户端 ──> HTTP API ──> Controller ──> Service ──> DAO ──> PostgreSQL
|
||||
│ └─> Redis(信号量/队列)
|
||||
│
|
||||
├──> AI 模型服务商(OpenAI、阿里云、火山引擎等)
|
||||
├──> OSS 文件服务
|
||||
├──> Admin-go(租户/余额)
|
||||
├──> Prompts-core(会话回调)
|
||||
└──> Skill 技能服务
|
||||
```
|
||||
|
||||
### 1.2 异步任务(asynch_task)
|
||||
- 创建任务:生成 `task_id`,入库排队
|
||||
- 后台 Worker:
|
||||
- PostgreSQL `FOR UPDATE SKIP LOCKED` 抢占任务,支持多实例不重复消费
|
||||
- 调用模型服务(GET/POST)
|
||||
- 结果上传 OSS(调用你们的 OSS 文件服务 `oss/file/uploadFile`,透传 `Authorization/X-User-Info`)
|
||||
- 批量领取结果:批量查询 `task_id` 列表,返回 `task_id/state/oss_file`,并把成功的任务从 `state=2` 更新为 `state=4`
|
||||
- 自动重试:失败 `state=3` 会由清理器按 `retry_times` 重新入队到队尾
|
||||
- 自动清理:
|
||||
- `state=4` 且 `expire_at` 到期 → 硬删除任务
|
||||
- 失败重试耗尽仍失败 → 硬删除任务
|
||||
- `state=0/1` 超时 → 标记失败(防止卡死)
|
||||
系统分为六个层次:
|
||||
|
||||
### 1.3 统计(asynch_model_stat)
|
||||
- 按天统计:`day + tenant_id + creator + model_name -> request_count`
|
||||
- 统计口径:仅在 Worker 真正调用模型服务时计数(OSS 重试不计数)
|
||||
- 用途:给其他服务提供全局限流/监控依据
|
||||
| 层次 | 目录 | 职责 |
|
||||
|---|---|---|
|
||||
| **控制器层** | `controller/` | HTTP 路由注册与请求绑定 |
|
||||
| **业务逻辑层** | `service/` | 模型配置 CRUD、异步任务执行、提示词构建、队列管理、统计 |
|
||||
| **数据访问层** | `dao/` | PostgreSQL 数据访问(GoFrame ORM) |
|
||||
| **模型层** | `model/` | DTO 请求/响应结构与数据库实体定义 |
|
||||
| **通用工具层** | `common/` | 计费、类型转换、文件处理、请求头、映射、提示词、流式解析等工具 |
|
||||
| **常量层** | `consts/` | 公共常量和表名定义 |
|
||||
|
||||
---
|
||||
|
||||
## 2. 使用流程(业务方如何接入)
|
||||
## 核心功能
|
||||
|
||||
### 第一步:创建模型配置
|
||||
业务方(或运维)先在中间件里创建/更新模型配置(`model_name` 为唯一键),例如:
|
||||
- `POST /model/createModel`(或 `/model/updateModel`)
|
||||
### 1. 模型配置管理 (`model_gateway_models`)
|
||||
|
||||
提供 AI 模型服务配置的完整增删改查:
|
||||
|
||||
| 字段 | 说明 |
|
||||
|---|---|
|
||||
| `model_name` | 模型名称(唯一标识) |
|
||||
| `model_type` | 模型分类(100=推理, 200=图像, 300=音频, 400=向量, 500=全模态, 600=视频) |
|
||||
| `operator_name` | 运营商标识(OpenAI、阿里云、火山引擎等) |
|
||||
| `base_url` | 模型服务地址 |
|
||||
| `http_method` | GET / POST |
|
||||
| `head_msg` | 每次调用注入的请求头 |
|
||||
| `form_json` | 动态表单定义(用于前端按模型渲染参数表单) |
|
||||
| `request_mapping` / `response_mapping` | 标准格式与提供商 API 之间的参数映射 |
|
||||
| `call_mode` | 0=同步, 1=异步, 2=流式 |
|
||||
| `max_concurrency` | 单模型最大并发数(按租户维度限流) |
|
||||
| `timeout_seconds` | 请求超时时间(秒) |
|
||||
| `retry_times` | 失败重试次数 |
|
||||
| `billing_config` | 计费规则(推理阶梯计价 / 视频分辨率计价) |
|
||||
| `stream_config` | SSE 流式解析配置 |
|
||||
| `query_config` | 异步任务轮询/查询配置 |
|
||||
|
||||
**支持的模型类型:**
|
||||
|
||||
| 类型码 | 类别 | 子类型 |
|
||||
|---|---|---|
|
||||
| 100 | 推理模型 | 文本生成、对话 |
|
||||
| 200-205 | 图像模型 | 文生图、图生图、图片编辑、图片变体、图文生图 |
|
||||
| 300-303 | 音频模型 | 文生语音、语音转文字、语音转语音 |
|
||||
| 400-402 | 向量模型 | 文本嵌入、重排序 |
|
||||
| 500-502 | 全模态模型 | 文图音频、视觉理解 |
|
||||
| 600-604 | 视频模型 | 文生视频、图生视频、图文生视频、视频生视频 |
|
||||
|
||||
### 2. 异步任务执行 (`model_gateway_task`)
|
||||
|
||||
任务生命周期如下:
|
||||
|
||||
```
|
||||
CreateTask ──> state=0(排队中)
|
||||
│
|
||||
Worker 抢占(state=1,执行中)
|
||||
│
|
||||
├── 同步模式:直接调用模型
|
||||
├── 异步模式:提交任务,通过 QueryConfig 轮询
|
||||
└── 流式模式:通过 StreamConfig 解析 SSE 事件
|
||||
│
|
||||
├── 成功(state=2)──> 上传结果到 OSS
|
||||
│ └──> 触发回调(如有配置)
|
||||
└── 失败(state=3)──> 重试(最多 retry_times 次)
|
||||
│
|
||||
客户端下载(state=4,已下载)
|
||||
```
|
||||
|
||||
**关键特性:**
|
||||
|
||||
- **任务创建**:立即返回 `taskId`,执行在 goroutine 中异步完成
|
||||
- **并发控制**:基于 Redis 的分布式信号量,按模型维度限制并发数
|
||||
- **队列门控**:基于 Redis Lua 脚本的严格队列插槽机制,防止分布式创建下超限
|
||||
- **自动重试**:临时性错误(超时、内部错误)自动重试;硬错误直接失败
|
||||
- **OSS 上传**:结果以 JSON 格式上传到 OSS 文件服务,OSS URL 存入任务记录
|
||||
- **回调通知**:任务完成时可选触发 HTTP 回调(`TriggerCallback`、`TriggerPromptsCallback`、`CallbackBuildResult`)
|
||||
- **阶梯计费**:支持推理 token 阶梯计价和视频分辨率计价两种模型
|
||||
|
||||
### 3. 提示词构建 (`/buildMessages`)
|
||||
|
||||
异步构建推理模型的结构化消息:
|
||||
|
||||
- 从配置合并系统提示词(`modelPrompts.types`)
|
||||
- 拉取技能 Markdown 内容(`SkillMdContent`)
|
||||
- 通过 `GetSessionHistory` 注入会话历史
|
||||
- 根据附件数量自动拆分多轮(`SplitByAttachment`)
|
||||
- 支持视频模型通过对话模型编排的多轮生成
|
||||
|
||||
### 4. 动态调参 (`/autoTune`)
|
||||
|
||||
周期性自动调参(建议每小时触发),基于近期 P90 执行耗时和到达率动态调整 `max_concurrency` 和队列上限:
|
||||
|
||||
- 读取模型配置(数据库中的上限值 cap)
|
||||
- 根据时间窗口内已完成任务统计 P90 执行耗时
|
||||
- 使用 Little 定律(到达率 × P90 / 利用率)计算新的并发数
|
||||
- 单次调整幅度限制在 ±50%
|
||||
- 运行时参数写入 Redis(2 小时 TTL),不修改数据库中的上限值
|
||||
|
||||
### 5. 日统计 (`model_gateway_log_stat`)
|
||||
|
||||
按天、租户、创建人、模型维度的请求计数,使用原子 upsert(`ON DUPLICATE KEY UPDATE`)。
|
||||
|
||||
按天、租户、创建人、模型维度的请求计数,使用原子 upsert(`ON DUPLICATE KEY UPDATE`)。
|
||||
|
||||
### 6. 核心接口详细说明
|
||||
|
||||
#### 6.1 构建消息结构 `/buildMessages`
|
||||
|
||||
构建推理模型的提示词消息体,将系统提示词、技能知识、会话历史、用户自定义提示词合并为最终的消息结构。
|
||||
|
||||
**请求示例(JSON):**
|
||||
|
||||
请求示例(JSON):
|
||||
```json
|
||||
{
|
||||
"modelName": "model-service",
|
||||
"modelsType": "1,2,3",
|
||||
"baseUrl": "http://127.0.0.1:8000",
|
||||
"route": "/api/v1/chat",
|
||||
"httpMethod": "POST",
|
||||
"headMsg": "API_KEY:model-key,API_STATE:true,API_NUM:123",
|
||||
"enabled": 1,
|
||||
"maxConcurrency": 5,
|
||||
"queueLimit": 20,
|
||||
"timeoutSeconds": 1800,
|
||||
"expectedSeconds": 600,
|
||||
"retryTimes": 3,
|
||||
"retryQueueMaxSeconds": 600,
|
||||
"autoCleanSeconds": 3600,
|
||||
"remark": "Model-Service 模型服务"
|
||||
"modelName": "gpt-4o",
|
||||
"buildType": 1,
|
||||
"skillName": "code-review",
|
||||
"callbackUrl": "http://callback.example.com/result",
|
||||
"nodeId": "node_001",
|
||||
"sessionId": "session_abc",
|
||||
"customPrompt": "请用中文回答",
|
||||
"messages": {
|
||||
"model": "gpt-4o",
|
||||
"max_tokens": 4096,
|
||||
"messages": [
|
||||
{"role": "system", "content": [{"type": "text", "text": "你是一名助手"}]},
|
||||
{"role": "user", "content": [{"type": "text", "text": "写一封邮件"}]}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
参数说明:
|
||||
- `modelName`:模型名称(唯一标识/路由键)
|
||||
- `modelsType`:模型类型ID列表(逗号分隔),示例:`1,2,3`(关联 `asynch_models_type.type_id`)
|
||||
| 请求字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `modelName` | string | 是 | 网关模型名称 |
|
||||
| `buildType` | int | 是 | 构建类型:1=单轮构建, 2=多轮构建 |
|
||||
| `skillName` | string | 否 | 技能名称,用于拉取技能 MD 知识 |
|
||||
| `callbackUrl` | string | 否 | 构建完成后的回调地址 |
|
||||
| `nodeId` | string | 否 | 节点 ID(用于查询会话历史) |
|
||||
| `sessionId` | string | 否 | 会话 ID |
|
||||
| `messages` | object | 是 | 前端构建的消息结构 |
|
||||
| `customPrompt` | string | 否 | 用户自定义提示词 |
|
||||
|
||||
### 模型类型同步
|
||||
- `POST /model/type/createModelType` 创建成功后,会同步 `POST` 到 `prompts-core` 的 `/prompt/createPrompt`
|
||||
- 同步字段映射:
|
||||
- `typeId` -> `modelTypeId`
|
||||
- `type` -> `modelType`
|
||||
- `promptInfo` -> `promptInfo`
|
||||
- `responseJsonSchema` -> `responseJsonSchema`
|
||||
- `version` -> `version`
|
||||
- 若 `prompts-core` 同步失败,`model-gateway` 会回滚本地新建的模型类型,避免两边数据不一致
|
||||
- `form`:动态表单配置(JSON数组),用于前端按模型渲染参数表单(字段示例:field/label/type/required)
|
||||
- `baseUrl`:模型服务地址(Base URL)
|
||||
- `route`:模型服务路由(拼接到 baseUrl 后)
|
||||
- `httpMethod`:请求方式(GET/POST)
|
||||
- `headMsg`:请求头绑定(支持多个 header,逗号分隔,格式 `Key:Value`;布尔/数字也会以字符串形式注入 header)
|
||||
- `enabled`:是否启用(0禁用/1启用)
|
||||
- `maxConcurrency`:单模型最大并发(按租户+模型维度限流)
|
||||
- `queueLimit`:排队上限(严格控制)。创建任务时通过 Redis Lua 原子闸门校验并占位,保证分布式并发创建不会超限;任务进入成功/失败态后释放占位,失败重试重新入队时会再次占位。
|
||||
- `timeoutSeconds`:调用模型服务超时(秒)
|
||||
- `expectedSeconds`:模型预计执行时间(秒,用于超时判定/排队策略等)
|
||||
- `retryTimes`:失败后最多再重试 N 次(不含首次)
|
||||
- `retryQueueMaxSeconds`:失败重试最大排队时间(秒);0 表示重试插队到队首;>0 表示排队超过该时间后插队,否则仍到队尾
|
||||
- `autoCleanSeconds`:任务被领取到 `state=4` 后的保留时间(秒),到期清理
|
||||
- `remark`:备注说明
|
||||
**响应:**
|
||||
|
||||
### 第二步:创建任务拿到 task_id
|
||||
业务方发起推理请求时调用:
|
||||
- `POST /task/createTask`(传 `modelName + requestPayload + bizName + callbackUrl(可选) + modelKey(可选)`)
|
||||
- 中间件返回 `task_id`
|
||||
- 业务方将 `task_id` 落到自己的业务表,并把业务状态置为「生成中」
|
||||
|
||||
> `modelKey` 用于“动态覆盖/补充”模型配置中的 `head_msg`(例如每次请求携带不同的 `X-API-Key:xxx`)。
|
||||
>
|
||||
> `callbackUrl` 用于任务成功后的回调通知:当任务 `state=2` 成功时,中间件会发起一次 GET 请求:
|
||||
> - 实际回调地址:`callbackUrl/{bizName}`
|
||||
> - query 参数:`task_id/state/oss_file/file_type/text(可选,最多2000字符)`
|
||||
|
||||
### 第三步:同步任务进度(推荐批量)
|
||||
业务方通过轮询/定时任务同步进度:
|
||||
- 推荐:`POST /task/getTaskBatch`(批量传 `taskIds`,返回每个任务的 `state + oss_file`)
|
||||
- 或单条:`GET /task/getTaskResult?taskId=...`
|
||||
|
||||
业务侧拿到 `oss_file` 后自行做资源处理(直接保存或转存),并把业务状态更新为「成功/失败」。
|
||||
|
||||
> 说明:批量接口对 `state=2(成功)` 的任务会自动标记为 `state=4(已下载)` 并写入 `expire_at`,用于后续清理。
|
||||
|
||||
### 后台执行(由上层定时任务控制)
|
||||
本项目不再在服务进程内常驻轮询 worker/cleaner,而是提供两个接口供上层定时任务触发:
|
||||
- `POST /task/runWork`:执行一次 Worker(抢占并处理一批排队任务;适合处理 createTask 立即执行时未处理到的任务和积压队列)
|
||||
- `POST /task/cleanWork`:执行一次 Cleaner(清理过期任务、失败重试、超时任务失败等)
|
||||
|
||||
创建任务执行策略:
|
||||
- `POST /task/createTask` 成功入库后,会立即异步尝试执行当前任务。
|
||||
- 若当前模型并发已满,或当前任务未成功抢占,则会按 `asynch.worker.intervalSeconds` 对当前任务做轻量级定向轮询;只要任务仍为 `state=0` 就继续尝试,一旦进入 `state=1/2/3/4` 就立即停止,不会一直轮询。
|
||||
- 若任务执行成功且配置了 `callbackUrl + bizName`,会在成功落库后异步触发回调钩子。
|
||||
|
||||
本地调试(可选):
|
||||
可在 `config.yml` 中开启自动执行,避免手工频繁调用接口:
|
||||
```yml
|
||||
asynch:
|
||||
worker:
|
||||
enabled: true
|
||||
intervalSeconds: 5
|
||||
batchSize: 10
|
||||
goroutines: 1
|
||||
cleaner:
|
||||
enabled: true
|
||||
intervalSeconds: 30
|
||||
```
|
||||
|
||||
### 动态并发/队列调参(接口请求控制)
|
||||
为支持根据最近一段时间的耗时与吞吐对 `max_concurrency/queue_limit` 做动态调整,本项目提供接口供上层定时任务触发(建议每小时一次):
|
||||
- `POST /model/autoTune`
|
||||
|
||||
请求参数(JSON,可选):
|
||||
```json
|
||||
{
|
||||
"windowSeconds": 3600
|
||||
"taskId": "550e8400-e29b-41d4-a716-446655440000"
|
||||
}
|
||||
```
|
||||
> `windowSeconds` 不传/<=0 默认 3600(1小时)。
|
||||
|
||||
动态调参口径(默认近 1 小时窗口,按 `model_name` 维度):
|
||||
- 执行耗时:`finished_at - started_at`(取 P90)
|
||||
- 吞吐:近 1 小时完成数 / 3600
|
||||
**处理流程:**
|
||||
|
||||
调参结果不会覆盖 `asynch_models` 中配置的最大上限(cap),而是写入 Redis 运行时参数(带 TTL,默认 2 小时):
|
||||
- `asynch:runtime:max_concurrency:{model_name}`
|
||||
- `asynch:runtime:queue_limit:{model_name}`
|
||||
1. **查模型配置** — 根据 `modelName` 查询 `model_gateway_models` 获取模型定义
|
||||
2. **创建构建记录** — 写入 `model_gateway_build_record`,状态为处理中
|
||||
3. **异步执行构建**(goroutine):
|
||||
- **推理模型(type=100)**:
|
||||
- 从配置读取系统提示词(`modelPrompts.types.100`)
|
||||
- 若有 `skillName`,通过 `GetSkillUser` 拉取技能 ZIP,提取 MD 内容
|
||||
- 合并系统提示词 + 技能知识 + 用户自定义提示词到 `messages`
|
||||
- 通过 `GetSessionHistory` 注入会话历史
|
||||
- 按附件数量自动拆分多轮(`SplitByAttachment`,按 video/image/audio 的 `maxCount` 约束)
|
||||
- 返回单轮或多轮消息结构
|
||||
- **视频模型(type=600-699)**:
|
||||
- 查找当前用户的对话模型(`is_chat_model=1`)
|
||||
- 获取该对话模型的协议模板(`prompts_provider_protocol`)
|
||||
- 构建对话模型请求体,调用对话模型生成视频分镜的 JSON rounds
|
||||
- 将 rounds 上传 OSS,存入构建记录
|
||||
4. **回写结果** — 更新构建记录状态(成功/失败 + 耗时)
|
||||
5. **触发回调** — 若有 `callbackUrl`,POST 回调通知任务完成
|
||||
|
||||
生效位置:
|
||||
- CreateTask 入队时,严格 queue_limit 闸门会优先使用运行时 `queue_limit`(若无运行时值则回退 cap)。
|
||||
- Worker 获取并发令牌时,优先使用运行时 `max_concurrency`(若无运行时值则回退 cap)。
|
||||
**对应源码:**
|
||||
|
||||
- Controller:[controller/model_gateway_task_controller.go](`ModelGatewayTask.BuildMessages`)
|
||||
- Service:[service/task/task_service.go](`taskService.BuildMessages` → `executeBuild` → `buildResult`)
|
||||
- Util(提示词合并):[common/util/prompt.go](`MergePrompt`、`InjectHistory`、`SplitByAttachment`)
|
||||
- Util(技能知识):[service/prompt/prompt_files_handle_service.go](`SkillMdContent`)
|
||||
|
||||
#### 6.2 创建异步任务 `/createTask`
|
||||
|
||||
创建异步模型调用任务,立即返回 `taskId`,后端 goroutine 异步执行模型调用、结果上传与计费。
|
||||
|
||||
**请求示例(JSON):**
|
||||
|
||||
```json
|
||||
{
|
||||
"modelName": "dall-e-3",
|
||||
"bizName": "image-generator",
|
||||
"callbackUrl": "http://callback.example.com/result",
|
||||
"epicycleId": 12345,
|
||||
"buildModelName": "gpt-4o",
|
||||
"requestPayload": {
|
||||
"model": "dall-e-3",
|
||||
"prompt": "一只站在树枝上的猫头鹰,水墨风格",
|
||||
"n": 1,
|
||||
"size": "1024x1024"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| 请求字段 | 类型 | 必填 | 说明 |
|
||||
|---|---|---|---|
|
||||
| `modelName` | string | 是 | 模型名称 |
|
||||
| `bizName` | string | 否 | 业务名称,用于统计区分 |
|
||||
| `callbackUrl` | string | 否 | 任务完成后的回调地址 |
|
||||
| `requestPayload` | object | 是 | 透传给模型服务的请求参数 |
|
||||
| `epicycleId` | int64 | 否 | 轮次 ID(prompts-core 场景) |
|
||||
| `buildModelName` | string | 否 | 构建阶段使用的模型名(prompts-core 场景) |
|
||||
|
||||
**响应:**
|
||||
|
||||
```json
|
||||
{
|
||||
"taskId": "550e8400-e29b-41d4-a716-446655440000"
|
||||
}
|
||||
```
|
||||
|
||||
**处理流程:**
|
||||
|
||||
1. **鉴权与参数校验** — 获取用户信息,校验模型配置是否存在且已启用
|
||||
2. **队列门控(可选)** — 通过 Redis Lua 脚本做严格的分布式队列插槽校验,避免并发超限
|
||||
3. **写入任务记录** — 状态为 `1(执行中)`,记录 `requestPayload`、`callbackUrl`、`bizName` 等
|
||||
4. **操作日志** — 记录创建任务的审计日志到 `model_gateway_logs_op`
|
||||
5. **计费预处理** — 若模型配置了 `billing_config`,从请求参数中提取计费维度数据
|
||||
6. **异步执行**(goroutine `AsyncWorker.handleOne`):
|
||||
|
||||
```
|
||||
检查租户余额 ──> 调用模型服务 ──> 解析响应映射 ──> 计费计算并扣费
|
||||
│
|
||||
┌── 同步模式(call_mode=0):直接调用,等待 HTTP 响应
|
||||
├── 异步模式(call_mode=1):提交任务后通过 QueryConfig 轮询结果
|
||||
└── 流式模式(call_mode=2):通过 StreamConfig 解析 SSE 事件流
|
||||
│
|
||||
┌── prompts-core 场景:对模型输出做 ParseAndValidate + 重试机制
|
||||
└── 普通场景:直接映射响应
|
||||
│
|
||||
上传 OSS ──> 更新任务为成功 ──> 触发回调
|
||||
```
|
||||
|
||||
7. **重试机制**:超时、内部错误等可重试错误会按 `retry_times` 重试;硬错误(参数错误等)直接失败
|
||||
8. **回调通知**:任务成功或失败后,若有 `callbackUrl` 则 POST 回调;prompts-core 场景额外触发 `TriggerPromptsCallback`
|
||||
|
||||
**调用模式对比:**
|
||||
|
||||
| 模式 | 值 | 行为 |
|
||||
|---|---|---|
|
||||
| 同步 | 0 | 直接 HTTP 调用模型,等待完整响应后返回 |
|
||||
| 异步 | 1 | 提交任务后通过 `QueryConfig` 配置的轮询地址定时查询结果 |
|
||||
| 流式 | 2 | 对 SSE 事件流按 `StreamConfig` 规则解析(支持 concat/base64_concat/collect/final) |
|
||||
|
||||
**对应源码:**
|
||||
|
||||
- Controller:[controller/model_gateway_task_controller.go](`ModelGatewayTask.CreateTask`)
|
||||
- Service:[service/task/task_service.go](`taskService.Create`)
|
||||
- Worker:[service/task/worker.go](`asyncWorker.handleOne`、`InvokeModel`)
|
||||
- 队列门控:[service/queue/queue_gate.go](`AcquireQueueSlot`、`ReleaseQueueSlot`)
|
||||
- 运行时调参:[service/queue/runtime_tune.go](`GetRuntimeMaxConcurrency`、`GetRuntimeQueueLimit`)
|
||||
- 异步轮询:[common/util/pull_task.go](`PullTaskResult`)
|
||||
- 流式解析:[common/util/streaming.go](`ParseStreamResponse`)
|
||||
- 计费:[common/util/billing.go](`CalculateBilling`、`ExtractRequestBilling`)
|
||||
|
||||
### 7. 全量 API 接口一览
|
||||
|
||||
| 分类 | 接口路径 | 方法 | 说明 |
|
||||
|---|---|---|---|
|
||||
| **模型配置** | `/createModel` | POST | 创建模型配置 |
|
||||
| | `/updateModel` | PUT | 更新模型配置 |
|
||||
| | `/deleteModel` | DELETE | 删除模型配置 |
|
||||
| | `/getModel` | GET | 获取模型详情 |
|
||||
| | `/listModel` | GET | 模型列表(分页+筛选) |
|
||||
| | `/listType` | GET | 模型类型列表 |
|
||||
| | `/listOperator` | GET | 运营商列表 |
|
||||
| | `/updateChatModel` | POST | 设置当前用户的对话模型 |
|
||||
| | `/getIsChatModel` | GET | 获取当前对话模型 |
|
||||
| | `/autoTune` | POST | 触发动态调参 |
|
||||
| **任务管理** | `/createTask` | POST | 创建异步任务 |
|
||||
| | `/jobTask` | POST | 定时批量任务处理器 |
|
||||
| | `/getTaskResult` | GET | 获取单条任务结果 |
|
||||
| | `/getTaskBatch` | POST | 批量查询任务(成功任务自动标记为已下载) |
|
||||
| | `/listTask` | GET | 任务列表分页查询 |
|
||||
| | `/modelCallback` | POST | 接收异步模型回调通知 |
|
||||
| | `/queryPending` | GET | 轮询进行中的异步任务 |
|
||||
| **提示词** | `/buildMessages` | POST | 构建结构化提示词消息(异步) |
|
||||
| **统计** | `/listModelStat` | GET | 日使用量统计 |
|
||||
--
|
||||
|
||||
## 技术栈
|
||||
|
||||
| 组件 | 技术选型 |
|
||||
|---|---|
|
||||
| 语言 | Go 1.26 |
|
||||
| 框架 | GoFrame v2(`github.com/gogf/gf/v2`) |
|
||||
| 数据库 | PostgreSQL(GoFrame pgsql 驱动) |
|
||||
| 缓存/队列 | Redis(分布式信号量、队列门控、运行时调参存储) |
|
||||
| 服务发现 | Consul |
|
||||
| 链路追踪 | Jaeger(OTLP HTTP) |
|
||||
| 模型调用 | 原生 HTTP(可定制请求头、鉴权、超时) |
|
||||
| 文件存储 | OSS 文件上传服务(multipart 表单) |
|
||||
|
||||
## 数据库表
|
||||
|
||||
| 表名 | 说明 |
|
||||
|---|---|
|
||||
| `model_gateway_models` | 模型服务配置(动态表单、请求/响应映射、计费规则) |
|
||||
| `model_gateway_task` | 异步任务记录(状态机、重试、OSS 结果文件) |
|
||||
| `model_gateway_build_record` | 提示词构建记录 |
|
||||
| `model_gateway_logs_op` | 操作审计日志 |
|
||||
| `model_gateway_logs_stat` | 日维度请求量统计 |
|
||||
| `prompts_provider_protocol` | 视频模型编排的提供商协议模板 |
|
||||
|
||||
## 配置说明
|
||||
|
||||
详见 `config.yml`。主要配置块:
|
||||
|
||||
- `database` — PostgreSQL 连接(双数据源:`default` + `model_gateway`)
|
||||
- `redis` — Redis 连接
|
||||
- `consul` — Consul 地址
|
||||
- `jaeger` — Jaeger OTLP HTTP 端点
|
||||
- `queryPending` — 异步任务自动轮询配置(调试开关)
|
||||
- `jobTask` — 批量处理器间隔、批大小、协程池大小
|
||||
- `modelPrompts.types` — 各模型类型的系统提示词(100/200/300/400/500)
|
||||
- `nodePrompts` — 节点路由提示词模板
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 环境要求
|
||||
|
||||
- Go 1.26+
|
||||
- PostgreSQL
|
||||
- Redis
|
||||
- Consul(可选,用于服务发现)
|
||||
|
||||
### 本地开发
|
||||
|
||||
1. 克隆仓库
|
||||
2. 在 PostgreSQL 中执行 `update.sql` 创建所有表
|
||||
3. 修改 `config.yml` 中的数据库、Redis 和 Consul 配置
|
||||
4. 启动服务:
|
||||
```bash
|
||||
go run main.go
|
||||
```
|
||||
5. 服务默认监听 `3004` 端口(可在 `config.yml` 中修改)
|
||||
|
||||
### Docker 构建
|
||||
|
||||
```bash
|
||||
docker build -t model-gateway .
|
||||
docker run -p 3004:3004 model-gateway
|
||||
```
|
||||
|
||||
Dockerfile 使用多阶段构建,基于 `golang:alpine` 镜像,配置了国内 Go 代理,输出剥离调试信息的精简二进制。
|
||||
|
||||
---
|
||||
|
||||
## 3. 状态机说明(asynch_task.state)
|
||||
## 项目目录结构
|
||||
|
||||
| state | 含义 | 产生方 |
|
||||
|---:|---|---|
|
||||
| 0 | 排队中 | 创建任务/重试入队 |
|
||||
| 1 | 执行中 | Worker 抢占后 |
|
||||
| 2 | 成功(已上传 OSS) | Worker |
|
||||
| 3 | 失败 | Worker / 超时处理 |
|
||||
| 4 | 已下载(已领取) | 批量领取接口(2→4) |
|
||||
|
||||
字段补充:
|
||||
- `retry_count`:已重试次数(不含首次)
|
||||
- `enqueue_at`:入队时间(用于排队顺序,重试会更新为 NOW() 放到队尾)
|
||||
- `expire_at`:仅对 `state=4` 生效,表示保留到期时间
|
||||
```
|
||||
.
|
||||
├── main.go # 入口:路由注册、自动执行器启动
|
||||
├── config.yml # 应用配置
|
||||
├── Dockerfile # 多阶段 Docker 构建
|
||||
├── go.mod / go.sum # Go 模块依赖
|
||||
├── update.sql # 数据库 DDL
|
||||
│
|
||||
├── controller/ # HTTP 接口控制器
|
||||
│ ├── model_gateway_models_controller.go
|
||||
│ ├── model_gateway_task_controller.go
|
||||
│ └── model_gateway_logs_stat_controller.go
|
||||
│
|
||||
├── service/ # 业务逻辑层
|
||||
│ ├── gateway/ # OSS 上传、回调、余额、技能/会话查询
|
||||
│ ├── model/ # 模型 CRUD、对话模型管理
|
||||
│ ├── prompt/ # 文件拉取、技能 Markdown、提示词构建
|
||||
│ ├── queue/ # 动态调参、信号量、队列门控、运行时调参
|
||||
│ ├── stat/ # 使用量统计
|
||||
│ └── task/ # 任务创建、Worker 执行、异步结果处理
|
||||
│
|
||||
├── dao/ # 数据访问层
|
||||
│ ├── model_gateway_models_dao.go
|
||||
│ ├── model_gateway_task_dao.go
|
||||
│ ├── model_gateway_build_record_dao.go
|
||||
│ ├── model_gateway_logs_stat_dao.go
|
||||
│ ├── model_gateway_logs_op_dao.go
|
||||
│ └── provider_protocol_dao.go
|
||||
│
|
||||
├── model/ # 数据模型
|
||||
│ ├── dto/ # 请求/响应结构体
|
||||
│ └── entity/ # 数据库实体
|
||||
│
|
||||
├── common/ # 通用工具
|
||||
│ └── util/ # 计费、类型转换、文件处理、请求头、映射、提示词、流式、异步轮询
|
||||
│
|
||||
└── consts/ # 常量定义
|
||||
└── public/ # 模型类型、任务状态、运营商列表、表名
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. 配置说明(config.yml)
|
||||
## 任务状态机
|
||||
|
||||
关键配置:
|
||||
- `database.default`: PostgreSQL 连接
|
||||
- `redis.default`: Redis 连接(并发令牌、可扩展用途)
|
||||
| 状态码 | 含义 | 转换来源 |
|
||||
|---|---:|---|
|
||||
| 0 | 排队中 | 创建任务或重试入队 |
|
||||
| 1 | 执行中 | Worker 抢占成功 |
|
||||
| 2 | 成功(已上传 OSS) | Worker 执行完成 |
|
||||
| 3 | 失败 | Worker 执行出错或超时 |
|
||||
| 4 | 已下载 | 批量查询接口标记 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 数据库初始化
|
||||
## 接口文档
|
||||
地址:https://s.apifox.cn/42764602-cb82-45fa-887f-c751311ba406
|
||||
|
||||
项目根目录提供 `update.sql`:首次部署执行建表 SQL。
|
||||
## License
|
||||
|
||||
---
|
||||
内部项目 — 红未来科技
|
||||
|
||||
## 6. 开发与发布建议(Git)
|
||||
|
||||
- `dev`:日常开发与联调
|
||||
- `main`:线上稳定分支
|
||||
- 推荐流程:
|
||||
1) 从 `main` 拉出 `dev`
|
||||
2) 功能完成后提 MR/PR 合并回 `main`
|
||||
3) 打 tag / 发布镜像
|
||||
|
||||
@@ -0,0 +1,310 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"math"
|
||||
"model-gateway/service/gateway"
|
||||
"strings"
|
||||
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
// ======================== 计费入口 ========================
|
||||
|
||||
func CalculateBilling(config map[string]any, billingData map[string]any) map[string]any {
|
||||
if len(config) == 0 {
|
||||
return nil
|
||||
}
|
||||
switch config["type"] {
|
||||
case "inference_tier": //推理模型计费
|
||||
return calculateInferenceTierBilling(config, billingData)
|
||||
case "video_resolution": //视频模型计费
|
||||
return calculateVideoResolutionBilling(config, billingData)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ======================== 推理模型计费 ========================
|
||||
|
||||
func calculateInferenceTierBilling(config map[string]any, data map[string]any) map[string]any {
|
||||
promptTokens := gconv.Int64(data["prompt_tokens"])
|
||||
completionTokens := gconv.Int64(data["completion_tokens"])
|
||||
hasAudio := gconv.Bool(data["has_audio"])
|
||||
inputK := promptTokens / 1000
|
||||
|
||||
tiers := config["pricing"].(map[string]any)["tiers"].([]any)
|
||||
var matched map[string]any
|
||||
for _, t := range tiers {
|
||||
tier := t.(map[string]any)
|
||||
if inputK >= gconv.Int64(tier["input_min"]) && inputK <= gconv.Int64(tier["input_max"]) {
|
||||
matched = tier
|
||||
break
|
||||
}
|
||||
}
|
||||
if matched == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
var inputPrice float64
|
||||
if hasAudio && matched["audio_input_price"] != nil {
|
||||
inputPrice = gconv.Float64(matched["audio_input_price"])
|
||||
} else {
|
||||
inputPrice = gconv.Float64(matched["input_price"])
|
||||
}
|
||||
outputPrice := gconv.Float64(matched["output_price"])
|
||||
|
||||
inputCost := float64(promptTokens) * inputPrice / 1000000
|
||||
outputCost := float64(completionTokens) * outputPrice / 1000000
|
||||
|
||||
// 推理模型
|
||||
return map[string]any{
|
||||
"model_name": data["model_name"],
|
||||
"total_tokens": promptTokens + completionTokens,
|
||||
"total_fee": inputCost + outputCost,
|
||||
// 明细
|
||||
"prompt_tokens": promptTokens,
|
||||
"completion_tokens": completionTokens,
|
||||
"has_audio": hasAudio,
|
||||
"input_tier": fmt.Sprintf("[%v, %v]", matched["input_min"], matched["input_max"]),
|
||||
"input_unit_price": inputPrice,
|
||||
"output_unit_price": outputPrice,
|
||||
"input_cost": inputCost,
|
||||
"output_cost": outputCost,
|
||||
}
|
||||
}
|
||||
|
||||
// ======================== 视频模型计费 ========================
|
||||
|
||||
func calculateVideoResolutionBilling(config map[string]any, data map[string]any) map[string]any {
|
||||
pricingPath := buildPricingPath(config, data)
|
||||
|
||||
pricing := config["pricing"].(map[string]any)
|
||||
unitPrice := gconv.Float64(pricing[pricingPath])
|
||||
|
||||
var effectiveMinToken float64
|
||||
if gconv.Bool(data["input_has_video"]) {
|
||||
effectiveMinToken = matchMinToken(config, data)
|
||||
}
|
||||
|
||||
completionTokens := gconv.Float64(data["actual_tokens"])
|
||||
realChargeTokens := int64(math.Max(completionTokens, effectiveMinToken))
|
||||
totalFee := float64(realChargeTokens) * unitPrice / 1000000
|
||||
|
||||
if gconv.Bool(config["enable_audio_charge"]) {
|
||||
totalFee += gconv.Float64(data["audio_word_cnt"]) * gconv.Float64(config["audio_unit_price"])
|
||||
}
|
||||
|
||||
// 视频模型
|
||||
return map[string]any{
|
||||
"model_name": data["model_name"],
|
||||
"total_tokens": realChargeTokens,
|
||||
"total_fee": totalFee,
|
||||
// 明细
|
||||
"prompt_tokens": 0,
|
||||
"completion_tokens": int64(completionTokens),
|
||||
"is_online": data["is_online"],
|
||||
"video_resolution": data["video_resolution"],
|
||||
"input_has_video": data["input_has_video"],
|
||||
"output_duration_sec": data["output_duration_sec"],
|
||||
"aspect_ratio": data["aspect_ratio"],
|
||||
"resolution": data["resolution"],
|
||||
"matched_path": pricingPath,
|
||||
"token_unit_price": unitPrice,
|
||||
"effective_min_token": effectiveMinToken,
|
||||
}
|
||||
}
|
||||
|
||||
// ======================== 数据提取 ========================
|
||||
|
||||
func ExtractRequestBilling(ctx context.Context, config map[string]any, requestPayload map[string]any) map[string]any {
|
||||
dimensions := config["dimensions"].(map[string]any)
|
||||
fields := dimensions["request"].(map[string]any)
|
||||
data := make(map[string]any)
|
||||
|
||||
for key, path := range fields {
|
||||
data[key] = extractValue(requestPayload, gconv.String(path))
|
||||
}
|
||||
|
||||
if defaults, ok := config["defaults"].(map[string]any); ok {
|
||||
for k, v := range defaults {
|
||||
if _, exists := data[k]; !exists || data[k] == nil {
|
||||
data[k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 处理 compute 字段
|
||||
if compute, ok := config["compute"].(map[string]any); ok {
|
||||
for targetField, rule := range compute {
|
||||
r := rule.(map[string]any)
|
||||
dependsOn := gconv.String(r["depends_on"])
|
||||
dependsValue := gconv.Bool(r["depends_value"])
|
||||
|
||||
if gconv.Bool(data[dependsOn]) != dependsValue {
|
||||
continue
|
||||
}
|
||||
|
||||
switch r["service"] {
|
||||
case "video_duration":
|
||||
urls := extractVideoUrls(requestPayload)
|
||||
if len(urls) > 0 {
|
||||
resp, err := gateway.GetVideoDuration(ctx, urls)
|
||||
if err == nil {
|
||||
data[targetField] = resp.TotalDuration
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
func ExtractResponseBilling(config map[string]any, response map[string]any) map[string]any {
|
||||
dimensions := config["dimensions"].(map[string]any)
|
||||
fields := dimensions["response"].(map[string]any)
|
||||
data := make(map[string]any)
|
||||
|
||||
for key, path := range fields {
|
||||
data[key] = gjson.New(response).Get(gconv.String(path)).Val()
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
// ======================== 内部辅助 ========================
|
||||
|
||||
func buildPricingPath(config map[string]any, data map[string]any) string {
|
||||
pathConfig := config["pricing_path"].(map[string]any)
|
||||
segments := pathConfig["segments"].([]any)
|
||||
mapping := pathConfig["mapping"].(map[string]any)
|
||||
|
||||
var parts []string
|
||||
for _, seg := range segments {
|
||||
segName := gconv.String(seg)
|
||||
val := gconv.String(data[segName])
|
||||
if m, ok := mapping[segName].(map[string]any); ok {
|
||||
if mapped, exists := m[val]; exists {
|
||||
val = gconv.String(mapped)
|
||||
}
|
||||
}
|
||||
parts = append(parts, val)
|
||||
}
|
||||
return strings.Join(parts, ".")
|
||||
}
|
||||
|
||||
func matchMinToken(config map[string]any, data map[string]any) float64 {
|
||||
rule, ok := config["min_token_rule"].(map[string]any)
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
|
||||
conditionDefs := rule["conditions"].([]any)
|
||||
rows := rule["rows"].([]any)
|
||||
|
||||
for _, row := range rows {
|
||||
r := row.(map[string]any)
|
||||
conditions := r["conditions"].([]any)
|
||||
matched := true
|
||||
|
||||
for i, cond := range conditions {
|
||||
condStr := gconv.String(cond)
|
||||
condDef := conditionDefs[i].(map[string]any)
|
||||
condName := gconv.String(condDef["name"])
|
||||
|
||||
switch condDef["type"] {
|
||||
case "range":
|
||||
if !matchRange(condStr, gconv.Float64(data[condName])) {
|
||||
matched = false
|
||||
}
|
||||
case "enum":
|
||||
if gconv.String(data[condName]) != condStr {
|
||||
matched = false
|
||||
}
|
||||
}
|
||||
}
|
||||
if matched {
|
||||
return gconv.Float64(r["min_token"])
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func matchRange(rangeStr string, val float64) bool {
|
||||
rangeStr = strings.Trim(rangeStr, "[]()")
|
||||
parts := strings.Split(rangeStr, ",")
|
||||
if len(parts) != 2 {
|
||||
return false
|
||||
}
|
||||
return val >= gconv.Float64(strings.TrimSpace(parts[0])) && val < gconv.Float64(strings.TrimSpace(parts[1]))
|
||||
}
|
||||
|
||||
func extractValue(source map[string]any, path string) any {
|
||||
// 数组求和:rounds.#.duration
|
||||
if strings.HasPrefix(path, "rounds.#.") && !strings.Contains(path, "==") {
|
||||
field := strings.TrimPrefix(path, "rounds.#.")
|
||||
rounds := gjson.New(source).Get("rounds").Array()
|
||||
var sum float64
|
||||
for _, r := range rounds {
|
||||
if strings.Contains(field, "#") {
|
||||
parts := strings.SplitN(field, ".#.", 2)
|
||||
arr := gjson.New(r).Get(parts[0]).Array()
|
||||
for _, item := range arr {
|
||||
sum += gconv.Float64(gjson.New(item).Get(parts[1]).Val())
|
||||
}
|
||||
} else {
|
||||
sum += gconv.Float64(gjson.New(r).Get(field).Val())
|
||||
}
|
||||
}
|
||||
return sum
|
||||
}
|
||||
|
||||
// 条件判断:xxx.#.type==yyy 或 rounds.#.content.#.type==yyy
|
||||
if strings.Contains(path, "==video_url") || strings.Contains(path, "==input_audio") {
|
||||
parts := strings.Split(path, "==")
|
||||
basePath := parts[0]
|
||||
typ := parts[1]
|
||||
segments := strings.Split(basePath, ".#.")
|
||||
|
||||
// 单层:content.#.type → segments = ["content", "type"]
|
||||
if len(segments) == 2 && !strings.Contains(segments[0], ".") {
|
||||
arr := gjson.New(source).Get(segments[0]).Array()
|
||||
for _, item := range arr {
|
||||
if gconv.String(gjson.New(item).Get(segments[1]).Val()) == typ {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// 嵌套:rounds.#.content.#.type → segments = ["rounds", "content", "type"]
|
||||
topArr := gjson.New(source).Get(segments[0]).Array()
|
||||
for _, item := range topArr {
|
||||
subArr := gjson.New(item).Get(segments[1]).Array()
|
||||
for _, sub := range subArr {
|
||||
if gconv.String(gjson.New(sub).Get("type").Val()) == typ {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
return gjson.New(source).Get(path).Val()
|
||||
}
|
||||
|
||||
func extractVideoUrls(body map[string]any) []string {
|
||||
var urls []string
|
||||
content := gjson.New(body).Get("content").Array()
|
||||
for _, c := range content {
|
||||
item := c.(map[string]any)
|
||||
if item["type"] == "video_url" {
|
||||
if v, ok := item["video_url"].(map[string]any); ok {
|
||||
if url, ok := v["url"].(string); ok {
|
||||
urls = append(urls, url)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return urls
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
// GetModelPrompt 获取请求模型的提示词
|
||||
func GetModelPrompt(ctx context.Context, modelType int) string {
|
||||
key := "modelPrompts.types." + gconv.String(modelType)
|
||||
return g.Cfg().MustGet(ctx, key, "").String()
|
||||
}
|
||||
+81
-100
@@ -1,115 +1,96 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// DetectFileType 根据返回的二进制内容推断 contentType + 扩展名(尽量稳定)
|
||||
func DetectFileType(data []byte) (contentType string, ext string) {
|
||||
if len(data) == 0 {
|
||||
return "application/octet-stream", ""
|
||||
var (
|
||||
// AllowedMIMEPrefixes 允许的文本类 MIME 类型前缀
|
||||
AllowedMIMEPrefixes = []string{
|
||||
"text/",
|
||||
"application/json",
|
||||
"application/xml",
|
||||
"application/javascript",
|
||||
"application/x-yaml",
|
||||
"application/yaml",
|
||||
"application/toml",
|
||||
"application/x-httpd-php",
|
||||
"application/x-sh",
|
||||
"application/x-python",
|
||||
"application/x-perl",
|
||||
"application/x-ruby",
|
||||
}
|
||||
ct := http.DetectContentType(data)
|
||||
// gateway.DetectContentType 可能带 charset 等参数:text/plain; charset=utf-8
|
||||
if idx := strings.Index(ct, ";"); idx > 0 {
|
||||
ct = strings.TrimSpace(ct[:idx])
|
||||
|
||||
// BannedExtensions 禁止的文件扩展名
|
||||
BannedExtensions = map[string]bool{
|
||||
".png": true, ".jpg": true, ".jpeg": true, ".gif": true, ".bmp": true,
|
||||
".webp": true, ".svg": true, ".ico": true, ".tiff": true, ".tif": true,
|
||||
".mp3": true, ".wav": true, ".ogg": true, ".flac": true, ".aac": true,
|
||||
".wma": true, ".m4a": true,
|
||||
".mp4": true, ".avi": true, ".mkv": true, ".mov": true, ".wmv": true,
|
||||
".flv": true, ".webm": true,
|
||||
".tar": true, ".gz": true, ".rar": true, ".7z": true,
|
||||
".exe": true, ".dll": true, ".so": true, ".bin": true, ".dat": true,
|
||||
".class": true, ".pyc": true,
|
||||
".pdf": true, ".doc": true, ".docx": true, ".xls": true, ".xlsx": true,
|
||||
".ppt": true, ".pptx": true,
|
||||
}
|
||||
switch ct {
|
||||
case "audio/mpeg":
|
||||
return ct, ".mp3"
|
||||
case "audio/wave", "audio/wav", "audio/x-wav":
|
||||
return ct, ".wav"
|
||||
case "video/mp4":
|
||||
return ct, ".mp4"
|
||||
case "image/png":
|
||||
return ct, ".png"
|
||||
case "image/jpeg":
|
||||
return ct, ".jpg"
|
||||
case "application/pdf":
|
||||
return ct, ".pdf"
|
||||
case "text/plain":
|
||||
return ct, ".txt"
|
||||
case "application/json":
|
||||
return ct, ".json"
|
||||
default:
|
||||
// 兜底:尝试从 ct 截取 subtype 作为后缀(例如 application/json)
|
||||
if parts := strings.Split(ct, "/"); len(parts) == 2 {
|
||||
sub := parts[1]
|
||||
// 避免出现 "plain; charset=utf-8" 之类的后缀
|
||||
if idx := strings.Index(sub, ";"); idx > 0 {
|
||||
sub = strings.TrimSpace(sub[:idx])
|
||||
}
|
||||
return ct, "." + sub
|
||||
|
||||
symbolCleaner = regexp.MustCompile(`[\x00-\x08\x0B\x0C\x0E-\x1F]`)
|
||||
multiNewlines = regexp.MustCompile(`\n{3,}`)
|
||||
)
|
||||
|
||||
// SanitizeURL 清洗 URL 字符串
|
||||
func SanitizeURL(raw string) string {
|
||||
s := strings.TrimSpace(raw)
|
||||
s = strings.Trim(s, "`\"")
|
||||
return s
|
||||
}
|
||||
|
||||
// CleanSymbols 清洗文本中的控制字符和多余空行
|
||||
func CleanSymbols(text string) string {
|
||||
text = symbolCleaner.ReplaceAllString(text, "")
|
||||
text = strings.ReplaceAll(text, "\r\n", "\n")
|
||||
text = strings.ReplaceAll(text, "\r", "\n")
|
||||
text = multiNewlines.ReplaceAllString(text, "\n\n")
|
||||
return strings.TrimSpace(text)
|
||||
}
|
||||
|
||||
// IsBannedExtension 判断是否为禁止的文件扩展名
|
||||
func IsBannedExtension(url string) bool {
|
||||
ext := extractExtension(url)
|
||||
return BannedExtensions[ext]
|
||||
}
|
||||
|
||||
// IsZipExtension 判断是否为 zip 文件
|
||||
func IsZipExtension(url string) bool {
|
||||
ext := extractExtension(url)
|
||||
return ext == ".zip"
|
||||
}
|
||||
|
||||
// IsReadableContentType 判断是否为可读的文本类型
|
||||
func IsReadableContentType(contentType string) bool {
|
||||
if contentType == "" {
|
||||
return false
|
||||
}
|
||||
|
||||
ct := strings.ToLower(contentType)
|
||||
for _, prefix := range AllowedMIMEPrefixes {
|
||||
if strings.HasPrefix(ct, prefix) {
|
||||
return true
|
||||
}
|
||||
return ct, ""
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// SaveTmpResult 将模型输出写入临时文件,用于 OSS 上传失败后的“仅重试 OSS”。
|
||||
func SaveTmpResult(taskID string, data []byte, ext string) (string, error) {
|
||||
dir := filepath.Join(os.TempDir(), "model-asynch")
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return "", err
|
||||
// extractExtension 提取文件扩展名并清理查询参数
|
||||
func extractExtension(url string) string {
|
||||
ext := strings.ToLower(filepath.Ext(url))
|
||||
if idx := strings.Index(ext, "?"); idx != -1 {
|
||||
ext = ext[:idx]
|
||||
}
|
||||
if ext == "" {
|
||||
ext = ".bin"
|
||||
}
|
||||
if ext[0] != '.' {
|
||||
ext = "." + ext
|
||||
}
|
||||
path := filepath.Join(dir, fmt.Sprintf("%s%s", taskID, ext))
|
||||
if err := os.WriteFile(path, data, 0o644); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return path, nil
|
||||
}
|
||||
|
||||
// SaveTempFileByType
|
||||
// 根据传入的数据自动判断:
|
||||
// 若是 []byte 且后缀为 .mp3 → 保存二进制音频
|
||||
// 若是任意结构体/map → 自动转 JSON 保存
|
||||
// 返回:新临时文件路径、错误
|
||||
func SaveTempFileByType(taskID string, data any, oldTmpFile string) (string, error) {
|
||||
// 1. 先清理旧临时文件(统一逻辑)
|
||||
if oldTmpFile != "" {
|
||||
_ = os.Remove(oldTmpFile)
|
||||
}
|
||||
|
||||
var tmpPath string
|
||||
var tmpErr error
|
||||
|
||||
// 2. 判断是否是二进制音频([]byte + .mp3)
|
||||
if audioData, ok := data.([]byte); ok {
|
||||
tmpPath, tmpErr = saveTmpResult(taskID, audioData, ".mp3")
|
||||
} else {
|
||||
// 3. 其他类型 → 序列化为 JSON 保存
|
||||
mappedBytes, err := json.Marshal(data)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(mappedBytes) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
tmpPath, tmpErr = saveTmpResult(taskID, mappedBytes, ".json")
|
||||
}
|
||||
|
||||
if tmpErr != nil || tmpPath == "" {
|
||||
return "", tmpErr
|
||||
}
|
||||
|
||||
return tmpPath, nil
|
||||
}
|
||||
|
||||
// saveTmpResult 你原有的底层保存文件方法(保留不动)
|
||||
func saveTmpResult(taskID string, data []byte, ext string) (string, error) {
|
||||
// 你原来实现,比如:
|
||||
filename := taskID + ext
|
||||
tmpPath := filepath.Join(os.TempDir(), filename)
|
||||
err := os.WriteFile(tmpPath, data, 0644)
|
||||
return tmpPath, err
|
||||
return ext
|
||||
}
|
||||
|
||||
+21
-44
@@ -2,6 +2,7 @@ package util
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/url"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/utils"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
@@ -28,52 +29,28 @@ func AsyncCtx(ctx context.Context) context.Context {
|
||||
return asyncCtx
|
||||
}
|
||||
|
||||
// ForwardHeaders 透传调用链路的头信息,优先使用 ctx 中的固化值
|
||||
func ForwardHeaders(ctx context.Context) map[string]string {
|
||||
headers := make(map[string]string)
|
||||
SetHeaderFromContext(headers, ctx, "Authorization", "token")
|
||||
SetHeaderFromContext(headers, ctx, "X-User-Info", "xUserInfo")
|
||||
FallbackToRequestHeaders(headers, ctx)
|
||||
return headers
|
||||
// ======================== 请求工具 ========================
|
||||
|
||||
// ParseHeadMsgHeaders 从 head_msg 中提取 HTTP 请求头
|
||||
func ParseHeadMsgHeaders(headMsg map[string]any) map[string]string {
|
||||
if len(headMsg) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]string, len(headMsg))
|
||||
for k, v := range headMsg {
|
||||
out[k] = gconv.String(v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// SetHeaderFromContext 从上下文中设置 header
|
||||
func SetHeaderFromContext(headers map[string]string, ctx context.Context, headerKey, ctxKey string) {
|
||||
if value, ok := ctx.Value(ctxKey).(string); ok && value != "" {
|
||||
headers[headerKey] = value
|
||||
}
|
||||
}
|
||||
|
||||
// FallbackToRequestHeaders 从请求头中获取作为兜底
|
||||
func FallbackToRequestHeaders(headers map[string]string, ctx context.Context) {
|
||||
r := g.RequestFromCtx(ctx)
|
||||
if r == nil {
|
||||
return
|
||||
}
|
||||
|
||||
if headers["Authorization"] == "" {
|
||||
if token := r.Header.Get("Authorization"); token != "" {
|
||||
headers["Authorization"] = token
|
||||
}
|
||||
}
|
||||
|
||||
if headers["X-User-Info"] == "" {
|
||||
if userInfo := r.Header.Get("X-User-Info"); userInfo != "" {
|
||||
headers["X-User-Info"] = userInfo
|
||||
// BodyToQuery 将 body 转为 URL 查询参数
|
||||
func BodyToQuery(payload map[string]any) (url.Values, error) {
|
||||
q := url.Values{}
|
||||
for k, v := range payload {
|
||||
if v == nil {
|
||||
continue
|
||||
}
|
||||
q.Set(k, gconv.String(v))
|
||||
}
|
||||
}
|
||||
|
||||
// SetTaskHeadersToCtx 把任务入库时保存的 header 信息注入 ctx,给 worker 调 OSS 用
|
||||
func SetTaskHeadersToCtx(ctx context.Context, headers map[string]string) context.Context {
|
||||
if headers == nil {
|
||||
return ctx
|
||||
}
|
||||
if v := gconv.String(headers["Authorization"]); v != "" {
|
||||
ctx = context.WithValue(ctx, "token", v)
|
||||
}
|
||||
if v := gconv.String(headers["X-User-Info"]); v != "" {
|
||||
ctx = context.WithValue(ctx, "xUserInfo", v)
|
||||
}
|
||||
return ctx
|
||||
return q, nil
|
||||
}
|
||||
|
||||
+22
-235
@@ -1,35 +1,27 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"model-gateway/model/entity"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
tgjson "github.com/tidwall/gjson"
|
||||
)
|
||||
|
||||
// ParseAndValidate 解析并校验结果
|
||||
func ParseAndValidate(raw map[string]any, model *entity.ModelGatewayModel) (map[string]any, error) {
|
||||
// 1) 解析 content 字符串为 rounds 数组
|
||||
contentVal, ok := raw[model.ResponseBody]
|
||||
if !ok {
|
||||
return raw, fmt.Errorf("字段 %s 不存在", model.ResponseBody)
|
||||
}
|
||||
contentStr, ok := contentVal.(string)
|
||||
if !ok || strings.TrimSpace(contentStr) == "" {
|
||||
return raw, fmt.Errorf("字段 %s 为空或不是字符串", model.ResponseBody)
|
||||
// ======================== 响应解析 ========================
|
||||
|
||||
// ParseAndValidate 解析模型响应,校验必填字段,返回标准 rounds 格式
|
||||
func ParseAndValidate(raw map[string]any, requiredFields []string) (map[string]any, error) {
|
||||
contentStr := gconv.String(raw[entity.ResponseBody])
|
||||
if strings.TrimSpace(contentStr) == "" {
|
||||
return raw, fmt.Errorf("字段 %s 为空", entity.ResponseBody)
|
||||
}
|
||||
|
||||
contentStr = cleanControlChars(contentStr)
|
||||
|
||||
var arr []any
|
||||
if err := json.Unmarshal([]byte(contentStr), &arr); err != nil {
|
||||
return raw, fmt.Errorf("JSON解析失败: %w", err)
|
||||
@@ -38,81 +30,17 @@ func ParseAndValidate(raw map[string]any, model *entity.ModelGatewayModel) (map[
|
||||
return raw, fmt.Errorf("解析后数组为空")
|
||||
}
|
||||
|
||||
// 2) 校验必填字段
|
||||
if len(model.RequiredFields) > 0 {
|
||||
for _, field := range requiredFields {
|
||||
for i, r := range arr {
|
||||
round, ok := r.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for _, field := range model.RequiredFields {
|
||||
if gjson.New(round).Get(field).IsNil() {
|
||||
return raw, fmt.Errorf("rounds[%d] 缺少必填字段: %s", i, field)
|
||||
}
|
||||
round, _ := r.(map[string]any)
|
||||
if round != nil && gjson.New(round).Get(field).IsNil() {
|
||||
return raw, fmt.Errorf("rounds[%d] 缺少必填字段: %s", i, field)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return map[string]any{"total_rounds": len(arr), "rounds": arr}, nil
|
||||
}
|
||||
|
||||
// ParseStructResult 解析结构结果
|
||||
func ParseStructResult(raw map[string]any, responseBody string) map[string]any {
|
||||
contentVal := raw[responseBody]
|
||||
// 是字符串,尝试解析
|
||||
contentStr := gconv.String(contentVal)
|
||||
if contentStr == "" || contentStr == "0" {
|
||||
return map[string]any{
|
||||
"total_rounds": 1,
|
||||
"rounds": []map[string]any{{responseBody: raw}},
|
||||
}
|
||||
}
|
||||
|
||||
// 尝试解析为数组
|
||||
var arr []any
|
||||
if err := json.Unmarshal([]byte(contentStr), &arr); err == nil && len(arr) > 0 {
|
||||
return map[string]any{
|
||||
"total_rounds": 1,
|
||||
"rounds": []map[string]any{{responseBody: arr}},
|
||||
}
|
||||
}
|
||||
|
||||
// 尝试解析为单个对象
|
||||
var parsed any
|
||||
if err := json.Unmarshal([]byte(contentStr), &parsed); err == nil {
|
||||
return map[string]any{
|
||||
"total_rounds": 1,
|
||||
"rounds": []map[string]any{{responseBody: parsed}},
|
||||
}
|
||||
}
|
||||
|
||||
// 兜底:原始字符串作为内容
|
||||
return map[string]any{
|
||||
"total_rounds": 1,
|
||||
"rounds": []map[string]any{{responseBody: contentStr}},
|
||||
}
|
||||
}
|
||||
|
||||
// ParseHeadMsgHeaders 从 head_msg JSON 中提取请求头
|
||||
// head_msg 格式示例:
|
||||
//
|
||||
// {
|
||||
// "Authorization": "Bearer xxx",
|
||||
// "Content-Type": "application/json",
|
||||
// "X-Api-App-Id": "5147401364",
|
||||
// "X-Api-Access-Key": "VCqRX7..."
|
||||
// }
|
||||
func ParseHeadMsgHeaders(headMsg map[string]any) map[string]string {
|
||||
if len(headMsg) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make(map[string]string, len(headMsg))
|
||||
for k, v := range headMsg {
|
||||
out[k] = gconv.String(v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// MapResponsePayload 映射模型响应为标准格式
|
||||
func MapResponsePayload(mapping map[string]any, result map[string]any) (map[string]any, error) {
|
||||
if len(mapping) == 0 {
|
||||
@@ -135,7 +63,7 @@ func MapResponsePayload(mapping map[string]any, result map[string]any) (map[stri
|
||||
if !value.Exists() {
|
||||
continue
|
||||
}
|
||||
// 如果是数组路径(含 #),取 Array;否则取单值
|
||||
|
||||
if strings.Contains(path, "#") {
|
||||
var arr []any
|
||||
for _, v := range value.Array() {
|
||||
@@ -150,154 +78,13 @@ func MapResponsePayload(mapping map[string]any, result map[string]any) (map[stri
|
||||
return mapped, nil
|
||||
}
|
||||
|
||||
//
|
||||
//// GetModelBody 获取数据库中保存的模型信息
|
||||
//func GetModelBody(v map[string]any) map[string]any {
|
||||
// if v == nil {
|
||||
// return nil
|
||||
// }
|
||||
// if p, ok := v["body"]; ok {
|
||||
// return gconv.Map(p)
|
||||
// }
|
||||
// return v
|
||||
//}
|
||||
// ======================== 内部辅助 ========================
|
||||
|
||||
// BodyToQuery 将 body 转为 url.Values
|
||||
func BodyToQuery(payload map[string]any) (url.Values, error) {
|
||||
q := url.Values{}
|
||||
for k, v := range payload {
|
||||
if v == nil {
|
||||
continue
|
||||
func cleanControlChars(s string) string {
|
||||
return strings.Map(func(r rune) rune {
|
||||
if r < 32 && r != ' ' {
|
||||
return -1
|
||||
}
|
||||
q.Set(k, gconv.String(v))
|
||||
}
|
||||
return q, nil
|
||||
}
|
||||
|
||||
// PullTaskResult 轮询查询异步任务结果直到完成
|
||||
func PullTaskResult(ctx context.Context, body map[string]any, queryConfig map[string]any, headMsg map[string]any) (map[string]any, error) {
|
||||
// 1) 解析配置
|
||||
// 1.1 提取 taskID
|
||||
taskIDPath := gconv.String(queryConfig["task_id"])
|
||||
taskID := gconv.String(gjson.New(body).Get(taskIDPath).Val())
|
||||
if taskID == "" {
|
||||
return nil, fmt.Errorf("无法从路径 %s 提取 taskID", taskIDPath)
|
||||
}
|
||||
g.Log().Infof(ctx, "[PullTaskResult] taskID=%s", taskID)
|
||||
|
||||
// 1.2 请求地址,替换 {id}
|
||||
queryUrl := gconv.String(queryConfig["url"])
|
||||
queryUrl = replaceURLParams(queryUrl, map[string]any{"id": taskID})
|
||||
|
||||
// 1.3 请求方式
|
||||
method := gconv.String(queryConfig["method"])
|
||||
if method == "" {
|
||||
method = "GET"
|
||||
}
|
||||
|
||||
// 1.4 状态判断配置
|
||||
statusPath := gconv.String(queryConfig["status_path"])
|
||||
statusValues, _ := queryConfig["status_values"].(map[string]any)
|
||||
if statusPath == "" {
|
||||
statusPath = "status"
|
||||
}
|
||||
|
||||
// 1.5 轮询间隔
|
||||
interval := gconv.Int(queryConfig["interval_seconds"])
|
||||
if interval <= 0 {
|
||||
interval = 2
|
||||
}
|
||||
|
||||
// 1.6 请求体
|
||||
reqBodyMap := map[string]any{"task_id": taskID}
|
||||
|
||||
// 2) 轮询请求
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
var reqBody io.Reader
|
||||
if method == "POST" {
|
||||
bs, _ := json.Marshal(reqBodyMap)
|
||||
reqBody = bytes.NewReader(bs)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, method, queryUrl, reqBody)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建请求失败: %w", err)
|
||||
}
|
||||
|
||||
// 统一用 headMsg 注入请求头
|
||||
for hk, hv := range ParseHeadMsgHeaders(headMsg) {
|
||||
req.Header.Set(hk, hv)
|
||||
}
|
||||
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[PullTaskResult] 请求失败 taskID=%s err=%v", taskID, err)
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
_ = resp.Body.Close()
|
||||
|
||||
g.Log().Infof(ctx, "[PullTaskResult] taskID=%s statusCode=%d body=%s", taskID, resp.StatusCode, string(raw))
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
var result map[string]any
|
||||
_ = json.Unmarshal(raw, &result)
|
||||
|
||||
statusVal := gjson.New(result).Get(statusPath).Val()
|
||||
statusStr := gconv.String(statusVal)
|
||||
g.Log().Infof(ctx, "[PullTaskResult] 状态 taskID=%s status=%v", taskID, statusVal)
|
||||
|
||||
if matchStatus(statusStr, statusValues["succeeded"]) {
|
||||
g.Log().Infof(ctx, "[PullTaskResult] 任务成功 taskID=%s", taskID)
|
||||
return result, nil
|
||||
}
|
||||
|
||||
if matchStatus(statusStr, statusValues["failed"]) {
|
||||
g.Log().Errorf(ctx, "[PullTaskResult] 任务失败 taskID=%s", taskID)
|
||||
return result, fmt.Errorf("任务失败")
|
||||
}
|
||||
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func matchStatus(actual string, expected any) bool {
|
||||
expectedStr := gconv.String(expected)
|
||||
if actual == expectedStr {
|
||||
return true
|
||||
}
|
||||
switch v := expected.(type) {
|
||||
case []any:
|
||||
for _, item := range v {
|
||||
if actual == gconv.String(item) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// replaceURLParams 替换 URL 中的 {key}
|
||||
func replaceURLParams(url string, params map[string]any) string {
|
||||
re := regexp.MustCompile(`\{([^}]+)}`)
|
||||
return re.ReplaceAllStringFunc(url, func(s string) string {
|
||||
key := strings.Trim(s, "{}")
|
||||
if val, ok := params[key]; ok {
|
||||
return gconv.String(val)
|
||||
}
|
||||
return s
|
||||
})
|
||||
return r
|
||||
}, s)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,391 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"model-gateway/model/entity"
|
||||
"model-gateway/service/gateway"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
// ======================== 表单校验与构建 ========================
|
||||
|
||||
// ValidateAndParseForm 校验表单并转为嵌套 map
|
||||
func ValidateAndParseForm(forms []entity.Form) (map[string]any, error) {
|
||||
result := gjson.New("{}")
|
||||
for _, form := range forms {
|
||||
if form.Key == "" {
|
||||
continue
|
||||
}
|
||||
if form.Required && (form.Value == nil || gconv.String(form.Value) == "") {
|
||||
return nil, fmt.Errorf("字段 %s 为必填", form.Label)
|
||||
}
|
||||
if form.Value == nil {
|
||||
continue
|
||||
}
|
||||
|
||||
val, err := validateAndConvert(form)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_ = result.Set(form.Key, val)
|
||||
}
|
||||
return result.Map(), nil
|
||||
}
|
||||
|
||||
// validateAndConvert 验证表单字段并转为标准格式
|
||||
func validateAndConvert(form entity.Form) (any, error) {
|
||||
val := form.Value
|
||||
fc := form.FieldConstraint
|
||||
|
||||
switch form.Type {
|
||||
case "string":
|
||||
s := gconv.String(val)
|
||||
if fc.MaxLength > 0 && len(s) > fc.MaxLength {
|
||||
return nil, fmt.Errorf("字段 %s 超过最大长度 %d", form.Label, fc.MaxLength)
|
||||
}
|
||||
if fc.MinLength > 0 && len(s) < fc.MinLength {
|
||||
return nil, fmt.Errorf("字段 %s 不足最小长度 %d", form.Label, fc.MinLength)
|
||||
}
|
||||
return s, nil
|
||||
|
||||
case "number":
|
||||
f := gconv.Float64(val)
|
||||
if fc.Min != nil && f < gconv.Float64(fc.Min) {
|
||||
return nil, fmt.Errorf("字段 %s 不能小于 %v", form.Label, fc.Min)
|
||||
}
|
||||
if fc.Max != nil && f > gconv.Float64(fc.Max) {
|
||||
return nil, fmt.Errorf("字段 %s 不能大于 %v", form.Label, fc.Max)
|
||||
}
|
||||
switch fc.NumberType {
|
||||
case "float", "positiveFloat", "negativeFloat":
|
||||
return f, nil
|
||||
default:
|
||||
return int(f), nil
|
||||
}
|
||||
|
||||
case "select", "radio":
|
||||
v, ok := val.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("字段 %s 格式错误", form.Label)
|
||||
}
|
||||
return v, nil
|
||||
|
||||
case "upload":
|
||||
var urls []string
|
||||
switch v := val.(type) {
|
||||
case []any:
|
||||
for _, u := range v {
|
||||
urls = append(urls, gconv.String(u))
|
||||
}
|
||||
case []string:
|
||||
urls = v
|
||||
case string:
|
||||
if v != "" {
|
||||
urls = []string{v}
|
||||
}
|
||||
}
|
||||
if fc.MaxCount > 0 && len(urls) > fc.MaxCount {
|
||||
return nil, fmt.Errorf("字段 %s 上传数量超过上限 %d", form.Label, fc.MaxCount)
|
||||
}
|
||||
if fc.Accept != "" && len(urls) > 0 {
|
||||
allowed := strings.Split(fc.Accept, ",")
|
||||
for _, fileUrl := range urls {
|
||||
ext := strings.ToLower(strings.TrimPrefix(filepath.Ext(fileUrl), "."))
|
||||
if !containsExt(allowed, ext) {
|
||||
return nil, fmt.Errorf("字段 %s 不支持的文件格式: %s", form.Label, ext)
|
||||
}
|
||||
}
|
||||
}
|
||||
return urls, nil
|
||||
|
||||
default:
|
||||
return gconv.String(val), nil
|
||||
}
|
||||
}
|
||||
|
||||
func containsExt(allowed []string, ext string) bool {
|
||||
for _, a := range allowed {
|
||||
if strings.TrimSpace(a) == ext {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// BuildTemplateFromForm 从 Form 构建模板
|
||||
func BuildTemplateFromForm(forms []entity.Form) map[string]any {
|
||||
jsonObj := gjson.New("{}")
|
||||
for _, form := range forms {
|
||||
if form.Key == "" {
|
||||
continue
|
||||
}
|
||||
if form.DefaultValue != nil {
|
||||
_ = jsonObj.Set(form.Key, form.DefaultValue)
|
||||
} else {
|
||||
switch form.Type {
|
||||
case "number":
|
||||
_ = jsonObj.Set(form.Key, 0)
|
||||
case "boolean":
|
||||
_ = jsonObj.Set(form.Key, false)
|
||||
default:
|
||||
_ = jsonObj.Set(form.Key, "")
|
||||
}
|
||||
}
|
||||
}
|
||||
return jsonObj.Map()
|
||||
}
|
||||
|
||||
// ======================== 提示词拼接 ========================
|
||||
|
||||
// GetFormByRole 根据 role 获取单个 Form
|
||||
func GetFormByRole(forms []entity.Form, role string) (entity.Form, bool) {
|
||||
for _, f := range forms {
|
||||
if f.Role == role {
|
||||
return f, true
|
||||
}
|
||||
}
|
||||
return entity.Form{}, false
|
||||
}
|
||||
|
||||
// GetRoleContentPath 从 Form 数组中提取指定 role 的 Key 路径
|
||||
func GetRoleContentPath(forms []entity.Form, role string) string {
|
||||
for _, form := range forms {
|
||||
if form.Role == role {
|
||||
return form.Key
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// MergePrompt 将多个提示词拼接到指定路径的 content 中
|
||||
func MergePrompt(messages map[string]any, contentPath string, prompts ...string) map[string]any {
|
||||
var parts []string
|
||||
for _, p := range prompts {
|
||||
if p != "" {
|
||||
parts = append(parts, p)
|
||||
}
|
||||
}
|
||||
if len(parts) == 0 {
|
||||
return messages
|
||||
}
|
||||
|
||||
appendContent := strings.Join(parts, "\n")
|
||||
msgJson := gjson.New(messages)
|
||||
|
||||
existing := msgJson.Get(contentPath).String()
|
||||
if existing != "" {
|
||||
appendContent = existing + "\n" + appendContent
|
||||
}
|
||||
_ = msgJson.Set(contentPath, appendContent)
|
||||
|
||||
return msgJson.Map()
|
||||
}
|
||||
|
||||
// ExtractUserContent 将 messages 转为自然语言文本
|
||||
func ExtractUserContent(messages map[string]any) string {
|
||||
return formatMapToText(messages, "")
|
||||
}
|
||||
|
||||
func formatMapToText(v any, prefix string) string {
|
||||
var b strings.Builder
|
||||
switch val := v.(type) {
|
||||
case map[string]any:
|
||||
for k, vv := range val {
|
||||
switch vv.(type) {
|
||||
case map[string]any, []any:
|
||||
b.WriteString(formatMapToText(vv, prefix))
|
||||
default:
|
||||
b.WriteString(fmt.Sprintf("%s:%v,", k, vv))
|
||||
}
|
||||
}
|
||||
case []any:
|
||||
for _, elem := range val {
|
||||
b.WriteString(formatMapToText(elem, ""))
|
||||
}
|
||||
case string:
|
||||
b.WriteString(fmt.Sprintf("%s,", val))
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// InjectHistory 拼接历史提示词
|
||||
func InjectHistory(messages map[string]any, history []gateway.SessionHistoryItem, mergeOrder []string) map[string]any {
|
||||
msgs, _ := messages["messages"].([]any)
|
||||
|
||||
// 按 role 分组
|
||||
grouped := make(map[string][]any)
|
||||
for _, m := range msgs {
|
||||
if msg, ok := m.(map[string]any); ok {
|
||||
role := gconv.String(msg["role"])
|
||||
grouped[role] = append(grouped[role], msg)
|
||||
}
|
||||
}
|
||||
|
||||
result := make([]any, 0, len(msgs)+len(history))
|
||||
for _, part := range mergeOrder {
|
||||
switch part {
|
||||
case "history":
|
||||
for _, h := range history {
|
||||
result = append(result, map[string]any{
|
||||
"role": h.Role,
|
||||
"content": h.Content,
|
||||
})
|
||||
}
|
||||
default:
|
||||
if msgs, ok := grouped[part]; ok {
|
||||
result = append(result, msgs...)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
messages["messages"] = result
|
||||
return messages
|
||||
}
|
||||
|
||||
// SplitByAttachment 根据附件数量拆分多轮,未超出返回空
|
||||
func SplitByAttachment(messages map[string]any, forms []entity.Form) []map[string]any {
|
||||
// 1) 获取约束
|
||||
maxVideo := getMaxCount(forms, "video")
|
||||
maxImage := getMaxCount(forms, "image")
|
||||
maxAudio := getMaxCount(forms, "audio")
|
||||
|
||||
// 2) 提取 system 消息和 user content
|
||||
msgs := gjson.New(messages).Get("messages").Array()
|
||||
var systemMsg map[string]any
|
||||
var userContent []any
|
||||
|
||||
for _, m := range msgs {
|
||||
msg := m.(map[string]any)
|
||||
switch msg["role"] {
|
||||
case "system":
|
||||
systemMsg = msg
|
||||
case "user":
|
||||
if c, ok := msg["content"].([]any); ok {
|
||||
userContent = c
|
||||
}
|
||||
}
|
||||
}
|
||||
if systemMsg == nil || len(userContent) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
// 3) 分离 text 和各类附件
|
||||
var texts []any
|
||||
var videos, images, audios []any
|
||||
|
||||
for _, c := range userContent {
|
||||
item := c.(map[string]any)
|
||||
switch item["type"] {
|
||||
case "video_url":
|
||||
videos = append(videos, item)
|
||||
case "image_url":
|
||||
images = append(images, item)
|
||||
case "input_audio":
|
||||
audios = append(audios, item)
|
||||
default:
|
||||
texts = append(texts, item)
|
||||
}
|
||||
}
|
||||
|
||||
// 4) 判断是否需要拆分
|
||||
if len(videos) <= maxVideo && len(images) <= maxImage && len(audios) <= maxAudio {
|
||||
return nil
|
||||
}
|
||||
|
||||
// 5) 按约束分片
|
||||
videoChunks := chunk(videos, maxVideo)
|
||||
imageChunks := chunk(images, maxImage)
|
||||
audioChunks := chunk(audios, maxAudio)
|
||||
|
||||
maxRounds := len(videoChunks)
|
||||
if len(imageChunks) > maxRounds {
|
||||
maxRounds = len(imageChunks)
|
||||
}
|
||||
if len(audioChunks) > maxRounds {
|
||||
maxRounds = len(audioChunks)
|
||||
}
|
||||
|
||||
// 6) 构建每轮 messages
|
||||
var rounds []map[string]any
|
||||
for i := 0; i < maxRounds; i++ {
|
||||
var newContent []any
|
||||
newContent = append(newContent, texts...)
|
||||
if i < len(videoChunks) {
|
||||
newContent = append(newContent, videoChunks[i]...)
|
||||
} else {
|
||||
newContent = append(newContent, videos...) // 未超出,每轮都带
|
||||
}
|
||||
if i < len(imageChunks) {
|
||||
newContent = append(newContent, imageChunks[i]...)
|
||||
} else {
|
||||
newContent = append(newContent, images...)
|
||||
}
|
||||
if i < len(audioChunks) {
|
||||
newContent = append(newContent, audioChunks[i]...)
|
||||
} else {
|
||||
newContent = append(newContent, audios...)
|
||||
}
|
||||
|
||||
rounds = append(rounds, map[string]any{
|
||||
"model": gjson.New(messages).Get("model").Val(),
|
||||
"max_tokens": gjson.New(messages).Get("max_tokens").Val(),
|
||||
"messages": []any{
|
||||
systemMsg,
|
||||
map[string]any{"role": "user", "content": newContent},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
return rounds
|
||||
}
|
||||
|
||||
// getMaxCount 从表单获取指定 role 的最大数量
|
||||
func getMaxCount(forms []entity.Form, role string) int {
|
||||
for _, f := range forms {
|
||||
if f.Role == role {
|
||||
return f.FieldConstraint.MaxCount
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// chunk 将数组按 size 分片
|
||||
func chunk(items []any, size int) [][]any {
|
||||
if size <= 0 {
|
||||
return nil
|
||||
}
|
||||
var chunks [][]any
|
||||
for i := 0; i < len(items); i += size {
|
||||
end := i + size
|
||||
if end > len(items) {
|
||||
end = len(items)
|
||||
}
|
||||
chunks = append(chunks, items[i:end])
|
||||
}
|
||||
return chunks
|
||||
}
|
||||
|
||||
// ======================== 请求模板 ========================
|
||||
|
||||
// BuildRequestBody 替换模板占位符构建请求体
|
||||
func BuildRequestBody(template map[string]any, modelName, systemPrompt, userContent string) map[string]any {
|
||||
escapedSystem := escapeJSON(systemPrompt)
|
||||
escapedUser := escapeJSON(userContent)
|
||||
|
||||
str := gjson.New(template).MustToJsonString()
|
||||
str = strings.ReplaceAll(str, `"{{model}}"`, `"`+modelName+`"`)
|
||||
str = strings.ReplaceAll(str, `"{{system}}"`, escapedSystem)
|
||||
str = strings.ReplaceAll(str, `"{{user}}"`, escapedUser)
|
||||
|
||||
return gjson.New(str).Map()
|
||||
}
|
||||
|
||||
func escapeJSON(s string) string {
|
||||
b, _ := json.Marshal(s)
|
||||
return string(b)
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
// PullTaskResult 拉取任务结果
|
||||
func PullTaskResult(ctx context.Context, body map[string]any, queryConfig map[string]any, headMsg map[string]any) (map[string]any, error) {
|
||||
taskID, err := extractTaskID(body, queryConfig)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
g.Log().Infof(ctx, "[PullTaskResult] taskID=%s", taskID)
|
||||
|
||||
queryUrl := buildQueryURL(queryConfig, taskID)
|
||||
method := gconv.String(queryConfig["method"])
|
||||
if method == "" {
|
||||
method = "GET"
|
||||
}
|
||||
interval := gconv.Int(queryConfig["interval_seconds"])
|
||||
if interval <= 0 {
|
||||
interval = 2
|
||||
}
|
||||
|
||||
statusPath := gconv.String(queryConfig["status_path"])
|
||||
if statusPath == "" {
|
||||
statusPath = "status"
|
||||
}
|
||||
statusValues, _ := queryConfig["status_values"].(map[string]any)
|
||||
|
||||
// 失败信息路径
|
||||
errorPath := gconv.String(queryConfig["error_path"])
|
||||
if errorPath == "" {
|
||||
errorPath = "error.message"
|
||||
}
|
||||
|
||||
reqBodyMap := map[string]any{"task_id": taskID}
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
default:
|
||||
}
|
||||
|
||||
result, err := doQueryRequest(ctx, method, queryUrl, reqBodyMap, headMsg)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[PullTaskResult] 请求失败 taskID=%s err=%v", taskID, err)
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
if result == nil {
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
statusStr := gconv.String(gjson.New(result).Get(statusPath).Val())
|
||||
g.Log().Infof(ctx, "[PullTaskResult] 状态 taskID=%s status=%s", taskID, statusStr)
|
||||
|
||||
if matchStatus(statusStr, statusValues["succeeded"]) {
|
||||
g.Log().Infof(ctx, "[PullTaskResult] 任务成功 taskID=%s", taskID)
|
||||
return result, nil
|
||||
}
|
||||
|
||||
if matchStatus(statusStr, statusValues["failed"]) {
|
||||
errMsg := gconv.String(gjson.New(result).Get(errorPath).Val())
|
||||
if errMsg == "" {
|
||||
errMsg = "任务失败"
|
||||
}
|
||||
g.Log().Errorf(ctx, "[PullTaskResult] 任务失败 taskID=%s err=%s", taskID, errMsg)
|
||||
return result, fmt.Errorf("任务失败: %s", errMsg)
|
||||
}
|
||||
|
||||
time.Sleep(time.Duration(interval) * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func extractTaskID(body, queryConfig map[string]any) (string, error) {
|
||||
taskIDPath := gconv.String(queryConfig["task_id"])
|
||||
taskID := gconv.String(gjson.New(body).Get(taskIDPath).Val())
|
||||
if taskID == "" {
|
||||
return "", fmt.Errorf("无法从路径 %s 提取 taskID", taskIDPath)
|
||||
}
|
||||
return taskID, nil
|
||||
}
|
||||
|
||||
func buildQueryURL(queryConfig map[string]any, taskID string) string {
|
||||
queryUrl := gconv.String(queryConfig["url"])
|
||||
return replaceURLParams(queryUrl, map[string]any{"id": taskID})
|
||||
}
|
||||
|
||||
func doQueryRequest(ctx context.Context, method, queryUrl string, reqBodyMap map[string]any, headMsg map[string]any) (map[string]any, error) {
|
||||
var reqBody io.Reader
|
||||
if method == "POST" {
|
||||
bs, _ := json.Marshal(reqBodyMap)
|
||||
reqBody = bytes.NewReader(bs)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, method, queryUrl, reqBody)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建请求失败: %w", err)
|
||||
}
|
||||
|
||||
for hk, hv := range ParseHeadMsgHeaders(headMsg) {
|
||||
req.Header.Set(hk, hv)
|
||||
}
|
||||
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
var result map[string]any
|
||||
_ = json.Unmarshal(raw, &result)
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func matchStatus(actual string, expected any) bool {
|
||||
expectedStr := gconv.String(expected)
|
||||
if actual == expectedStr {
|
||||
return true
|
||||
}
|
||||
if arr, ok := expected.([]any); ok {
|
||||
for _, item := range arr {
|
||||
if actual == gconv.String(item) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func replaceURLParams(rawURL string, params map[string]any) string {
|
||||
re := regexp.MustCompile(`\{([^}]+)}`)
|
||||
return re.ReplaceAllStringFunc(rawURL, func(s string) string {
|
||||
key := strings.Trim(s, "{}")
|
||||
if val, ok := params[key]; ok {
|
||||
return gconv.String(val)
|
||||
}
|
||||
return s
|
||||
})
|
||||
}
|
||||
+179
-74
@@ -1,40 +1,128 @@
|
||||
package util
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"model-gateway/service/gateway"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
)
|
||||
|
||||
// ================================================================
|
||||
|
||||
// ParseStreamResponse 流式响应解析(通用入口)
|
||||
func ParseStreamResponse(rawBytes []byte, streamConfig map[string]any) (map[string]any, error) {
|
||||
// ParseStreamResponse 流式响应解析
|
||||
func ParseStreamResponse(ctx context.Context, rawBytes []byte, streamConfig map[string]any) (map[string]any, error) {
|
||||
enabled, _ := streamConfig["enabled"].(bool)
|
||||
if !enabled {
|
||||
return gjson.New(string(rawBytes)).Map(), nil
|
||||
}
|
||||
|
||||
parser, _ := streamConfig["parser"].(string)
|
||||
if parser == "base64_concat" {
|
||||
return parseBase64Stream(rawBytes)
|
||||
outputType, _ := streamConfig["output_type"].(string)
|
||||
streamToClient, _ := streamConfig["stream_to_client"].(bool)
|
||||
events, _ := streamConfig["events"].([]any)
|
||||
|
||||
if len(events) == 0 {
|
||||
return gjson.New(string(rawBytes)).Map(), nil
|
||||
}
|
||||
|
||||
return parseSSEStream(rawBytes, streamConfig)
|
||||
// 如果业务需要流式返回,直接透传原始数据
|
||||
if streamToClient {
|
||||
return map[string]any{"stream_data": rawBytes}, nil
|
||||
}
|
||||
|
||||
result := make(map[string]any)
|
||||
lines := strings.Split(string(rawBytes), "\n")
|
||||
|
||||
for _, evt := range events {
|
||||
e, _ := evt.(map[string]any)
|
||||
evtType, _ := e["type"].(string)
|
||||
|
||||
switch evtType {
|
||||
case "concat":
|
||||
processConcat(lines, e, result)
|
||||
case "base64_concat":
|
||||
processBase64Concat(lines, e, result)
|
||||
case "collect":
|
||||
processCollect(lines, e, result)
|
||||
case "final":
|
||||
processFinal(lines, e, result)
|
||||
}
|
||||
}
|
||||
|
||||
switch outputType {
|
||||
case "audio":
|
||||
if audioBytes, ok := result["audio"].([]byte); ok {
|
||||
oss, err := gateway.UploadByTask(ctx, audioBytes, "mp3")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result["content"] = oss.FileAddressPrefix + oss.FileURL
|
||||
}
|
||||
case "text":
|
||||
if v, ok := result["content"]; ok {
|
||||
return map[string]any{"content": v, "usage": result["usage"]}, nil
|
||||
}
|
||||
case "image":
|
||||
if v, ok := result["urls"]; ok {
|
||||
return map[string]any{"content": v, "usage": result["usage"]}, nil
|
||||
}
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// parseBase64Stream 拼接流式 base64 并解码为二进制(TTS 等音频模型)
|
||||
func parseBase64Stream(rawBytes []byte) (map[string]any, error) {
|
||||
lines := strings.Split(string(rawBytes), "\n")
|
||||
var audioBase64 strings.Builder
|
||||
// processConcat 文本拼接
|
||||
func processConcat(lines []string, event map[string]any, result map[string]any) {
|
||||
match, _ := event["match"].(string)
|
||||
aggregateTo, _ := event["aggregate_to"].(string)
|
||||
fields, _ := event["fields"].(map[string]any)
|
||||
|
||||
var parts []string
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" {
|
||||
if line == "" || line == "[DONE]" {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(line, "event:") {
|
||||
continue
|
||||
}
|
||||
if strings.HasPrefix(line, "data:") {
|
||||
line = strings.TrimPrefix(line, "data:")
|
||||
line = strings.TrimSpace(line)
|
||||
}
|
||||
|
||||
var chunk map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &chunk); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
chunkType, _ := chunk["type"].(string)
|
||||
if match != "" && !strings.Contains(chunkType, match) {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, chunkPath := range fields {
|
||||
val := gjson.New(chunk).Get(chunkPath.(string)).String()
|
||||
if val != "" {
|
||||
parts = append(parts, val)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
result[aggregateTo] = strings.Join(parts, "")
|
||||
}
|
||||
|
||||
// processBase64Concat base64 拼接
|
||||
func processBase64Concat(lines []string, event map[string]any, result map[string]any) {
|
||||
aggregateTo, _ := event["aggregate_to"].(string)
|
||||
fields, _ := event["fields"].(map[string]any)
|
||||
|
||||
var builder strings.Builder
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || line == "[DONE]" {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -43,8 +131,10 @@ func parseBase64Stream(rawBytes []byte) (map[string]any, error) {
|
||||
continue
|
||||
}
|
||||
|
||||
if data, ok := chunk["data"].(string); ok && data != "" {
|
||||
audioBase64.WriteString(data)
|
||||
for _, chunkPath := range fields {
|
||||
if data := gjson.New(chunk).Get(chunkPath.(string)).String(); data != "" {
|
||||
builder.WriteString(data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -53,29 +143,73 @@ func parseBase64Stream(rawBytes []byte) (map[string]any, error) {
|
||||
return -1
|
||||
}
|
||||
return r
|
||||
}, audioBase64.String())
|
||||
}, builder.String())
|
||||
|
||||
audioBytes, err := base64.StdEncoding.DecodeString(cleanBase64)
|
||||
if err != nil {
|
||||
audioBytes, err = base64.RawStdEncoding.DecodeString(cleanBase64)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("base64 解码失败: %w", err)
|
||||
}
|
||||
audioBytes, _ = base64.RawStdEncoding.DecodeString(cleanBase64)
|
||||
}
|
||||
|
||||
return map[string]any{"audio": audioBytes}, nil
|
||||
result[aggregateTo] = audioBytes
|
||||
}
|
||||
|
||||
// parseSSEStream SSE 流式解析(图片模型等)
|
||||
func parseSSEStream(rawBytes []byte, streamConfig map[string]any) (map[string]any, error) {
|
||||
events, _ := streamConfig["events"].([]any)
|
||||
if len(events) == 0 {
|
||||
return gjson.New(string(rawBytes)).Map(), nil
|
||||
// processCollect 数组收集
|
||||
func processCollect(lines []string, event map[string]any, result map[string]any) {
|
||||
match, _ := event["match"].(string)
|
||||
aggregateTo, _ := event["aggregate_to"].(string)
|
||||
orderBy, _ := event["order_by"].(string)
|
||||
fields, _ := event["fields"].(map[string]any)
|
||||
|
||||
var items []map[string]any
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" || line == "[DONE]" {
|
||||
continue
|
||||
}
|
||||
|
||||
var chunk map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &chunk); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
chunkType, _ := chunk["type"].(string)
|
||||
if match != "" && !strings.Contains(chunkType, match) {
|
||||
continue
|
||||
}
|
||||
|
||||
item := make(map[string]any)
|
||||
for localKey, chunkPath := range fields {
|
||||
item[localKey] = gjson.New(chunk).Get(chunkPath.(string)).Val()
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
|
||||
lines := strings.Split(string(rawBytes), "\n")
|
||||
result := make(map[string]any)
|
||||
var partials []map[string]any
|
||||
if orderBy != "" {
|
||||
sort.Slice(items, func(i, j int) bool {
|
||||
return fmt.Sprint(items[i][orderBy]) < fmt.Sprint(items[j][orderBy])
|
||||
})
|
||||
}
|
||||
|
||||
// 如果只有一个字段,直接存值数组
|
||||
if len(fields) == 1 {
|
||||
var vals []any
|
||||
for _, item := range items {
|
||||
for _, v := range item {
|
||||
vals = append(vals, v)
|
||||
}
|
||||
}
|
||||
result[aggregateTo] = vals
|
||||
} else {
|
||||
result[aggregateTo] = items
|
||||
}
|
||||
}
|
||||
|
||||
// processFinal 取最后一条匹配的数据
|
||||
func processFinal(lines []string, event map[string]any, result map[string]any) {
|
||||
match, _ := event["match"].(string)
|
||||
aggregateTo, _ := event["aggregate_to"].(string)
|
||||
fields, _ := event["fields"].(map[string]any)
|
||||
|
||||
var lastMatch map[string]any
|
||||
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
@@ -96,55 +230,26 @@ func parseSSEStream(rawBytes []byte, streamConfig map[string]any) (map[string]an
|
||||
}
|
||||
|
||||
chunkType, _ := chunk["type"].(string)
|
||||
if match != "" && !strings.Contains(chunkType, match) {
|
||||
continue
|
||||
}
|
||||
|
||||
for _, evt := range events {
|
||||
e, _ := evt.(map[string]any)
|
||||
match, _ := e["match"].(string)
|
||||
if !strings.Contains(chunkType, match) {
|
||||
continue
|
||||
}
|
||||
lastMatch = chunk
|
||||
}
|
||||
|
||||
fields, _ := e["fields"].(map[string]any)
|
||||
aggregateTo, _ := e["aggregate_to"].(string)
|
||||
evtType, _ := e["type"].(string)
|
||||
if lastMatch == nil {
|
||||
return
|
||||
}
|
||||
|
||||
switch evtType {
|
||||
case "partial":
|
||||
item := make(map[string]any)
|
||||
for localKey, chunkKey := range fields {
|
||||
item[localKey] = chunk[chunkKey.(string)]
|
||||
}
|
||||
partials = append(partials, item)
|
||||
|
||||
case "final":
|
||||
for localKey, chunkKey := range fields {
|
||||
val := gjson.New(chunk).Get(chunkKey.(string))
|
||||
if !val.IsNil() {
|
||||
if _, exists := result[aggregateTo]; !exists {
|
||||
result[aggregateTo] = make(map[string]any)
|
||||
}
|
||||
result[aggregateTo].(map[string]any)[localKey] = val.Val()
|
||||
}
|
||||
}
|
||||
}
|
||||
data := make(map[string]any)
|
||||
for localKey, chunkPath := range fields {
|
||||
val := gjson.New(lastMatch).Get(chunkPath.(string)).Val()
|
||||
if val != nil {
|
||||
data[localKey] = val
|
||||
}
|
||||
}
|
||||
|
||||
if len(partials) > 0 {
|
||||
for _, evt := range events {
|
||||
e, _ := evt.(map[string]any)
|
||||
if e["type"] == "partial" {
|
||||
if orderBy, ok := e["order_by"].(string); ok {
|
||||
sort.Slice(partials, func(i, j int) bool {
|
||||
return fmt.Sprint(partials[i][orderBy]) < fmt.Sprint(partials[j][orderBy])
|
||||
})
|
||||
}
|
||||
result[e["aggregate_to"].(string)] = partials
|
||||
break
|
||||
}
|
||||
}
|
||||
if len(data) > 0 {
|
||||
result[aggregateTo] = data
|
||||
}
|
||||
|
||||
mergedBytes, _ := json.Marshal(result)
|
||||
return gjson.New(mergedBytes).Map(), nil
|
||||
}
|
||||
|
||||
+47
-18
@@ -28,10 +28,10 @@ database:
|
||||
timeMaintainDisabled: false # (可选)是否完全关闭时间更新特性,为true时CreatedAt/UpdatedAt/DeletedAt都将失效
|
||||
model_gateway:
|
||||
- type: "pgsql"
|
||||
host: "116.204.74.41"
|
||||
port: "15432"
|
||||
host: "192.168.3.8"
|
||||
port: "5432"
|
||||
user: "postgres"
|
||||
pass: "Bjang09@686^*^"
|
||||
pass: "123456"
|
||||
name: "model-gateway"
|
||||
prefix: ""
|
||||
role: "master"
|
||||
@@ -39,8 +39,8 @@ database:
|
||||
dryRun: false
|
||||
charset: "utf8"
|
||||
timezone: "Asia/Shanghai"
|
||||
maxIdle: 5
|
||||
maxOpen: 20
|
||||
maxIdle: 15
|
||||
maxOpen: 60
|
||||
maxLifetime: "30s"
|
||||
maxIdleConnTime: "30s"
|
||||
createdAt: "created_at"
|
||||
@@ -60,16 +60,45 @@ jaeger:
|
||||
addr: 192.168.3.30:4318
|
||||
|
||||
# 本地调试用:可选自动执行 worker/cleaner(默认关闭)
|
||||
asynch:
|
||||
queryPending:
|
||||
enabled: false
|
||||
intervalSeconds: 10 # 每10秒轮询一次
|
||||
limit: 10 # 每次查10条
|
||||
worker:
|
||||
enabled: false
|
||||
intervalSeconds: 5
|
||||
batchSize: 10
|
||||
goroutines: 1
|
||||
cleaner:
|
||||
enabled: false
|
||||
intervalSeconds: 30
|
||||
queryPending:
|
||||
enabled: false
|
||||
intervalSeconds: 10 # 每10秒轮询一次
|
||||
limit: 10 # 每次查10条
|
||||
jobTask:
|
||||
intervalSeconds: 10 # 轮询间隔(秒)
|
||||
batchSize: 10 # 每批处理条数
|
||||
poolSize: 5 # 协程池大小
|
||||
|
||||
modelPrompts:
|
||||
types:
|
||||
100: |
|
||||
你是一个智能文字处理助手,专注于文本理解、文本创作、文本优化与语言表达任务,能够根据不同场景完成文章撰写、商业文案、报告总结、邮件通知、脚本创作、内容改写、信息提炼、语言翻译等多种文字处理工作,并能够理解上下文语义关系,保持内容逻辑完整、结构清晰、表达自然。
|
||||
在执行文本任务时,你需要以专业内容创作者、编辑顾问、语言优化专家的身份完成输出,严格保证语言准确性、逻辑连贯性、表达一致性与阅读体验,根据不同用户场景自动适配正式、口语化、专业化、营销化等表达风格,同时避免空洞表达、重复描述与机械化生成内容。
|
||||
当用户提供具体需求时,需要结合用户输入、上下文信息、参数条件与目标场景生成最终文本结果;若涉及改写、扩写、摘要、总结、标题、营销内容等任务,需要保证核心语义不偏离,并根据用户真实目的完成结构化输出。
|
||||
200: |
|
||||
你是一个智能图片处理助手,专注于视觉内容生成、图像编辑、画面分析与风格控制任务,能够根据文字描述生成不同风格的图片内容,包括写实、插画、动漫、水彩、电影感、商业海报等多种视觉形式,并支持图片局部修改、风格迁移、画面扩展、背景处理与视觉增强等操作。
|
||||
在执行图片相关任务时,你需要以专业视觉设计师、插画师、摄影指导、美术导演的身份进行画面构建,重点关注主体构图、色彩关系、光影氛围、镜头语言、视觉层次与整体风格统一性,确保生成结果具备明确视觉主题与稳定审美表现,而不是简单关键词堆砌。
|
||||
当用户提供图片需求时,需要结合用户描述、场景用途、风格方向、尺寸比例、主体元素、氛围要求等信息生成完整视觉方案;若存在图片编辑任务,则必须保留原图核心特征,仅对用户指定区域或效果进行修改。
|
||||
300: |
|
||||
你是一个智能音频处理助手,专注于语音生成、语音识别、音频分析与声音编辑任务,能够完成文字转语音、语音转文本、多语言识别、音频降噪、音色处理、混音剪辑、情绪识别与声音特征分析等多种音频相关工作,并能够根据不同场景匹配对应语音风格与声音表现形式。
|
||||
在执行音频任务时,你需要以专业配音导演、声音工程师、语音分析专家、后期音频制作人员的身份进行处理,重点保证语音自然度、情绪一致性、识别准确率、音频清晰度与输出稳定性,同时确保不同格式、采样率与播放场景下具备良好兼容性。
|
||||
当用户提供具体音频需求时,需要结合音色、语速、语言类型、情绪风格、背景环境、输出格式等参数完成对应处理;若涉及语音识别或音频分析,则需要尽可能保留原始语义与声音特征,并明确标注不确定内容。
|
||||
400: |
|
||||
你是一个智能向量化处理助手,专注于文本向量化、语义检索、知识索引、相似度计算与语义聚类任务,能够将文本内容转换为高维语义向量,并基于向量相似度完成语义搜索、知识召回、内容聚类、文档匹配与知识库构建等处理流程。
|
||||
在执行向量化任务时,你需要以语义检索工程师、知识库架构师、AI检索系统专家的身份进行处理,重点保证语义表达准确性、向量一致性、检索稳定性与召回有效性,同时确保不同文本之间的语义关系能够被正确表达与计算。
|
||||
当用户提供文本集合、知识内容或检索需求时,需要结合文本上下文、主题方向、检索目标、相似度要求与业务场景生成最终结果;若涉及聚类或知识库构建,则必须明确类别关系、索引结构与召回逻辑。
|
||||
500: |
|
||||
你是一个全模态智能处理助手,能够同时理解、分析与生成文本、图片、音频、视频等多种模态内容,并支持跨模态转换、多模态融合推理、联合内容生成与复杂场景交互,能够根据不同输入形式自动匹配最合理的处理策略与输出方式。
|
||||
在执行多模态任务时,你需要以全链路AI内容架构师、多模态交互专家、综合内容生成系统的身份完成处理,重点保证不同模态之间的语义一致性、风格统一性、信息完整性与交互连贯性,避免出现跨模态语义断裂或输出不一致的问题。
|
||||
当用户提供混合输入内容时,需要结合文本、图片、音频、视频等多种信息共同分析用户真实目标,并根据任务场景自动决定最终输出形式;若涉及跨模态生成,则必须保证生成结果能够准确映射原始语义与核心信息。
|
||||
|
||||
nodePrompts: |
|
||||
你是流程路由助手,你的任务是根据上下文,选择一个正确的节点ID返回。
|
||||
规则:
|
||||
1. 只允许从下面的可选节点ID列表中选择一个返回
|
||||
2. 不要返回任何多余文字、标点、解释、标题
|
||||
3. 只返回纯节点ID
|
||||
可选节点ID(ID: 节点描述):
|
||||
%s
|
||||
上下文内容:
|
||||
%s
|
||||
@@ -15,9 +15,8 @@ const (
|
||||
)
|
||||
|
||||
const (
|
||||
BuildTypePrompt = 1 //提示词构建
|
||||
BuildTypeNode = 2 //节点构建
|
||||
BuildTypeStruct = 3 //结构构建
|
||||
BuildTypeSingle = 1 // 单轮构建
|
||||
BuildTypeMulti = 2 // 多轮构建
|
||||
)
|
||||
|
||||
// ModelType 模型类型常量
|
||||
|
||||
@@ -5,8 +5,10 @@ const (
|
||||
)
|
||||
|
||||
const (
|
||||
TableNameModel = "model_gateway_models" // 模型表
|
||||
TableNameTask = "model_gateway_task" // 任务表
|
||||
TableNameOpLog = "model_gateway_logs_op" // 操作日志表
|
||||
TableNameStat = "model_gateway_logs_stat" // 按天统计表
|
||||
TableNameModel = "model_gateway_models" // 模型表
|
||||
TableNameTask = "model_gateway_task" // 任务表
|
||||
TableNameBuildRecord = "model_gateway_build_record" // 构建记录表
|
||||
TableNameOpLog = "model_gateway_logs_op" // 操作日志表
|
||||
TableNameStat = "model_gateway_logs_stat" // 按天统计表
|
||||
TableNameProtocol = "prompts_provider_protocol" // 模型协议表
|
||||
)
|
||||
|
||||
@@ -17,6 +17,11 @@ func (c *model) CreateModel(ctx context.Context, req *dto.CreateModelReq) (res *
|
||||
return modelService.ModelGatewayModels.Create(ctx, req)
|
||||
}
|
||||
|
||||
// GetModel 获取配置详情
|
||||
func (c *model) GetModel(ctx context.Context, req *dto.GetModelReq) (res *dto.GetModelRes, err error) {
|
||||
return modelService.ModelGatewayModels.Get(ctx, req)
|
||||
}
|
||||
|
||||
// UpdateModel 更改配置
|
||||
func (c *model) UpdateModel(ctx context.Context, req *dto.UpdateModelReq) (res *dto.UpdateModelRes, err error) {
|
||||
err = modelService.ModelGatewayModels.Update(ctx, req)
|
||||
@@ -29,11 +34,6 @@ func (c *model) DeleteModel(ctx context.Context, req *dto.DeleteModelReq) (res *
|
||||
return
|
||||
}
|
||||
|
||||
// GetModel 获取配置详情
|
||||
func (c *model) GetModel(ctx context.Context, req *dto.GetModelReq) (res *dto.GetModelRes, err error) {
|
||||
return modelService.ModelGatewayModels.Get(ctx, req)
|
||||
}
|
||||
|
||||
// ListModel 配置列表
|
||||
func (c *model) ListModel(ctx context.Context, req *dto.ListModelReq) (res *dto.ListModelRes, err error) {
|
||||
return modelService.ModelGatewayModels.List(ctx, req)
|
||||
|
||||
@@ -12,11 +12,21 @@ var ModelGatewayTask = new(task)
|
||||
|
||||
type task struct{}
|
||||
|
||||
// BuildMessages 构建请求模型的数据结构
|
||||
func (c *task) BuildMessages(ctx context.Context, req *dto.BuildMessagesReq) (res *dto.BuildMessagesRes, err error) {
|
||||
return taskService.ModelGatewayTask.BuildMessages(ctx, req)
|
||||
}
|
||||
|
||||
// CreateTask 根据 modelName 创建异步任务,返回 taskId
|
||||
func (c *task) CreateTask(ctx context.Context, req *dto.CreateTaskReq) (res *dto.CreateTaskRes, err error) {
|
||||
return taskService.ModelGatewayTask.Create(ctx, req)
|
||||
}
|
||||
|
||||
// JobTask 定时任务:循环执行待处理任务
|
||||
func (c *task) JobTask(ctx context.Context, req *dto.JobTaskReq) (res *dto.JobTaskRes, err error) {
|
||||
return taskService.ModelGatewayTask.JobTask(ctx, req)
|
||||
}
|
||||
|
||||
// GetTaskResult 获取单条任务结果(返回 *dto.GetTaskResultRes)
|
||||
func (c *task) GetTaskResult(ctx context.Context, req *dto.GetTaskResultReq) (res *dto.GetTaskResultRes, err error) {
|
||||
return taskService.ModelGatewayTask.GetResult(ctx, req.TaskID)
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
package dao
|
||||
|
||||
import (
|
||||
"context"
|
||||
"model-gateway/consts/public"
|
||||
"model-gateway/model/entity"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
var ModelGatewayBuildRecord = &buildRecordDao{}
|
||||
|
||||
type buildRecordDao struct{}
|
||||
|
||||
// Insert 插入构建记录
|
||||
func (d *buildRecordDao) Insert(ctx context.Context, req *entity.ModelGatewayBuildRecord) (id int64, err error) {
|
||||
m := new(entity.ModelGatewayBuildRecord)
|
||||
err = gconv.Struct(req, &m)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameBuildRecord).Insert(m)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
return r.LastInsertId()
|
||||
}
|
||||
|
||||
// Update 更新构建记录
|
||||
func (d *buildRecordDao) Update(ctx context.Context, req *entity.ModelGatewayBuildRecord) (rows int64, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameBuildRecord).
|
||||
OmitEmpty().
|
||||
Data(req).
|
||||
Where(entity.ModelGatewayBuildRecordCol.Id, req.Id).
|
||||
Where(entity.ModelGatewayBuildRecordCol.TaskID, req.TaskID).
|
||||
Update()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
return r.RowsAffected()
|
||||
}
|
||||
|
||||
// Get 获取构建记录
|
||||
func (d *buildRecordDao) Get(ctx context.Context, req *entity.ModelGatewayBuildRecord) (m *entity.ModelGatewayBuildRecord, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameBuildRecord).
|
||||
OmitEmpty().
|
||||
Where(entity.ModelGatewayBuildRecordCol.TaskID, req.TaskID).
|
||||
Where(entity.ModelGatewayBuildRecordCol.Id, req.Id).
|
||||
One()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
err = r.Struct(&m)
|
||||
return
|
||||
}
|
||||
|
||||
// List 分页查询
|
||||
func (d *buildRecordDao) List(ctx context.Context, pageNum, pageSize int, req *entity.ModelGatewayBuildRecord) (list []*entity.ModelGatewayBuildRecord, total int64, err error) {
|
||||
model := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameBuildRecord).
|
||||
OmitEmpty().
|
||||
Where(entity.ModelGatewayBuildRecordCol.Creator, req.Creator).
|
||||
Where(entity.ModelGatewayBuildRecordCol.ModelName, req.ModelName).
|
||||
Where(entity.ModelGatewayBuildRecordCol.Status, req.Status).
|
||||
Where(entity.ModelGatewayBuildRecordCol.TaskID, req.TaskID).
|
||||
OrderDesc(entity.ModelGatewayBuildRecordCol.CreatedAt)
|
||||
if pageNum > 0 && pageSize > 0 {
|
||||
model = model.Page(pageNum, pageSize)
|
||||
}
|
||||
r, totalInt, err := model.AllAndCount(false)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
total = gconv.Int64(totalInt)
|
||||
err = r.Structs(&list)
|
||||
return
|
||||
}
|
||||
|
||||
// Delete 删除构建记录
|
||||
func (d *buildRecordDao) Delete(ctx context.Context, req *entity.ModelGatewayBuildRecord) (rows int64, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameBuildRecord).
|
||||
Where(entity.ModelGatewayBuildRecordCol.Id, req.Id).
|
||||
Delete()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
return r.RowsAffected()
|
||||
}
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"model-gateway/consts/public"
|
||||
"model-gateway/model/dto"
|
||||
"model-gateway/model/entity"
|
||||
"model-gateway/service/gateway"
|
||||
"strconv"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
|
||||
@@ -56,6 +57,7 @@ func (d *modelGatewayModelsDao) Get(ctx context.Context, req *entity.ModelGatewa
|
||||
Where(entity.ModelGatewayModelCol.Id, req.Id).
|
||||
Where(entity.ModelGatewayModelCol.Creator, req.Creator).
|
||||
Where(entity.ModelGatewayModelCol.ModelName, req.ModelName).
|
||||
Where(entity.ModelGatewayModelCol.IsChatModel, req.IsChatModel).
|
||||
Fields(fields).One()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -118,11 +120,14 @@ func (d *modelGatewayModelsDao) GetByAcrossTenant(ctx context.Context, req *enti
|
||||
return &m, err
|
||||
}
|
||||
|
||||
// GetByCreatorAndPlatform 按创建者、平台获取
|
||||
// GetByCreatorAndPlatform 获取模型列表
|
||||
func (d *modelGatewayModelsDao) GetByCreatorAndPlatform(ctx context.Context, req *dto.ListModelReq) (list []*entity.ModelGatewayModel, total int, err error) {
|
||||
// 判断是否管理员
|
||||
isAdmin, _ := gateway.IsSuperAdmin(ctx)
|
||||
|
||||
sql := `
|
||||
SELECT DISTINCT ON (model_name) *
|
||||
FROM asynch_models
|
||||
FROM ` + public.TableNameModel + `
|
||||
WHERE deleted_at IS NULL
|
||||
AND (? = '' OR model_name LIKE ?)
|
||||
`
|
||||
@@ -130,9 +135,8 @@ WHERE deleted_at IS NULL
|
||||
req.ModelName, "%" + req.ModelName + "%",
|
||||
}
|
||||
|
||||
// modelType: 传 6 模糊匹配 6%
|
||||
if req.ModelType > 0 {
|
||||
prefix := strconv.Itoa(req.ModelType)[:1] // 截取第一位
|
||||
prefix := strconv.Itoa(req.ModelType)[:1]
|
||||
sql += ` AND model_type::text LIKE ? `
|
||||
args = append(args, prefix+"%")
|
||||
}
|
||||
@@ -142,27 +146,18 @@ WHERE deleted_at IS NULL
|
||||
args = append(args, req.IsPrivate)
|
||||
}
|
||||
|
||||
if req.IsOwner != nil && *req.IsOwner == 0 {
|
||||
if req.Enabled != nil && *req.Enabled == 1 {
|
||||
sql += ` AND creator = ? AND is_owner = ? AND enabled=1 `
|
||||
} else if req.Enabled != nil && *req.Enabled == 0 {
|
||||
sql += ` AND creator = ? AND is_owner = ? AND enabled=0 `
|
||||
} else {
|
||||
sql += ` AND creator = ? AND is_owner = ? `
|
||||
}
|
||||
args = append(args, req.Creator, req.IsOwner)
|
||||
} else if req.IsOwner != nil && *req.IsOwner == 1 {
|
||||
if req.Enabled != nil && *req.Enabled == 1 {
|
||||
sql += ` AND ((creator = ? AND is_owner = ? AND enabled=1) OR (is_owner = 0 AND enabled=1)) `
|
||||
} else if req.Enabled != nil && *req.Enabled == 0 {
|
||||
sql += ` AND ((creator = ? AND is_owner = ? AND enabled=0) OR (is_owner = 0 AND enabled=1)) `
|
||||
} else {
|
||||
sql += ` AND ((creator = ? AND is_owner = ?) OR (is_owner = 0 AND enabled=1)) `
|
||||
}
|
||||
args = append(args, req.Creator, req.IsOwner)
|
||||
// 非管理员只看自己的 + 公共启用的
|
||||
if !isAdmin {
|
||||
sql += ` AND ((creator = ?) OR (is_private = 1 AND enabled = 1)) `
|
||||
args = append(args, req.Creator)
|
||||
}
|
||||
|
||||
sql += ` ORDER BY model_name, is_owner DESC, created_at DESC`
|
||||
if req.Enabled != nil {
|
||||
sql += ` AND enabled = ? `
|
||||
args = append(args, *req.Enabled)
|
||||
}
|
||||
|
||||
sql += ` ORDER BY model_name, created_at DESC`
|
||||
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).GetAll(ctx, sql, args...)
|
||||
if err != nil {
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"model-gateway/model/entity"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
|
||||
"github.com/gogf/gf/v2/database/gdb"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
)
|
||||
|
||||
@@ -104,6 +103,23 @@ func (d *modelGatewayTaskDao) ListByTaskIDs(ctx context.Context, taskIDs []strin
|
||||
return
|
||||
}
|
||||
|
||||
// ListPending 查询待处理任务
|
||||
func (d *modelGatewayTaskDao) ListPending(ctx context.Context, limit int) (list []*entity.ModelGatewayTask, err error) {
|
||||
if limit <= 0 {
|
||||
limit = 10
|
||||
}
|
||||
model := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameTask).
|
||||
Where(entity.ModelGatewayTaskCol.State, 0).
|
||||
OrderAsc(entity.ModelGatewayTaskCol.CreatedAt).
|
||||
Limit(limit)
|
||||
r, err := model.All()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
err = r.Structs(&list)
|
||||
return
|
||||
}
|
||||
|
||||
// MarkDownloadedByID 标记已下载
|
||||
func (d *modelGatewayTaskDao) MarkDownloadedByID(ctx context.Context, id int64) error {
|
||||
_, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameTask).
|
||||
@@ -128,32 +144,32 @@ func (d *modelGatewayTaskDao) GetPendingAsyncTasks(ctx context.Context, limit in
|
||||
|
||||
// ClaimByID 按主键抢占,返回抢占后的任务
|
||||
func (d *modelGatewayTaskDao) ClaimByID(ctx context.Context, id int64) (*entity.ModelGatewayTask, error) {
|
||||
// 1) 先查任务
|
||||
var task entity.ModelGatewayTask
|
||||
err := gfdb.DB(ctx, public.DbNameModelGateway).Transaction(ctx, func(ctx context.Context, tx gdb.TX) error {
|
||||
r, err := tx.Model(public.TableNameTask).
|
||||
Where(entity.ModelGatewayTaskCol.Id, id).
|
||||
Where(entity.ModelGatewayTaskCol.State, public.TaskStatusPending).
|
||||
Limit(1).
|
||||
LockUpdate().
|
||||
One()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if r.IsEmpty() {
|
||||
return fmt.Errorf("任务已被抢占或不存在: id=%d", id)
|
||||
}
|
||||
if err := r.Struct(&task); err != nil {
|
||||
return err
|
||||
}
|
||||
_, err = tx.Model(public.TableNameTask).
|
||||
Data(&entity.ModelGatewayTask{State: public.TaskStatusRunning}).
|
||||
Where(entity.ModelGatewayTaskCol.Id, id).
|
||||
OmitEmpty().
|
||||
Update()
|
||||
return err
|
||||
})
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameTask).
|
||||
Where(entity.ModelGatewayTaskCol.Id, id).
|
||||
Where(entity.ModelGatewayTaskCol.State, public.TaskStatusPending).
|
||||
One()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r.IsEmpty() {
|
||||
return nil, fmt.Errorf("任务已被抢占或不存在: id=%d", id)
|
||||
}
|
||||
if err = r.Struct(&task); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2) 改为执行中
|
||||
_, err = gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameTask).
|
||||
Data(&entity.ModelGatewayTask{State: public.TaskStatusRunning}).
|
||||
Where(entity.ModelGatewayTaskCol.Id, id).
|
||||
Where(entity.ModelGatewayTaskCol.State, public.TaskStatusPending). // 防并发
|
||||
OmitEmpty().
|
||||
Update()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &task, nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
package dao
|
||||
|
||||
import (
|
||||
"context"
|
||||
"model-gateway/consts/public"
|
||||
"model-gateway/model/entity"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
|
||||
)
|
||||
|
||||
var ProviderProtocol = &providerProtocolDao{}
|
||||
|
||||
type providerProtocolDao struct{}
|
||||
|
||||
// Insert 新增协议配置
|
||||
func (d *providerProtocolDao) Insert(ctx context.Context, req *entity.ProviderProtocol) (id int64, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameProtocol).Insert(req)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return r.LastInsertId()
|
||||
}
|
||||
|
||||
// Get 获取协议配置
|
||||
func (d *providerProtocolDao) Get(ctx context.Context, req *entity.ProviderProtocol) (res *entity.ProviderProtocol, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameProtocol).
|
||||
NoTenantId(ctx).
|
||||
OmitEmpty().
|
||||
Where(entity.ProviderProtocolCol.Id, req.Id).
|
||||
Where(entity.ProviderProtocolCol.ProviderName, req.ProviderName).
|
||||
Where(entity.ProviderProtocolCol.Status, 1).
|
||||
One()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r.IsEmpty() {
|
||||
return nil, nil
|
||||
}
|
||||
err = r.Struct(&res)
|
||||
return
|
||||
}
|
||||
|
||||
// List 列表查询
|
||||
func (d *providerProtocolDao) List(ctx context.Context, req *entity.ProviderProtocol, page, size int) (list []*entity.ProviderProtocol, total int, err error) {
|
||||
if page <= 0 {
|
||||
page = 1
|
||||
}
|
||||
if size <= 0 {
|
||||
size = 10
|
||||
}
|
||||
model := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameProtocol).OmitEmpty()
|
||||
if req.ProviderName != "" {
|
||||
model = model.Where(entity.ProviderProtocolCol.ProviderName, req.ProviderName)
|
||||
}
|
||||
if req.Status > 0 {
|
||||
model = model.Where(entity.ProviderProtocolCol.Status, req.Status)
|
||||
}
|
||||
model = model.OrderDesc(entity.ProviderProtocolCol.CreatedAt).Page(page, size)
|
||||
r, totalInt, err := model.AllAndCount(false)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
err = r.Structs(&list)
|
||||
total = totalInt
|
||||
return
|
||||
}
|
||||
|
||||
// Update 更新协议配置
|
||||
func (d *providerProtocolDao) Update(ctx context.Context, req *entity.ProviderProtocol) (rows int64, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameProtocol).
|
||||
OmitEmpty().
|
||||
Where(entity.ProviderProtocolCol.Id, req.Id).
|
||||
Data(req).
|
||||
Update()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return r.RowsAffected()
|
||||
}
|
||||
|
||||
// Delete 软删除协议配置
|
||||
func (d *providerProtocolDao) Delete(ctx context.Context, id int64) (rows int64, err error) {
|
||||
r, err := gfdb.DB(ctx, public.DbNameModelGateway).Model(ctx, public.TableNameProtocol).
|
||||
Where(entity.ProviderProtocolCol.Id, id).
|
||||
Data(map[string]any{"deleted_at": "NOW()"}).
|
||||
Update()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return r.RowsAffected()
|
||||
}
|
||||
@@ -3,7 +3,7 @@ module model-gateway
|
||||
go 1.26.1
|
||||
|
||||
require (
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.23
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.29
|
||||
github.com/gogf/gf/contrib/drivers/pgsql/v2 v2.10.2
|
||||
github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.2
|
||||
github.com/gogf/gf/v2 v2.10.2
|
||||
@@ -12,22 +12,25 @@ require (
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/BurntSushi/toml v1.5.0 // indirect
|
||||
github.com/BurntSushi/toml v1.6.0 // indirect
|
||||
github.com/armon/go-metrics v0.4.1 // indirect
|
||||
github.com/bwmarrin/snowflake v0.3.0 // indirect
|
||||
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/clbanning/mxj/v2 v2.7.0 // indirect
|
||||
github.com/clipperhouse/displaywidth v0.11.0 // indirect
|
||||
github.com/clipperhouse/uax29/v2 v2.7.0 // indirect
|
||||
github.com/dgraph-io/badger/v4 v4.2.0 // indirect
|
||||
github.com/dgraph-io/ristretto v0.1.1 // indirect
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/emirpasic/gods/v2 v2.0.0-alpha // indirect
|
||||
github.com/fatih/color v1.18.0 // indirect
|
||||
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
||||
github.com/fatih/color v1.19.0 // indirect
|
||||
github.com/fsnotify/fsnotify v1.10.1 // indirect
|
||||
github.com/go-ego/gse v1.0.2 // indirect
|
||||
github.com/go-logr/logr v1.4.3 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
||||
github.com/goccy/go-json v0.10.6 // indirect
|
||||
github.com/gogf/gf/contrib/registry/consul/v2 v2.9.5 // indirect
|
||||
github.com/gogf/gf/contrib/trace/otlphttp/v2 v2.9.5 // indirect
|
||||
github.com/gogo/protobuf v1.3.2 // indirect
|
||||
@@ -36,57 +39,62 @@ require (
|
||||
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
|
||||
github.com/golang/protobuf v1.5.4 // indirect
|
||||
github.com/golang/snappy v1.0.0 // indirect
|
||||
github.com/google/flatbuffers v1.12.1 // indirect
|
||||
github.com/google/btree v1.1.3 // indirect
|
||||
github.com/google/flatbuffers v25.12.19+incompatible // indirect
|
||||
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
|
||||
github.com/grokify/html-strip-tags-go v0.1.0 // indirect
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 // indirect
|
||||
github.com/hashicorp/consul/api v1.26.1 // indirect
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
|
||||
github.com/hashicorp/consul/api v1.33.5 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
|
||||
github.com/hashicorp/go-hclog v1.5.0 // indirect
|
||||
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/hashicorp/go-rootcerts v1.0.2 // indirect
|
||||
github.com/hashicorp/go-version v1.9.0 // indirect
|
||||
github.com/hashicorp/golang-lru v1.0.2 // indirect
|
||||
github.com/hashicorp/serf v0.10.1 // indirect
|
||||
github.com/klauspost/compress v1.18.0 // indirect
|
||||
github.com/lib/pq v1.10.9 // indirect
|
||||
github.com/klauspost/compress v1.18.6 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
|
||||
github.com/lib/pq v1.12.3 // indirect
|
||||
github.com/magiconair/properties v1.8.10 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.16 // indirect
|
||||
github.com/mattn/go-colorable v0.1.15 // indirect
|
||||
github.com/mattn/go-isatty v0.0.22 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.24 // indirect
|
||||
github.com/miekg/dns v1.1.72 // indirect
|
||||
github.com/mitchellh/go-homedir v1.1.0 // indirect
|
||||
github.com/mitchellh/mapstructure v1.5.0 // indirect
|
||||
github.com/olekukonko/errors v1.1.0 // indirect
|
||||
github.com/olekukonko/ll v0.0.9 // indirect
|
||||
github.com/olekukonko/tablewriter v1.1.0 // indirect
|
||||
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 // indirect
|
||||
github.com/olekukonko/errors v1.3.0 // indirect
|
||||
github.com/olekukonko/ll v0.1.8 // indirect
|
||||
github.com/olekukonko/tablewriter v1.1.4 // indirect
|
||||
github.com/pkg/errors v0.9.1 // indirect
|
||||
github.com/r3labs/diff/v2 v2.15.1 // indirect
|
||||
github.com/redis/go-redis/v9 v9.12.1 // indirect
|
||||
github.com/rivo/uniseg v0.4.7 // indirect
|
||||
github.com/tidwall/match v1.1.1 // indirect
|
||||
github.com/tidwall/pretty v1.2.0 // indirect
|
||||
github.com/redis/go-redis/v9 v9.21.0 // indirect
|
||||
github.com/stretchr/objx v0.5.2 // indirect
|
||||
github.com/tidwall/match v1.2.0 // indirect
|
||||
github.com/tidwall/pretty v1.2.1 // indirect
|
||||
github.com/tiger1103/gfast-token v1.0.10 // indirect
|
||||
github.com/vcaesar/cedar v0.30.0 // indirect
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
|
||||
go.mongodb.org/mongo-driver/v2 v2.4.0 // indirect
|
||||
go.opencensus.io v0.23.0 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
|
||||
go.opentelemetry.io/otel v1.38.0 // indirect
|
||||
go.mongodb.org/mongo-driver/v2 v2.6.0 // indirect
|
||||
go.opencensus.io v0.24.0 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||
go.opentelemetry.io/otel v1.44.0 // indirect
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/metric v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/sdk v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.38.0 // indirect
|
||||
go.opentelemetry.io/otel/metric v1.44.0 // indirect
|
||||
go.opentelemetry.io/otel/sdk v1.44.0 // indirect
|
||||
go.opentelemetry.io/otel/trace v1.44.0 // indirect
|
||||
go.opentelemetry.io/proto/otlp v1.7.1 // indirect
|
||||
golang.org/x/exp v0.0.0-20250305212735-054e65f0b394 // indirect
|
||||
golang.org/x/net v0.47.0 // indirect
|
||||
golang.org/x/sys v0.38.0 // indirect
|
||||
golang.org/x/text v0.31.0 // indirect
|
||||
go.uber.org/atomic v1.11.0 // indirect
|
||||
golang.org/x/exp v0.0.0-20260611194520-c48552f49976 // indirect
|
||||
golang.org/x/net v0.56.0 // indirect
|
||||
golang.org/x/sys v0.46.0 // indirect
|
||||
golang.org/x/text v0.38.0 // indirect
|
||||
google.golang.org/appengine v1.6.7 // indirect
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250825161204-c5933d9347a5 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5 // indirect
|
||||
google.golang.org/grpc v1.75.0 // indirect
|
||||
google.golang.org/protobuf v1.36.8 // indirect
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57 // indirect
|
||||
google.golang.org/grpc v1.79.3 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
)
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.23 h1:xieoA00iKOCDm5SO9iXn+cSyMKBAlZwI0fuEVPWrHLg=
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.23/go.mod h1:50U1Xi+Ie56z09S5LQbZvaken0Mxv3OeS9LgR7U/ZRY=
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.29 h1:5McaN5pSewvrLUHQzWMX6EaUvD+B5I5bMYoU+clHJk4=
|
||||
gitea.redpowerfuture.com/red-future/common v0.0.29/go.mod h1:50U1Xi+Ie56z09S5LQbZvaken0Mxv3OeS9LgR7U/ZRY=
|
||||
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
|
||||
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
|
||||
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
|
||||
github.com/BurntSushi/toml v1.6.0 h1:dRaEfpa2VI55EwlIW72hMRHdWouJeRF7TPYhI+AUQjk=
|
||||
github.com/BurntSushi/toml v1.6.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
|
||||
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
|
||||
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
|
||||
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
|
||||
@@ -36,6 +36,10 @@ github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp
|
||||
github.com/clbanning/mxj/v2 v2.7.0 h1:WA/La7UGCanFe5NpHF0Q3DNtnCsVoxbPKuyBNHWRyME=
|
||||
github.com/clbanning/mxj/v2 v2.7.0/go.mod h1:hNiWqW14h+kc+MdF9C6/YoRfjEJoR3ou6tn/Qo+ve2s=
|
||||
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
|
||||
github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8=
|
||||
github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0=
|
||||
github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
|
||||
github.com/clipperhouse/uax29/v2 v2.7.0/go.mod h1:EFJ2TJMRUaplDxHKj1qAEhCtQPW2tJSwu5BF98AuoVM=
|
||||
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f/go.mod h1:M8M6+tZqaGXZJjfX53e64911xZQV5JYwmTeXPW+k8Sc=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
@@ -47,8 +51,6 @@ github.com/dgraph-io/ristretto v0.1.1 h1:6CWw5tJNgpegArSHpNHJKldNeq03FQCwYvfMVWa
|
||||
github.com/dgraph-io/ristretto v0.1.1/go.mod h1:S1GPSBCYCIhmVNfcth17y2zZtQT6wzkzgwUve0VDWWA=
|
||||
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2 h1:tdlZCpZ/P9DhczCTSixgIKmwPv6+wP5DGjqLYw5SUiA=
|
||||
github.com/dgryski/go-farm v0.0.0-20190423205320-6a90982ecee2/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
|
||||
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
@@ -61,10 +63,10 @@ github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7
|
||||
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
|
||||
github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
|
||||
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
|
||||
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
|
||||
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
|
||||
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
||||
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
||||
github.com/fatih/color v1.19.0 h1:Zp3PiM21/9Ld6FzSKyL5c/BULoe/ONr9KlbYVOfG8+w=
|
||||
github.com/fatih/color v1.19.0/go.mod h1:zNk67I0ZUT1bEGsSGyCZYZNrHuTkJJB+r6Q9VuMi0LE=
|
||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||
github.com/go-ego/gse v1.0.2 h1:+27lYFPhQEhA9igtdOsJPRKYL/k3TwYsxBF5jr6KFv4=
|
||||
github.com/go-ego/gse v1.0.2/go.mod h1:Fy35G+q7VV7Et1zIKO8o/sW1kkugV3znXap/lF/11zc=
|
||||
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
|
||||
@@ -77,6 +79,10 @@ github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ4
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
|
||||
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
|
||||
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
||||
github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU=
|
||||
github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/gogf/gf/contrib/drivers/pgsql/v2 v2.10.2 h1:u8EpP24GkprogROnJ7htMov9Fc66pTP1eVYrWxiCYOs=
|
||||
github.com/gogf/gf/contrib/drivers/pgsql/v2 v2.10.2/go.mod h1:GmvM3r8GVByVMi4RD2+MCs5+CfxVXPMeT8mVDkAaAXE=
|
||||
github.com/gogf/gf/contrib/nosql/redis/v2 v2.10.2 h1:iTQegT+lEg/wDKvj2mi3W1wrdrwFarjokf88EXVVgu4=
|
||||
@@ -114,10 +120,10 @@ github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6
|
||||
github.com/golang/snappy v1.0.0 h1:Oy607GVXHs7RtbggtPBnr2RmDArIsAefDwvrdWvRhGs=
|
||||
github.com/golang/snappy v1.0.0/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
|
||||
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
|
||||
github.com/google/btree v1.0.1 h1:gK4Kx5IaGY9CD5sPJ36FHiBJ6ZXl0kilRiiCj+jdYp4=
|
||||
github.com/google/btree v1.0.1/go.mod h1:xXMiIv4Fb/0kKde4SpL7qlzvu5cMJDRkFDxJfI9uaxA=
|
||||
github.com/google/flatbuffers v1.12.1 h1:MVlul7pQNoDzWRLTw5imwYsl+usrS1TXG2H4jg6ImGw=
|
||||
github.com/google/flatbuffers v1.12.1/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
|
||||
github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg=
|
||||
github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4=
|
||||
github.com/google/flatbuffers v25.12.19+incompatible h1:haMV2JRRJCe1998HeW/p0X9UaMTK6SDo0ffLn2+DbLs=
|
||||
github.com/google/flatbuffers v25.12.19+incompatible/go.mod h1:1AeVuKshWv4vARoZatz6mlQ0JxURH0Kv5+zNeJKJCa8=
|
||||
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
|
||||
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
@@ -134,12 +140,12 @@ github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 h1:JeSE6pjso5T
|
||||
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674/go.mod h1:r4w70xmWCQKmi1ONH4KIaBptdivuRPyosB9RmPlGEwA=
|
||||
github.com/grokify/html-strip-tags-go v0.1.0 h1:03UrQLjAny8xci+R+qjCce/MYnpNXCtgzltlQbOBae4=
|
||||
github.com/grokify/html-strip-tags-go v0.1.0/go.mod h1:ZdzgfHEzAfz9X6Xe5eBLVblWIxXfYSQ40S/VKrAOGpc=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2 h1:8Tjv8EJ+pM1xP8mK6egEbD1OgnVTyacbefKhmbLhIhU=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.27.2/go.mod h1:pkJQ2tZHJ0aFOVEEot6oZmaVEZcRme73eIFmhiVuRWs=
|
||||
github.com/hashicorp/consul/api v1.26.1 h1:5oSXOO5fboPZeW5SN+TdGFP/BILDgBm19OrPZ/pICIM=
|
||||
github.com/hashicorp/consul/api v1.26.1/go.mod h1:B4sQTeaSO16NtynqrAdwOlahJ7IUDZM9cj2420xYL8A=
|
||||
github.com/hashicorp/consul/sdk v0.15.0 h1:2qK9nDrr4tiJKRoxPGhm6B7xJjLVIQqkjiab2M4aKjU=
|
||||
github.com/hashicorp/consul/sdk v0.15.0/go.mod h1:r/OmRRPbHOe0yxNahLw7G9x5WG17E1BIECMtCjcPSNo=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 h1:HWRh5R2+9EifMyIHV7ZV+MIZqgz+PMpZ14Jynv3O2Zs=
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0/go.mod h1:JfhWUomR1baixubs02l85lZYYOm7LV6om4ceouMv45c=
|
||||
github.com/hashicorp/consul/api v1.33.5 h1:Nn6q87zudRU1rLBTJEgaWxz9STCNadilLCD7B8OA5aI=
|
||||
github.com/hashicorp/consul/api v1.33.5/go.mod h1:pa6fJOSHKLOzNHpUVeqLDtxA5+J1D7NNzLasuk8eRXA=
|
||||
github.com/hashicorp/consul/sdk v0.17.3 h1:oZMMxzQGSsiT+ToOH50y3Qcs0nc9Ud+7L5lRx+EmMU0=
|
||||
github.com/hashicorp/consul/sdk v0.17.3/go.mod h1:jnOmYjiNfVRpBaujQ1DFFVs0N6g3S1y6wygSjLTzYfc=
|
||||
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
@@ -169,8 +175,8 @@ github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/b
|
||||
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
|
||||
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-version v1.2.1 h1:zEfKbn2+PDgroKdiOzqiE8rsmLqU2uwi5PB5pBJ3TkI=
|
||||
github.com/hashicorp/go-version v1.2.1/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
|
||||
github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaXPSCnA=
|
||||
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
|
||||
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
|
||||
github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c=
|
||||
github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
|
||||
@@ -185,8 +191,10 @@ github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/u
|
||||
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
|
||||
github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8=
|
||||
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
|
||||
github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
|
||||
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
|
||||
github.com/klauspost/compress v1.18.6 h1:2jupLlAwFm95+YDR+NwD2MEfFO9d4z4Prjl1XXDjuao=
|
||||
github.com/klauspost/compress v1.18.6/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
|
||||
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
|
||||
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
|
||||
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
|
||||
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
|
||||
@@ -196,8 +204,8 @@ github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
|
||||
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
|
||||
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
|
||||
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
|
||||
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
|
||||
@@ -205,39 +213,39 @@ github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVc
|
||||
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-colorable v0.1.15 h1:+u9SLTRGnXv73cEsnsmoZBom+dMU88B2M0aDcWy0/jY=
|
||||
github.com/mattn/go-colorable v0.1.15/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
|
||||
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
|
||||
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
|
||||
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
|
||||
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
|
||||
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
|
||||
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
|
||||
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
|
||||
github.com/mattn/go-runewidth v0.0.24 h1:cpokDiIn0MGnhdHwuWnJBITySJ20QyNGnY2kR/ay2DU=
|
||||
github.com/mattn/go-runewidth v0.0.24/go.mod h1:XBkDxAl56ILZc9knddidhrOlY5R/pDhgLpndooCuJAs=
|
||||
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
|
||||
github.com/miekg/dns v1.1.26/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso=
|
||||
github.com/miekg/dns v1.1.41/go.mod h1:p6aan82bvRIyn+zDIv9xYNUpwa73JcSh9BKwknJysuI=
|
||||
github.com/miekg/dns v1.1.63 h1:8M5aAw6OMZfFXTT7K5V0Eu5YiiL8l7nUAkyN6C9YwaY=
|
||||
github.com/miekg/dns v1.1.63/go.mod h1:6NGHfjhpmr5lt3XPLuyfDJi5AXbNIPM9PY6H6sF1Nfs=
|
||||
github.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=
|
||||
github.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=
|
||||
github.com/mitchellh/cli v1.1.0/go.mod h1:xcISNoH86gajksDmfB23e/pu+B+GeFRMYmoHXxx3xhI=
|
||||
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
|
||||
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
|
||||
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
|
||||
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
|
||||
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
|
||||
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
|
||||
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
|
||||
github.com/olekukonko/errors v1.1.0 h1:RNuGIh15QdDenh+hNvKrJkmxxjV4hcS50Db478Ou5sM=
|
||||
github.com/olekukonko/errors v1.1.0/go.mod h1:ppzxA5jBKcO1vIpCXQ9ZqgDh8iwODz6OXIGKU8r5m4Y=
|
||||
github.com/olekukonko/ll v0.0.9 h1:Y+1YqDfVkqMWuEQMclsF9HUR5+a82+dxJuL1HHSRpxI=
|
||||
github.com/olekukonko/ll v0.0.9/go.mod h1:En+sEW0JNETl26+K8eZ6/W4UQ7CYSrrgg/EdIYT2H8g=
|
||||
github.com/olekukonko/tablewriter v1.1.0 h1:N0LHrshF4T39KvI96fn6GT8HEjXRXYNDrDjKFDB7RIY=
|
||||
github.com/olekukonko/tablewriter v1.1.0/go.mod h1:5c+EBPeSqvXnLLgkm9isDdzR3wjfBkHR9Nhfp3NWrzo=
|
||||
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6 h1:zrbMGy9YXpIeTnGj4EljqMiZsIcE09mmF8XsD5AYOJc=
|
||||
github.com/olekukonko/cat v0.0.0-20250911104152-50322a0618f6/go.mod h1:rEKTHC9roVVicUIfZK7DYrdIoM0EOr8mK1Hj5s3JjH0=
|
||||
github.com/olekukonko/errors v1.3.0 h1:teJvgLGUEqMzBUms+Dj3/3szNqCG/Jdw9iDbum8fR6U=
|
||||
github.com/olekukonko/errors v1.3.0/go.mod h1:ppzxA5jBKcO1vIpCXQ9ZqgDh8iwODz6OXIGKU8r5m4Y=
|
||||
github.com/olekukonko/ll v0.1.8 h1:ysHCJRGHYKzmBSdz9w5AySztx7lG8SQY+naTGYUbsz8=
|
||||
github.com/olekukonko/ll v0.1.8/go.mod h1:RPRC6UcscfFZgjo1nulkfMH5IM0QAYim0LfnMvUuozw=
|
||||
github.com/olekukonko/tablewriter v1.1.4 h1:ORUMI3dXbMnRlRggJX3+q7OzQFDdvgbN9nVWj1drm6I=
|
||||
github.com/olekukonko/tablewriter v1.1.4/go.mod h1:+kedxuyTtgoZLwif3P1Em4hARJs+mVnzKxmsCL/C5RY=
|
||||
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
|
||||
github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY=
|
||||
github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
|
||||
@@ -264,13 +272,10 @@ github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsT
|
||||
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
|
||||
github.com/r3labs/diff/v2 v2.15.1 h1:EOrVqPUzi+njlumoqJwiS/TgGgmZo83619FNDB9xQUg=
|
||||
github.com/r3labs/diff/v2 v2.15.1/go.mod h1:I8noH9Fc2fjSaMxqF3G2lhDdC0b+JXCfyx85tWFM9kc=
|
||||
github.com/redis/go-redis/v9 v9.12.1 h1:k5iquqv27aBtnTm2tIkROUDp8JBXhXZIVu1InSgvovg=
|
||||
github.com/redis/go-redis/v9 v9.12.1/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw=
|
||||
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
|
||||
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
|
||||
github.com/rivo/uniseg v0.4.7/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
|
||||
github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII=
|
||||
github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o=
|
||||
github.com/redis/go-redis/v9 v9.21.0 h1:FPBE4hhbAke+TLmcY3WkpbDffJEomdqPn3HYiqAtL9E=
|
||||
github.com/redis/go-redis/v9 v9.21.0/go.mod h1:v/M13XI1PVCDcm01VtPFOADfZtHf8YW3baQf57KlIkA=
|
||||
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
|
||||
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I=
|
||||
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
|
||||
@@ -278,22 +283,26 @@ github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPx
|
||||
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.5.0 h1:1zr/of2m5FGMsad5YfcqgdqdWrIhu+EBEJRhR1U7z/c=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
|
||||
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
|
||||
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
|
||||
github.com/tidwall/gjson v1.19.0 h1:xwxm7n691Uf3u5OFjzngavjGTh55KX5q/9w9xHW88JU=
|
||||
github.com/tidwall/gjson v1.19.0/go.mod h1:V37/opeE/JbLUOfH0QTXiNez2l0RUjYUhpT4szFQAfc=
|
||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.0 h1:RWIZEg2iJ8/g6fDDYzMpobmaoGh5OLl4AXtGUGPcqCs=
|
||||
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=
|
||||
github.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
|
||||
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
|
||||
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
|
||||
github.com/tiger1103/gfast-token v1.0.10 h1:fNiBE/Dq5iTHvTGlCx3DmXa2o4hr0NtumFpffZ39k6s=
|
||||
github.com/tiger1103/gfast-token v1.0.10/go.mod h1:a/21mxmj7zFeNvjhZSC0XpEAFHfb1aT2k6DXnufFU1s=
|
||||
github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM=
|
||||
@@ -305,28 +314,32 @@ github.com/vmihailenco/msgpack v4.0.4+incompatible h1:dSLoQfGFAo3F6OoNhwUmLwVgaU
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
|
||||
github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
|
||||
go.mongodb.org/mongo-driver/v2 v2.4.0 h1:Oq6BmUAAFTzMeh6AonuDlgZMuAuEiUxoAD1koK5MuFo=
|
||||
go.mongodb.org/mongo-driver/v2 v2.4.0/go.mod h1:jHeEDJHJq7tm6ZF45Issun9dbogjfnPySb1vXA7EeAI=
|
||||
go.opencensus.io v0.23.0 h1:gqCw0LfLxScz8irSi8exQc7fyQ0fKQU/qnC/X8+V/1M=
|
||||
go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
|
||||
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
|
||||
go.opentelemetry.io/otel v1.38.0 h1:RkfdswUDRimDg0m2Az18RKOsnI8UDzppJAtj01/Ymk8=
|
||||
go.opentelemetry.io/otel v1.38.0/go.mod h1:zcmtmQ1+YmQM9wrNsTGV/q/uyusom3P8RxwExxkZhjM=
|
||||
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
|
||||
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
|
||||
go.mongodb.org/mongo-driver/v2 v2.6.0 h1:b9sJOYrkmt4l8bY43ZenFBcPlhYIjaOfYHLtbB/5qi8=
|
||||
go.mongodb.org/mongo-driver/v2 v2.6.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
|
||||
go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
|
||||
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
|
||||
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0 h1:GqRJVj7UmLjCVyVJ3ZFLdPRmhDUp2zFmQe3RHIOsw24=
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.38.0/go.mod h1:ri3aaHSmCTVYu2AWv44YMauwAQc0aqI9gHKIcSbI1pU=
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0 h1:aTL7F04bJHUlztTsNGJ2l+6he8c+y/b//eR0jjjemT4=
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.38.0/go.mod h1:kldtb7jDTeol0l3ewcmd8SDvx3EmIE7lyvqbasU3QC4=
|
||||
go.opentelemetry.io/otel/metric v1.38.0 h1:Kl6lzIYGAh5M159u9NgiRkmoMKjvbsKtYRwgfrA6WpA=
|
||||
go.opentelemetry.io/otel/metric v1.38.0/go.mod h1:kB5n/QoRM8YwmUahxvI3bO34eVtQf2i4utNVLr9gEmI=
|
||||
go.opentelemetry.io/otel/sdk v1.38.0 h1:l48sr5YbNf2hpCUj/FoGhW9yDkl+Ma+LrVl8qaM5b+E=
|
||||
go.opentelemetry.io/otel/sdk v1.38.0/go.mod h1:ghmNdGlVemJI3+ZB5iDEuk4bWA3GkTpW+DOoZMYBVVg=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.38.0 h1:aSH66iL0aZqo//xXzQLYozmWrXxyFkBJ6qT5wthqPoM=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.38.0/go.mod h1:dg9PBnW9XdQ1Hd6ZnRz689CbtrUp0wMMs9iPcgT9EZA=
|
||||
go.opentelemetry.io/otel/trace v1.38.0 h1:Fxk5bKrDZJUH+AMyyIXGcFAPah0oRcT+LuNtJrmcNLE=
|
||||
go.opentelemetry.io/otel/trace v1.38.0/go.mod h1:j1P9ivuFsTceSWe1oY+EeW3sc+Pp42sO++GHkg4wwhs=
|
||||
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
|
||||
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
|
||||
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
|
||||
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
|
||||
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
|
||||
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
|
||||
go.opentelemetry.io/proto/otlp v1.7.1 h1:gTOMpGDb0WTBOP8JaO72iL3auEZhVmAQg4ipjOVAtj4=
|
||||
go.opentelemetry.io/proto/otlp v1.7.1/go.mod h1:b2rVh6rfI/s2pHWNlB7ILJcRALpcNDzKhACevjI+ZnE=
|
||||
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
|
||||
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
|
||||
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
|
||||
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
|
||||
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
|
||||
@@ -335,15 +348,15 @@ golang.org/x/crypto v0.0.0-20190923035154-9ee001bba392/go.mod h1:/lpIB1dKB+9EgE3
|
||||
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
|
||||
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
|
||||
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
|
||||
golang.org/x/exp v0.0.0-20250305212735-054e65f0b394 h1:nDVHiLt8aIbd/VzvPWN6kSOPE7+F/fNFDSXLVYkE/Iw=
|
||||
golang.org/x/exp v0.0.0-20250305212735-054e65f0b394/go.mod h1:sIifuuw/Yco/y6yb6+bDNfyeQ/MdPUy/hKEMYQV17cM=
|
||||
golang.org/x/exp v0.0.0-20260611194520-c48552f49976 h1:X8Hz2ImujgbmetVuW+w2YkyZChE3cBpZi2P158rTG9M=
|
||||
golang.org/x/exp v0.0.0-20260611194520-c48552f49976/go.mod h1:vnf4pv9iKZXY58sQE1L86zmNWJ4159e1RkcWiLCkeEY=
|
||||
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
|
||||
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
|
||||
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
|
||||
golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
|
||||
golang.org/x/mod v0.29.0 h1:HV8lRxZC4l2cr3Zq1LvtOsi/ThTgWnUk/y64QSs8GwA=
|
||||
golang.org/x/mod v0.29.0/go.mod h1:NyhrlYXJ2H4eJiRy/WDBO6HMqZQ6q9nk4JzS3NuCK+w=
|
||||
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
|
||||
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
|
||||
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
@@ -359,8 +372,8 @@ golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwY
|
||||
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8=
|
||||
golang.org/x/net v0.47.0 h1:Mx+4dIFzqraBXUugkia1OOvlD6LemFo1ALMHjrXDOhY=
|
||||
golang.org/x/net v0.47.0/go.mod h1:/jNxtkgq5yWUGYkaZGqo27cfGZ1c5Nen03aYrrKpVRU=
|
||||
golang.org/x/net v0.56.0 h1:Rw8j/hFzGvJUZwNBXnAtf5sVDVt+65SK2C7IxCxZt5o=
|
||||
golang.org/x/net v0.56.0/go.mod h1:D3Ku6r+V6JROoZK144D2XfMHFcMq/0zSfLelVTCFKec=
|
||||
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
|
||||
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
@@ -369,8 +382,8 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ
|
||||
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.18.0 h1:kr88TuHDroi+UVf+0hZnirlk8o8T+4MrK6mr60WkH/I=
|
||||
golang.org/x/sync v0.18.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||
golang.org/x/sync v0.21.0 h1:HLII4xRRTtCRkxYp4HNFF0Js/Og6q2i++KXbg0gHCwM=
|
||||
golang.org/x/sync v0.21.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
@@ -394,16 +407,15 @@ golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBc
|
||||
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220728004956-3c1f35247d10/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20221010170243-090e33056c14/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
|
||||
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||
golang.org/x/sys v0.46.0 h1:noSf2Fq6F8DBgS+LysIkx7rIExoNHJsxOAtPp4rthXw=
|
||||
golang.org/x/sys v0.46.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.6/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.31.0 h1:aC8ghyu4JhP8VojJ2lEHBnochRno1sgL6nEi9WGFGMM=
|
||||
golang.org/x/text v0.31.0/go.mod h1:tKRAlv61yKIjGGHX/4tP1LTbc13YSec1pxVEWXzfoeM=
|
||||
golang.org/x/text v0.38.0 h1:sXmwo9DwP3OK9EZ7PqAdaooSGozfl/3a6/xJcbzPRhE=
|
||||
golang.org/x/text v0.38.0/go.mod h1:YXZt3QhHUKYT53r2lLKFIVi6Ao1jdzrTR/KQ09qyxF4=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
|
||||
@@ -413,8 +425,8 @@ golang.org/x/tools v0.0.0-20190907020128-2ca718005c18/go.mod h1:b+2E5dAYhXwXZwtn
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
|
||||
golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA=
|
||||
golang.org/x/tools v0.38.0 h1:Hx2Xv8hISq8Lm16jvBZ2VQf+RLmbd7wVUsALibYI/IQ=
|
||||
golang.org/x/tools v0.38.0/go.mod h1:yEsQ/d/YK8cjh0L6rZlY8tgtlKiBNTL14pGDJPJpYQs=
|
||||
golang.org/x/tools v0.46.0 h1:7jTurBkPZu4moS/Uy4OQT1M+QBlsj3wejyZwsT8Z7rk=
|
||||
golang.org/x/tools v0.46.0/go.mod h1:FrD85F8l+NWL+9XWBSyVSHO6Ne4jutsfIFba7AWQ5Ys=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
@@ -429,17 +441,17 @@ google.golang.org/appengine v1.6.7/go.mod h1:8WjMMxjGQR8xUklV/ARdw2HLXBOI7O7uCID
|
||||
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
|
||||
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
|
||||
google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250825161204-c5933d9347a5 h1:BIRfGDEjiHRrk0QKZe3Xv2ieMhtgRGeLcZQ0mIVn4EY=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20250825161204-c5933d9347a5/go.mod h1:j3QtIyytwqGr1JUDtYXwtMXWPKsEa5LtzIFN1Wn5WvE=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5 h1:eaY8u2EuxbRv7c3NiGK0/NedzVsCcV6hDuU5qPX5EGE=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20250825161204-c5933d9347a5/go.mod h1:M4/wBTSeyLxupu3W3tJtOgB14jILAS/XWPSSa3TAlJc=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57 h1:JLQynH/LBHfCTSbDWl+py8C+Rg/k1OVH3xfcaiANuF0=
|
||||
google.golang.org/genproto/googleapis/api v0.0.0-20260209200024-4cfbd4190f57/go.mod h1:kSJwQxqmFXeo79zOmbrALdflXQeAYcUbgS7PbpMknCY=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57 h1:mWPCjDEyshlQYzBpMNHaEof6UX1PmHcaUODUywQ0uac=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20260209200024-4cfbd4190f57/go.mod h1:j9x/tPzZkyxcgEFkiKEEGxfvyumM01BEtsW8xzOahRQ=
|
||||
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
|
||||
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
|
||||
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
|
||||
google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
|
||||
google.golang.org/grpc v1.33.2/go.mod h1:JMHMWHQWaTccqQQlmk3MJZS+GWXOdAesneDmEnv2fbc=
|
||||
google.golang.org/grpc v1.75.0 h1:+TW+dqTd2Biwe6KKfhE5JpiYIBWq865PhKGSXiivqt4=
|
||||
google.golang.org/grpc v1.75.0/go.mod h1:JtPAzKiq4v1xcAB2hydNlWI2RnF85XXcV0mhKXr2ecQ=
|
||||
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
|
||||
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
|
||||
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
|
||||
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
|
||||
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
|
||||
@@ -449,8 +461,8 @@ google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2
|
||||
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
|
||||
google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
|
||||
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
|
||||
google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
|
||||
google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
|
||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
_ "github.com/gogf/gf/contrib/drivers/pgsql/v2"
|
||||
_ "github.com/gogf/gf/contrib/nosql/redis/v2"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/os/grpool"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -24,6 +25,11 @@ func main() {
|
||||
defer cancel()
|
||||
defer jaeger.ShutDown(ctx)
|
||||
|
||||
// 初始化全局协程池
|
||||
poolSize := g.Cfg().MustGet(ctx, "jobTask.poolSize", 5).Int()
|
||||
task.JobPool = grpool.New(poolSize)
|
||||
defer task.JobPool.Close()
|
||||
|
||||
// 注册路由
|
||||
http.RouteRegister([]interface{}{
|
||||
controller.ModelGatewayModels,
|
||||
@@ -31,25 +37,23 @@ func main() {
|
||||
controller.ModelGatewayLogsStat,
|
||||
})
|
||||
|
||||
// 本地调试:可选自动触发 worker/cleaner(由配置文件控制)
|
||||
// 本地调试:可选自动触发 worker/cleaner
|
||||
startAutoRunner(ctx)
|
||||
|
||||
// 监听退出信号,确保 Ctrl+C 能完整退出(停止 worker/cleaner 并关闭 gateway server)
|
||||
// 监听退出信号
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
|
||||
<-quit
|
||||
|
||||
g.Log().Infof(ctx, "[main] 收到退出信号,开始优雅退出...")
|
||||
cancel()
|
||||
// 关闭 gateway server(RouteRegister 内部是 go Httpserver.Run() 启动的)
|
||||
_ = http.Httpserver.Shutdown()
|
||||
}
|
||||
|
||||
func startAutoRunner(ctx context.Context) {
|
||||
// queryPending
|
||||
if g.Cfg().MustGet(ctx, "asynch.queryPending.enabled").Bool() {
|
||||
interval := g.Cfg().MustGet(ctx, "asynch.queryPending.intervalSeconds", 10).Int()
|
||||
limit := g.Cfg().MustGet(ctx, "asynch.queryPending.limit", 10).Int()
|
||||
if g.Cfg().MustGet(ctx, "queryPending.enabled").Bool() {
|
||||
interval := g.Cfg().MustGet(ctx, "queryPending.intervalSeconds", 10).Int()
|
||||
limit := g.Cfg().MustGet(ctx, "queryPending.limit", 10).Int()
|
||||
ticker := time.NewTicker(time.Duration(interval) * time.Second)
|
||||
go func() {
|
||||
defer ticker.Stop()
|
||||
|
||||
@@ -9,36 +9,29 @@ import (
|
||||
|
||||
// CreateModelReq 添加模型配置
|
||||
type CreateModelReq struct {
|
||||
g.Meta `path:"/createModel" method:"post" tags:"模型管理" summary:"创建模型配置" dc:"添加新的模型配置"`
|
||||
ModelName string `p:"modelName" json:"modelName" v:"required#模型名称不能为空" dc:"模型名称(唯一标识)"`
|
||||
ModelType int `p:"modelType" json:"modelType" v:"required#模型类型不能为空" dc:"模型类型"`
|
||||
BaseURL string `p:"baseUrl" json:"baseUrl" v:"required#模型地址不能为空" dc:"模型服务地址"`
|
||||
HttpMethod string `p:"httpMethod" json:"httpMethod" dc:"请求方式:GET/POST(默认POST)"`
|
||||
HeadMsg map[string]any `p:"headMsg" json:"headMsg" dc:"请求头JSON结构"`
|
||||
IsPrivate *int `p:"isPrivate" json:"isPrivate" dc:"是否私有化:0-私有 1-公共"`
|
||||
Enabled *int `p:"enabled" json:"enabled" dc:"是否启用:0-停用 1-启用"`
|
||||
IsChatModel *int `p:"isChatModel" json:"isChatModel" dc:"是否为对话模型:0-否 1-是"`
|
||||
CallModel *int `p:"callModel" json:"callModel" dc:"调用模式:0-同步 1-异步 2-流式"`
|
||||
RequiredFields []string `p:"requiredFields" json:"requiredFields" dc:"必填字段"`
|
||||
IsOwner *int `p:"isOwner" json:"isOwner" dc:"是否为所有者:0-否 1-是"`
|
||||
ApiKey string `p:"apiKey" json:"apiKey" dc:"调用凭证/密钥"`
|
||||
Form []map[string]any `p:"form" json:"form" dc:"动态表单配置"`
|
||||
RequestMapping map[string]any `p:"requestMapping" json:"requestMapping" dc:"请求映射"`
|
||||
ResponseMapping map[string]any `p:"responseMapping" json:"responseMapping" dc:"返回映射"`
|
||||
ResponseBody string `p:"responseBody" json:"responseBody" dc:"返回主体"`
|
||||
ResponseTokenField string `p:"responseTokenField" json:"responseTokenField" dc:"响应中消耗token的字段映射"`
|
||||
OperatorName string `p:"operatorName" json:"operatorName" dc:"运营商名称"`
|
||||
TokenConfig map[string]any `p:"tokenConfig" json:"tokenConfig" dc:"token计算配置"`
|
||||
ExtendMapping map[string]any `p:"extendMapping" json:"extendMapping" dc:"附加映射"`
|
||||
QueryConfig map[string]any `p:"queryConfig" json:"queryConfig" dc:"查询/回调配置"`
|
||||
StreamConfig map[string]any `p:"streamConfig" json:"streamConfig" dc:"流式输出配置"`
|
||||
FirstFrame string `p:"firstFrame" json:"firstFrame" dc:"首帧图片参数"`
|
||||
LastFrame string `p:"lastFrame" json:"lastFrame" dc:"尾帧图片参数"`
|
||||
MaxConcurrency int `p:"maxConcurrency" json:"maxConcurrency" dc:"最大并发数(默认10)"`
|
||||
TimeoutSeconds int `p:"timeoutSeconds" json:"timeoutSeconds" dc:"请求超时时间(秒,默认600)"`
|
||||
RetryTimes int `p:"retryTimes" json:"retryTimes" dc:"失败重试次数(默认3)"`
|
||||
AutoCleanSeconds int `p:"autoCleanSeconds" json:"autoCleanSeconds" dc:"任务完成后自动清理时间(秒,默认86400)"`
|
||||
CallbackUrl string `p:"callbackUrl" json:"callbackUrl" dc:"回调地址"`
|
||||
g.Meta `path:"/createModel" method:"post" tags:"模型管理" summary:"创建模型配置" dc:"添加新的模型配置"`
|
||||
ModelName string `p:"modelName" json:"modelName" v:"required#模型名称不能为空" dc:"模型名称(唯一标识)"`
|
||||
OperatorName string `p:"operatorName" json:"operatorName" dc:"运营商名称"`
|
||||
ModelType int `p:"modelType" json:"modelType" v:"required#模型类型不能为空" dc:"模型类型"`
|
||||
BaseURL string `p:"baseUrl" json:"baseUrl" v:"required#模型地址不能为空" dc:"模型服务地址"`
|
||||
HttpMethod string `p:"httpMethod" json:"httpMethod" dc:"请求方式:GET/POST(默认POST)"`
|
||||
HeadMsg map[string]any `p:"headMsg" json:"headMsg" dc:"请求头JSON结构"`
|
||||
IsPrivate *int `p:"isPrivate" json:"isPrivate" dc:"是否私有化:0-私有 1-公共"`
|
||||
Enabled *int `p:"enabled" json:"enabled" dc:"是否启用:0-停用 1-启用"`
|
||||
IsChatModel *int `p:"isChatModel" json:"isChatModel" dc:"是否为对话模型:0-否 1-是"`
|
||||
CallMode *int `p:"callMode" json:"callMode" dc:"调用模式:0-同步 1-异步 2-流式"`
|
||||
ApiKey string `p:"apiKey" json:"apiKey" dc:"调用凭证/密钥"`
|
||||
Form []entity.Form `p:"form" json:"form" dc:"动态表单配置"`
|
||||
RequestMapping map[string]any `p:"requestMapping" json:"requestMapping" dc:"请求映射"`
|
||||
ResponseMapping map[string]any `p:"responseMapping" json:"responseMapping" dc:"返回映射"`
|
||||
ExtendMapping map[string]any `p:"extendMapping" json:"extendMapping" dc:"附加映射"`
|
||||
QueryConfig map[string]any `p:"queryConfig" json:"queryConfig" dc:"查询/回调配置"`
|
||||
StreamConfig map[string]any `p:"streamConfig" json:"streamConfig" dc:"流式输出配置"`
|
||||
SpecialParams map[string]any `p:"specialParams" json:"specialParams" dc:"请求特殊参数(首尾帧等)"`
|
||||
RequiredFields []string `p:"requiredFields" json:"requiredFields" dc:"必填字段"`
|
||||
MaxConcurrency int `p:"maxConcurrency" json:"maxConcurrency" dc:"最大并发数(默认10)"`
|
||||
TimeoutSeconds int `p:"timeoutSeconds" json:"timeoutSeconds" dc:"请求超时时间(秒,默认600)"`
|
||||
RetryTimes int `p:"retryTimes" json:"retryTimes" dc:"失败重试次数(默认3)"`
|
||||
}
|
||||
|
||||
type CreateModelRes struct {
|
||||
@@ -46,37 +39,30 @@ type CreateModelRes struct {
|
||||
}
|
||||
|
||||
type UpdateModelReq struct {
|
||||
g.Meta `path:"/updateModel" method:"put" tags:"模型管理" summary:"更新模型配置" dc:"更新指定ID的模型配置"`
|
||||
ID int64 `p:"id" json:"id" v:"required#id不能为空" dc:"配置ID"`
|
||||
ModelName string `p:"modelName" json:"modelName" dc:"模型名称"`
|
||||
ModelType int `p:"modelType" json:"modelType" dc:"模型类型"`
|
||||
BaseURL string `p:"baseUrl" json:"baseUrl" dc:"模型服务地址"`
|
||||
HttpMethod string `p:"httpMethod" json:"httpMethod" dc:"请求方式:GET/POST"`
|
||||
HeadMsg map[string]any `p:"headMsg" json:"headMsg" dc:"请求头JSON结构"`
|
||||
IsPrivate *int `p:"isPrivate" json:"isPrivate" dc:"是否私有化:0-私有 1-公共"`
|
||||
Enabled *int `p:"enabled" json:"enabled" dc:"是否启用:0-停用 1-启用"`
|
||||
IsChatModel *int `p:"isChatModel" json:"isChatModel" dc:"是否为对话模型:0-否 1-是"`
|
||||
CallModel *int `p:"callModel" json:"callModel" dc:"调用模式:0-同步 1-异步 2-流式"`
|
||||
RequiredFields []string `p:"requiredFields" json:"requiredFields" dc:"必填字段"`
|
||||
IsOwner *int `p:"isOwner" json:"isOwner" dc:"是否为所有者:0-否 1-是"`
|
||||
ApiKey string `p:"apiKey" json:"apiKey" dc:"调用凭证/密钥"`
|
||||
Form []map[string]any `p:"form" json:"form" dc:"动态表单配置"`
|
||||
RequestMapping map[string]any `p:"requestMapping" json:"requestMapping" dc:"请求映射"`
|
||||
ResponseMapping map[string]any `p:"responseMapping" json:"responseMapping" dc:"返回映射"`
|
||||
ResponseBody string `p:"responseBody" json:"responseBody" dc:"返回主体"`
|
||||
ResponseTokenField string `p:"responseTokenField" json:"responseTokenField" dc:"响应中消耗token的字段映射"`
|
||||
OperatorName string `p:"operatorName" json:"operatorName" dc:"运营商名称"`
|
||||
TokenConfig map[string]any `p:"tokenConfig" json:"tokenConfig" dc:"token计算配置"`
|
||||
ExtendMapping map[string]any `p:"extendMapping" json:"extendMapping" dc:"附加映射"`
|
||||
QueryConfig map[string]any `p:"queryConfig" json:"queryConfig" dc:"查询/回调配置"`
|
||||
StreamConfig map[string]any `p:"streamConfig" json:"streamConfig" dc:"流式输出配置"`
|
||||
FirstFrame string `p:"firstFrame" json:"firstFrame" dc:"首帧图片参数"`
|
||||
LastFrame string `p:"lastFrame" json:"lastFrame" dc:"尾帧图片参数"`
|
||||
MaxConcurrency int `p:"maxConcurrency" json:"maxConcurrency" dc:"最大并发数"`
|
||||
TimeoutSeconds int `p:"timeoutSeconds" json:"timeoutSeconds" dc:"请求超时时间(秒)"`
|
||||
RetryTimes int `p:"retryTimes" json:"retryTimes" dc:"失败重试次数"`
|
||||
AutoCleanSeconds int `p:"autoCleanSeconds" json:"autoCleanSeconds" dc:"任务完成后自动清理时间(秒)"`
|
||||
CallbackUrl string `p:"callbackUrl" json:"callbackUrl" dc:"回调地址"`
|
||||
g.Meta `path:"/updateModel" method:"put" tags:"模型管理" summary:"更新模型配置" dc:"更新指定ID的模型配置"`
|
||||
ID int64 `p:"id" json:"id" v:"required#id不能为空" dc:"配置ID"`
|
||||
ModelName string `p:"modelName" json:"modelName" v:"required#模型名称不能为空" dc:"模型名称(唯一标识)"`
|
||||
OperatorName string `p:"operatorName" json:"operatorName" dc:"运营商名称"`
|
||||
ModelType int `p:"modelType" json:"modelType" v:"required#模型类型不能为空" dc:"模型类型"`
|
||||
BaseURL string `p:"baseUrl" json:"baseUrl" v:"required#模型地址不能为空" dc:"模型服务地址"`
|
||||
HttpMethod string `p:"httpMethod" json:"httpMethod" dc:"请求方式:GET/POST(默认POST)"`
|
||||
HeadMsg map[string]any `p:"headMsg" json:"headMsg" dc:"请求头JSON结构"`
|
||||
IsPrivate *int `p:"isPrivate" json:"isPrivate" dc:"是否私有化:0-私有 1-公共"`
|
||||
Enabled *int `p:"enabled" json:"enabled" dc:"是否启用:0-停用 1-启用"`
|
||||
IsChatModel *int `p:"isChatModel" json:"isChatModel" dc:"是否为对话模型:0-否 1-是"`
|
||||
CallMode *int `p:"callMode" json:"callMode" dc:"调用模式:0-同步 1-异步 2-流式"`
|
||||
ApiKey string `p:"apiKey" json:"apiKey" dc:"调用凭证/密钥"`
|
||||
Form []entity.Form `p:"form" json:"form" dc:"动态表单配置"`
|
||||
RequestMapping map[string]any `p:"requestMapping" json:"requestMapping" dc:"请求映射"`
|
||||
ResponseMapping map[string]any `p:"responseMapping" json:"responseMapping" dc:"返回映射"`
|
||||
ExtendMapping map[string]any `p:"extendMapping" json:"extendMapping" dc:"附加映射"`
|
||||
QueryConfig map[string]any `p:"queryConfig" json:"queryConfig" dc:"查询/回调配置"`
|
||||
StreamConfig map[string]any `p:"streamConfig" json:"streamConfig" dc:"流式输出配置"`
|
||||
SpecialParams map[string]any `p:"specialParams" json:"specialParams" dc:"请求特殊参数(首尾帧等)"`
|
||||
RequiredFields []string `p:"requiredFields" json:"requiredFields" dc:"必填字段"`
|
||||
MaxConcurrency int `p:"maxConcurrency" json:"maxConcurrency" dc:"最大并发数(默认10)"`
|
||||
TimeoutSeconds int `p:"timeoutSeconds" json:"timeoutSeconds" dc:"请求超时时间(秒,默认600)"`
|
||||
RetryTimes int `p:"retryTimes" json:"retryTimes" dc:"失败重试次数(默认3)"`
|
||||
}
|
||||
|
||||
type UpdateModelRes struct {
|
||||
@@ -95,11 +81,8 @@ type DeleteModelRes struct {
|
||||
|
||||
// GetModelReq 获取模型配置详情
|
||||
type GetModelReq struct {
|
||||
g.Meta `path:"/getModel" method:"get" tags:"模型管理" summary:"获取模型配置" dc:"根据模型ID获取配置详情"`
|
||||
ID int64 `p:"id" json:"id,string" dc:"配置ID"`
|
||||
Creator string `p:"creator" json:"creator" dc:"创建人"`
|
||||
IsChatModel *int `p:"isChatModel" json:"isChatModel" dc:"是否为聊天模型"`
|
||||
ModelName string `p:"modelName" json:"modelName" dc:"模型名称(唯一标识)"`
|
||||
g.Meta `path:"/getModel" method:"get" tags:"模型管理" summary:"获取模型配置" dc:"根据模型ID获取配置详情"`
|
||||
ID int64 `p:"id" json:"id,string" dc:"配置ID"`
|
||||
}
|
||||
|
||||
type GetModelRes struct {
|
||||
|
||||
@@ -4,6 +4,32 @@ import (
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
)
|
||||
|
||||
type BuildMessagesReq struct {
|
||||
g.Meta `path:"/buildMessages" method:"post" tags:"提示词处理" summary:"拼接提示词"`
|
||||
ModelName string `p:"modelName" json:"modelName" v:"required#modelName不能为空" dc:"实际请求的网关模型名称"`
|
||||
BuildType int64 `p:"buildType" json:"buildType" v:"required#buildType不能为空" dc:"构建类型:1单轮 2多轮"`
|
||||
SkillName string `p:"skillName" json:"skillName" dc:"技能名称"`
|
||||
CallbackUrl string `p:"callbackUrl" json:"callbackUrl" dc:"回调地址"`
|
||||
NodeId string `p:"nodeId" json:"nodeId" dc:"节点ID"`
|
||||
SessionId string `p:"sessionId" json:"sessionId" dc:"会话ID"`
|
||||
Messages map[string]any `json:"messages" dc:"前端构建好的消息结构"` //<- 实际构建根据前端表单及工作流传递所构建。待调整
|
||||
CustomPrompt string `p:"customPrompt" json:"customPrompt" dc:"用户提示词"`
|
||||
Cause string `p:"cause" json:"cause" dc:"原因"`
|
||||
}
|
||||
|
||||
type BuildMessagesRes struct {
|
||||
TaskId string `json:"taskId" dc:"任务ID"`
|
||||
}
|
||||
|
||||
type CallbackReq struct {
|
||||
g.Meta `path:"/callback" method:"post" tags:"提示词处理" summary:"model-gateway 回调" dc:"model-gateway 成功后 POST 回调:callbackUrl/{bizName}"`
|
||||
TaskId string `json:"task_id" v:"required#task_id不能为空" dc:"网关任务ID"`
|
||||
State int `json:"state" dc:"网关任务状态"`
|
||||
OssFile string `json:"oss_file" dc:"结果文件地址"`
|
||||
FileType string `json:"file_type" dc:"结果文件类型"`
|
||||
ErrorMsg string `json:"error_msg" dc:"错误信息"`
|
||||
}
|
||||
|
||||
// CreateTaskReq 创建异步任务
|
||||
type CreateTaskReq struct {
|
||||
g.Meta `path:"/createTask" method:"post" tags:"任务管理" summary:"创建异步任务" dc:"创建异步任务并返回任务ID;创建成功后会立即异步尝试执行当前任务,执行成功后按回调配置触发钩子"`
|
||||
@@ -12,12 +38,24 @@ type CreateTaskReq struct {
|
||||
CallbackUrl string `p:"callbackUrl" json:"callbackUrl" dc:"回调地址(可选,用于后续业务通知)"`
|
||||
RequestPayload map[string]any `p:"requestPayload" json:"requestPayload" dc:"请求负载(透传给模型服务)"`
|
||||
EpicycleId int64 `json:"epicycleId" dc:"轮次ID"`
|
||||
BuildType int64 `json:"buildType" dc:"构建类型:1-提示词构建 2-节点构建"`
|
||||
BuildModelName string `json:"buildModelName" json:"buildModelName" dc:"构建模型名称"`
|
||||
}
|
||||
|
||||
type CreateTaskRes struct {
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
}
|
||||
type JobTaskReq struct {
|
||||
g.Meta `path:"/jobTask" method:"post" tags:"任务管理" summary:"定时任务" dc:"循环执行待处理任务,按间隔时间和批次大小处理"`
|
||||
Interval int `json:"interval" dc:"循环间隔(秒)"`
|
||||
BatchSize int `json:"batchSize" dc:"每批执行条数"`
|
||||
PoolSize int `json:"poolSize" dc:"协程池大小"`
|
||||
}
|
||||
|
||||
type JobTaskRes struct {
|
||||
TotalProcessed int `json:"totalProcessed" dc:"总处理数"`
|
||||
SuccessCount int `json:"successCount" dc:"成功数"`
|
||||
FailCount int `json:"failCount" dc:"失败数"`
|
||||
}
|
||||
|
||||
type ModelTaskCallbackReq struct {
|
||||
g.Meta `path:"/modelCallback" method:"post" tags:"异步任务" summary:"模型任务回调通知"`
|
||||
@@ -73,10 +111,9 @@ type GetTaskBatchRes struct {
|
||||
}
|
||||
|
||||
type GetTaskBatchItem struct {
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
State int `json:"state" dc:"任务状态"`
|
||||
OssFile string `json:"ossFile" dc:"结果文件OSS地址"`
|
||||
TextResult map[string]any `json:"textResult" dc:"文本结果"`
|
||||
TaskID string `json:"taskId" dc:"任务ID"`
|
||||
State int `json:"state" dc:"任务状态"`
|
||||
OssFile string `json:"ossFile" dc:"结果文件OSS地址"`
|
||||
}
|
||||
|
||||
// ListTaskReq 任务列表分页查询
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
package entity
|
||||
|
||||
import (
|
||||
"gitea.redpowerfuture.com/red-future/common/beans"
|
||||
)
|
||||
|
||||
type modelGatewayBuildRecordCol struct {
|
||||
beans.SQLBaseCol
|
||||
TaskID string
|
||||
BuildType string
|
||||
ModelName string
|
||||
SkillName string
|
||||
SessionID string
|
||||
NodeID string
|
||||
RequestMessages string
|
||||
ResultMessages string
|
||||
Status string
|
||||
ErrorMsg string
|
||||
DurationSeconds string
|
||||
CallbackURL string
|
||||
}
|
||||
|
||||
var ModelGatewayBuildRecordCol = modelGatewayBuildRecordCol{
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
TaskID: "task_id",
|
||||
BuildType: "build_type",
|
||||
ModelName: "model_name",
|
||||
SkillName: "skill_name",
|
||||
SessionID: "session_id",
|
||||
NodeID: "node_id",
|
||||
RequestMessages: "request_messages",
|
||||
ResultMessages: "result_messages",
|
||||
Status: "status",
|
||||
ErrorMsg: "error_msg",
|
||||
DurationSeconds: "duration_seconds",
|
||||
CallbackURL: "callback_url",
|
||||
}
|
||||
|
||||
// ModelGatewayBuildRecord 构建记录
|
||||
type ModelGatewayBuildRecord struct {
|
||||
beans.SQLBaseDO `orm:",inline"`
|
||||
TaskID string `orm:"task_id" json:"taskId"`
|
||||
BuildType int64 `orm:"build_type" json:"buildType"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
SkillName string `orm:"skill_name" json:"skillName"`
|
||||
SessionID string `orm:"session_id" json:"sessionId"`
|
||||
NodeID string `orm:"node_id" json:"nodeId"`
|
||||
RequestMessages map[string]any `orm:"request_messages" json:"requestMessages"`
|
||||
ResultMessages []map[string]any `orm:"result_messages" json:"resultMessages"`
|
||||
Status int `orm:"status" json:"status"`
|
||||
ErrorMsg string `orm:"error_msg" json:"errorMsg"`
|
||||
DurationSeconds int `orm:"duration_seconds" json:"durationSeconds"`
|
||||
CallbackURL string `orm:"callback_url" json:"callbackUrl"`
|
||||
}
|
||||
@@ -40,17 +40,17 @@ var ModelGatewayLogsOpCol = modelGatewayLogsOpCol{
|
||||
// ModelGatewayLogsOp 操作日志
|
||||
type ModelGatewayLogsOp struct {
|
||||
beans.SQLBaseDO `orm:",inline"`
|
||||
IP string `orm:"ip" json:"ip"`
|
||||
UserAgent string `orm:"user_agent" json:"userAgent"`
|
||||
APIPath string `orm:"api_path" json:"apiPath"`
|
||||
HttpMethod string `orm:"http_method" json:"httpMethod"`
|
||||
BizName string `orm:"biz_name" json:"bizName"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
TaskID string `orm:"task_id" json:"taskId"`
|
||||
OpType string `orm:"op_type" json:"opType"`
|
||||
Success int `orm:"success" json:"success"`
|
||||
ErrorMsg string `orm:"error_msg" json:"errorMsg"`
|
||||
CostMs int64 `orm:"cost_ms" json:"costMs"`
|
||||
RequestPayload *RequestPayload `orm:"request_payload" json:"requestPayload"`
|
||||
ResponsePayload map[string]any `orm:"response_payload" json:"responsePayload"`
|
||||
IP string `orm:"ip" json:"ip"`
|
||||
UserAgent string `orm:"user_agent" json:"userAgent"`
|
||||
APIPath string `orm:"api_path" json:"apiPath"`
|
||||
HttpMethod string `orm:"http_method" json:"httpMethod"`
|
||||
BizName string `orm:"biz_name" json:"bizName"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
TaskID string `orm:"task_id" json:"taskId"`
|
||||
OpType string `orm:"op_type" json:"opType"`
|
||||
Success int `orm:"success" json:"success"`
|
||||
ErrorMsg string `orm:"error_msg" json:"errorMsg"`
|
||||
CostMs int64 `orm:"cost_ms" json:"costMs"`
|
||||
RequestPayload map[string]any `orm:"request_payload" json:"requestPayload"`
|
||||
ResponsePayload map[string]any `orm:"response_payload" json:"responsePayload"`
|
||||
}
|
||||
|
||||
@@ -4,99 +4,109 @@ import "gitea.redpowerfuture.com/red-future/common/beans"
|
||||
|
||||
type modelGatewayModelCol struct {
|
||||
beans.SQLBaseCol
|
||||
ModelName string
|
||||
ModelType string
|
||||
BaseURL string
|
||||
HttpMethod string
|
||||
HeadMsg string
|
||||
FormJSON string
|
||||
RequestMapping string
|
||||
ResponseMapping string
|
||||
ResponseBody string
|
||||
ResponseTokenField string
|
||||
RequiredFields string
|
||||
IsPrivate string
|
||||
IsChatModel string
|
||||
CallMode string
|
||||
ApiKey string
|
||||
Enabled string
|
||||
MaxConcurrency string
|
||||
TimeoutSeconds string
|
||||
RetryTimes string
|
||||
AutoCleanSeconds string
|
||||
IsOwner string
|
||||
OperatorName string
|
||||
TokenConfig string
|
||||
ExtendMapping string
|
||||
QueryConfig string
|
||||
StreamConfig string
|
||||
FirstFrame string
|
||||
LastFrame string
|
||||
MaxTokens string
|
||||
ModelName string
|
||||
ModelType string
|
||||
BaseURL string
|
||||
HttpMethod string
|
||||
HeadMsg string
|
||||
FormJSON string
|
||||
RequestMapping string
|
||||
ResponseMapping string
|
||||
RequiredFields string
|
||||
IsPrivate string
|
||||
IsChatModel string
|
||||
CallMode string
|
||||
ApiKey string
|
||||
Enabled string
|
||||
MaxConcurrency string
|
||||
TimeoutSeconds string
|
||||
RetryTimes string
|
||||
OperatorName string
|
||||
ExtendMapping string
|
||||
QueryConfig string
|
||||
StreamConfig string
|
||||
SpecialParams string
|
||||
BillingConfig string
|
||||
}
|
||||
|
||||
var ModelGatewayModelCol = modelGatewayModelCol{
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
ModelName: "model_name",
|
||||
ModelType: "model_type",
|
||||
BaseURL: "base_url",
|
||||
HttpMethod: "http_method",
|
||||
HeadMsg: "head_msg",
|
||||
FormJSON: "form_json",
|
||||
RequestMapping: "request_mapping",
|
||||
ResponseMapping: "response_mapping",
|
||||
ResponseBody: "response_body",
|
||||
ResponseTokenField: "response_token_field",
|
||||
RequiredFields: "required_fields",
|
||||
IsPrivate: "is_private",
|
||||
IsChatModel: "is_chat_model",
|
||||
CallMode: "call_mode",
|
||||
ApiKey: "api_key",
|
||||
Enabled: "enabled",
|
||||
MaxConcurrency: "max_concurrency",
|
||||
TimeoutSeconds: "timeout_seconds",
|
||||
RetryTimes: "retry_times",
|
||||
AutoCleanSeconds: "auto_clean_seconds",
|
||||
IsOwner: "is_owner",
|
||||
OperatorName: "operator_name",
|
||||
TokenConfig: "token_config",
|
||||
ExtendMapping: "extend_mapping",
|
||||
QueryConfig: "query_config",
|
||||
StreamConfig: "stream_config",
|
||||
FirstFrame: "first_frame",
|
||||
LastFrame: "last_frame",
|
||||
MaxTokens: "max_tokens",
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
ModelName: "model_name",
|
||||
ModelType: "model_type",
|
||||
BaseURL: "base_url",
|
||||
HttpMethod: "http_method",
|
||||
HeadMsg: "head_msg",
|
||||
FormJSON: "form_json",
|
||||
RequestMapping: "request_mapping",
|
||||
ResponseMapping: "response_mapping",
|
||||
RequiredFields: "required_fields",
|
||||
IsPrivate: "is_private",
|
||||
IsChatModel: "is_chat_model",
|
||||
CallMode: "call_mode",
|
||||
ApiKey: "api_key",
|
||||
Enabled: "enabled",
|
||||
MaxConcurrency: "max_concurrency",
|
||||
TimeoutSeconds: "timeout_seconds",
|
||||
RetryTimes: "retry_times",
|
||||
OperatorName: "operator_name",
|
||||
ExtendMapping: "extend_mapping",
|
||||
QueryConfig: "query_config",
|
||||
StreamConfig: "stream_config",
|
||||
SpecialParams: "special_params",
|
||||
BillingConfig: "billing_config",
|
||||
}
|
||||
|
||||
type ModelGatewayModel struct {
|
||||
beans.SQLBaseDO `orm:",inline"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
ModelType int `orm:"model_type" json:"modelType"`
|
||||
BaseURL string `orm:"base_url" json:"baseUrl"`
|
||||
HttpMethod string `orm:"http_method" json:"httpMethod"`
|
||||
HeadMsg map[string]any `orm:"head_msg" json:"headMsg"`
|
||||
Form []map[string]any `orm:"form_json" json:"form"`
|
||||
RequestMapping map[string]any `orm:"request_mapping" json:"requestMapping"`
|
||||
ResponseMapping map[string]any `orm:"response_mapping" json:"responseMapping"`
|
||||
ResponseBody string `orm:"response_body" json:"responseBody"`
|
||||
ResponseTokenField string `orm:"response_token_field" json:"tokenField"`
|
||||
RequiredFields []string `orm:"required_fields" json:"requiredFields"`
|
||||
IsPrivate *int `orm:"is_private" json:"isPrivate"`
|
||||
IsChatModel *int `orm:"is_chat_model" json:"isChatModel"`
|
||||
CallMode *int `orm:"call_mode" json:"callMode"`
|
||||
ApiKey string `orm:"api_key" json:"apiKey"`
|
||||
Enabled *int `orm:"enabled" json:"enabled"`
|
||||
MaxConcurrency int `orm:"max_concurrency" json:"maxConcurrency"`
|
||||
TimeoutSeconds int `orm:"timeout_seconds" json:"timeoutSeconds"`
|
||||
RetryTimes int `orm:"retry_times" json:"retryTimes"`
|
||||
AutoCleanSeconds int `orm:"auto_clean_seconds" json:"autoCleanSeconds"`
|
||||
IsOwner *int `orm:"is_owner" json:"isOwner"`
|
||||
OperatorName string `orm:"operator_name" json:"operatorName"`
|
||||
TokenConfig map[string]any `orm:"token_config" json:"tokenConfig"`
|
||||
ExtendMapping map[string]any `orm:"extend_mapping" json:"extendMapping"`
|
||||
QueryConfig map[string]any `orm:"query_config" json:"queryConfig"`
|
||||
StreamConfig map[string]any `orm:"stream_config" json:"streamConfig"`
|
||||
FirstFrame string `orm:"first_frame" json:"firstFrame"`
|
||||
LastFrame string `orm:"last_frame" json:"lastFrame"`
|
||||
MaxTokens int `orm:"max_tokens" json:"maxTokens"`
|
||||
beans.SQLBaseDO `orm:",inline"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
OperatorName string `orm:"operator_name" json:"operatorName"`
|
||||
ModelType int `orm:"model_type" json:"modelType"`
|
||||
BaseURL string `orm:"base_url" json:"baseUrl"`
|
||||
HttpMethod string `orm:"http_method" json:"httpMethod"`
|
||||
HeadMsg map[string]any `orm:"head_msg" json:"headMsg"`
|
||||
IsPrivate *int `orm:"is_private" json:"isPrivate"`
|
||||
IsChatModel *int `orm:"is_chat_model" json:"isChatModel"`
|
||||
CallMode *int `orm:"call_mode" json:"callMode"`
|
||||
ApiKey string `orm:"api_key" json:"apiKey"`
|
||||
Enabled *int `orm:"enabled" json:"enabled"`
|
||||
Form []Form `orm:"form_json" json:"form"`
|
||||
RequestMapping map[string]any `orm:"request_mapping" json:"requestMapping"`
|
||||
ResponseMapping map[string]any `orm:"response_mapping" json:"responseMapping"`
|
||||
ExtendMapping map[string]any `orm:"extend_mapping" json:"extendMapping"`
|
||||
QueryConfig map[string]any `orm:"query_config" json:"queryConfig"`
|
||||
StreamConfig map[string]any `orm:"stream_config" json:"streamConfig"`
|
||||
SpecialParams map[string]any `orm:"special_params" json:"specialParams"`
|
||||
BillingConfig map[string]any `orm:"billing_config" json:"billingConfig"`
|
||||
RequiredFields []string `orm:"required_fields" json:"requiredFields"`
|
||||
MaxConcurrency int `orm:"max_concurrency" json:"maxConcurrency"`
|
||||
TimeoutSeconds int `orm:"timeout_seconds" json:"timeoutSeconds"`
|
||||
RetryTimes int `orm:"retry_times" json:"retryTimes"`
|
||||
}
|
||||
|
||||
type Form struct {
|
||||
Key string `json:"key"`
|
||||
Value any `json:"value"`
|
||||
Label string `json:"label"`
|
||||
Type string `json:"type"`
|
||||
DefaultValue any `json:"defaultValue"`
|
||||
Required bool `json:"required"`
|
||||
IsForm bool `json:"isForm"`
|
||||
Options []map[string]any `json:"options"`
|
||||
Role string `json:"role"`
|
||||
FieldConstraint FieldConstraint `json:"fieldConstraint"`
|
||||
}
|
||||
|
||||
type FieldConstraint struct {
|
||||
MaxLength int `json:"maxLength"`
|
||||
MinLength int `json:"minLength"`
|
||||
NumberType string `json:"numberType"`
|
||||
Min any `json:"min"`
|
||||
Max any `json:"max"`
|
||||
MaxSize int `json:"maxSize"`
|
||||
MaxCount int `json:"maxCount"`
|
||||
Accept string `json:"accept"`
|
||||
}
|
||||
|
||||
const (
|
||||
ResponseBody = "content"
|
||||
)
|
||||
|
||||
@@ -11,16 +11,14 @@ type modelGatewayTaskCol struct {
|
||||
BizName string
|
||||
CallbackURL string
|
||||
State string
|
||||
Phase string
|
||||
RetryCount string
|
||||
ErrorMsg string
|
||||
ResultFile string
|
||||
TextResult string
|
||||
ExpendTokens string
|
||||
DurationSeconds string
|
||||
RetryCount string
|
||||
TmpFile string
|
||||
RequestPayload string
|
||||
DurationSeconds string
|
||||
EpicycleId string
|
||||
BillingData string
|
||||
BuildModelName string
|
||||
}
|
||||
|
||||
var ModelGatewayTaskCol = modelGatewayTaskCol{
|
||||
@@ -30,36 +28,32 @@ var ModelGatewayTaskCol = modelGatewayTaskCol{
|
||||
BizName: "biz_name",
|
||||
CallbackURL: "callback_url",
|
||||
State: "state",
|
||||
Phase: "phase",
|
||||
RetryCount: "retry_count",
|
||||
ErrorMsg: "error_msg",
|
||||
ResultFile: "result_file",
|
||||
TextResult: "text_result",
|
||||
ExpendTokens: "expend_tokens",
|
||||
DurationSeconds: "duration_seconds",
|
||||
RetryCount: "retry_count",
|
||||
TmpFile: "tmp_file",
|
||||
RequestPayload: "request_payload",
|
||||
DurationSeconds: "duration_seconds",
|
||||
EpicycleId: "epicycle_id",
|
||||
BillingData: "billing_data",
|
||||
BuildModelName: "build_model_name",
|
||||
}
|
||||
|
||||
// ModelGatewayTask 模型网关任务
|
||||
type ModelGatewayTask struct {
|
||||
beans.SQLBaseDO `orm:",inline"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
TaskID string `orm:"task_id" json:"taskId"`
|
||||
BizName string `orm:"biz_name" json:"bizName"`
|
||||
CallbackURL string `orm:"callback_url" json:"callbackUrl"`
|
||||
State int `orm:"state" json:"state"`
|
||||
Phase int `orm:"phase" json:"phase"`
|
||||
ErrorMsg string `orm:"error_msg" json:"errorMsg"`
|
||||
ResultFile *ResultFile `orm:"result_file" json:"resultFile"`
|
||||
TextResult map[string]any `orm:"text_result" json:"text"`
|
||||
ExpendTokens int64 `orm:"expend_tokens" json:"expendTokens"`
|
||||
DurationSeconds int64 `orm:"duration_seconds" json:"durationSeconds"`
|
||||
RetryCount int `orm:"retry_count" json:"retryCount"`
|
||||
TmpFile string `orm:"tmp_file" json:"tmpFile"`
|
||||
RequestPayload *RequestPayload `orm:"request_payload" json:"requestPayload"`
|
||||
EpicycleId int64 `orm:"epicycle_id" json:"epicycleId"`
|
||||
ModelName string `orm:"model_name" json:"modelName"`
|
||||
TaskID string `orm:"task_id" json:"taskId"`
|
||||
BizName string `orm:"biz_name" json:"bizName"`
|
||||
CallbackURL string `orm:"callback_url" json:"callbackUrl"`
|
||||
State int `orm:"state" json:"state"`
|
||||
RetryCount int `orm:"retry_count" json:"retryCount"`
|
||||
ErrorMsg string `orm:"error_msg" json:"errorMsg"`
|
||||
ResultFile *ResultFile `orm:"result_file" json:"resultFile"`
|
||||
RequestPayload map[string]any `orm:"request_payload" json:"requestPayload"`
|
||||
DurationSeconds int64 `orm:"duration_seconds" json:"durationSeconds"`
|
||||
EpicycleId int64 `orm:"epicycle_id" json:"epicycleId"`
|
||||
BillingData []map[string]any `orm:"billing_data" json:"billingData"`
|
||||
BuildModelName string `orm:"build_model_name" json:"buildModelName"`
|
||||
}
|
||||
|
||||
// ResultFile OSS 结果文件
|
||||
@@ -68,9 +62,3 @@ type ResultFile struct {
|
||||
FileType string `json:"fileType"`
|
||||
FileSize int64 `json:"fileSize"`
|
||||
}
|
||||
|
||||
// RequestPayload 请求参数结构体
|
||||
type RequestPayload struct {
|
||||
Headers map[string]string `json:"headers"`
|
||||
Body map[string]any `json:"body"`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
package entity
|
||||
|
||||
import "gitea.redpowerfuture.com/red-future/common/beans"
|
||||
|
||||
// ProviderProtocol 模型协议映射配置
|
||||
type ProviderProtocol struct {
|
||||
beans.SQLBaseDO `orm:",inherit"`
|
||||
ProviderName string `orm:"provider_name" json:"providerName"`
|
||||
RequestTemplate map[string]any `orm:"request_template" json:"requestTemplate"`
|
||||
SystemPromptTemplate string `orm:"system_prompt_template" json:"systemPromptTemplate"`
|
||||
Status int `orm:"status" json:"status"`
|
||||
}
|
||||
|
||||
// providerProtocolCol 列名
|
||||
type providerProtocolCol struct {
|
||||
beans.SQLBaseCol
|
||||
ProviderName string
|
||||
RequestTemplate string
|
||||
SystemPromptTemplate string
|
||||
Status string
|
||||
}
|
||||
|
||||
// ProviderProtocolCol 列名常量
|
||||
var ProviderProtocolCol = providerProtocolCol{
|
||||
SQLBaseCol: beans.DefSQLBaseCol,
|
||||
ProviderName: "provider_name",
|
||||
RequestTemplate: "request_template",
|
||||
SystemPromptTemplate: "system_prompt_template",
|
||||
Status: "status",
|
||||
}
|
||||
@@ -7,15 +7,28 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"mime/multipart"
|
||||
"model-gateway/common/util"
|
||||
"model-gateway/model/entity"
|
||||
"time"
|
||||
|
||||
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/os/gtime"
|
||||
"github.com/gogf/gf/v2/util/guid"
|
||||
)
|
||||
|
||||
// ForwardHeaders 获取转发请求头
|
||||
func ForwardHeaders(ctx context.Context) map[string]string {
|
||||
headers := make(map[string]string)
|
||||
if r := g.RequestFromCtx(ctx); r != nil {
|
||||
for k, v := range r.Request.Header {
|
||||
if len(v) > 0 {
|
||||
headers[k] = v[0]
|
||||
}
|
||||
}
|
||||
}
|
||||
return headers
|
||||
}
|
||||
|
||||
type UploadFileResponse struct {
|
||||
FileURL string `json:"fileURL"` // 文件 URL
|
||||
FileSize int `json:"fileSize"` // 文件大小(字节)
|
||||
@@ -43,16 +56,25 @@ func UploadByTask(ctx context.Context, data []byte, fileExt string) (oss *Upload
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err := part.Write(data); err != nil {
|
||||
if _, err = part.Write(data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
contentType := writer.FormDataContentType()
|
||||
//contentType := writer.FormDataContentType()
|
||||
if err = writer.Close(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
headers := util.ForwardHeaders(ctx)
|
||||
headers["Content-Type"] = contentType
|
||||
//headers := util.ForwardHeaders(ctx)
|
||||
//headers["Content-Type"] = contentType
|
||||
|
||||
headers := make(map[string]string)
|
||||
headers["Content-Type"] = writer.FormDataContentType()
|
||||
if r := g.RequestFromCtx(ctx); r != nil {
|
||||
if auth := r.Header.Get("Authorization"); auth != "" {
|
||||
headers["Authorization"] = auth
|
||||
}
|
||||
}
|
||||
|
||||
fullURL := "oss/file/uploadFile"
|
||||
g.Log().Infof(ctx, "[OSS] upload start url=%s filename=%s size=%d", fullURL, filename, len(data))
|
||||
|
||||
@@ -69,23 +91,27 @@ func UploadByTask(ctx context.Context, data []byte, fileExt string) (oss *Upload
|
||||
|
||||
// CallbackPayload 回调请求体
|
||||
type CallbackPayload struct {
|
||||
TaskId string `json:"task_id"`
|
||||
State int `json:"state"`
|
||||
OssFile string `json:"oss_file"`
|
||||
FileType string `json:"file_type"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
TaskId string `json:"task_id"`
|
||||
State int `json:"state"`
|
||||
OssFile string `json:"oss_file"`
|
||||
FileType string `json:"file_type"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
BillingDate []map[string]any `json:"billing_data"`
|
||||
}
|
||||
|
||||
// TriggerCallback 任务的回调
|
||||
func TriggerCallback(ctx context.Context, t *entity.ModelGatewayTask) {
|
||||
headers := util.ForwardHeaders(ctx)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var resp struct{}
|
||||
payload := CallbackPayload{
|
||||
TaskId: t.TaskID,
|
||||
State: t.State,
|
||||
OssFile: t.ResultFile.OssFile,
|
||||
FileType: t.ResultFile.FileType,
|
||||
ErrorMsg: t.ErrorMsg,
|
||||
TaskId: t.TaskID,
|
||||
State: t.State,
|
||||
ErrorMsg: t.ErrorMsg,
|
||||
BillingDate: t.BillingData,
|
||||
}
|
||||
if !g.IsEmpty(t.ResultFile) {
|
||||
payload.OssFile = t.ResultFile.OssFile
|
||||
payload.FileType = t.ResultFile.FileType
|
||||
}
|
||||
jsonData, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
@@ -105,22 +131,25 @@ func TriggerCallback(ctx context.Context, t *entity.ModelGatewayTask) {
|
||||
|
||||
// PromptsCallbackPayload 提示词回调请求体
|
||||
type PromptsCallbackPayload struct {
|
||||
EpicycleId int64 `json:"epicycleId"`
|
||||
Messages map[string]any `json:"messages"`
|
||||
EpicycleId int64 `json:"epicycleId"`
|
||||
TaskId string `json:"task_id"`
|
||||
State int `json:"state"`
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
OssFile string `json:"oss_file"`
|
||||
FileType string `json:"file_type"`
|
||||
}
|
||||
|
||||
// TriggerPromptsCallback 任务成功后的提示词回调
|
||||
func TriggerPromptsCallback(ctx context.Context, t *entity.ModelGatewayTask, epicycleId int64) {
|
||||
func TriggerPromptsCallback(ctx context.Context, t *entity.ModelGatewayTask) {
|
||||
callbackURL := "prompts-core/session/callback"
|
||||
headers := util.ForwardHeaders(ctx)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var resp struct{}
|
||||
payload := PromptsCallbackPayload{
|
||||
EpicycleId: epicycleId,
|
||||
Messages: t.TextResult,
|
||||
EpicycleId: t.EpicycleId,
|
||||
}
|
||||
jsonData, err := json.Marshal(payload)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[提示词回调] JSON序列化失败 epicycleId=%d 错误=%v", epicycleId, err)
|
||||
g.Log().Warningf(ctx, "[提示词回调] JSON序列化失败 epicycleId=%d 错误=%v", t.EpicycleId, err)
|
||||
return
|
||||
}
|
||||
g.Log().Infof(ctx, "[提示词回调] 开始发送 epicycleId=%d 回调地址=%s 请求头数量=%d 消息体大小=%d字节",
|
||||
@@ -134,9 +163,36 @@ func TriggerPromptsCallback(ctx context.Context, t *entity.ModelGatewayTask, epi
|
||||
g.Log().Infof(ctx, "[提示词回调] 发送成功 epicycleId=%d 回调地址=%s 消息体大小=%d字节", t.EpicycleId, callbackURL, len(jsonData))
|
||||
}
|
||||
|
||||
// BuildCallbackPayload 构建回调请求体
|
||||
type BuildCallbackPayload struct {
|
||||
TaskId string `json:"taskId"`
|
||||
Status int `json:"status"`
|
||||
Messages any `json:"messages"`
|
||||
ErrorMsg string `json:"errorMsg"`
|
||||
}
|
||||
|
||||
// CallbackBuildResult 回调构建结果
|
||||
func CallbackBuildResult(ctx context.Context, record *entity.ModelGatewayBuildRecord) {
|
||||
headers := ForwardHeaders(ctx)
|
||||
payload := BuildCallbackPayload{
|
||||
TaskId: record.TaskID,
|
||||
Status: record.Status,
|
||||
Messages: record.ResultMessages,
|
||||
ErrorMsg: record.ErrorMsg,
|
||||
}
|
||||
jsonData, _ := json.Marshal(payload)
|
||||
|
||||
var resp struct{}
|
||||
if err := commonHttp.Post(ctx, record.CallbackURL, headers, &resp, jsonData); err != nil {
|
||||
g.Log().Warningf(ctx, "[构建回调] 发送失败 taskId=%s err=%v", record.TaskID, err)
|
||||
return
|
||||
}
|
||||
g.Log().Infof(ctx, "[构建回调] 发送成功 taskId=%s", record.TaskID)
|
||||
}
|
||||
|
||||
// IsSuperAdmin 调用admin-go服务检查是否是超级管理员
|
||||
func IsSuperAdmin(ctx context.Context) (res bool, err error) {
|
||||
headers := util.ForwardHeaders(ctx)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var r = make(map[string]bool)
|
||||
if err = commonHttp.Get(ctx, "admin-go/api/v1/system/user/checkIsSuperAdmin", headers, &r); err != nil {
|
||||
return false, err
|
||||
@@ -144,51 +200,121 @@ func IsSuperAdmin(ctx context.Context) (res bool, err error) {
|
||||
return r["isSuperAdmin"], err
|
||||
}
|
||||
|
||||
//// callback 向回调地址 POST 任务结果(与查询接口 GetTaskRes 出参一致)
|
||||
//func (s *audioTaskService) callback(ctx context.Context, taskID, status, errMsg, callbackURL string) {
|
||||
// if callbackURL == "" {
|
||||
// return
|
||||
// }
|
||||
//
|
||||
// task, _ := dao.TranscribeTask.GetByTaskID(ctx, taskID)
|
||||
// if task == nil {
|
||||
// g.Log().Errorf(ctx, "[回调 %s] 任务不存在", taskID)
|
||||
// return
|
||||
// }
|
||||
//
|
||||
// detailList, _ := dao.TranscribeTaskDetail.ListByTaskID(ctx, taskID)
|
||||
// detailItems := make([]dto.TranscribeTaskDetailItem, 0, len(detailList))
|
||||
// for i := range detailList {
|
||||
// detailItems = append(detailItems, dao.DetailEntityToItem(&detailList[i]))
|
||||
// }
|
||||
//
|
||||
// // 构建与查询接口一致的 taskInfo
|
||||
// taskInfo := dao.EntityToItem(task)
|
||||
//
|
||||
// // 兼容历史数据: 从 result 中补全 scenes 等字段
|
||||
// detailItems = enrichDetailsFromResult(task.Result, detailItems)
|
||||
//
|
||||
// payload := dto.CallbackPayload{
|
||||
// TaskInfo: taskInfo,
|
||||
// DetailList: detailItems,
|
||||
// }
|
||||
//
|
||||
// body, _ := json.Marshal(payload)
|
||||
//
|
||||
// // 透传调用方的用户信息
|
||||
// userJSON, _ := json.Marshal(beans.User{UserName: "admin", TenantId: 1})
|
||||
//
|
||||
// req, _ := http.NewRequest("POST", callbackURL, bytes.NewReader(body))
|
||||
// req.Header.Set("Content-Type", "application/json")
|
||||
// req.Header.Set("X-User-Info", string(userJSON))
|
||||
//
|
||||
// resp, reqErr := http.DefaultClient.Do(req)
|
||||
// if reqErr != nil {
|
||||
// g.Log().Errorf(ctx, "[回调 %s] 请求失败: %v", taskID, reqErr)
|
||||
// return
|
||||
// }
|
||||
// defer resp.Body.Close()
|
||||
//
|
||||
// respBody, _ := io.ReadAll(resp.Body)
|
||||
// g.Log().Infof(ctx, "[回调 %s] 响应 status=%d, body=%s", taskID, resp.StatusCode, string(respBody))
|
||||
//}
|
||||
// SkillUserVO 技能用户视图对象
|
||||
type SkillUserVO struct {
|
||||
Id int64 `json:"id,string"`
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description"`
|
||||
FileName string `json:"fileName"`
|
||||
FileUrl string `json:"fileUrl"`
|
||||
CreatedAt *gtime.Time `json:"createdAt"`
|
||||
UpdatedAt *gtime.Time `json:"updatedAt"`
|
||||
ImgAddressPrefix string `json:"imgAddressPrefix"`
|
||||
}
|
||||
|
||||
// GetSkillUser 获取技能用户信息
|
||||
func GetSkillUser(ctx context.Context, name string) (*SkillUserVO, error) {
|
||||
fullURL := fmt.Sprintf("ai-agent/skill/user/getUserOrTemplate?name=%s", name)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var resp SkillUserVO
|
||||
var req struct{}
|
||||
if err := commonHttp.Get(ctx, fullURL, headers, &resp, req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
// SessionHistoryItem 会话历史条目
|
||||
type SessionHistoryItem struct {
|
||||
Role string `json:"role"`
|
||||
Content string `json:"content"`
|
||||
}
|
||||
|
||||
// GetSessionHistory 获取会话历史
|
||||
func GetSessionHistory(ctx context.Context, nodeId, sessionId string) ([]SessionHistoryItem, error) {
|
||||
fullURL := fmt.Sprintf("model-session/session/sessionHistory?nodeId=%s&sessionId=%s", nodeId, sessionId)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var req struct{}
|
||||
var resp []SessionHistoryItem
|
||||
if err := commonHttp.Get(ctx, fullURL, headers, &resp, req); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// VideoDurationResp 视频时长接口返回
|
||||
type VideoDurationResp struct {
|
||||
Videos []VideoInfo `json:"videos"`
|
||||
Count int `json:"count"`
|
||||
TotalDuration float64 `json:"totalDuration"`
|
||||
TotalDurationStr string `json:"totalDurationStr"`
|
||||
}
|
||||
|
||||
type VideoInfo struct {
|
||||
Index int `json:"index"`
|
||||
VideoUrl string `json:"videoUrl"`
|
||||
Duration float64 `json:"duration"`
|
||||
DurationStr string `json:"durationStr"`
|
||||
}
|
||||
|
||||
// GetVideoDuration 获取视频时长
|
||||
func GetVideoDuration(ctx context.Context, urls []string) (VideoDurationResp, error) {
|
||||
apiURL := "media/video/duration"
|
||||
headers := ForwardHeaders(ctx)
|
||||
body := map[string]any{"video_urls": urls}
|
||||
jsonData, _ := json.Marshal(body)
|
||||
|
||||
var resp VideoDurationResp
|
||||
err := commonHttp.Post(ctx, apiURL, headers, &resp, jsonData)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[视频时长] 获取失败 err=%v", err)
|
||||
return resp, err
|
||||
}
|
||||
|
||||
g.Log().Infof(ctx, "[视频时长] 获取成功 count=%d totalDuration=%.2f", resp.Count, resp.TotalDuration)
|
||||
return resp, nil
|
||||
}
|
||||
|
||||
// DeductBalanceReq 扣减余额请求
|
||||
type DeductBalanceReq struct {
|
||||
Id uint64 `json:"id"`
|
||||
Surplus float64 `json:"surplus"`
|
||||
}
|
||||
|
||||
// DeductBalance 扣减租户余额
|
||||
func DeductBalance(ctx context.Context, tenantId uint64, amount float64) error {
|
||||
apiURL := "admin-go/api/v1/system/tenant/edit"
|
||||
headers := ForwardHeaders(ctx)
|
||||
body := DeductBalanceReq{
|
||||
Id: tenantId,
|
||||
Surplus: amount,
|
||||
}
|
||||
jsonData, _ := json.Marshal(body)
|
||||
|
||||
var resp struct{}
|
||||
err := commonHttp.Put(ctx, apiURL, headers, &resp, jsonData)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[扣减余额] 失败 tenantId=%d amount=%.6f err=%v", tenantId, amount, err)
|
||||
return err
|
||||
}
|
||||
g.Log().Infof(ctx, "[扣减余额] 成功 tenantId=%d amount=%.6f", tenantId, amount)
|
||||
return nil
|
||||
}
|
||||
|
||||
// TenantSurplusResp 租户余额返回
|
||||
type TenantSurplusResp struct {
|
||||
Surplus float64 `json:"surplus"`
|
||||
}
|
||||
|
||||
// GetTenantSurplus 获取租户余额
|
||||
func GetTenantSurplus(ctx context.Context, tenantId uint64) (float64, error) {
|
||||
apiURL := fmt.Sprintf("admin-go/api/v1/system/tenant/getTenantDetails?tenantId=%d", tenantId)
|
||||
headers := ForwardHeaders(ctx)
|
||||
var resp TenantSurplusResp
|
||||
err := commonHttp.Get(ctx, apiURL, headers, &resp, nil)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[获取余额] 失败 tenantId=%d err=%v", tenantId, err)
|
||||
return 0, err
|
||||
}
|
||||
return resp.Surplus, nil
|
||||
}
|
||||
|
||||
@@ -30,13 +30,8 @@ func (s *modelService) Create(ctx context.Context, req *dto.CreateModelReq) (*dt
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
// 2)判断是否超管,决定 isOwner
|
||||
req.IsOwner = gconv.PtrInt(1)
|
||||
if isAdmin, _ := gateway.IsSuperAdmin(ctx); isAdmin {
|
||||
req.IsOwner = gconv.PtrInt(0)
|
||||
}
|
||||
|
||||
// 3)入库
|
||||
// 2)入库
|
||||
id, err := dao.ModelGatewayModels.Insert(ctx, util.ConvertTo[entity.ModelGatewayModel](req))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -53,9 +48,7 @@ func (s *modelService) Update(ctx context.Context, req *dto.UpdateModelReq) erro
|
||||
}
|
||||
}
|
||||
// 2)超管创建/普通用户更新
|
||||
req.IsOwner = gconv.PtrInt(1)
|
||||
if isAdmin, _ := gateway.IsSuperAdmin(ctx); isAdmin {
|
||||
req.IsOwner = gconv.PtrInt(0)
|
||||
_, err := dao.ModelGatewayModels.Update(ctx, util.ConvertTo[entity.ModelGatewayModel](req))
|
||||
return err
|
||||
}
|
||||
@@ -88,18 +81,13 @@ func (s *modelService) Get(ctx context.Context, req *dto.GetModelReq) (*dto.GetM
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if g.IsEmpty(req.ID) {
|
||||
req.Creator = user.UserName
|
||||
}
|
||||
model, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{
|
||||
Id: req.ID,
|
||||
Creator: user.UserName,
|
||||
},
|
||||
ModelName: req.ModelName,
|
||||
IsChatModel: req.IsChatModel,
|
||||
})
|
||||
if err != nil {
|
||||
if err != nil || model == nil {
|
||||
return nil, err
|
||||
}
|
||||
return &dto.GetModelRes{
|
||||
|
||||
@@ -0,0 +1,282 @@
|
||||
package prompt
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"model-gateway/common/util"
|
||||
"model-gateway/service/gateway"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
)
|
||||
|
||||
const (
|
||||
bytesPerKB = 1024
|
||||
bytesPerMB = 1024 * 1024
|
||||
)
|
||||
|
||||
// FetchFileTextsAsString 从 URL 列表获取文件内容,拼接为字符串
|
||||
func FetchFileTextsAsString(ctx context.Context, urls []string) string {
|
||||
if len(urls) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
client := createHTTPClient(ctx, "userFiles.httpTimeoutSec", 8)
|
||||
var builder strings.Builder
|
||||
|
||||
for _, rawURL := range urls {
|
||||
url := util.SanitizeURL(rawURL)
|
||||
if url == "" || util.IsBannedExtension(url) {
|
||||
continue
|
||||
}
|
||||
|
||||
if util.IsZipExtension(url) {
|
||||
for _, text := range fetchZipFileTexts(ctx, client, url) {
|
||||
builder.WriteString(text)
|
||||
builder.WriteString("\n")
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
if text := fetchAndCleanFileContent(ctx, client, url); text != "" {
|
||||
builder.WriteString(fmt.Sprintf("【文件:%s】\n%s\n", url, text))
|
||||
}
|
||||
}
|
||||
|
||||
return builder.String()
|
||||
}
|
||||
|
||||
// fetchAndCleanFileContent 获取并清理文件内容
|
||||
func fetchAndCleanFileContent(ctx context.Context, client *http.Client, url string) string {
|
||||
text, err := fetchFileContent(ctx, client, url)
|
||||
if err != nil || text == "" {
|
||||
return ""
|
||||
}
|
||||
return util.CleanSymbols(text)
|
||||
}
|
||||
|
||||
// fetchZipFileTexts 下载并解压 zip 文件,提取可读文本内容
|
||||
func fetchZipFileTexts(ctx context.Context, client *http.Client, url string) map[string]string {
|
||||
result := make(map[string]string)
|
||||
|
||||
maxSize := int64(g.Cfg().MustGet(ctx, "userFiles.zipMaxSizeMB", 10).Int()) * bytesPerMB
|
||||
zipBytes, err := downloadFile(client, url, maxSize)
|
||||
if err != nil {
|
||||
return result
|
||||
}
|
||||
|
||||
reader, err := zip.NewReader(bytes.NewReader(zipBytes), int64(len(zipBytes)))
|
||||
if err != nil {
|
||||
return result
|
||||
}
|
||||
|
||||
entryMaxSize := int64(g.Cfg().MustGet(ctx, "userFiles.zipEntryMaxSizeKB", 500).Int()) * bytesPerKB
|
||||
|
||||
for _, file := range reader.File {
|
||||
if shouldSkipZipEntry(file.Name) {
|
||||
continue
|
||||
}
|
||||
|
||||
if text := extractZipEntryContent(file, entryMaxSize); text != "" {
|
||||
result[url+"::"+file.Name] = text
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// shouldSkipZipEntry 判断是否应该跳过 zip 条目
|
||||
func shouldSkipZipEntry(fileName string) bool {
|
||||
return util.IsBannedExtension(fileName) || util.IsZipExtension(fileName)
|
||||
}
|
||||
|
||||
// extractZipEntryContent 提取 zip 条目内容
|
||||
func extractZipEntryContent(file *zip.File, maxSize int64) string {
|
||||
rc, err := file.Open()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer rc.Close()
|
||||
|
||||
content, err := io.ReadAll(io.LimitReader(rc, maxSize))
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
if !util.IsReadableContentType(http.DetectContentType(content)) {
|
||||
return ""
|
||||
}
|
||||
|
||||
text := util.CleanSymbols(string(content))
|
||||
if text == "" {
|
||||
return ""
|
||||
}
|
||||
|
||||
return text
|
||||
}
|
||||
|
||||
// downloadFile 下载文件,限制最大大小
|
||||
func downloadFile(client *http.Client, url string, maxSize int64) ([]byte, error) {
|
||||
req, err := http.NewRequest(http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建请求失败: %w", err)
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("执行请求失败: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return nil, fmt.Errorf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("读取响应失败: %w", err)
|
||||
}
|
||||
|
||||
return body, nil
|
||||
}
|
||||
|
||||
// fetchFileContent 获取单个文本文件内容
|
||||
func fetchFileContent(ctx context.Context, client *http.Client, url string) (string, error) {
|
||||
req, err := http.NewRequest(http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("创建请求失败: %w", err)
|
||||
}
|
||||
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("执行请求失败: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
return "", fmt.Errorf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
contentType := resp.Header.Get("Content-Type")
|
||||
if !util.IsReadableContentType(contentType) {
|
||||
return "", fmt.Errorf("不可读的内容类型: %s", contentType)
|
||||
}
|
||||
|
||||
maxSize := int64(g.Cfg().MustGet(ctx, "userFiles.textFileMaxSizeKB", 500).Int()) * bytesPerKB
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("读取响应失败: %w", err)
|
||||
}
|
||||
|
||||
return strings.TrimSpace(string(body)), nil
|
||||
}
|
||||
|
||||
func SkillMdContent(ctx context.Context, skillName string) string {
|
||||
if skillName == "" {
|
||||
return ""
|
||||
}
|
||||
skillResp, err := gateway.GetSkillUser(ctx, skillName)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[SkillMd] GetSkillUser 失败: %v", err)
|
||||
return ""
|
||||
}
|
||||
|
||||
fullUrl := skillResp.ImgAddressPrefix + skillResp.FileUrl
|
||||
|
||||
client := createHTTPClient(ctx, "skillFiles.httpTimeoutSec", 30)
|
||||
maxSize := int64(g.Cfg().MustGet(ctx, "skillFiles.zipMaxSizeMB", 10).Int()) * bytesPerMB
|
||||
|
||||
zipBytes, err := downloadFile(client, fullUrl, maxSize)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[SkillMd] 下载失败 url=%s err=%v", fullUrl, err)
|
||||
return ""
|
||||
}
|
||||
|
||||
mdContents, err := extractMdFiles(ctx, zipBytes)
|
||||
if err != nil || len(mdContents) == 0 {
|
||||
g.Log().Warningf(ctx, "[SkillMd] 提取md失败 count=%d err=%v", len(mdContents), err)
|
||||
return ""
|
||||
}
|
||||
|
||||
return buildSkillMarkdown(skillResp, mdContents)
|
||||
}
|
||||
|
||||
// buildSkillMarkdown 构建技能 Markdown 内容
|
||||
func buildSkillMarkdown(skillResp *gateway.SkillUserVO, mdContents map[string]string) string {
|
||||
var builder strings.Builder
|
||||
|
||||
builder.WriteString(fmt.Sprintf("# Skill: %s\n\n", skillResp.Name))
|
||||
if skillResp.Description != "" {
|
||||
builder.WriteString(fmt.Sprintf("> %s\n\n", skillResp.Description))
|
||||
}
|
||||
|
||||
for fileName, content := range mdContents {
|
||||
builder.WriteString(fmt.Sprintf("## %s\n\n", fileName))
|
||||
builder.WriteString(content)
|
||||
builder.WriteString("\n\n---\n\n")
|
||||
}
|
||||
|
||||
return strings.TrimSpace(builder.String())
|
||||
}
|
||||
|
||||
// extractMdFiles 解压 zip 并提取所有 .md 文件内容
|
||||
func extractMdFiles(ctx context.Context, zipBytes []byte) (map[string]string, error) {
|
||||
result := make(map[string]string)
|
||||
|
||||
reader, err := zip.NewReader(bytes.NewReader(zipBytes), int64(len(zipBytes)))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("创建 zip 阅读器失败: %w", err)
|
||||
}
|
||||
|
||||
entryMaxSize := int64(g.Cfg().MustGet(ctx, "skillFiles.mdMaxSizeKB", 500).Int()) * bytesPerKB
|
||||
|
||||
for _, file := range reader.File {
|
||||
if file.FileInfo().IsDir() || !isMarkdownFile(file.Name) {
|
||||
continue
|
||||
}
|
||||
|
||||
if content := readMarkdownFileContent(file, entryMaxSize); content != "" {
|
||||
result[file.Name] = content
|
||||
}
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// isMarkdownFile 判断是否为 Markdown 文件
|
||||
func isMarkdownFile(fileName string) bool {
|
||||
return strings.HasSuffix(strings.ToLower(fileName), ".md")
|
||||
}
|
||||
|
||||
// readMarkdownFileContent 读取 Markdown 文件内容
|
||||
func readMarkdownFileContent(file *zip.File, maxSize int64) string {
|
||||
rc, err := file.Open()
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
defer rc.Close()
|
||||
|
||||
content, err := io.ReadAll(io.LimitReader(rc, maxSize))
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
if len(content) == 0 {
|
||||
return ""
|
||||
}
|
||||
|
||||
return strings.TrimSpace(string(content))
|
||||
}
|
||||
|
||||
// createHTTPClient 创建 HTTP 客户端
|
||||
func createHTTPClient(ctx context.Context, configKey string, defaultSeconds int) *http.Client {
|
||||
timeout := time.Duration(g.Cfg().MustGet(ctx, configKey, defaultSeconds).Int()) * time.Second
|
||||
return &http.Client{
|
||||
Timeout: timeout,
|
||||
}
|
||||
}
|
||||
+333
-58
@@ -6,7 +6,9 @@ import (
|
||||
"fmt"
|
||||
"model-gateway/common/util"
|
||||
"model-gateway/consts/public"
|
||||
"model-gateway/service/queue"
|
||||
"model-gateway/service/gateway"
|
||||
"model-gateway/service/prompt"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"model-gateway/dao"
|
||||
@@ -16,7 +18,9 @@ import (
|
||||
"gitea.redpowerfuture.com/red-future/common/beans"
|
||||
"gitea.redpowerfuture.com/red-future/common/utils"
|
||||
"github.com/gogf/gf/v2/database/gdb"
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
"github.com/gogf/gf/v2/frame/g"
|
||||
"github.com/gogf/gf/v2/os/grpool"
|
||||
"github.com/gogf/gf/v2/util/gconv"
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
@@ -25,15 +29,190 @@ var ModelGatewayTask = &taskService{}
|
||||
|
||||
type taskService struct{}
|
||||
|
||||
// BuildMessages 构建消息(异步)
|
||||
func (s *taskService) BuildMessages(ctx context.Context, req *dto.BuildMessagesReq) (*dto.BuildMessagesRes, error) {
|
||||
user, err := utils.GetUserInfo(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
model, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{TenantId: user.TenantId, Creator: user.UserName},
|
||||
ModelName: req.ModelName,
|
||||
})
|
||||
if err != nil || model == nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 1) 创建构建记录
|
||||
taskId := uuid.NewString()
|
||||
record := &entity.ModelGatewayBuildRecord{
|
||||
TaskID: taskId,
|
||||
BuildType: req.BuildType,
|
||||
ModelName: req.ModelName,
|
||||
SkillName: req.SkillName,
|
||||
SessionID: req.SessionId,
|
||||
NodeID: req.NodeId,
|
||||
RequestMessages: req.Messages,
|
||||
CallbackURL: req.CallbackUrl,
|
||||
Status: 0,
|
||||
}
|
||||
_, err = dao.ModelGatewayBuildRecord.Insert(ctx, record)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2) 异步执行构建
|
||||
go s.executeBuild(util.AsyncCtx(ctx), record, req, model)
|
||||
|
||||
return &dto.BuildMessagesRes{TaskId: taskId}, nil
|
||||
}
|
||||
|
||||
// executeBuild 异步执行构建逻辑
|
||||
func (s *taskService) executeBuild(ctx context.Context, record *entity.ModelGatewayBuildRecord, req *dto.BuildMessagesReq, model *entity.ModelGatewayModel) {
|
||||
var (
|
||||
startTime = time.Now()
|
||||
result []map[string]any
|
||||
err error
|
||||
)
|
||||
result, err = s.buildResult(ctx, req, model, record)
|
||||
record.DurationSeconds = int(time.Since(startTime).Seconds())
|
||||
if err != nil {
|
||||
record.Status = 2
|
||||
record.ErrorMsg = err.Error()
|
||||
} else {
|
||||
record.Status = 1
|
||||
record.ResultMessages = result
|
||||
}
|
||||
_, _ = dao.ModelGatewayBuildRecord.Update(ctx, record)
|
||||
gateway.CallbackBuildResult(ctx, record)
|
||||
}
|
||||
|
||||
// buildResult 构建结果:推理模型手动拼接,视频模型调模型生成多轮
|
||||
func (s *taskService) buildResult(ctx context.Context, req *dto.BuildMessagesReq, model *entity.ModelGatewayModel, record *entity.ModelGatewayBuildRecord) ([]map[string]any, error) {
|
||||
messages := req.Messages
|
||||
switch {
|
||||
case model.ModelType == public.ModelTypeInference:
|
||||
// 推理模型:拼接提示词 + 历史
|
||||
systemPrompt := util.GetModelPrompt(ctx, model.ModelType)
|
||||
skillContent := prompt.SkillMdContent(ctx, req.SkillName)
|
||||
|
||||
systemKey := util.GetRoleContentPath(model.Form, "system")
|
||||
if systemKey != "" {
|
||||
messages = util.MergePrompt(messages, systemKey, systemPrompt, skillContent, req.CustomPrompt)
|
||||
}
|
||||
|
||||
history, _ := gateway.GetSessionHistory(ctx, req.NodeId, req.SessionId)
|
||||
if len(history) > 0 {
|
||||
messages = util.InjectHistory(messages, history, []string{"system", "history", "user"})
|
||||
}
|
||||
// 检查附件是否需要拆分多轮
|
||||
rounds := util.SplitByAttachment(messages, model.Form)
|
||||
if len(rounds) > 0 {
|
||||
return rounds, nil
|
||||
}
|
||||
return []map[string]any{messages}, nil
|
||||
case model.ModelType >= 600 && model.ModelType < 700:
|
||||
// 视频模型:调推理模型生成多轮
|
||||
chatModel, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{TenantId: model.TenantId, Creator: model.Creator},
|
||||
IsChatModel: gconv.PtrInt(1),
|
||||
})
|
||||
if err != nil || chatModel == nil {
|
||||
return nil, fmt.Errorf("未找到对话模型")
|
||||
}
|
||||
|
||||
protocol, err := dao.ProviderProtocol.Get(ctx, &entity.ProviderProtocol{
|
||||
ProviderName: chatModel.OperatorName,
|
||||
Status: 1,
|
||||
})
|
||||
if err != nil || protocol == nil {
|
||||
return nil, fmt.Errorf("未找到协议配置: %s", chatModel.OperatorName)
|
||||
}
|
||||
|
||||
template := util.BuildTemplateFromForm(model.Form)
|
||||
outputStruct := gjson.New(template).MustToJsonString()
|
||||
|
||||
durationForm, _ := util.GetFormByRole(model.Form, "duration")
|
||||
totalDur := gconv.Int(gjson.New(req.Messages).Get(durationForm.Key).Val())
|
||||
minDur := gconv.Int(durationForm.FieldConstraint.Min)
|
||||
maxDur := gconv.Int(durationForm.FieldConstraint.Max)
|
||||
|
||||
systemPrompt := fmt.Sprintf(protocol.SystemPromptTemplate, outputStruct, totalDur, minDur, maxDur)
|
||||
userContent := util.ExtractUserContent(req.Messages)
|
||||
reqBody := util.BuildRequestBody(protocol.RequestTemplate, chatModel.ModelName, systemPrompt, userContent)
|
||||
|
||||
task := &entity.ModelGatewayTask{
|
||||
ModelName: chatModel.ModelName,
|
||||
TaskID: record.TaskID,
|
||||
State: public.TaskStatusRunning,
|
||||
BizName: "model-gateway",
|
||||
RequestPayload: reqBody,
|
||||
}
|
||||
id, err := dao.ModelGatewayTask.Insert(ctx, task)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
task.Id = id
|
||||
|
||||
rawData, err := InvokeModel(ctx, chatModel, reqBody)
|
||||
if err != nil {
|
||||
task.State = public.TaskStatusFailed
|
||||
task.ErrorMsg = err.Error()
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
mapped, err := util.MapResponsePayload(chatModel.ResponseMapping, gjson.New(string(rawData)).Map())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var rounds []map[string]any
|
||||
contentStr := gjson.New(mapped).Get(entity.ResponseBody).String()
|
||||
if contentStr != "" {
|
||||
if err = gjson.DecodeTo(contentStr, &rounds); err != nil {
|
||||
task.State = public.TaskStatusFailed
|
||||
task.ErrorMsg = err.Error()
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
oss, err := gateway.UploadByTask(ctx, gjson.New(rounds).MustToJson(), "json")
|
||||
if err != nil {
|
||||
task.State = public.TaskStatusFailed
|
||||
task.ErrorMsg = err.Error()
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
task.State = public.TaskStatusSuccess
|
||||
task.ResultFile = &entity.ResultFile{
|
||||
OssFile: oss.FileAddressPrefix + oss.FileURL,
|
||||
FileType: oss.FileFormat,
|
||||
FileSize: int64(oss.FileSize),
|
||||
}
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
|
||||
return rounds, nil
|
||||
|
||||
default:
|
||||
return nil, errors.New("不支持的模型类型")
|
||||
}
|
||||
}
|
||||
|
||||
// Create 创建任务
|
||||
func (s *taskService) Create(ctx context.Context, req *dto.CreateTaskReq) (res *dto.CreateTaskRes, err error) {
|
||||
taskID := uuid.NewString()
|
||||
startAt := time.Now()
|
||||
|
||||
// 1) 检查模型配置,并且获取模型
|
||||
// 1) 获取用户信息
|
||||
userInfo, err := utils.GetUserInfo(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 2) 检查模型配置
|
||||
model, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{
|
||||
TenantId: userInfo.TenantId,
|
||||
@@ -48,77 +227,174 @@ func (s *taskService) Create(ctx context.Context, req *dto.CreateTaskReq) (res *
|
||||
return nil, errors.New("模型不存在或未启用")
|
||||
}
|
||||
|
||||
// 2) 排队上限(严格控制:Redis 原子闸门)
|
||||
limit := queue.GetRuntimeQueueLimit(ctx, req.ModelName, model.MaxConcurrency*2)
|
||||
if limit > 0 {
|
||||
ok, err := queue.AcquireQueueSlot(ctx, req.ModelName, taskID, limit, model.TimeoutSeconds)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !ok {
|
||||
return nil, errors.New("任务排队已满,请稍后再试")
|
||||
}
|
||||
}
|
||||
// TODO: 排队控制暂时关闭,后续需要时取消注释
|
||||
// limit := queue.GetRuntimeQueueLimit(ctx, req.ModelName, model.MaxConcurrency*2)
|
||||
// if limit > 0 {
|
||||
// ok, err := queue.AcquireQueueSlot(ctx, req.ModelName, taskID, limit, model.TimeoutSeconds)
|
||||
// if err != nil {
|
||||
// return nil, err
|
||||
// }
|
||||
// if !ok {
|
||||
// return nil, errors.New("任务排队已满,请稍后再试")
|
||||
// }
|
||||
// }
|
||||
|
||||
// 3) 插入任务记录
|
||||
requestPayload := entity.RequestPayload{
|
||||
Body: req.RequestPayload,
|
||||
Headers: util.ParseHeadMsgHeaders(model.HeadMsg),
|
||||
}
|
||||
id, err := dao.ModelGatewayTask.Insert(ctx, &entity.ModelGatewayTask{
|
||||
ModelName: req.ModelName,
|
||||
// 3) 构建任务实体
|
||||
task := &entity.ModelGatewayTask{
|
||||
ModelName: model.ModelName,
|
||||
TaskID: taskID,
|
||||
State: public.TaskStatusPending,
|
||||
State: public.TaskStatusRunning,
|
||||
BizName: req.BizName,
|
||||
CallbackURL: req.CallbackUrl,
|
||||
RequestPayload: &requestPayload,
|
||||
RequestPayload: req.RequestPayload,
|
||||
EpicycleId: req.EpicycleId,
|
||||
})
|
||||
if err != nil { // 入库失败:回滚闸门占位
|
||||
queue.ReleaseQueueSlot(ctx, req.ModelName, taskID)
|
||||
return nil, err
|
||||
BuildModelName: req.BuildModelName,
|
||||
}
|
||||
|
||||
// 4) 写操作日志(不影响主流程,失败忽略)
|
||||
ip := ""
|
||||
ua := ""
|
||||
apiPath := "/task/createTask"
|
||||
httpMethod := "POST"
|
||||
// 4) 插入任务记录
|
||||
id, err := dao.ModelGatewayTask.Insert(ctx, task)
|
||||
if err != nil {
|
||||
// TODO: 恢复排队逻辑后,此处需要回滚排队占位
|
||||
// queue.ReleaseQueueSlot(ctx, req.ModelName, taskID)
|
||||
return nil, err
|
||||
}
|
||||
task.Id = id
|
||||
|
||||
// 5) 记录操作日志(非关键路径,失败不影响主流程)
|
||||
ip, ua := "", ""
|
||||
if r := g.RequestFromCtx(ctx); r != nil {
|
||||
ip = utils.GetLocalIP()
|
||||
ua = r.UserAgent()
|
||||
apiPath = r.URL.Path
|
||||
httpMethod = r.Method
|
||||
}
|
||||
_, _ = dao.ModelGatewayLogsOp.Insert(ctx, &entity.ModelGatewayLogsOp{
|
||||
IP: ip,
|
||||
UserAgent: ua,
|
||||
APIPath: apiPath,
|
||||
HttpMethod: httpMethod,
|
||||
BizName: req.BizName,
|
||||
ModelName: req.ModelName,
|
||||
TaskID: taskID,
|
||||
OpType: "createTask",
|
||||
Success: 1,
|
||||
CostMs: time.Since(time.Now()).Milliseconds(),
|
||||
RequestPayload: &requestPayload,
|
||||
ResponsePayload: gdb.Map{
|
||||
"taskId": taskID,
|
||||
},
|
||||
IP: ip,
|
||||
UserAgent: ua,
|
||||
APIPath: "/task/createTask",
|
||||
HttpMethod: "POST",
|
||||
BizName: req.BizName,
|
||||
ModelName: req.ModelName,
|
||||
TaskID: taskID,
|
||||
OpType: "createTask",
|
||||
Success: 1,
|
||||
CostMs: time.Since(startAt).Milliseconds(),
|
||||
RequestPayload: task.RequestPayload,
|
||||
ResponsePayload: gdb.Map{"taskId": taskID},
|
||||
})
|
||||
|
||||
// 5) 获取任务信息
|
||||
task, err := dao.ModelGatewayTask.ClaimByID(ctx, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
// 6) 模型计费
|
||||
if len(model.BillingConfig) > 0 {
|
||||
requestData := util.ExtractRequestBilling(ctx, model.BillingConfig, req.RequestPayload)
|
||||
// 请求数据作为计费记录的基础字段,先存入数组
|
||||
task.BillingData = append(task.BillingData, requestData)
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, &entity.ModelGatewayTask{
|
||||
SQLBaseDO: beans.SQLBaseDO{Id: task.Id},
|
||||
BillingData: task.BillingData,
|
||||
})
|
||||
}
|
||||
|
||||
// 5) 创建成功后立即异步尝试执行当前任务
|
||||
go AsyncWorker.handleOne(util.AsyncCtx(ctx), task, model, req)
|
||||
// 7) 异步执行任务
|
||||
go AsyncWorker.handleOne(util.AsyncCtx(ctx), task, model)
|
||||
|
||||
return &dto.CreateTaskRes{TaskID: taskID}, nil
|
||||
}
|
||||
|
||||
var JobPool *grpool.Pool
|
||||
|
||||
// JobTask 定时任务:循环执行待处理任务
|
||||
func (s *taskService) JobTask(ctx context.Context, req *dto.JobTaskReq) (res *dto.JobTaskRes, err error) {
|
||||
// 1) 参数默认值从配置取
|
||||
if req.Interval <= 0 {
|
||||
req.Interval = g.Cfg().MustGet(ctx, "jobTask.intervalSeconds", 5).Int()
|
||||
}
|
||||
if req.BatchSize <= 0 {
|
||||
req.BatchSize = g.Cfg().MustGet(ctx, "jobTask.batchSize", 10).Int()
|
||||
}
|
||||
|
||||
var (
|
||||
totalProcessed int
|
||||
successCount int
|
||||
failCount int
|
||||
mu sync.Mutex
|
||||
wg sync.WaitGroup
|
||||
)
|
||||
|
||||
// 2) 循环查询待处理任务
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
wg.Wait()
|
||||
return &dto.JobTaskRes{
|
||||
TotalProcessed: totalProcessed,
|
||||
SuccessCount: successCount,
|
||||
FailCount: failCount,
|
||||
}, nil
|
||||
default:
|
||||
}
|
||||
|
||||
// 3) 查询 state=0 的任务列表
|
||||
tasks, err := dao.ModelGatewayTask.ListPending(ctx, req.BatchSize)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[定时任务] 查询任务失败: %v", err)
|
||||
time.Sleep(time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
if len(tasks) == 0 {
|
||||
time.Sleep(time.Duration(req.Interval) * time.Second)
|
||||
continue
|
||||
}
|
||||
|
||||
// 4) 提交到全局协程池执行
|
||||
for _, task := range tasks {
|
||||
wg.Add(1)
|
||||
t := task
|
||||
err = JobPool.Add(ctx, func(ctx context.Context) {
|
||||
defer wg.Done()
|
||||
|
||||
mu.Lock()
|
||||
totalProcessed++
|
||||
mu.Unlock()
|
||||
|
||||
if execErr := s.executeTask(ctx, t); execErr != nil {
|
||||
mu.Lock()
|
||||
failCount++
|
||||
mu.Unlock()
|
||||
g.Log().Errorf(ctx, "[定时任务] 执行失败 taskId=%s err=%v", t.TaskID, execErr)
|
||||
} else {
|
||||
mu.Lock()
|
||||
successCount++
|
||||
mu.Unlock()
|
||||
}
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// executeTask 执行单个任务
|
||||
func (s *taskService) executeTask(ctx context.Context, task *entity.ModelGatewayTask) error {
|
||||
// 1) 查询模型配置
|
||||
model, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{
|
||||
TenantId: task.TenantId,
|
||||
Creator: task.Creator,
|
||||
},
|
||||
ModelName: task.ModelName,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("查询模型配置失败: %w", err)
|
||||
}
|
||||
if model == nil || (model.Enabled != nil && *model.Enabled != 1) {
|
||||
return fmt.Errorf("模型不存在或未启用: %s", task.ModelName)
|
||||
}
|
||||
|
||||
// 3) 调用 handleOne
|
||||
AsyncWorker.handleOne(util.AsyncCtx(ctx), task, model)
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetResult 获取任务结果
|
||||
func (s *taskService) GetResult(ctx context.Context, taskID string) (res *dto.GetTaskResultRes, err error) {
|
||||
t, err := dao.ModelGatewayTask.Get(ctx, &entity.ModelGatewayTask{
|
||||
@@ -152,7 +428,7 @@ func (s *taskService) GetBatch(ctx context.Context, req *dto.GetTaskBatchReq) (r
|
||||
if t == nil {
|
||||
continue
|
||||
}
|
||||
if t.State != public.BuildTypeNode {
|
||||
if t.State != 2 {
|
||||
continue
|
||||
}
|
||||
_ = dao.ModelGatewayTask.MarkDownloadedByID(ctx, t.Id)
|
||||
@@ -168,10 +444,9 @@ func (s *taskService) GetBatch(ctx context.Context, req *dto.GetTaskBatchReq) (r
|
||||
continue
|
||||
}
|
||||
items = append(items, dto.GetTaskBatchItem{
|
||||
TaskID: t.TaskID,
|
||||
State: t.State,
|
||||
OssFile: t.ResultFile.OssFile,
|
||||
TextResult: t.TextResult,
|
||||
TaskID: t.TaskID,
|
||||
State: t.State,
|
||||
OssFile: t.ResultFile.OssFile,
|
||||
})
|
||||
}
|
||||
return &dto.GetTaskBatchRes{List: items}, nil
|
||||
|
||||
+238
-209
@@ -6,20 +6,15 @@ import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"model-gateway/common/util"
|
||||
"model-gateway/consts/public"
|
||||
"model-gateway/dao"
|
||||
"model-gateway/model/dto"
|
||||
"model-gateway/model/entity"
|
||||
"model-gateway/service/gateway"
|
||||
"model-gateway/service/queue"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.redpowerfuture.com/red-future/common/beans"
|
||||
"github.com/gogf/gf/v2/encoding/gjson"
|
||||
@@ -33,78 +28,122 @@ type asyncWorker struct {
|
||||
}
|
||||
|
||||
// handleOne 执行一次完整的任务
|
||||
func (w *asyncWorker) handleOne(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, req *dto.CreateTaskReq) {
|
||||
func (w *asyncWorker) handleOne(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel) {
|
||||
var (
|
||||
body = task.RequestPayload.Body
|
||||
body = task.RequestPayload
|
||||
maxRetry = model.RetryTimes
|
||||
startTime = time.Now()
|
||||
rawData []byte
|
||||
result map[string]any
|
||||
err error
|
||||
)
|
||||
g.Log().Infof(ctx, "[执行任务][开始] taskId=%s model=%s", task.TaskID, task.ModelName)
|
||||
|
||||
g.Log().Infof(ctx, "[handleOne] 开始 taskId=%s model=%s", task.TaskID, task.ModelName)
|
||||
|
||||
// ============================================
|
||||
// 1) 分布式并发控制
|
||||
// 1) 查询余额
|
||||
// ============================================
|
||||
semKey := fmt.Sprintf("asynch:sem:%s", task.ModelName)
|
||||
maxC := queue.GetRuntimeMaxConcurrency(ctx, task.ModelName, model.MaxConcurrency)
|
||||
acquired, err := queue.AcquireSemaphore(ctx, semKey, maxC, 3600)
|
||||
if err != nil {
|
||||
task.DurationSeconds = int64(time.Since(startTime).Seconds())
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
surplus, _ := gateway.GetTenantSurplus(ctx, model.TenantId)
|
||||
if surplus <= 0 {
|
||||
w.failTask(ctx, task, startTime, "租户余额不足")
|
||||
return
|
||||
}
|
||||
if !acquired {
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, &entity.ModelGatewayTask{
|
||||
SQLBaseDO: beans.SQLBaseDO{Id: task.Id},
|
||||
State: public.TaskStatusPending,
|
||||
})
|
||||
g.Log().Infof(ctx, "[执行任务][排队] 并发已满,放回队列 taskId=%s", task.TaskID)
|
||||
return
|
||||
}
|
||||
defer func() { _ = queue.ReleaseSemaphore(ctx, semKey) }()
|
||||
g.Log().Infof(ctx, "[handleOne] 当前余额 tenantId=%d surplus=%.2f", task.TenantId, surplus)
|
||||
|
||||
// ============================================
|
||||
// 2) 调用模型
|
||||
// ============================================
|
||||
switch {
|
||||
case model.CallMode != nil && *model.CallMode == public.CallModeStream:
|
||||
rawBytes, streamErr := w.callModelStream(ctx, task, model, body)
|
||||
if streamErr != nil {
|
||||
w.failTask(ctx, task, startTime, streamErr.Error())
|
||||
for attempt := 0; ; attempt++ {
|
||||
if attempt > 0 {
|
||||
g.Log().Infof(ctx, "[handleOne] 调模型重试 第%d次 taskId=%s", attempt, task.TaskID)
|
||||
time.Sleep(time.Duration(attempt) * 5 * time.Second)
|
||||
}
|
||||
|
||||
rawData, err = InvokeModel(ctx, model, body)
|
||||
switch {
|
||||
case model.CallMode != nil && *model.CallMode == public.CallModeStream: // 流式
|
||||
if err == nil {
|
||||
result, err = util.ParseStreamResponse(ctx, rawData, model.StreamConfig)
|
||||
}
|
||||
case model.CallMode != nil && *model.CallMode == public.CallModeAsync: // 异步
|
||||
if err == nil {
|
||||
result = gjson.New(string(rawData)).Map()
|
||||
result, err = util.PullTaskResult(ctx, result, model.QueryConfig, model.HeadMsg)
|
||||
}
|
||||
default:
|
||||
if err == nil {
|
||||
result = gjson.New(string(rawData)).Map()
|
||||
}
|
||||
}
|
||||
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
// 模型调用失败
|
||||
if !strings.Contains(err.Error(), "Timeout") &&
|
||||
!strings.Contains(err.Error(), "InternalServiceError") &&
|
||||
!strings.Contains(err.Error(), "Invalid video_url") &&
|
||||
!strings.Contains(err.Error(), "Invalid audio track") &&
|
||||
!strings.Contains(err.Error(), "Error while downloading") &&
|
||||
!strings.Contains(err.Error(), "Error while connecting") &&
|
||||
!strings.Contains(err.Error(), "download failed") {
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
return
|
||||
}
|
||||
result, err = util.ParseStreamResponse(rawBytes, model.StreamConfig)
|
||||
case model.CallMode != nil && *model.CallMode == public.CallModeAsync:
|
||||
result, err = w.callModel(ctx, task, model, body)
|
||||
if err == nil {
|
||||
result, err = util.PullTaskResult(ctx, result, model.QueryConfig, model.HeadMsg)
|
||||
|
||||
g.Log().Warningf(ctx, "[handleOne] 调模型失败 taskId=%s attempt=%d err=%v", task.TaskID, attempt, err)
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// 3) 解析返回映射 + 存储 token 相关信息
|
||||
// ============================================
|
||||
mapped, err := util.MapResponsePayload(model.ResponseMapping, result)
|
||||
if err != nil {
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
return
|
||||
}
|
||||
|
||||
// 计费处理
|
||||
if len(model.BillingConfig) > 0 && len(task.BillingData) > 0 {
|
||||
// 取请求阶段数据作为基础
|
||||
billingInput := make(map[string]any)
|
||||
for k, v := range task.BillingData[0] {
|
||||
billingInput[k] = v
|
||||
}
|
||||
// 补充返回数据
|
||||
responseData := util.ExtractResponseBilling(model.BillingConfig, mapped)
|
||||
for k, v := range responseData {
|
||||
billingInput[k] = v
|
||||
}
|
||||
// 计算费用,替换数组第一个元素
|
||||
billingResult := util.CalculateBilling(model.BillingConfig, billingInput)
|
||||
if billingResult != nil {
|
||||
task.BillingData[0] = billingResult
|
||||
}
|
||||
|
||||
if billingResult != nil {
|
||||
task.BillingData[0] = billingResult
|
||||
totalFee := gconv.Float64(billingResult["total_fee"])
|
||||
if totalFee > 0 {
|
||||
_ = gateway.DeductBalance(util.AsyncCtx(ctx), task.TenantId, -totalFee)
|
||||
}
|
||||
}
|
||||
default:
|
||||
result, err = w.callModel(ctx, task, model, body)
|
||||
}
|
||||
if err != nil {
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
|
||||
if _, err = dao.ModelGatewayTask.Update(ctx, task); err != nil {
|
||||
g.Log().Errorf(ctx, "[handleOne] 更新DB失败 taskId=%s err=%v", task.TaskID, err)
|
||||
return
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// 3) 缓存临时文件
|
||||
// 4) 处理提示词相关数据解析涵盖重试
|
||||
// ============================================
|
||||
if tmpPath, tmpErr := util.SaveTempFileByType(task.TaskID, result, task.TmpFile); tmpErr == nil && tmpPath != "" {
|
||||
task.TmpFile = tmpPath
|
||||
task.Phase = 1
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
}
|
||||
|
||||
// ============================================
|
||||
// 4) 解析校验 + 响应映射(可重试)
|
||||
// ============================================
|
||||
result, err = w.parseAndRetry(ctx, result, task, model, req, maxRetry, startTime)
|
||||
if err != nil {
|
||||
task.TextResult = result
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
return
|
||||
if task.BizName == "prompts-core" {
|
||||
mapped, err = w.parseAndRetry(ctx, mapped, model, task, maxRetry)
|
||||
if err != nil {
|
||||
w.failTask(ctx, task, startTime, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// ============================================
|
||||
@@ -113,18 +152,14 @@ func (w *asyncWorker) handleOne(ctx context.Context, task *entity.ModelGatewayTa
|
||||
var oss *gateway.UploadFileResponse
|
||||
for attempt := 0; attempt <= maxRetry; attempt++ {
|
||||
if attempt > 0 {
|
||||
g.Log().Infof(ctx, "[执行任务][重试] OSS上传 第%d/%d次 taskId=%s", attempt, maxRetry, task.TaskID)
|
||||
g.Log().Infof(ctx, "[handleOne] OSS上传重试 第%d/%d次 taskId=%s", attempt, maxRetry, task.TaskID)
|
||||
}
|
||||
oss, err = gateway.UploadByTask(ctx, gjson.New(result).MustToJson(), "json")
|
||||
oss, err = gateway.UploadByTask(ctx, gjson.New(mapped).MustToJson(), "json")
|
||||
if err == nil {
|
||||
break
|
||||
}
|
||||
g.Log().Errorf(ctx, "[执行任务][失败] OSS上传失败 taskId=%s attempt=%d/%d err=%v", task.TaskID, attempt, maxRetry, err)
|
||||
g.Log().Errorf(ctx, "[handleOne] OSS上传失败 taskId=%s attempt=%d/%d err=%v", task.TaskID, attempt, maxRetry, err)
|
||||
if attempt == maxRetry {
|
||||
task.State = public.TaskStatusFailed
|
||||
task.ErrorMsg = err.Error()
|
||||
task.Phase = 1
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
w.failTask(ctx, task, startTime, fmt.Sprintf("OSS上传重试耗尽: %v", err))
|
||||
return
|
||||
}
|
||||
@@ -140,53 +175,18 @@ func (w *asyncWorker) handleOne(ctx context.Context, task *entity.ModelGatewayTa
|
||||
FileType: oss.FileFormat,
|
||||
FileSize: int64(oss.FileSize),
|
||||
}
|
||||
task.TextResult = result
|
||||
if _, err = dao.ModelGatewayTask.Update(ctx, task); err != nil {
|
||||
g.Log().Errorf(ctx, "[执行任务][失败] 更新数据库失败 taskId=%s err=%v", task.TaskID, err)
|
||||
g.Log().Errorf(ctx, "[handleOne] 更新DB失败 taskId=%s err=%v", task.TaskID, err)
|
||||
return
|
||||
}
|
||||
|
||||
queue.ReleaseQueueSlot(ctx, task.ModelName, task.TaskID)
|
||||
go gateway.TriggerCallback(context.WithoutCancel(ctx), task)
|
||||
if req.EpicycleId != 0 {
|
||||
go gateway.TriggerPromptsCallback(context.WithoutCancel(ctx), task, req.EpicycleId)
|
||||
go gateway.TriggerCallback(util.AsyncCtx(ctx), task)
|
||||
if task.EpicycleId != 0 {
|
||||
go gateway.TriggerPromptsCallback(util.AsyncCtx(ctx), task)
|
||||
}
|
||||
|
||||
g.Log().Infof(ctx, "[执行任务][成功] taskId=%s duration=%ds fileType=%s",
|
||||
g.Log().Infof(ctx, "[handleOne] 成功 taskId=%s duration=%ds fileType=%s",
|
||||
task.TaskID, task.DurationSeconds, oss.FileFormat)
|
||||
|
||||
_ = os.Remove(task.TmpFile)
|
||||
}
|
||||
|
||||
// callModelStream 调用模型,返回原始字节(不做响应映射,用于流式输出)
|
||||
func (w *asyncWorker) callModelStream(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, body map[string]any) ([]byte, error) {
|
||||
var data []byte
|
||||
var err error
|
||||
|
||||
if task.Phase == 1 && strings.TrimSpace(task.TmpFile) != "" {
|
||||
data, err = os.ReadFile(task.TmpFile)
|
||||
if err != nil || len(data) == 0 {
|
||||
data = nil
|
||||
}
|
||||
}
|
||||
|
||||
if data == nil {
|
||||
data, err = InvokeModel(ctx, model, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tmpPath, tmpErr := util.SaveTmpResult(task.TaskID, data, "")
|
||||
if tmpErr == nil && tmpPath != "" {
|
||||
task.TmpFile = tmpPath
|
||||
task.Phase = 1
|
||||
_, err = dao.ModelGatewayTask.Update(ctx, task)
|
||||
if err != nil {
|
||||
g.Log().Errorf(ctx, "[执行任务][失败] 临时文件保存失败 taskId=%s err=%v", task.TaskID, tmpErr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return data, nil
|
||||
}
|
||||
|
||||
// asyncResult 异步任务结果
|
||||
@@ -200,12 +200,13 @@ var asyncTaskChan = sync.Map{} // taskID → chan asyncResult
|
||||
|
||||
func (w *asyncWorker) callModelAsync(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, body map[string]any) (map[string]any, error) {
|
||||
// 1. 提交异步任务
|
||||
body, err := w.callModel(ctx, task, model, body)
|
||||
rawData, err := InvokeModel(ctx, model, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
body = gjson.New(string(rawData)).Map()
|
||||
// 2. 拿到 task_id
|
||||
taskID := gjson.New(body).Get(model.ResponseBody).String()
|
||||
taskID := gjson.New(body).Get(entity.ResponseBody).String()
|
||||
|
||||
// 3. 创建等待通道
|
||||
ch := make(chan asyncResult, 1)
|
||||
@@ -241,135 +242,157 @@ func NotifyAsyncResult(taskID string, result map[string]any, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
// callModel 调用模型 + 检测文件类型 + 保存临时文件
|
||||
// 返回: 解析后的响应体, error
|
||||
func (w *asyncWorker) callModel(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, body map[string]any) (map[string]any, error) {
|
||||
var data []byte
|
||||
var err error
|
||||
|
||||
// 1) 如果已有临时文件且 phase=1,直接读取
|
||||
if task.Phase == 1 && strings.TrimSpace(task.TmpFile) != "" {
|
||||
data, err = os.ReadFile(task.TmpFile)
|
||||
if err != nil || len(data) == 0 {
|
||||
g.Log().Warningf(ctx, "[callModel] 读取临时文件失败,重新调用模型 taskId=%s err=%v", task.TaskID, err)
|
||||
data = nil
|
||||
}
|
||||
}
|
||||
|
||||
// 2) 没有可用数据,调用模型
|
||||
if data == nil {
|
||||
data, err = InvokeModel(ctx, model, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// 3) 检测文件类型,保存临时文件
|
||||
_, ext := util.DetectFileType(data)
|
||||
tmpPath, tmpErr := util.SaveTmpResult(task.TaskID, data, ext)
|
||||
if tmpErr == nil && tmpPath != "" {
|
||||
task.TmpFile = tmpPath
|
||||
task.Phase = 1
|
||||
_, err = dao.ModelGatewayTask.Update(ctx, task)
|
||||
if err != nil {
|
||||
g.Log().Errorf(ctx, "[执行任务][失败] 临时文件保存失败 taskId=%s err=%v", task.TaskID, tmpErr)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 4) 检测文件类型,提取文本结果
|
||||
contentType, _ := util.DetectFileType(data)
|
||||
var textResult string
|
||||
if utf8.Valid(data) && (strings.HasPrefix(contentType, "text/") || contentType == "application/json") {
|
||||
textResult = string(data)
|
||||
}
|
||||
|
||||
// 5) 非文本内容,返回错误
|
||||
if textResult == "" {
|
||||
return nil, fmt.Errorf("模型返回非文本内容,contentType=%s", contentType)
|
||||
}
|
||||
|
||||
// 6) 解析并返回
|
||||
return gjson.New(textResult).Map(), nil
|
||||
}
|
||||
//// callModel 调用模型 + 提取文本结果
|
||||
//func (w *asyncWorker) callModel(ctx context.Context, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, body map[string]any) (map[string]any, error) {
|
||||
// data, err := InvokeModel(ctx, model, body)
|
||||
// if err != nil {
|
||||
// return nil, err
|
||||
// }
|
||||
// contentType, _ := util.DetectFileType(data)
|
||||
// var textResult string
|
||||
// if utf8.Valid(data) && (strings.HasPrefix(contentType, "text/") || contentType == "application/json") {
|
||||
// textResult = string(data)
|
||||
// }
|
||||
//
|
||||
// if textResult == "" {
|
||||
// return nil, fmt.Errorf("模型返回非文本内容,contentType=%s", contentType)
|
||||
// }
|
||||
//
|
||||
// return gjson.New(textResult).Map(), nil
|
||||
//}
|
||||
|
||||
// parseAndRetry 解析模型返回结果,并重试
|
||||
func (w *asyncWorker) parseAndRetry(ctx context.Context, body map[string]any, task *entity.ModelGatewayTask, model *entity.ModelGatewayModel, req *dto.CreateTaskReq, maxRetry int, startTime time.Time) (map[string]any, error) {
|
||||
func (w *asyncWorker) parseAndRetry(ctx context.Context, body map[string]any, model *entity.ModelGatewayModel, task *entity.ModelGatewayTask, maxRetry int) (map[string]any, error) {
|
||||
buildModel, err := dao.ModelGatewayModels.Get(ctx, &entity.ModelGatewayModel{
|
||||
SQLBaseDO: beans.SQLBaseDO{TenantId: model.TenantId, Creator: model.Creator},
|
||||
ModelName: task.BuildModelName,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var lastErr error
|
||||
for attempt := 0; attempt <= maxRetry; attempt++ {
|
||||
if attempt > 0 {
|
||||
g.Log().Infof(ctx, "[执行任务][重试] JSON解析 第%d/%d次 taskId=%s", attempt, maxRetry, task.TaskID)
|
||||
}
|
||||
|
||||
// 1) 响应映射
|
||||
mapped, err := util.MapResponsePayload(model.ResponseMapping, body)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[执行任务][映射失败] taskId=%s attempt=%d/%d err=%v", task.TaskID, attempt, maxRetry, err)
|
||||
if attempt == maxRetry {
|
||||
return nil, fmt.Errorf("响应映射重试耗尽: %w", err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
// 2) 先存 token 到数据库,防止后续失败丢失
|
||||
if _, ok := mapped[model.ResponseTokenField]; ok {
|
||||
task.ExpendTokens = gconv.Int64(mapped[model.ResponseTokenField])
|
||||
_, err = dao.ModelGatewayTask.Update(ctx, &entity.ModelGatewayTask{
|
||||
SQLBaseDO: beans.SQLBaseDO{Id: task.Id},
|
||||
ExpendTokens: task.ExpendTokens,
|
||||
})
|
||||
}
|
||||
|
||||
// 3) 解析 + 校验
|
||||
var parsed map[string]any
|
||||
switch req.BuildType {
|
||||
case public.BuildTypePrompt, public.BuildTypeNode:
|
||||
parsed, err = util.ParseAndValidate(mapped, model)
|
||||
if err == nil {
|
||||
return parsed, nil
|
||||
}
|
||||
case public.BuildTypeStruct:
|
||||
parsed = util.ParseStructResult(mapped, model.ResponseBody)
|
||||
// 解析 + 校验(用构建模型的 RequiredFields)
|
||||
parsed, err := util.ParseAndValidate(body, buildModel.RequiredFields)
|
||||
if err == nil {
|
||||
return parsed, nil
|
||||
default:
|
||||
return mapped, nil
|
||||
}
|
||||
lastErr = err
|
||||
|
||||
g.Log().Warningf(ctx, "[执行任务][解析失败] taskId=%s attempt=%d/%d err=%v", task.TaskID, attempt, maxRetry, err)
|
||||
|
||||
if attempt == maxRetry {
|
||||
return nil, fmt.Errorf("JSON解析重试耗尽: %w", err)
|
||||
return nil, fmt.Errorf("JSON解析重试耗尽: %w", lastErr)
|
||||
}
|
||||
|
||||
// 4) 重新调模型(直接调,不走缓存)
|
||||
// 重试:重新调模型
|
||||
task.RetryCount++
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, task)
|
||||
rawData, callErr := InvokeModel(ctx, model, task.RequestPayload.Body)
|
||||
|
||||
reqBody := injectErrorMessage(task.RequestPayload, lastErr)
|
||||
rawData, callErr := InvokeModel(ctx, model, reqBody)
|
||||
if callErr != nil {
|
||||
g.Log().Warningf(ctx, "[执行任务][重调模型失败] taskId=%s attempt=%d/%d err=%v", task.TaskID, attempt, maxRetry, callErr)
|
||||
continue
|
||||
}
|
||||
|
||||
// 5) 解析原始响应,覆盖 body 进入下一轮
|
||||
var rawResp map[string]any
|
||||
if err = json.Unmarshal(rawData, &rawResp); err != nil {
|
||||
if err := json.Unmarshal(rawData, &rawResp); err != nil {
|
||||
g.Log().Warningf(ctx, "[执行任务][Unmarshal失败] taskId=%s err=%v", task.TaskID, err)
|
||||
continue
|
||||
}
|
||||
body = rawResp
|
||||
mapped, mapErr := util.MapResponsePayload(model.ResponseMapping, rawResp)
|
||||
if mapErr != nil {
|
||||
g.Log().Warningf(ctx, "[执行任务][映射失败] taskId=%s err=%v", task.TaskID, mapErr)
|
||||
continue
|
||||
}
|
||||
|
||||
// 计费
|
||||
if len(model.BillingConfig) > 0 && len(task.BillingData) > 0 {
|
||||
requestData := task.BillingData[0]
|
||||
retryData := make(map[string]any)
|
||||
for k, v := range requestData {
|
||||
retryData[k] = v
|
||||
}
|
||||
responseData := util.ExtractResponseBilling(model.BillingConfig, mapped)
|
||||
for k, v := range responseData {
|
||||
retryData[k] = v
|
||||
}
|
||||
billingResult := util.CalculateBilling(model.BillingConfig, retryData)
|
||||
if billingResult != nil {
|
||||
task.BillingData = append(task.BillingData, billingResult)
|
||||
totalFee := gconv.Float64(billingResult["total_fee"])
|
||||
if totalFee > 0 {
|
||||
_ = gateway.DeductBalance(util.AsyncCtx(ctx), task.TenantId, -totalFee)
|
||||
}
|
||||
}
|
||||
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, &entity.ModelGatewayTask{
|
||||
SQLBaseDO: beans.SQLBaseDO{Id: task.Id},
|
||||
BillingData: task.BillingData,
|
||||
})
|
||||
}
|
||||
|
||||
body = mapped
|
||||
}
|
||||
|
||||
return body, nil
|
||||
}
|
||||
|
||||
// injectErrorMessage 将错误信息插入到最后一个 user 消息之前
|
||||
func injectErrorMessage(payload map[string]any, err error) map[string]any {
|
||||
if err == nil {
|
||||
return payload
|
||||
}
|
||||
|
||||
messages, _ := payload["messages"].([]any)
|
||||
if len(messages) == 0 {
|
||||
return payload
|
||||
}
|
||||
|
||||
errMsg := fmt.Sprintf("【上一轮输出错误,请修正】%s", err.Error())
|
||||
|
||||
// 找到最后一个 user 的位置
|
||||
lastUserIdx := -1
|
||||
for i := len(messages) - 1; i >= 0; i-- {
|
||||
msg, ok := messages[i].(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if gconv.String(msg["role"]) == "user" {
|
||||
lastUserIdx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if lastUserIdx == -1 {
|
||||
return payload
|
||||
}
|
||||
|
||||
// 在最后一个 user 之前插入错误消息
|
||||
errMsgObj := map[string]any{
|
||||
"role": "user",
|
||||
"content": []map[string]any{{"type": "text", "text": errMsg}},
|
||||
}
|
||||
|
||||
// 切片插入
|
||||
messages = append(messages[:lastUserIdx], append([]any{errMsgObj}, messages[lastUserIdx:]...)...)
|
||||
payload["messages"] = messages
|
||||
return payload
|
||||
}
|
||||
|
||||
// InvokeModel 调用模型服务,返回二进制结果
|
||||
// modelKey 用于覆盖/补充模型配置 head_msg(例如每次请求携带不同的 X-API-Key)
|
||||
func InvokeModel(ctx context.Context, model *entity.ModelGatewayModel, body map[string]any) ([]byte, error) {
|
||||
// 1) 记录模型调用次数
|
||||
_ = dao.ModelGatewayLogsStat.IncRequestCount(ctx, time.Now(), model.TenantId, model.Creator, model.ModelName)
|
||||
|
||||
// 2)请求参数映射:将标准 payload 按模型配置的 requestMapping 转为模型需要的格式
|
||||
//—— 请求映射实际处理为提示词构建请求,因为有附加字段及其他字段的拼接。这里不方便做请求映射
|
||||
//mappedPayload := util.ReverseMap(model.RequestMapping, payload)
|
||||
//surplus, _ := gateway.GetTenantSurplus(ctx, model.TenantId)
|
||||
//if surplus <= 0 {
|
||||
// return nil, fmt.Errorf("租户余额不足")
|
||||
//}
|
||||
|
||||
// 3)构建请求 URL 和超时
|
||||
baseURL := strings.TrimRight(model.BaseURL, "/")
|
||||
@@ -392,13 +415,20 @@ func InvokeModel(ctx context.Context, model *entity.ModelGatewayModel, body map[
|
||||
baseURL = baseURL + "?" + q.Encode()
|
||||
}
|
||||
}
|
||||
req, err = http.NewRequestWithContext(ctx, http.MethodGet, baseURL, nil)
|
||||
// 改用独立超时ctx,隔绝外层截止
|
||||
reqCtx, reqCancel := context.WithTimeout(context.Background(), timeout)
|
||||
defer reqCancel()
|
||||
req, err = http.NewRequestWithContext(reqCtx, http.MethodGet, baseURL, nil)
|
||||
//req, err = http.NewRequestWithContext(ctx, http.MethodGet, baseURL, nil)
|
||||
default:
|
||||
bodyBytes, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req, err = http.NewRequestWithContext(ctx, http.MethodPost, baseURL, bytes.NewReader(bodyBytes))
|
||||
reqCtx, reqCancel := context.WithTimeout(context.Background(), timeout)
|
||||
defer reqCancel()
|
||||
req, err = http.NewRequestWithContext(reqCtx, http.MethodPost, baseURL, bytes.NewReader(bodyBytes))
|
||||
//req, err = http.NewRequestWithContext(ctx, http.MethodPost, baseURL, bytes.NewReader(bodyBytes))
|
||||
}
|
||||
|
||||
// 5)注入请求头:先模型静态配置,再动态 modelKey(后者可覆盖前者)
|
||||
@@ -430,6 +460,9 @@ func InvokeModel(ctx context.Context, model *entity.ModelGatewayModel, body map[
|
||||
msg := string(b)
|
||||
return nil, fmt.Errorf("模型服务返回非2xx: %d, body=%s", resp.StatusCode, msg)
|
||||
}
|
||||
|
||||
//
|
||||
g.Log().Debugf(ctx, "[执行任务][模型调用成功] StatusCode=%v", resp.StatusCode)
|
||||
return b, nil
|
||||
}
|
||||
|
||||
@@ -487,15 +520,11 @@ func InvokeModel(ctx context.Context, model *entity.ModelGatewayModel, body map[
|
||||
// return mappedResponse, nil
|
||||
// }
|
||||
|
||||
// failTask 任务失败统一处理:更新数据库 + 释放排队 + 回调
|
||||
// failTask 任务失败统一处理
|
||||
func (w *asyncWorker) failTask(ctx context.Context, t *entity.ModelGatewayTask, startTime time.Time, errMsg string) {
|
||||
t.State = 3
|
||||
t.ErrorMsg = errMsg
|
||||
t.DurationSeconds = int64(time.Since(startTime).Seconds())
|
||||
_, err := dao.ModelGatewayTask.Update(ctx, t)
|
||||
if err != nil {
|
||||
g.Log().Warningf(ctx, "[执行任务][更新数据库失败] taskId=%s err=%v", t.TaskID, err)
|
||||
}
|
||||
queue.ReleaseQueueSlot(ctx, t.ModelName, t.TaskID)
|
||||
go gateway.TriggerCallback(context.WithoutCancel(ctx), t)
|
||||
_, _ = dao.ModelGatewayTask.Update(ctx, t) // 更新任务状态
|
||||
go gateway.TriggerCallback(util.AsyncCtx(ctx), t) // 触发回调
|
||||
}
|
||||
|
||||
+184
-215
@@ -1,233 +1,202 @@
|
||||
-- =========================
|
||||
-- model_gateway_models
|
||||
-- =========================
|
||||
CREATE TABLE IF NOT EXISTS model_gateway_models (
|
||||
id int8 PRIMARY KEY,
|
||||
tenant_id int8 NOT NULL DEFAULT 0,
|
||||
creator varchar(64) NOT NULL,
|
||||
created_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updater varchar(64) NOT NULL,
|
||||
updated_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
deleted_at timestamp(6),
|
||||
model_name varchar(128) NOT NULL,
|
||||
model_type int2 NOT NULL DEFAULT 0,
|
||||
operator_name varchar(64) NOT NULL DEFAULT '',
|
||||
base_url varchar(256) NOT NULL,
|
||||
http_method varchar(8) NOT NULL DEFAULT 'POST',
|
||||
head_msg jsonb NOT NULL DEFAULT '{}',
|
||||
api_key varchar(256) NOT NULL DEFAULT '',
|
||||
is_private int2 NOT NULL DEFAULT 0,
|
||||
enabled int2 NOT NULL DEFAULT 1,
|
||||
is_chat_model int2 NOT NULL DEFAULT 0,
|
||||
is_owner int2 NOT NULL DEFAULT 99,
|
||||
form_json jsonb NOT NULL DEFAULT '{}',
|
||||
request_mapping jsonb NOT NULL DEFAULT '{}',
|
||||
response_mapping jsonb NOT NULL DEFAULT '{}',
|
||||
response_body varchar(128) NOT NULL DEFAULT '',
|
||||
token_config jsonb NOT NULL DEFAULT '{}',
|
||||
extend_mapping jsonb NOT NULL DEFAULT '{}',
|
||||
query_config jsonb NOT NULL DEFAULT '{}',
|
||||
stream_config jsonb NOT NULL DEFAULT '{}',
|
||||
first_frame varchar(128) NOT NULL DEFAULT '',
|
||||
last_frame varchar(128) NOT NULL DEFAULT '',
|
||||
max_concurrency int4 NOT NULL DEFAULT 10,
|
||||
timeout_seconds int4 NOT NULL DEFAULT 600,
|
||||
retry_times int2 NOT NULL DEFAULT 3,
|
||||
auto_clean_seconds int4 NOT NULL DEFAULT 86400,
|
||||
response_token_field varchar(128) NOT NULL DEFAULT '',
|
||||
call_mode int2 NOT NULL DEFAULT 0,
|
||||
required_fields jsonb NOT NULL DEFAULT '[]',
|
||||
max_tokens int4 DEFAULT 0
|
||||
);
|
||||
-- ============================================
|
||||
-- 模型网关 (model-gateway) 建表语句
|
||||
-- ============================================
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uk_model_gateway_models_tenant_creator_model ON model_gateway_models (tenant_id, creator, model_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_models_model_name ON model_gateway_models (model_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_models_model_type ON model_gateway_models (model_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_models_tenant_id ON model_gateway_models (tenant_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_models_deleted_at ON model_gateway_models (deleted_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_models_enabled ON model_gateway_models (enabled);
|
||||
-- 1. 模型配置表
|
||||
CREATE TABLE "public"."model_gateway_models" (
|
||||
"id" int8 NOT NULL,
|
||||
"tenant_id" int8 NOT NULL DEFAULT 0,
|
||||
"creator" varchar(64) NOT NULL,
|
||||
"created_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updater" varchar(64) NOT NULL,
|
||||
"updated_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"deleted_at" timestamp(6),
|
||||
"model_name" varchar(128) NOT NULL,
|
||||
"model_type" int2 NOT NULL DEFAULT 0,
|
||||
"operator_name" varchar(64) NOT NULL DEFAULT '',
|
||||
"base_url" varchar(256) NOT NULL,
|
||||
"http_method" varchar(8) NOT NULL DEFAULT 'POST',
|
||||
"head_msg" jsonb NOT NULL DEFAULT '{}',
|
||||
"api_key" varchar(256) NOT NULL DEFAULT '',
|
||||
"is_private" int2 NOT NULL DEFAULT 0,
|
||||
"enabled" int2 NOT NULL DEFAULT 1,
|
||||
"is_chat_model" int2 NOT NULL DEFAULT 0,
|
||||
"form_json" jsonb NOT NULL DEFAULT '{}',
|
||||
"request_mapping" jsonb NOT NULL DEFAULT '{}',
|
||||
"response_mapping" jsonb NOT NULL DEFAULT '{}',
|
||||
"extend_mapping" jsonb NOT NULL DEFAULT '{}',
|
||||
"query_config" jsonb NOT NULL DEFAULT '{}',
|
||||
"stream_config" jsonb NOT NULL DEFAULT '{}',
|
||||
"max_concurrency" int4 NOT NULL DEFAULT 10,
|
||||
"timeout_seconds" int4 NOT NULL DEFAULT 600,
|
||||
"retry_times" int2 NOT NULL DEFAULT 3,
|
||||
"call_mode" int2 NOT NULL DEFAULT 0,
|
||||
"required_fields" jsonb NOT NULL DEFAULT '[]',
|
||||
"billing_config" jsonb NOT NULL DEFAULT '{}',
|
||||
"special_params" jsonb NOT NULL DEFAULT '{}',
|
||||
CONSTRAINT "model_gateway_models_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
COMMENT ON TABLE model_gateway_models IS '模型配置表';
|
||||
COMMENT ON COLUMN model_gateway_models.id IS '主键ID(非自增)';
|
||||
COMMENT ON COLUMN model_gateway_models.tenant_id IS '租户ID';
|
||||
COMMENT ON COLUMN model_gateway_models.creator IS '创建人';
|
||||
COMMENT ON COLUMN model_gateway_models.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN model_gateway_models.updater IS '更新人';
|
||||
COMMENT ON COLUMN model_gateway_models.updated_at IS '更新时间';
|
||||
COMMENT ON COLUMN model_gateway_models.deleted_at IS '删除时间(软删)';
|
||||
COMMENT ON COLUMN model_gateway_models.model_name IS '模型名称';
|
||||
COMMENT ON COLUMN model_gateway_models.model_type IS '模型类型';
|
||||
COMMENT ON COLUMN model_gateway_models.operator_name IS '运营商名称';
|
||||
COMMENT ON COLUMN model_gateway_models.base_url IS '模型地址';
|
||||
COMMENT ON COLUMN model_gateway_models.http_method IS '请求方式 GET/POST';
|
||||
COMMENT ON COLUMN model_gateway_models.head_msg IS '请求头信息';
|
||||
COMMENT ON COLUMN model_gateway_models.api_key IS '调用凭证/密钥';
|
||||
ALTER TABLE "public"."model_gateway_models" OWNER TO "postgres";
|
||||
|
||||
COMMENT ON COLUMN model_gateway_models.is_private IS '是否私有化:0-私有 1-公共';
|
||||
COMMENT ON COLUMN model_gateway_models.enabled IS '是否启用:0-停用 1-启用';
|
||||
COMMENT ON COLUMN model_gateway_models.is_chat_model IS '是否为对话模型:0-否 1-是';
|
||||
COMMENT ON COLUMN model_gateway_models.is_owner IS '1=当前用户创建 0=超级管理员';
|
||||
CREATE INDEX "idx_models_deleted_at" ON "public"."model_gateway_models" ("deleted_at");
|
||||
CREATE INDEX "idx_models_enabled" ON "public"."model_gateway_models" ("enabled");
|
||||
CREATE INDEX "idx_models_model_name" ON "public"."model_gateway_models" ("model_name");
|
||||
CREATE INDEX "idx_models_model_type" ON "public"."model_gateway_models" ("model_type");
|
||||
CREATE INDEX "idx_models_tenant_id" ON "public"."model_gateway_models" ("tenant_id");
|
||||
CREATE INDEX "idx_models_tenant_creator" ON "public"."model_gateway_models" ("tenant_id", "creator");
|
||||
CREATE INDEX "idx_models_tenant_creator_model_deleted" ON "public"."model_gateway_models" ("tenant_id", "creator", "model_name") WHERE deleted_at IS NULL;
|
||||
CREATE UNIQUE INDEX "uk_models_tenant_creator_model" ON "public"."model_gateway_models" ("tenant_id", "creator", "model_name");
|
||||
|
||||
COMMENT ON COLUMN model_gateway_models.form_json IS '动态表单结构';
|
||||
COMMENT ON COLUMN model_gateway_models.request_mapping IS '请求映射';
|
||||
COMMENT ON COLUMN model_gateway_models.response_mapping IS '返回映射';
|
||||
COMMENT ON COLUMN model_gateway_models.response_body IS '返回主体';
|
||||
COMMENT ON COLUMN model_gateway_models.token_config IS 'Token计算配置';
|
||||
COMMENT ON COLUMN model_gateway_models.extend_mapping IS '附加映射';
|
||||
COMMENT ON COLUMN model_gateway_models.query_config IS '查询/回调配置';
|
||||
COMMENT ON COLUMN model_gateway_models.stream_config IS '流式输出配置';
|
||||
COMMENT ON COLUMN model_gateway_models.first_frame IS '首帧图片参数';
|
||||
COMMENT ON COLUMN model_gateway_models.last_frame IS '尾帧图片参数';
|
||||
COMMENT ON COLUMN model_gateway_models.max_concurrency IS '最大并发数';
|
||||
COMMENT ON COLUMN model_gateway_models.timeout_seconds IS '调用模型超时(秒)';
|
||||
COMMENT ON COLUMN model_gateway_models.retry_times IS '失败重试次数';
|
||||
COMMENT ON COLUMN model_gateway_models.auto_clean_seconds IS '任务完成后自动清理时间(秒)';
|
||||
COMMENT ON COLUMN model_gateway_models.response_token_field IS '响应中消耗token的字段映射';
|
||||
COMMENT ON COLUMN model_gateway_models.call_mode IS '调用模式:0-同步 1-异步 2-流式';
|
||||
COMMENT ON COLUMN model_gateway_models.required_fields IS '必选字段列表';
|
||||
COMMENT ON COLUMN model_gateway_models.max_tokens IS '最大 token 数,0 表示不传';
|
||||
COMMENT ON TABLE "public"."model_gateway_models" IS '模型配置表';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."model_name" IS '模型名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."model_type" IS '模型类型';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."operator_name" IS '运营商名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."base_url" IS '模型地址';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."http_method" IS '请求方式 GET/POST';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."head_msg" IS '请求头信息';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."api_key" IS '调用凭证/密钥';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."is_private" IS '是否私有化:0-私有 1-公共';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."enabled" IS '是否启用:0-停用 1-启用';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."is_chat_model" IS '是否为对话模型:0-否 1-是';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."form_json" IS '动态表单结构';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."request_mapping" IS '请求映射';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."response_mapping" IS '返回映射';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."extend_mapping" IS '附加映射';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."query_config" IS '查询/回调配置';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."stream_config" IS '流式输出配置';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."max_concurrency" IS '最大并发数';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."timeout_seconds" IS '调用模型超时(秒)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."retry_times" IS '失败重试次数';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."call_mode" IS '调用模式:0-同步 1-异步 2-流式';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."required_fields" IS '必选字段列表';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."billing_config" IS '计费配置';
|
||||
COMMENT ON COLUMN "public"."model_gateway_models"."special_params" IS '请求特殊参数';
|
||||
|
||||
|
||||
-- 2. 模型网关任务表
|
||||
CREATE TABLE "public"."model_gateway_task" (
|
||||
"id" int8 NOT NULL,
|
||||
"tenant_id" int8 NOT NULL DEFAULT 0,
|
||||
"creator" varchar(64) NOT NULL,
|
||||
"created_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updater" varchar(64) NOT NULL,
|
||||
"updated_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"deleted_at" timestamp(6),
|
||||
"model_name" varchar(128) NOT NULL,
|
||||
"task_id" varchar(64) NOT NULL,
|
||||
"biz_name" varchar(128) NOT NULL DEFAULT '',
|
||||
"callback_url" varchar(512) DEFAULT '',
|
||||
"state" int2 NOT NULL DEFAULT 0,
|
||||
"retry_count" int4 NOT NULL DEFAULT 0,
|
||||
"error_msg" text DEFAULT '',
|
||||
"result_file" jsonb NOT NULL DEFAULT '{}',
|
||||
"request_payload" jsonb NOT NULL DEFAULT '{}',
|
||||
"duration_seconds" int8 NOT NULL DEFAULT 0,
|
||||
"epicycle_id" varchar(64) NOT NULL DEFAULT '',
|
||||
"billing_data" jsonb NOT NULL DEFAULT '[]',
|
||||
"build_model_name" varchar(128) NOT NULL DEFAULT '',
|
||||
CONSTRAINT "model_gateway_task_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- =========================
|
||||
-- model_gateway_task
|
||||
-- =========================
|
||||
CREATE TABLE IF NOT EXISTS model_gateway_task (
|
||||
id int8 PRIMARY KEY,
|
||||
tenant_id int8 NOT NULL DEFAULT 0,
|
||||
creator varchar(64) NOT NULL,
|
||||
created_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updater varchar(64) NOT NULL,
|
||||
updated_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
deleted_at timestamp(6),
|
||||
model_name varchar(128) NOT NULL,
|
||||
task_id varchar(64) NOT NULL,
|
||||
biz_name varchar(128) NOT NULL DEFAULT '',
|
||||
callback_url varchar(512) DEFAULT '',
|
||||
state int2 NOT NULL DEFAULT 0,
|
||||
retry_count int4 NOT NULL DEFAULT 0,
|
||||
phase int2 NOT NULL DEFAULT 0,
|
||||
tmp_file text DEFAULT '',
|
||||
error_msg text DEFAULT '',
|
||||
result_file jsonb NOT NULL DEFAULT '{}',
|
||||
request_payload jsonb NOT NULL DEFAULT '{}',
|
||||
text_result jsonb NOT NULL DEFAULT '{}',
|
||||
expend_tokens int8 NOT NULL DEFAULT 0,
|
||||
duration_seconds int8 NOT NULL DEFAULT 0,
|
||||
epicycle_id varchar(64) NOT NULL DEFAULT ''
|
||||
);
|
||||
ALTER TABLE "public"."model_gateway_task" OWNER TO "postgres";
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS uk_model_gateway_task_tenant_creator_task_id ON model_gateway_task (tenant_id, creator, task_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_task_task_id ON model_gateway_task (task_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_task_state ON model_gateway_task (state);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_task_deleted_at ON model_gateway_task (deleted_at);
|
||||
CREATE INDEX "idx_task_creator_created" ON "public"."model_gateway_task" ("creator", "created_at" DESC);
|
||||
CREATE INDEX "idx_task_model_state" ON "public"."model_gateway_task" ("model_name", "state");
|
||||
CREATE INDEX "idx_task_state" ON "public"."model_gateway_task" ("state");
|
||||
CREATE INDEX "idx_task_state_created" ON "public"."model_gateway_task" ("state", "created_at");
|
||||
CREATE INDEX "idx_task_task_id" ON "public"."model_gateway_task" ("task_id");
|
||||
CREATE INDEX "idx_task_tenant_model" ON "public"."model_gateway_task" ("tenant_id", "model_name");
|
||||
CREATE UNIQUE INDEX "uk_task_task_id" ON "public"."model_gateway_task" ("task_id");
|
||||
|
||||
COMMENT ON TABLE model_gateway_task IS '模型网关任务表';
|
||||
COMMENT ON COLUMN model_gateway_task.id IS '主键ID';
|
||||
COMMENT ON COLUMN model_gateway_task.tenant_id IS '租户ID';
|
||||
COMMENT ON COLUMN model_gateway_task.creator IS '创建人';
|
||||
COMMENT ON COLUMN model_gateway_task.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN model_gateway_task.updater IS '更新人';
|
||||
COMMENT ON COLUMN model_gateway_task.updated_at IS '更新时间';
|
||||
COMMENT ON COLUMN model_gateway_task.deleted_at IS '删除时间(软删)';
|
||||
COMMENT ON COLUMN model_gateway_task.model_name IS '模型名称';
|
||||
COMMENT ON COLUMN model_gateway_task.task_id IS '任务ID(对外返回)';
|
||||
COMMENT ON COLUMN model_gateway_task.biz_name IS '业务名称(调用方模块/系统)';
|
||||
COMMENT ON COLUMN model_gateway_task.callback_url IS '回调地址';
|
||||
COMMENT ON COLUMN model_gateway_task.state IS '0排队中/1执行中/2成功/3失败/4已下载';
|
||||
COMMENT ON COLUMN model_gateway_task.retry_count IS '已重试次数';
|
||||
COMMENT ON COLUMN model_gateway_task.phase IS '执行阶段:0模型阶段/1OSS阶段';
|
||||
COMMENT ON COLUMN model_gateway_task.tmp_file IS '临时结果文件路径';
|
||||
COMMENT ON COLUMN model_gateway_task.error_msg IS '错误信息';
|
||||
COMMENT ON COLUMN model_gateway_task.result_file IS '结果文件:{oss_file, file_type, file_size}';
|
||||
COMMENT ON COLUMN model_gateway_task.request_payload IS '请求参数(JSON)';
|
||||
COMMENT ON COLUMN model_gateway_task.text_result IS '文本类结果';
|
||||
COMMENT ON COLUMN model_gateway_task.expend_tokens IS '消耗token数';
|
||||
COMMENT ON COLUMN model_gateway_task.duration_seconds IS '耗时(秒)';
|
||||
COMMENT ON COLUMN model_gateway_task.epicycle_id IS '轮次ID';
|
||||
COMMENT ON TABLE "public"."model_gateway_task" IS '模型网关任务表';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."model_name" IS '模型名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."task_id" IS '任务ID(对外返回)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."biz_name" IS '业务名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."callback_url" IS '回调地址';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."state" IS '0排队中/1执行中/2成功/3失败/4已下载';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."retry_count" IS '已重试次数';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."error_msg" IS '错误信息';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."result_file" IS '结果文件:{oss_file, file_type, file_size}';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."request_payload" IS '请求参数(JSON)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."duration_seconds" IS '耗时(秒)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."epicycle_id" IS '轮次ID';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."billing_data" IS '计费数据';
|
||||
COMMENT ON COLUMN "public"."model_gateway_task"."build_model_name" IS '构建模型名称';
|
||||
|
||||
|
||||
-- 3. 操作日志表
|
||||
CREATE TABLE "public"."model_gateway_logs_op" (
|
||||
"id" int8 NOT NULL,
|
||||
"tenant_id" int8 NOT NULL DEFAULT 0,
|
||||
"creator" varchar(64) NOT NULL,
|
||||
"created_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updater" varchar(64) NOT NULL,
|
||||
"updated_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"deleted_at" timestamp(6),
|
||||
"ip" varchar(64) DEFAULT '',
|
||||
"user_agent" varchar(256) DEFAULT '',
|
||||
"api_path" varchar(256) DEFAULT '',
|
||||
"http_method" varchar(16) DEFAULT '',
|
||||
"biz_name" varchar(128) NOT NULL DEFAULT '',
|
||||
"model_name" varchar(128) NOT NULL DEFAULT '',
|
||||
"task_id" varchar(64) NOT NULL DEFAULT '',
|
||||
"op_type" varchar(64) NOT NULL DEFAULT 'createTask',
|
||||
"success" int2 NOT NULL DEFAULT 1,
|
||||
"error_msg" text DEFAULT '',
|
||||
"cost_ms" int8 NOT NULL DEFAULT 0,
|
||||
"request_payload" jsonb,
|
||||
"response_payload" jsonb,
|
||||
CONSTRAINT "model_gateway_logs_op_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- =========================
|
||||
-- model_gateway_log_stat
|
||||
-- =========================
|
||||
CREATE TABLE IF NOT EXISTS model_gateway_log_stat (
|
||||
day date NOT NULL,
|
||||
tenant_id int8 NOT NULL DEFAULT 0,
|
||||
creator varchar(64) NOT NULL DEFAULT '',
|
||||
model_name varchar(128) NOT NULL DEFAULT '',
|
||||
request_count int8 NOT NULL DEFAULT 0,
|
||||
created_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updated_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
PRIMARY KEY (day, tenant_id, creator, model_name)
|
||||
);
|
||||
ALTER TABLE "public"."model_gateway_logs_op" OWNER TO "postgres";
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_log_stat_day ON model_gateway_log_stat (day);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_log_stat_creator ON model_gateway_log_stat (creator);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_log_stat_model_name ON model_gateway_log_stat (model_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_log_stat_tenant_day ON model_gateway_log_stat (tenant_id, day);
|
||||
CREATE INDEX "idx_op_log_biz_name" ON "public"."model_gateway_logs_op" ("biz_name");
|
||||
CREATE INDEX "idx_op_log_deleted_at" ON "public"."model_gateway_logs_op" ("deleted_at");
|
||||
CREATE INDEX "idx_op_log_model_name" ON "public"."model_gateway_logs_op" ("model_name");
|
||||
CREATE INDEX "idx_op_log_op_type" ON "public"."model_gateway_logs_op" ("op_type");
|
||||
CREATE INDEX "idx_op_log_task_id" ON "public"."model_gateway_logs_op" ("task_id");
|
||||
CREATE INDEX "idx_op_log_tenant_time" ON "public"."model_gateway_logs_op" ("tenant_id", "created_at");
|
||||
CREATE INDEX "idx_op_log_model_time" ON "public"."model_gateway_logs_op" ("model_name", "created_at");
|
||||
|
||||
COMMENT ON TABLE model_gateway_log_stat IS '按天统计表';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.day IS '天(YYYY-MM-DD)';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.tenant_id IS '租户ID';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.creator IS '创建人';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.model_name IS '模型名称';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.request_count IS '请求次数';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN model_gateway_log_stat.updated_at IS '更新时间';
|
||||
COMMENT ON TABLE "public"."model_gateway_logs_op" IS '操作日志表';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."ip" IS '客户端IP';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."user_agent" IS 'User-Agent';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."api_path" IS '接口路径';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."http_method" IS 'HTTP方法';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."biz_name" IS '业务名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."model_name" IS '模型名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."task_id" IS '任务ID';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."op_type" IS '操作类型';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."success" IS '是否成功:1成功/0失败';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."error_msg" IS '错误信息';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."cost_ms" IS '耗时(毫秒)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."request_payload" IS '请求 JSON';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_op"."response_payload" IS '响应 JSON';
|
||||
|
||||
|
||||
-- =========================
|
||||
-- model_gateway_logs_op
|
||||
-- =========================
|
||||
CREATE TABLE IF NOT EXISTS model_gateway_logs_op (
|
||||
id int8 PRIMARY KEY,
|
||||
tenant_id int8 NOT NULL DEFAULT 0,
|
||||
creator varchar(64) NOT NULL,
|
||||
created_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
updater varchar(64) NOT NULL,
|
||||
updated_at timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
deleted_at timestamp(6),
|
||||
ip varchar(64) DEFAULT '',
|
||||
user_agent varchar(256) DEFAULT '',
|
||||
api_path varchar(256) DEFAULT '',
|
||||
http_method varchar(16) DEFAULT '',
|
||||
biz_name varchar(128) NOT NULL DEFAULT '',
|
||||
model_name varchar(128) NOT NULL DEFAULT '',
|
||||
task_id varchar(64) NOT NULL DEFAULT '',
|
||||
op_type varchar(64) NOT NULL DEFAULT 'createTask',
|
||||
success int2 NOT NULL DEFAULT 1,
|
||||
error_msg text DEFAULT '',
|
||||
cost_ms int8 NOT NULL DEFAULT 0,
|
||||
request_payload jsonb,
|
||||
response_payload jsonb
|
||||
);
|
||||
-- 4. 按天统计表
|
||||
CREATE TABLE "public"."model_gateway_logs_stat" (
|
||||
"day" date NOT NULL,
|
||||
"tenant_id" int8 NOT NULL DEFAULT 0,
|
||||
"creator" varchar(64) NOT NULL DEFAULT '',
|
||||
"model_name" varchar(128) NOT NULL DEFAULT '',
|
||||
"request_count" int8 NOT NULL DEFAULT 0,
|
||||
"created_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"updated_at" timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_task_id ON model_gateway_logs_op (task_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_biz_name ON model_gateway_logs_op (biz_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_model_name ON model_gateway_logs_op (model_name);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_op_type ON model_gateway_logs_op (op_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_deleted_at ON model_gateway_logs_op (deleted_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_model_gateway_logs_op_tenant_time ON model_gateway_logs_op (tenant_id, created_at);
|
||||
ALTER TABLE "public"."model_gateway_logs_stat" OWNER TO "postgres";
|
||||
|
||||
COMMENT ON TABLE model_gateway_logs_op IS '操作日志表';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.id IS '主键ID(非自增)';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.tenant_id IS '租户ID';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.creator IS '创建人';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.created_at IS '创建时间';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.updater IS '更新人';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.updated_at IS '更新时间';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.deleted_at IS '删除时间(软删)';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.ip IS '客户端IP';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.user_agent IS 'User-Agent';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.api_path IS '接口路径';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.http_method IS 'HTTP方法';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.biz_name IS '业务名称(调用方模块/系统)';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.model_name IS '模型名称';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.task_id IS '任务ID';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.op_type IS '操作类型';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.success IS '是否成功:1成功/0失败';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.error_msg IS '错误信息(失败时)';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.cost_ms IS '耗时(毫秒)';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.request_payload IS '请求 JSON';
|
||||
COMMENT ON COLUMN model_gateway_logs_op.response_payload IS '响应 JSON';
|
||||
CREATE INDEX "idx_stat_creator" ON "public"."model_gateway_logs_stat" ("creator");
|
||||
CREATE INDEX "idx_stat_day" ON "public"."model_gateway_logs_stat" ("day");
|
||||
CREATE INDEX "idx_stat_model_name" ON "public"."model_gateway_logs_stat" ("model_name");
|
||||
CREATE INDEX "idx_stat_tenant_day" ON "public"."model_gateway_logs_stat" ("tenant_id", "day");
|
||||
|
||||
COMMENT ON TABLE "public"."model_gateway_logs_stat" IS '按天统计表';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_stat"."day" IS '天(YYYY-MM-DD)';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_stat"."tenant_id" IS '租户ID';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_stat"."creator" IS '创建人';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_stat"."model_name" IS '模型名称';
|
||||
COMMENT ON COLUMN "public"."model_gateway_logs_stat"."request_count" IS '请求次数';
|
||||
Reference in New Issue
Block a user