cid适配ppgo_job

This commit is contained in:
lmk
2026-07-18 10:10:23 +08:00
parent d930266fbf
commit 2fd858c97e
20 changed files with 810 additions and 884 deletions
+5 -1
View File
@@ -1,2 +1,6 @@
/.idea/*
/docs/*
/docs/*
# 运行日志
/resource/log/*
+90
View File
@@ -0,0 +1,90 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Build & Run
```bash
# Build (Docker)
docker build -t cid .
# Run locally (requires Go 1.26)
go mod download && go mod tidy
go run main.go
# Build binary
go build -ldflags="-s -w" -o main ./main.go
# Test
go test ./...
```
## GoFrame CLI (optional)
```bash
# Install gf CLI
go install github.com/gogf/gf/v2/cmd/gf/v2@latest
# Generate DAO/Model from DB
gf gen dao -c config.yml
```
## Project Overview
CID is a **content moderation service** that submits advertising images/videos to Netease Yidun (易盾) for automated content review. It provides REST APIs, a scheduled batch-checking scheduler, and a frontend management UI.
### Core Flow
1. **Material sources**: `tencent_image` / `tencent_video` tables in the `dataengine` PostgreSQL database (external system)
2. **Detection**: Submit media to Yidun API for content moderation (async callback mode or sync polling mode)
3. **Result handling**: Yidun pushes results to callback endpoints, or the service polls Yidun for results
4. **Logging**: Results recorded in `material_verify_log` table (cid DB) with status backfilled to source tables
### Architecture
```
controller/ HTTP handlers (GoFrame controller + ghttp.Request handlers)
├── yidun/ Content detection API (text/image/video submit, callback receive, result polling)
├── dataengine/ Material verification UI API (list, stats, manual verify, batch verify, export, callback)
service/ Business logic layer
├── yidun/ Yidun SDK wrappers (text/image/video detection, callback processing)
├── dataengine/ Material verification pipeline, content check scheduler, callback handling
dao/ Data access layer (GoFrame ORM + custom gfdb wrapper)
├── dataengine/ DAOs for tencent_image, tencent_video, material_verify_log, etc.
model/ Entity/DTO definitions
├── entity/dataengine/ DB entity structs with field constants
├── dto/yidun/ Request/response DTOs
consts/ Constants
├── dataengine/ Table names, check statuses, suggestion values
├── public/ Shared constants (currently empty)
resource/frontend/ Static frontend HTML (material-verify.html)
sql/ DDL and migration scripts
```
### Two Detection Modes
Configured via `yidun.callback_mode` in config.yml:
- **Callback mode** (true): Yidun pushes results to `/yidun/callback/receiveImage` / `/yidun/callback/receiveVideo` after async detection completes. Requires a public-facing callback URL configured in `yidun.image.callback_url` / `yidun.video.callback_url`.
- **Polling mode** (false): After submitting content, the service immediately calls Yidun's sync image API (for images) or queries results via callback API (for video). Manual polling endpoints are also available.
### Scheduled Content Check
The `content_check` section in config.yml controls a background scheduler (`tencent_content_check_service.go`) that periodically fetches pending media from source tables and submits them for verification. Configurable via `batch_size`, `interval_seconds`, `image_enabled`, `video_enabled`, `scheduler_enabled`.
## Key Design Decisions
- **Snowflake IDs**: DAO `Create()` methods generate IDs client-side via Snowflake because GoFrame v2.10's pgsql driver doesn't support `RETURNING` / `LastInsertId`.
- **Dual database**: The `cid` DB holds verification logs; the `dataengine` DB holds source material tables. The `db.go` helper (`Model(tableName)`) selects the `dataEngine` DB group.
- **Mixed ORM**: Some DAOs use `g.DB("dataEngine").Model()` (standard GoFrame), others use `gfdb.DB(ctx, "cid").Model(ctx, ...)` (custom common library wrapper that passes context to DB operations). Both access the same underlying gdb engine.
- **Singleton pattern**: All controllers, services, and DAOs are package-level global singletons (e.g., `var MaterialVerify = new(MaterialVerifyService)`).
- **Context user**: Each controller method injects a hardcoded `beans.User` into context (`ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})`). No real auth.
- **Verify status values**: `PENDING``SUBMITTING``VERIFIED` / `REJECTED` / `REVIEW`. Source tables use a different set: `PENDING` / `SUBMITTING` / `SUCCESS` / `FAILED` / `COMPLETED`.
## API Routes
Routes are registered in `main.go` via `http.RouteRegister()` (from the common library). Key endpoint groups:
- **Yidun detection** (`controller/yidun`): POST `/yidun/detectText`, `/yidun/detectImage`, `/yidun/detectVideo` — submit content for detection
- **Yidun callback** (`controller/yidun`): POST `/yidun/callback/receiveImage`, `/yidun/callback/receiveVideo` — receive Yidun push results; POST `/yidun/callback/poll`, `/yidun/callback/pollImage`, `/yidun/callback/pollVideo` — manual polling
- **Material verify** (`controller/dataengine`): POST `/dataengine/listImage`, `/dataengine/listVideo`, `/dataengine/statsImage`, `/dataengine/statsVideo`, `/dataengine/manualVerifyImage`, `/dataengine/batchVerifyImage`, `/dataengine/exportRejected`, `/dataengine/imageCallback`, etc.
- **Content check** (`controller/yidun`): POST `/contentCheck/start`, `/contentCheck/stop`, `/contentCheck/status`, `/contentCheck/manualSubmitImageByID`, etc.
+3 -2
View File
@@ -36,7 +36,7 @@ database:
port: "15432"
user: "postgres"
pass: "Bjang09@686^*^"
name: "dataengine"
name: "engine"
role: "master"
maxIdle: "5"
maxOpen: "20"
@@ -119,4 +119,5 @@ content_check:
# 是否启用视频检测
video_enabled: false
# 定时任务执行间隔(秒)
interval_seconds: 30
interval_seconds: 30
poll_interval: 60
-1
View File
@@ -1 +0,0 @@
package public
@@ -2,17 +2,15 @@ package dataengine
import (
consts "cid/consts/dataengine"
internal "cid/controller/internal"
dao "cid/dao/dataengine"
entity "cid/model/entity/dataengine"
serviceDataengine "cid/service/dataengine"
"context"
"fmt"
"strings"
"time"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
)
// MaterialVerifyController 素材校验控制器
@@ -107,7 +105,7 @@ type BatchVerifyReq struct {
// ListImage 图片素材列表
func (c *MaterialVerifyController) ListImage(ctx context.Context, req *ImageListReq) (res *ImageListRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.Page == 0 {
req.Page = 1
@@ -137,11 +135,20 @@ func (c *MaterialVerifyController) ListImage(ctx context.Context, req *ImageList
// StatsImage 图片素材统计
func (c *MaterialVerifyController) StatsImage(ctx context.Context, req *ImageListReq) (res *StatsRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
// 使用实体中定义的正确状态值:PENDING=待校验, VERIFIED=校验通过, REJECTED=校验不通过
pending, _ := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusPending)
verified, _ := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusVerified)
rejected, _ := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusRejected)
pending, err := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusPending)
if err != nil {
g.Log().Errorf(ctx, "统计待校验图片数量失败: %v", err)
}
verified, err := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusVerified)
if err != nil {
g.Log().Errorf(ctx, "统计已通过图片数量失败: %v", err)
}
rejected, err := dao.TencentImage.CountByStatus(ctx, entity.VerifyStatusRejected)
if err != nil {
g.Log().Errorf(ctx, "统计不通过图片数量失败: %v", err)
}
return &StatsRes{
Pending: pending,
@@ -156,7 +163,7 @@ func (c *MaterialVerifyController) StatsImage(ctx context.Context, req *ImageLis
// ListVideo 视频素材列表
func (c *MaterialVerifyController) ListVideo(ctx context.Context, req *VideoListReq) (res *VideoListRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.Page == 0 {
req.Page = 1
@@ -186,11 +193,20 @@ func (c *MaterialVerifyController) ListVideo(ctx context.Context, req *VideoList
// StatsVideo 视频素材统计
func (c *MaterialVerifyController) StatsVideo(ctx context.Context, req *VideoListReq) (res *StatsRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
// 使用实体中定义的正确状态值:PENDING=待校验, VERIFIED=校验通过, REJECTED=校验不通过
pending, _ := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusPending)
verified, _ := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusVerified)
rejected, _ := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusRejected)
pending, err := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusPending)
if err != nil {
g.Log().Errorf(ctx, "统计待校验视频数量失败: %v", err)
}
verified, err := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusVerified)
if err != nil {
g.Log().Errorf(ctx, "统计已通过视频数量失败: %v", err)
}
rejected, err := dao.TencentVideo.CountByStatus(ctx, entity.VerifyStatusRejected)
if err != nil {
g.Log().Errorf(ctx, "统计不通过视频数量失败: %v", err)
}
return &StatsRes{
Pending: pending,
@@ -211,7 +227,7 @@ type ListLogRes struct {
// ListLog 日志列表
func (c *MaterialVerifyController) ListLog(ctx context.Context, req *LogListReq) (res *ListLogRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.Page == 0 {
req.Page = 1
@@ -258,7 +274,7 @@ type GetLogDetailReq struct {
// GetLogDetail 日志详情
func (c *MaterialVerifyController) GetLogDetail(ctx context.Context, req *GetLogDetailReq) (res *LogDetailRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
log, err := serviceDataengine.MaterialVerify.GetLogByID(ctx, req.Id)
if err != nil {
@@ -297,7 +313,7 @@ type StatsLogRes struct {
// StatsLog 日志统计
func (c *MaterialVerifyController) StatsLog(ctx context.Context, req *LogListReq) (res *StatsLogRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
stats, err := serviceDataengine.MaterialVerify.GetStats(ctx)
if err != nil {
return nil, err
@@ -324,7 +340,7 @@ type ManualVerifyImageRes struct {
// ManualVerifyImage 手动校验图片
func (c *MaterialVerifyController) ManualVerifyImage(ctx context.Context, req *ManualVerifyReq) (res *ManualVerifyImageRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
log, err := serviceDataengine.MaterialVerify.VerifyImageByID(ctx, req.MaterialID)
if err != nil {
@@ -340,7 +356,7 @@ func (c *MaterialVerifyController) ManualVerifyImage(ctx context.Context, req *M
// ManualVerifyVideo 手动校验视频
func (c *MaterialVerifyController) ManualVerifyVideo(ctx context.Context, req *ManualVerifyReq) (res *ManualVerifyImageRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
log, err := serviceDataengine.MaterialVerify.VerifyVideoByID(ctx, req.MaterialID)
if err != nil {
@@ -368,7 +384,7 @@ type BatchVerifyRes struct {
// BatchVerifyImage 批量校验图片
func (c *MaterialVerifyController) BatchVerifyImage(ctx context.Context, req *BatchVerifyReq) (res *BatchVerifyRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.Limit <= 0 {
req.Limit = 100
@@ -403,7 +419,7 @@ func (c *MaterialVerifyController) BatchVerifyImage(ctx context.Context, req *Ba
// BatchVerifyVideo 批量校验视频
func (c *MaterialVerifyController) BatchVerifyVideo(ctx context.Context, req *BatchVerifyReq) (res *BatchVerifyRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.Limit <= 0 {
req.Limit = 100
@@ -456,7 +472,7 @@ type ListAccountsRes struct {
// ListAccounts 获取所有启用的广告账户列表(用于前端下拉筛选)
func (c *MaterialVerifyController) ListAccounts(ctx context.Context, req *ListAccountsReq) (res *ListAccountsRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
accounts, err := dao.TencentAccountRelation.GetAll(ctx)
if err != nil {
return nil, err
@@ -506,7 +522,7 @@ type ExportRejectedRes struct {
// ExportRejected 导出不通过的图片/视频数据(含失败原因)
func (c *MaterialVerifyController) ExportRejected(ctx context.Context, req *ExportRejectedReq) (res *ExportRejectedRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
items, err := serviceDataengine.MaterialVerify.ExportRejectedData(ctx, req.MaterialType)
if err != nil {
@@ -546,32 +562,12 @@ type CallbackRes struct {
Msg string `json:"msg"`
}
// checkIP 校验请求IP是否在白名单内(context模式)
func checkIP(ctx context.Context) bool {
r := ghttp.RequestFromCtx(ctx)
if r == nil {
return true
}
allowedIPs := g.Cfg().MustGet(ctx, "yidun.callback_allowed_ips", "").String()
if allowedIPs == "" {
return true
}
clientIP := r.GetClientIp()
for _, ip := range strings.Split(allowedIPs, ",") {
if strings.TrimSpace(ip) == clientIP {
return true
}
}
g.Log().Warningf(ctx, "回调IP不在白名单中, clientIP=%s", clientIP)
return false
}
// ImageCallback 图片校验回调
func (c *MaterialVerifyController) ImageCallback(ctx context.Context, req *ImageCallbackReq) (res *CallbackRes, err error) {
if !checkIP(ctx) {
if !internal.CheckCallbackIP(ctx) {
return &CallbackRes{Code: 403, Msg: "IP not allowed"}, nil
}
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "yidun_callback", TenantId: 1})
ctx = internal.WithCallbackUser(ctx)
if req.CallbackData == "" {
return &CallbackRes{Code: 400, Msg: "callbackData不能为空"}, nil
}
@@ -581,15 +577,15 @@ func (c *MaterialVerifyController) ImageCallback(ctx context.Context, req *Image
return &CallbackRes{Code: 500, Msg: err.Error()}, nil
}
return &CallbackRes{Code: 200, Msg: "处理成功"}, nil
return &CallbackRes{Code: 0, Msg: "处理成功"}, nil
}
// VideoCallback 视频校验回调
func (c *MaterialVerifyController) VideoCallback(ctx context.Context, req *VideoCallbackReq) (res *CallbackRes, err error) {
if !checkIP(ctx) {
if !internal.CheckCallbackIP(ctx) {
return &CallbackRes{Code: 403, Msg: "IP not allowed"}, nil
}
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "yidun_callback", TenantId: 1})
ctx = internal.WithCallbackUser(ctx)
if req.CallbackData == "" {
return &CallbackRes{Code: 400, Msg: "callbackData不能为空"}, nil
}
@@ -599,7 +595,7 @@ func (c *MaterialVerifyController) VideoCallback(ctx context.Context, req *Video
return &CallbackRes{Code: 500, Msg: err.Error()}, nil
}
return &CallbackRes{Code: 200, Msg: "处理成功"}, nil
return &CallbackRes{Code: 0, Msg: "处理成功"}, nil
}
// ResultRes 结果查询响应
@@ -610,7 +606,7 @@ type ResultRes struct {
// ImageResult 图片校验结果查询(轮询模式)
func (c *MaterialVerifyController) ImageResult(ctx context.Context, req *TaskIDReq) (res *ResultRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.TaskID == "" {
return &ResultRes{Code: 400, Msg: "taskId不能为空"}, nil
}
@@ -620,12 +616,12 @@ func (c *MaterialVerifyController) ImageResult(ctx context.Context, req *TaskIDR
return &ResultRes{Code: 500, Msg: err.Error()}, nil
}
return &ResultRes{Code: 200, Msg: "处理成功"}, nil
return &ResultRes{Code: 0, Msg: "处理成功"}, nil
}
// VideoResult 视频校验结果查询(轮询模式)
func (c *MaterialVerifyController) VideoResult(ctx context.Context, req *TaskIDReq) (res *ResultRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.TaskID == "" {
return &ResultRes{Code: 400, Msg: "taskId不能为空"}, nil
}
@@ -635,5 +631,5 @@ func (c *MaterialVerifyController) VideoResult(ctx context.Context, req *TaskIDR
return &ResultRes{Code: 500, Msg: err.Error()}, nil
}
return &ResultRes{Code: 200, Msg: "处理成功"}, nil
return &ResultRes{Code: 0, Msg: "处理成功"}, nil
}
+42
View File
@@ -0,0 +1,42 @@
// Package internal 提供 controller 层的共享工具函数
package internal
import (
"context"
"strings"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
)
// WithAdminUser 在 context 中注入 admin 用户信息
func WithAdminUser(ctx context.Context) context.Context {
return context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
}
// WithCallbackUser 在 context 中注入 yidun_callback 用户信息
func WithCallbackUser(ctx context.Context) context.Context {
return context.WithValue(ctx, "user", &beans.User{UserName: "yidun_callback", TenantId: 1})
}
// CheckCallbackIP 校验回调请求 IP 是否在白名单内
// 读取配置 yidun.callback_allowed_ips,若未配置则跳过校验
func CheckCallbackIP(ctx context.Context) bool {
r := ghttp.RequestFromCtx(ctx)
if r == nil {
return true
}
allowedIPs := g.Cfg().MustGet(ctx, "yidun.callback_allowed_ips", "").String()
if allowedIPs == "" {
return true
}
clientIP := r.GetClientIp()
for _, ip := range strings.Split(allowedIPs, ",") {
if strings.TrimSpace(ip) == clientIP {
return true
}
}
g.Log().Warningf(ctx, "回调IP不在白名单中, clientIP=%s, allowedIPs=%s", clientIP, allowedIPs)
return false
}
+12 -11
View File
@@ -1,6 +1,7 @@
package yidun
import (
internal "cid/controller/internal"
dto "cid/model/dto/yidun"
serviceDataengine "cid/service/dataengine"
"context"
@@ -24,7 +25,7 @@ type StatusRes struct {
// Start 启动送检服务
func (c *ContentCheckController) Start(ctx context.Context, req *dto.StartCheckReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if serviceDataengine.TencentContentCheck.IsRunning() {
return &beans.ResponseEmpty{}, nil
@@ -50,14 +51,14 @@ func (c *ContentCheckController) Start(ctx context.Context, req *dto.StartCheckR
// Stop 停止送检服务
func (c *ContentCheckController) Stop(ctx context.Context, req *dto.EmptyReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
serviceDataengine.TencentContentCheck.Stop(ctx)
return
}
// Status 获取送检服务状态
func (c *ContentCheckController) Status(ctx context.Context, req *dto.EmptyReq) (res *StatusRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
res = &StatusRes{
Running: serviceDataengine.TencentContentCheck.IsRunning(),
@@ -69,7 +70,7 @@ func (c *ContentCheckController) Status(ctx context.Context, req *dto.EmptyReq)
// ProcessImageCallback 处理图片检测回调
func (c *ContentCheckController) ProcessImageCallback(ctx context.Context, req *dto.ProcessImageCallbackReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.CallbackData == "" {
return nil, fmt.Errorf("callbackData不能为空")
@@ -81,7 +82,7 @@ func (c *ContentCheckController) ProcessImageCallback(ctx context.Context, req *
// ProcessVideoCallback 处理视频检测回调
func (c *ContentCheckController) ProcessVideoCallback(ctx context.Context, req *dto.ProcessVideoCallbackReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.CallbackData == "" {
return nil, fmt.Errorf("callbackData不能为空")
@@ -93,7 +94,7 @@ func (c *ContentCheckController) ProcessVideoCallback(ctx context.Context, req *
// ProcessImageResult 查询并处理图片检测结果(轮询模式)
func (c *ContentCheckController) ProcessImageResult(ctx context.Context, req *dto.ProcessImageResultReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.TaskID == "" {
return nil, fmt.Errorf("taskId不能为空")
@@ -105,7 +106,7 @@ func (c *ContentCheckController) ProcessImageResult(ctx context.Context, req *dt
// ProcessVideoResult 查询并处理视频检测结果(轮询模式)
func (c *ContentCheckController) ProcessVideoResult(ctx context.Context, req *dto.ProcessVideoResultReq) (res *beans.ResponseEmpty, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
if req.TaskID == "" {
return nil, fmt.Errorf("taskId不能为空")
@@ -117,7 +118,7 @@ func (c *ContentCheckController) ProcessVideoResult(ctx context.Context, req *dt
// ManualSubmitImageByID 根据图片ID手动提交送检
func (c *ContentCheckController) ManualSubmitImageByID(ctx context.Context, req *dto.ManualSubmitImageByIDReq) (res *dto.ManualSubmitRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
result, err := serviceDataengine.TencentContentCheck.SubmitImageByID(ctx, req.ImageID)
if err != nil {
@@ -132,7 +133,7 @@ func (c *ContentCheckController) ManualSubmitImageByID(ctx context.Context, req
// ManualSubmitVideoByID 根据视频ID手动提交送检
func (c *ContentCheckController) ManualSubmitVideoByID(ctx context.Context, req *dto.ManualSubmitVideoByIDReq) (res *dto.ManualSubmitRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
result, err := serviceDataengine.TencentContentCheck.SubmitVideoByID(ctx, req.VideoID)
if err != nil {
@@ -147,7 +148,7 @@ func (c *ContentCheckController) ManualSubmitVideoByID(ctx context.Context, req
// GetImageCheckLogs 获取图片的送检日志
func (c *ContentCheckController) GetImageCheckLogs(ctx context.Context, req *dto.GetImageCheckLogsReq) (res *dto.GetCheckLogsRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
logs, err := serviceDataengine.TencentContentCallback.GetCheckLogsByImageID(ctx, req.ImageID)
if err != nil {
@@ -162,7 +163,7 @@ func (c *ContentCheckController) GetImageCheckLogs(ctx context.Context, req *dto
// GetVideoCheckLogs 获取视频的送检日志
func (c *ContentCheckController) GetVideoCheckLogs(ctx context.Context, req *dto.GetVideoCheckLogsReq) (res *dto.GetCheckLogsRes, err error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
logs, err := serviceDataengine.TencentContentCallback.GetCheckLogsByVideoID(ctx, req.VideoID)
if err != nil {
+63 -49
View File
@@ -1,12 +1,10 @@
package yidun
import (
internal "cid/controller/internal"
dataengineService "cid/service/dataengine"
"context"
"fmt"
"strings"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
)
@@ -42,13 +40,13 @@ type PollResult struct {
// Body: callbackData={"antispam":{...}}
func (c *YidunCallbackController) ReceiveImageCallback(r *ghttp.Request) {
// IP 白名单校验
if !checkCallbackIP(r) {
if !internal.CheckCallbackIP(r.Context()) {
r.Response.WriteJson(CallbackResult{Code: 403, Msg: "IP not allowed"})
return
}
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "yidun_callback", TenantId: 1})
ctx = internal.WithCallbackUser(ctx)
// 易盾推送的数据在请求体中
var callbackData string
@@ -86,7 +84,7 @@ func (c *YidunCallbackController) ReceiveImageCallback(r *ghttp.Request) {
return
}
r.Response.WriteJson(CallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(CallbackResult{Code: 0, Msg: "success"})
}
// ReceiveVideoCallback 接收易盾视频检测结果推送
@@ -94,13 +92,13 @@ func (c *YidunCallbackController) ReceiveImageCallback(r *ghttp.Request) {
// Body: callbackData={"antispam":{...}}
func (c *YidunCallbackController) ReceiveVideoCallback(r *ghttp.Request) {
// IP 白名单校验
if !checkCallbackIP(r) {
if !internal.CheckCallbackIP(r.Context()) {
r.Response.WriteJson(CallbackResult{Code: 403, Msg: "IP not allowed"})
return
}
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "yidun_callback", TenantId: 1})
ctx = internal.WithCallbackUser(ctx)
// 易盾推送的数据在请求体中
var callbackData string
@@ -138,7 +136,7 @@ func (c *YidunCallbackController) ReceiveVideoCallback(r *ghttp.Request) {
return
}
r.Response.WriteJson(CallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(CallbackResult{Code: 0, Msg: "success"})
}
// =============================================================================
@@ -149,35 +147,40 @@ func (c *YidunCallbackController) ReceiveVideoCallback(r *ghttp.Request) {
// 格式: POST /yidun/callback/poll
func (c *YidunCallbackController) PollAllResults(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
g.Log().Info(ctx, "开始轮询所有待查询的检测结果...")
// 先获取待处理数量
pendingCount, _ := dataengineService.MaterialVerify.GetPendingResultsCount(ctx)
// 执行轮询
successCount, failCount, err := dataengineService.MaterialVerify.PollPendingResults(ctx)
result := PollResult{
SuccessCount: successCount,
FailCount: failCount,
PendingCount: pendingCount - successCount,
}
// 轮询后再查一下剩余待处理的明细
pendingItems, _ := dataengineService.MaterialVerify.GetPendingResultsDetail(ctx, 50)
msg := fmt.Sprintf("✅ 成功处理 %d 条 | ❌ 失败 %d 条 | ⏳ 还剩 %d 条待处理",
successCount, failCount, len(pendingItems))
if err != nil {
r.Response.WriteJson(CallbackResult{
Code: 500,
Msg: fmt.Sprintf("轮询完成但有错误: %v", err),
Data: result,
Msg: fmt.Sprintf("部分完成但有错误: %v", err),
Data: g.Map{
"summary": g.Map{"success": successCount, "fail": failCount, "pending": len(pendingItems)},
"pending_detail": pendingItems,
},
})
return
}
r.Response.WriteJson(CallbackResult{
Code: 200,
Msg: fmt.Sprintf("轮询完成,成功处理 %d 条,失败 %d 条", successCount, failCount),
Data: result,
Code: 0,
Msg: msg,
Data: g.Map{
"summary": g.Map{"success": successCount, "fail": failCount, "pending": len(pendingItems)},
"pending_detail": pendingItems,
},
})
}
@@ -185,7 +188,7 @@ func (c *YidunCallbackController) PollAllResults(r *ghttp.Request) {
// 格式: POST /yidun/callback/pollImage
func (c *YidunCallbackController) PollImageResults(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
g.Log().Info(ctx, "开始轮询图片待查询的检测结果...")
@@ -201,7 +204,7 @@ func (c *YidunCallbackController) PollImageResults(r *ghttp.Request) {
}
r.Response.WriteJson(CallbackResult{
Code: 200,
Code: 0,
Msg: fmt.Sprintf("轮询完成,成功处理 %d 条,失败 %d 条", successCount, failCount),
Data: PollResult{SuccessCount: successCount, FailCount: failCount},
})
@@ -211,7 +214,7 @@ func (c *YidunCallbackController) PollImageResults(r *ghttp.Request) {
// 格式: POST /yidun/callback/pollVideo
func (c *YidunCallbackController) PollVideoResults(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
g.Log().Info(ctx, "开始轮询视频待查询的检测结果...")
@@ -227,7 +230,7 @@ func (c *YidunCallbackController) PollVideoResults(r *ghttp.Request) {
}
r.Response.WriteJson(CallbackResult{
Code: 200,
Code: 0,
Msg: fmt.Sprintf("轮询完成,成功处理 %d 条,失败 %d 条", successCount, failCount),
Data: PollResult{SuccessCount: successCount, FailCount: failCount},
})
@@ -237,7 +240,7 @@ func (c *YidunCallbackController) PollVideoResults(r *ghttp.Request) {
// 格式: POST /yidun/callback/pollTask
func (c *YidunCallbackController) PollByTaskID(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
taskID := r.Get("taskId", "").String()
taskType := r.Get("type", "").String() // image 或 video
@@ -268,14 +271,41 @@ func (c *YidunCallbackController) PollByTaskID(r *ghttp.Request) {
return
}
r.Response.WriteJson(CallbackResult{Code: 200, Msg: "查询并处理成功"})
r.Response.WriteJson(CallbackResult{Code: 0, Msg: "查询并处理成功"})
}
// PendingListRes 待查询结果明细
type PendingListRes struct {
Total int `json:"total"`
List []dataengineService.PendingResultItem `json:"list"`
}
// GetPendingDetail 获取待查询结果的明细
// 格式: GET /yidun/callback/pendingDetail
func (c *YidunCallbackController) GetPendingDetail(r *ghttp.Request) {
ctx := r.Context()
ctx = internal.WithAdminUser(ctx)
items, err := dataengineService.MaterialVerify.GetPendingResultsDetail(ctx, 50)
if err != nil {
r.Response.WriteJson(CallbackResult{Code: 500, Msg: err.Error()})
return
}
r.Response.WriteJson(g.Map{
"code": 0,
"data": PendingListRes{
Total: len(items),
List: items,
},
})
}
// GetPendingCount 获取待查询结果的数量
// 格式: GET /yidun/callback/pendingCount
func (c *YidunCallbackController) GetPendingCount(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
count, err := dataengineService.MaterialVerify.GetPendingResultsCount(ctx)
if err != nil {
@@ -284,7 +314,7 @@ func (c *YidunCallbackController) GetPendingCount(r *ghttp.Request) {
}
r.Response.WriteJson(g.Map{
"code": 200,
"code": 0,
"data": g.Map{
"pending_count": count,
"description": "待查询结果的日志数量(状态为pending且有taskID",
@@ -300,7 +330,7 @@ func (c *YidunCallbackController) GetPendingCount(r *ghttp.Request) {
// 格式: POST /yidun/callback/processImage
func (c *YidunCallbackController) ProcessImageCallback(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
var req struct {
CallbackData string `json:"callbackData" v:"required#回调数据不能为空"`
@@ -317,14 +347,14 @@ func (c *YidunCallbackController) ProcessImageCallback(r *ghttp.Request) {
return
}
r.Response.WriteJson(CallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(CallbackResult{Code: 0, Msg: "success"})
}
// ProcessVideoCallback 手动处理视频回调(兼容旧接口)
// 格式: POST /yidun/callback/processVideo
func (c *YidunCallbackController) ProcessVideoCallback(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
var req struct {
CallbackData string `json:"callbackData" v:"required#回调数据不能为空"`
@@ -341,7 +371,7 @@ func (c *YidunCallbackController) ProcessVideoCallback(r *ghttp.Request) {
return
}
r.Response.WriteJson(CallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(CallbackResult{Code: 0, Msg: "success"})
}
// toString 转换interface{}为string
@@ -351,19 +381,3 @@ func toString(v interface{}) string {
}
return ""
}
// checkCallbackIP 校验回调请求IP是否在白名单内
func checkCallbackIP(r *ghttp.Request) bool {
allowedIPs := g.Cfg().MustGet(r.Context(), "yidun.callback_allowed_ips", "").String()
if allowedIPs == "" {
return true // 未配置白名单,跳过校验
}
clientIP := r.GetClientIp()
for _, ip := range strings.Split(allowedIPs, ",") {
if strings.TrimSpace(ip) == clientIP {
return true
}
}
g.Log().Warningf(r.Context(), "回调IP不在白名单中, clientIP=%s, allowedIPs=%s", clientIP, allowedIPs)
return false
}
+15 -14
View File
@@ -1,10 +1,11 @@
package yidun
import (
internal "cid/controller/internal"
serviceDataengine "cid/service/dataengine"
"cid/service/yidun"
"context"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/net/ghttp"
"github.com/yidun/yidun-golang-sdk/yidun/service/antispam/text/v5/check/async/single"
@@ -39,7 +40,7 @@ type DetectVideoReq struct {
// DetectText 文本检测
func (c *yidunController) DetectText(ctx context.Context, req *DetectTextReq) (string, error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
businessId := g.Cfg().MustGet(ctx, "yidun.text.business_id").String()
sdkReq := single.NewTextAsyncCheckRequest(businessId)
@@ -57,13 +58,13 @@ func (c *yidunController) DetectText(ctx context.Context, req *DetectTextReq) (s
// DetectImage 图片检测
func (c *yidunController) DetectImage(ctx context.Context, req *DetectImageReq) (*yidun.ImageSubmitResult, error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
return yidun.ImageDetection.DetectImage(ctx, req.ImageURL, req.DataID, req.CallbackURL)
}
// DetectVideo 视频检测
func (c *yidunController) DetectVideo(ctx context.Context, req *DetectVideoReq) (*yidun.VideoSubmitResult, error) {
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
return yidun.VideoDetection.DetectVideo(ctx, req.VideoURL, req.DataID, req.CallbackURL)
}
@@ -76,13 +77,13 @@ type ImageCallbackResult struct {
// ReceiveImageCallback 接收图片检测结果推送
func (c *yidunController) ReceiveImageCallback(r *ghttp.Request) {
// IP 白名单校验
if !checkCallbackIP(r) {
if !internal.CheckCallbackIP(r.Context()) {
r.Response.WriteJson(ImageCallbackResult{Code: 403, Msg: "IP not allowed"})
return
}
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
callbackData := r.GetForm("callbackData", "").String()
if callbackData == "" {
@@ -90,14 +91,14 @@ func (c *yidunController) ReceiveImageCallback(r *ghttp.Request) {
return
}
err := yidun.ImageDetection.ProcessImageCallback(ctx, callbackData)
err := serviceDataengine.MaterialVerify.ProcessImageCallback(ctx, callbackData)
if err != nil {
g.Log().Errorf(ctx, "处理图片检测回调失败: %v", err)
r.Response.WriteJson(ImageCallbackResult{Code: 500, Msg: err.Error()})
return
}
r.Response.WriteJson(ImageCallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(ImageCallbackResult{Code: 0, Msg: "success"})
}
// VideoCallbackResult 视频检测回调响应
@@ -109,13 +110,13 @@ type VideoCallbackResult struct {
// ReceiveVideoCallback 接收视频检测结果推送
func (c *yidunController) ReceiveVideoCallback(r *ghttp.Request) {
// IP 白名单校验
if !checkCallbackIP(r) {
if !internal.CheckCallbackIP(r.Context()) {
r.Response.WriteJson(VideoCallbackResult{Code: 403, Msg: "IP not allowed"})
return
}
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
callbackData := r.GetForm("callbackData", "").String()
if callbackData == "" {
@@ -123,20 +124,20 @@ func (c *yidunController) ReceiveVideoCallback(r *ghttp.Request) {
return
}
err := yidun.VideoDetection.ProcessVideoCallback(ctx, callbackData)
err := serviceDataengine.MaterialVerify.ProcessVideoCallback(ctx, callbackData)
if err != nil {
g.Log().Errorf(ctx, "处理视频检测回调失败: %v", err)
r.Response.WriteJson(VideoCallbackResult{Code: 500, Msg: err.Error()})
return
}
r.Response.WriteJson(VideoCallbackResult{Code: 200, Msg: "success"})
r.Response.WriteJson(VideoCallbackResult{Code: 0, Msg: "success"})
}
// GetVideoResult 获取视频检测结果
func (c *yidunController) GetVideoResult(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
taskId := r.Get("taskId", "").String()
if taskId == "" {
@@ -157,7 +158,7 @@ func (c *yidunController) GetVideoResult(r *ghttp.Request) {
// GetImageResult 获取图片检测结果
func (c *yidunController) GetImageResult(r *ghttp.Request) {
ctx := r.Context()
ctx = context.WithValue(ctx, "user", &beans.User{UserName: "admin", TenantId: 1})
ctx = internal.WithAdminUser(ctx)
taskId := r.Get("taskId", "").String()
if taskId == "" {
+30 -18
View File
@@ -1,11 +1,9 @@
package dataengine
import (
consts "cid/consts/dataengine"
daoEntity "cid/model/entity/dataengine"
"context"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/bwmarrin/snowflake"
"github.com/gogf/gf/v2/frame/g"
)
@@ -51,7 +49,7 @@ func (d *MaterialVerifyLogDAO) Create(ctx context.Context, log *daoEntity.Materi
// GetByID 根据ID获取日志
func (d *MaterialVerifyLogDAO) GetByID(ctx context.Context, id int64) (*daoEntity.MaterialVerifyLog, error) {
var result daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
One()
if err != nil {
@@ -69,7 +67,7 @@ func (d *MaterialVerifyLogDAO) GetByID(ctx context.Context, id int64) (*daoEntit
// GetByTaskID 根据任务ID获取日志
func (d *MaterialVerifyLogDAO) GetByTaskID(ctx context.Context, taskID string) (*daoEntity.MaterialVerifyLog, error) {
var result daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.TaskID, taskID).
One()
if err != nil {
@@ -87,7 +85,7 @@ func (d *MaterialVerifyLogDAO) GetByTaskID(ctx context.Context, taskID string) (
// GetByMaterialID 根据素材ID获取日志列表
func (d *MaterialVerifyLogDAO) GetByMaterialID(ctx context.Context, materialID string) ([]daoEntity.MaterialVerifyLog, error) {
var result []daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.MaterialID, materialID).
OrderDesc(daoEntity.MaterialVerifyLogCols.CreatedAt).
All()
@@ -103,7 +101,7 @@ func (d *MaterialVerifyLogDAO) GetByMaterialID(ctx context.Context, materialID s
// GetBySource 根据来源获取日志
func (d *MaterialVerifyLogDAO) GetBySource(ctx context.Context, sourceTable string, sourceID int64) ([]daoEntity.MaterialVerifyLog, error) {
var result []daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.SourceTable, sourceTable).
Where(daoEntity.MaterialVerifyLogCols.SourceID, sourceID).
OrderDesc(daoEntity.MaterialVerifyLogCols.CreatedAt).
@@ -119,7 +117,7 @@ func (d *MaterialVerifyLogDAO) GetBySource(ctx context.Context, sourceTable stri
// UpdateVerifyResult 更新校验结果
func (d *MaterialVerifyLogDAO) UpdateVerifyResult(ctx context.Context, id int64, verifyStatus string, suggestion, label, resultType int, responseResult string, checkTime int64) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.VerifyStatus: verifyStatus,
@@ -138,7 +136,7 @@ func (d *MaterialVerifyLogDAO) UpdateVerifyResult(ctx context.Context, id int64,
// UpdateError 更新错误信息
func (d *MaterialVerifyLogDAO) UpdateError(ctx context.Context, id int64, verifyStatus string, errorMsg string) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.VerifyStatus: verifyStatus,
@@ -153,7 +151,7 @@ func (d *MaterialVerifyLogDAO) UpdateError(ctx context.Context, id int64, verify
// UpdateTaskID 更新任务ID
func (d *MaterialVerifyLogDAO) UpdateTaskID(ctx context.Context, id int64, taskID string) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.TaskID: taskID,
@@ -166,7 +164,7 @@ func (d *MaterialVerifyLogDAO) UpdateTaskID(ctx context.Context, id int64, taskI
// UpdateDuration 更新处理耗时
func (d *MaterialVerifyLogDAO) UpdateDuration(ctx context.Context, id int64, durationMs int64) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.DurationMs: durationMs,
@@ -179,7 +177,7 @@ func (d *MaterialVerifyLogDAO) UpdateDuration(ctx context.Context, id int64, dur
// UpdateRequestParams 更新请求参数
func (d *MaterialVerifyLogDAO) UpdateRequestParams(ctx context.Context, id int64, requestParams string) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.RequestParams: requestParams,
@@ -193,7 +191,7 @@ func (d *MaterialVerifyLogDAO) UpdateRequestParams(ctx context.Context, id int64
// GetByCondition 根据条件分页查询
func (d *MaterialVerifyLogDAO) GetByCondition(ctx context.Context, condition map[string]interface{}, page, pageSize int) ([]daoEntity.MaterialVerifyLog, int, error) {
var result []daoEntity.MaterialVerifyLog
m := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable)
m := g.DB("cid").Model(MaterialVerifyLogTable)
for k, v := range condition {
m.Where(k, v)
@@ -219,7 +217,7 @@ func (d *MaterialVerifyLogDAO) GetByCondition(ctx context.Context, condition map
// CountByStatus 按状态统计
func (d *MaterialVerifyLogDAO) CountByStatus(ctx context.Context, verifyStatus string) (int, error) {
count, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
count, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, verifyStatus).
Count()
if err != nil {
@@ -259,8 +257,8 @@ func (d *MaterialVerifyLogDAO) GetStats(ctx context.Context) (map[string]int, er
func (d *MaterialVerifyLogDAO) GetPendingResults(ctx context.Context, limit int) ([]daoEntity.MaterialVerifyLog, error) {
var result []daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, consts.CheckStatusPending).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, daoEntity.VerifyStatusPending).
WhereNotNull(daoEntity.MaterialVerifyLogCols.TaskID).
Where(daoEntity.MaterialVerifyLogCols.TaskID + " != ''").
OrderAsc(daoEntity.MaterialVerifyLogCols.CreatedAt).
@@ -280,7 +278,7 @@ func (d *MaterialVerifyLogDAO) GetPendingResults(ctx context.Context, limit int)
// GetLastRejectedLogByMaterialID 根据素材ID获取最后一条失败的校验日志
func (d *MaterialVerifyLogDAO) GetLastRejectedLogByMaterialID(ctx context.Context, materialID string, verifyStatus string) (*daoEntity.MaterialVerifyLog, error) {
var result daoEntity.MaterialVerifyLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
r, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.MaterialID, materialID).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, verifyStatus).
OrderDesc(daoEntity.MaterialVerifyLogCols.CreatedAt).
@@ -297,10 +295,24 @@ func (d *MaterialVerifyLogDAO) GetLastRejectedLogByMaterialID(ctx context.Contex
return &result, nil
}
// UpdateRiskDescription 更新风险描述
func (d *MaterialVerifyLogDAO) UpdateRiskDescription(ctx context.Context, id int64, riskDescription string) error {
_, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.Id, id).
Data(g.Map{
daoEntity.MaterialVerifyLogCols.RiskDescription: riskDescription,
}).Update()
if err != nil {
g.Log().Errorf(ctx, "更新风险描述失败: %v", err)
return err
}
return nil
}
// CountPendingResults 统计待查询结果的数量
func (d *MaterialVerifyLogDAO) CountPendingResults(ctx context.Context) (int, error) {
count, err := gfdb.DB(ctx, "cid").Model(ctx, MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, consts.CheckStatusPending).
count, err := g.DB("cid").Model(MaterialVerifyLogTable).
Where(daoEntity.MaterialVerifyLogCols.VerifyStatus, daoEntity.VerifyStatusPending).
WhereNotNull(daoEntity.MaterialVerifyLogCols.TaskID).
Where(daoEntity.MaterialVerifyLogCols.TaskID + " != ''").
Count()
@@ -7,7 +7,6 @@ import (
"context"
"encoding/json"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/bwmarrin/snowflake"
"github.com/gogf/gf/v2/frame/g"
)
@@ -53,7 +52,7 @@ func (d *TencentContentCheckLogDAO) Create(ctx context.Context, log *entity.Tenc
// UpdateStatus 更新送检状态
func (d *TencentContentCheckLogDAO) UpdateStatus(ctx context.Context, id int64, status string, responseData string, failReason string) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
_, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("id", id).
Data(g.Map{
"status": status,
@@ -65,7 +64,7 @@ func (d *TencentContentCheckLogDAO) UpdateStatus(ctx context.Context, id int64,
// UpdateCheckResult 更新检测结果
func (d *TencentContentCheckLogDAO) UpdateCheckResult(ctx context.Context, id int64, suggestion, label, resultType int, checkTime int64) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
_, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("id", id).
Data(g.Map{
"status": consts.CheckStatusCompleted,
@@ -80,7 +79,7 @@ func (d *TencentContentCheckLogDAO) UpdateCheckResult(ctx context.Context, id in
// GetByID 根据ID获取日志
func (d *TencentContentCheckLogDAO) GetByID(ctx context.Context, id int64) (*entity.TencentContentCheckLog, error) {
var result entity.TencentContentCheckLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
r, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("id", id).
One()
if err != nil {
@@ -98,7 +97,7 @@ func (d *TencentContentCheckLogDAO) GetByID(ctx context.Context, id int64) (*ent
// GetBySourceID 根据来源ID获取日志
func (d *TencentContentCheckLogDAO) GetBySourceID(ctx context.Context, sourceTable string, sourceID int64) ([]entity.TencentContentCheckLog, error) {
var result []entity.TencentContentCheckLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
r, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("source_table", sourceTable).
Where("source_id", sourceID).
OrderDesc("created_at").
@@ -115,7 +114,7 @@ func (d *TencentContentCheckLogDAO) GetBySourceID(ctx context.Context, sourceTab
// GetByTaskID 根据任务ID获取日志
func (d *TencentContentCheckLogDAO) GetByTaskID(ctx context.Context, taskID string) (*entity.TencentContentCheckLog, error) {
var result entity.TencentContentCheckLog
r, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
r, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("task_id", taskID).
One()
if err != nil {
@@ -133,7 +132,7 @@ func (d *TencentContentCheckLogDAO) GetByTaskID(ctx context.Context, taskID stri
// ListByStatus 根据状态获取日志列表
func (d *TencentContentCheckLogDAO) ListByStatus(ctx context.Context, status string, page, pageSize int) ([]entity.TencentContentCheckLog, int, error) {
var result []entity.TencentContentCheckLog
m := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable)
m := g.DB("cid").Model(consts.TencentContentCheckLogTable)
if status != "" {
m.Where("status", status)
@@ -159,7 +158,7 @@ func (d *TencentContentCheckLogDAO) ListByStatus(ctx context.Context, status str
// UpdateDuration 更新耗时
func (d *TencentContentCheckLogDAO) UpdateDuration(ctx context.Context, id int64, duration int64) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
_, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("id", id).
Data("duration", duration).
Update()
@@ -168,7 +167,7 @@ func (d *TencentContentCheckLogDAO) UpdateDuration(ctx context.Context, id int64
// UpdateTaskID 更新任务ID
func (d *TencentContentCheckLogDAO) UpdateTaskID(ctx context.Context, id int64, taskID string) error {
_, err := gfdb.DB(ctx, "cid").Model(ctx, consts.TencentContentCheckLogTable).
_, err := g.DB("cid").Model(consts.TencentContentCheckLogTable).
Where("id", id).
Data("task_id", taskID).
Update()
+16
View File
@@ -121,6 +121,22 @@ func (d *TencentImageDAO) GetByCondition(ctx context.Context, condition map[stri
return result, int(total), nil
}
// ClaimPending 原子地尝试将图片从 PENDING 状态转为 SUBMITTING
// 返回 true 表示成功抢到处理权,false 表示已被其他进程处理
func (d *TencentImageDAO) ClaimPending(ctx context.Context, id int64) (bool, error) {
result, err := Model(consts.TencentImageTable).
Where(entity.TencentImageCols.Id, id).
Where(entity.TencentImageCols.VerifyStatus, consts.CheckStatusPending).
Data(entity.TencentImageCols.VerifyStatus, consts.CheckStatusSubmitting).
Update()
if err != nil {
g.Log().Errorf(ctx, "原子认领图片送检失败: %v", err)
return false, err
}
affected, _ := result.RowsAffected()
return affected > 0, nil
}
// UpdateStatus 更新图片校验状态
func (d *TencentImageDAO) UpdateStatus(ctx context.Context, id int64, verifyStatus string) (int64, error) {
result, err := Model(consts.TencentImageTable).
+16
View File
@@ -121,6 +121,22 @@ func (d *TencentVideoDAO) GetByCondition(ctx context.Context, condition map[stri
return result, int(total), nil
}
// ClaimPending 原子地尝试将视频从 PENDING 状态转为 SUBMITTING
// 返回 true 表示成功抢到处理权,false 表示已被其他进程处理
func (d *TencentVideoDAO) ClaimPending(ctx context.Context, id int64) (bool, error) {
result, err := Model(consts.TencentVideoTable).
Where(entity.TencentVideoCols.Id, id).
Where(entity.TencentVideoCols.VerifyStatus, consts.CheckStatusPending).
Data(entity.TencentVideoCols.VerifyStatus, consts.CheckStatusSubmitting).
Update()
if err != nil {
g.Log().Errorf(ctx, "原子认领视频送检失败: %v", err)
return false, err
}
affected, _ := result.RowsAffected()
return affected > 0, nil
}
// UpdateStatus 更新视频校验状态
func (d *TencentVideoDAO) UpdateStatus(ctx context.Context, id int64, verifyStatus string) (int64, error) {
result, err := Model(consts.TencentVideoTable).
+1
View File
@@ -126,6 +126,7 @@ func startContentCheckService(ctx context.Context) {
ImageEnabled: g.Cfg().MustGet(ctx, "content_check.image_enabled", true).Bool(),
VideoEnabled: g.Cfg().MustGet(ctx, "content_check.video_enabled", true).Bool(),
IntervalSeconds: g.Cfg().MustGet(ctx, "content_check.interval_seconds", 30).Int(),
PollInterval: g.Cfg().MustGet(ctx, "content_check.poll_interval", 60).Int(),
}
serviceDataengine.TencentContentCheck.SetConfig(config)
+52 -49
View File
@@ -8,22 +8,23 @@ import (
type MaterialVerifyLog struct {
beans.SQLBaseDO `orm:",inherit"`
// 业务字段
TenantID int64 `orm:"tenant_id" json:"tenantId" description:"租户ID"`
MaterialType string `orm:"material_type" json:"materialType" description:"素材类型 IMAGE/VIDEO"`
MaterialID string `orm:"material_id" json:"materialId" description:"素材ID"`
SourceTable string `orm:"source_table" json:"sourceTable" description:"来源表"`
SourceID int64 `orm:"source_id" json:"sourceId" description:"原表主键ID"`
AccountID int64 `orm:"account_id" json:"accountId" description:"账户ID"`
TaskID string `orm:"task_id" json:"taskId" description:"易盾任务ID"`
RequestParams string `orm:"request_params" json:"requestParams" description:"请求入参"`
ResponseResult string `orm:"response_result" json:"responseResult" description:"响应出参"`
VerifyStatus string `orm:"verify_status" json:"verifyStatus" description:"校验状态"`
Suggestion int `orm:"suggestion" json:"suggestion" description:"处置建议"`
Label int `orm:"label" json:"label" description:"垃圾类型"`
ResultType int `orm:"result_type" json:"resultType" description:"结果类型"`
ErrorMsg string `orm:"error_msg" json:"errorMsg" description:"错误信息"`
CheckTime int64 `orm:"check_time" json:"checkTime" description:"审核时间戳"`
DurationMs int64 `orm:"duration_ms" json:"durationMs" description:"处理耗时(毫秒)"`
TenantID int64 `orm:"tenant_id" json:"tenantId" description:"租户ID"`
MaterialType string `orm:"material_type" json:"materialType" description:"素材类型 IMAGE/VIDEO"`
MaterialID string `orm:"material_id" json:"materialId" description:"素材ID"`
SourceTable string `orm:"source_table" json:"sourceTable" description:"来源表"`
SourceID int64 `orm:"source_id" json:"sourceId" description:"原表主键ID"`
AccountID int64 `orm:"account_id" json:"accountId" description:"账户ID"`
TaskID string `orm:"task_id" json:"taskId" description:"易盾任务ID"`
RequestParams string `orm:"request_params" json:"requestParams" description:"请求入参"`
ResponseResult string `orm:"response_result" json:"responseResult" description:"响应出参"`
VerifyStatus string `orm:"verify_status" json:"verifyStatus" description:"校验状态"`
Suggestion int `orm:"suggestion" json:"suggestion" description:"处置建议"`
Label int `orm:"label" json:"label" description:"垃圾类型"`
ResultType int `orm:"result_type" json:"resultType" description:"结果类型"`
ErrorMsg string `orm:"error_msg" json:"errorMsg" description:"错误信息"`
CheckTime int64 `orm:"check_time" json:"checkTime" description:"审核时间戳"`
DurationMs int64 `orm:"duration_ms" json:"durationMs" description:"处理耗时(毫秒)"`
RiskDescription string `orm:"risk_description" json:"riskDescription" description:"风险描述(易盾返回)"`
// 扩展字段(用于展示)
PreviewURL string `orm:"-" json:"previewUrl" description:"预览URL"`
@@ -32,43 +33,45 @@ type MaterialVerifyLog struct {
// MaterialVerifyLogCol 日志表字段定义
type MaterialVerifyLogCol struct {
beans.SQLBaseCol
TenantID string
MaterialType string
MaterialID string
SourceTable string
SourceID string
AccountID string
TaskID string
RequestParams string
ResponseResult string
VerifyStatus string
Suggestion string
Label string
ResultType string
ErrorMsg string
CheckTime string
DurationMs string
TenantID string
MaterialType string
MaterialID string
SourceTable string
SourceID string
AccountID string
TaskID string
RequestParams string
ResponseResult string
VerifyStatus string
Suggestion string
Label string
ResultType string
ErrorMsg string
CheckTime string
DurationMs string
RiskDescription string
}
// MaterialVerifyLogCols 日志表字段常量
var MaterialVerifyLogCols = MaterialVerifyLogCol{
SQLBaseCol: beans.DefSQLBaseCol,
TenantID: "tenant_id",
MaterialType: "material_type",
MaterialID: "material_id",
SourceTable: "source_table",
SourceID: "source_id",
AccountID: "account_id",
TaskID: "task_id",
RequestParams: "request_params",
ResponseResult: "response_result",
VerifyStatus: "verify_status",
Suggestion: "suggestion",
Label: "label",
ResultType: "result_type",
ErrorMsg: "error_msg",
CheckTime: "check_time",
DurationMs: "duration_ms",
SQLBaseCol: beans.DefSQLBaseCol,
TenantID: "tenant_id",
MaterialType: "material_type",
MaterialID: "material_id",
SourceTable: "source_table",
SourceID: "source_id",
AccountID: "account_id",
TaskID: "task_id",
RequestParams: "request_params",
ResponseResult: "response_result",
VerifyStatus: "verify_status",
Suggestion: "suggestion",
Label: "label",
ResultType: "result_type",
ErrorMsg: "error_msg",
CheckTime: "check_time",
DurationMs: "duration_ms",
RiskDescription: "risk_description",
}
// 素材类型常量
+165 -96
View File
@@ -20,12 +20,6 @@ const (
PollBatchSize = 20
)
// 状态常量
const (
// 原表状态 - 与 tencent_image/tencent_video 表的 status 字段对应
StatusSubmitting = consts.CheckStatusSubmitting // 送检中
)
// MaterialVerifyService 素材校验服务
type MaterialVerifyService struct{}
@@ -55,31 +49,39 @@ func SuggestionToVerifyStatus(suggestion int) string {
// =============================================================================
// VerifyImageByID 根据图片ID执行校验
// 使用原子 CAS 防止并发重复送检
func (s *MaterialVerifyService) VerifyImageByID(ctx context.Context, imageID string) (*entity.MaterialVerifyLog, error) {
image, err := dao.TencentImage.GetByImageID(ctx, imageID)
if err != nil {
return nil, fmt.Errorf("unknown error: %w", err)
return nil, fmt.Errorf("查询图片数据失败, imageID=%s: %w", imageID, err)
}
if image == nil {
return nil, fmt.Errorf("未找到图片数据, imageID=%s", imageID)
}
// 幂等性检查:如果已在送检中,直接返回已有日志
if image.VerifyStatus == consts.CheckStatusSubmitting {
// 原子 CASPENDING → SUBMITTING,确保只有第一个调用者能抢到处理权
claimed, err := dao.TencentImage.ClaimPending(ctx, image.Id)
if err != nil {
return nil, fmt.Errorf("认领图片送检失败, imageID=%s: %w", imageID, err)
}
if !claimed {
logs, err := dao.MaterialVerifyLog.GetByMaterialID(ctx, imageID)
if err == nil && len(logs) > 0 {
g.Log().Infof(ctx, "图片已送检, imageID=%s, logId=%d", imageID, logs[0].Id)
g.Log().Infof(ctx, "图片已被其他进程送检, imageID=%s, logId=%d", imageID, logs[0].Id)
return &logs[0], nil
}
return nil, fmt.Errorf("图片正在送检中且无校验日志, imageID=%s", imageID)
}
log := s.createVerifyLog(ctx, entity.MaterialTypeImage, imageID, consts.SourceTableTencentImage, image.Id, image.AccountID)
if log == nil {
dao.TencentImage.UpdateStatus(ctx, image.Id, entity.VerifyStatusPending)
return nil, fmt.Errorf("创建校验日志失败")
}
err = s.submitImageCheck(ctx, image, log)
if err != nil {
dao.TencentImage.UpdateStatus(ctx, image.Id, entity.VerifyStatusPending)
return nil, err
}
@@ -113,13 +115,14 @@ func (s *MaterialVerifyService) submitImageCheck(ctx context.Context, image *ent
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending, err.Error())
dao.MaterialVerifyLog.UpdateDuration(ctx, log.Id, duration)
g.Log().Warningf(ctx, "图片异步检测失败(保持待检验), id=%d, imageId=%s, error=%v", image.Id, image.ImageID, err)
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("图片异步检测提交失败, imageId=%s: %w", image.ImageID, err)
}
taskID = result.TaskID
dao.MaterialVerifyLog.UpdateTaskID(ctx, log.Id, taskID)
dao.MaterialVerifyLog.UpdateRequestParams(ctx, log.Id, string(requestParamsJSON))
s.updateImageStatus(ctx, image.Id, StatusSubmitting)
TencentContentCheck.writeAuditLog(ctx, consts.SourceTableTencentImage, image.Id, image.ImageID, image.PreviewURL, taskID, -1, 0, 0, "", duration)
g.Log().Infof(ctx, "图片异步检测已提交, id=%d, imageId=%s, taskId=%s, duration=%dms",
image.Id, image.ImageID, taskID, duration)
@@ -130,7 +133,7 @@ func (s *MaterialVerifyService) submitImageCheck(ctx context.Context, image *ent
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending, err.Error())
dao.MaterialVerifyLog.UpdateDuration(ctx, log.Id, duration)
g.Log().Warningf(ctx, "图片同步检测失败(保持待检验), id=%d, imageId=%s, error=%v", image.Id, image.ImageID, err)
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("图片同步检测失败, imageId=%s: %w", image.ImageID, err)
}
taskID = syncResult.TaskID
@@ -143,6 +146,8 @@ func (s *MaterialVerifyService) submitImageCheck(ctx context.Context, image *ent
syncResult.Suggestion, syncResult.Label, syncResult.ResultType, string(responseJSON), syncResult.CensorTime)
s.updateImageStatus(ctx, image.Id, verifyStatus)
TencentContentCheck.writeAuditLog(ctx, consts.SourceTableTencentImage, image.Id, image.ImageID, image.PreviewURL, taskID, syncResult.Suggestion, syncResult.Label, syncResult.ResultType, string(responseJSON), duration)
g.Log().Infof(ctx, "图片同步检测完成, id=%d, imageId=%s, taskId=%s, suggestion=%d, verifyStatus=%s, duration=%dms",
image.Id, image.ImageID, taskID, syncResult.Suggestion, verifyStatus, duration)
}
@@ -155,31 +160,38 @@ func (s *MaterialVerifyService) submitImageCheck(ctx context.Context, image *ent
// =============================================================================
// VerifyVideoByID 根据视频ID执行校验
// 使用原子 CAS 防止并发重复送检
func (s *MaterialVerifyService) VerifyVideoByID(ctx context.Context, videoID string) (*entity.MaterialVerifyLog, error) {
video, err := dao.TencentVideo.GetByVideoID(ctx, videoID)
if err != nil {
return nil, fmt.Errorf("unknown error: %w", err)
return nil, fmt.Errorf("查询视频数据失败, videoID=%s: %w", videoID, err)
}
if video == nil {
return nil, fmt.Errorf("未找到视频数据, videoID=%s", videoID)
}
// 幂等性检查:如果已在送检中,直接返回已有日志
if video.VerifyStatus == consts.CheckStatusSubmitting {
claimed, err := dao.TencentVideo.ClaimPending(ctx, video.Id)
if err != nil {
return nil, fmt.Errorf("认领视频送检失败, videoID=%s: %w", videoID, err)
}
if !claimed {
logs, err := dao.MaterialVerifyLog.GetByMaterialID(ctx, videoID)
if err == nil && len(logs) > 0 {
g.Log().Infof(ctx, "视频已送检, videoID=%s, logId=%d", videoID, logs[0].Id)
g.Log().Infof(ctx, "视频已被其他进程送检, videoID=%s, logId=%d", videoID, logs[0].Id)
return &logs[0], nil
}
return nil, fmt.Errorf("视频正在送检中且无校验日志, videoID=%s", videoID)
}
log := s.createVerifyLog(ctx, entity.MaterialTypeVideo, videoID, consts.SourceTableTencentVideo, video.Id, video.AccountID)
if log == nil {
dao.TencentVideo.UpdateStatus(ctx, video.Id, entity.VerifyStatusPending)
return nil, fmt.Errorf("创建校验日志失败")
}
err = s.submitVideoCheck(ctx, video, log)
if err != nil {
dao.TencentVideo.UpdateStatus(ctx, video.Id, entity.VerifyStatusPending)
return nil, err
}
@@ -211,21 +223,18 @@ func (s *MaterialVerifyService) submitVideoCheck(ctx context.Context, video *ent
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending, err.Error())
dao.MaterialVerifyLog.UpdateDuration(ctx, log.Id, duration)
g.Log().Warningf(ctx, "视频校验接口调用失败(保持待检验), id=%d, videoId=%s, error=%v", video.Id, video.VideoID, err)
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("视频检测提交失败, videoId=%s: %w", video.VideoID, err)
}
dao.MaterialVerifyLog.UpdateTaskID(ctx, log.Id, result.TaskID)
dao.MaterialVerifyLog.UpdateRequestParams(ctx, log.Id, string(requestParamsJSON))
s.updateVideoStatus(ctx, video.Id, StatusSubmitting)
if !callbackMode {
g.Log().Infof(ctx, "轮询模式:提交后立即查询结果, taskId=%s", result.TaskID)
time.Sleep(500 * time.Millisecond)
if err := s.ProcessVideoResultByTask(ctx, result.TaskID); err != nil {
g.Log().Warningf(ctx, "提交后立即查询结果失败(不影响状态,后续轮询继续), taskId=%s, error=%v", result.TaskID, err)
}
g.Log().Infof(ctx, "轮询模式:视频检测已提交, taskId=%s, 请通过轮询接口获取结果", result.TaskID)
}
TencentContentCheck.writeAuditLog(ctx, consts.SourceTableTencentVideo, video.Id, video.VideoID, video.PreviewURL, result.TaskID, -1, 0, 0, "", duration)
g.Log().Infof(ctx, "视频校验已提交, id=%d, videoId=%s, taskId=%s, duration=%dms",
video.Id, video.VideoID, result.TaskID, duration)
@@ -243,7 +252,7 @@ func (s *MaterialVerifyService) ProcessImageCallback(ctx context.Context, callba
var callback yidunService.ImageCallbackData
if err := json.Unmarshal([]byte(callbackData), &callback); err != nil {
g.Log().Errorf(ctx, "解析图片回调数据失败: %v", err)
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("解析图片回调数据失败: %w", err)
}
if callback.Antispam == nil {
@@ -256,7 +265,7 @@ func (s *MaterialVerifyService) ProcessImageCallback(ctx context.Context, callba
log, err := dao.MaterialVerifyLog.GetByTaskID(ctx, antispam.TaskId)
if err != nil {
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("查询图片校验日志失败, taskId=%s: %w", antispam.TaskId, err)
}
if log == nil {
g.Log().Warningf(ctx, "未找到校验日志, taskId=%s", antispam.TaskId)
@@ -268,13 +277,20 @@ func (s *MaterialVerifyService) ProcessImageCallback(ctx context.Context, callba
err = dao.MaterialVerifyLog.UpdateVerifyResult(ctx, log.Id, verifyStatus,
antispam.Suggestion, antispam.Label, antispam.ResultType, callbackData, antispam.CensorTime)
if err != nil {
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("更新图片校验日志结果失败, logId=%d: %w", log.Id, err)
}
if log.SourceTable == consts.SourceTableTencentImage {
s.updateImageStatus(ctx, log.SourceID, verifyStatus)
}
// 更新送检审计日志
s.updateCheckLogResult(ctx, antispam.TaskId, antispam.Suggestion, antispam.Label, antispam.ResultType, callbackData)
// 提取风险描述
if antispam.RiskDescription != "" {
_ = dao.MaterialVerifyLog.UpdateRiskDescription(ctx, log.Id, antispam.RiskDescription)
}
g.Log().Infof(ctx, "图片校验回调处理完成, taskId=%s, verifyStatus=%s, suggestion=%d",
antispam.TaskId, verifyStatus, antispam.Suggestion)
@@ -288,7 +304,7 @@ func (s *MaterialVerifyService) ProcessVideoCallback(ctx context.Context, callba
var callback yidunService.VideoCallbackData
if err := json.Unmarshal([]byte(callbackData), &callback); err != nil {
g.Log().Errorf(ctx, "解析视频回调数据失败: %v", err)
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("解析视频回调数据失败: %w", err)
}
if callback.Antispam == nil {
@@ -301,7 +317,7 @@ func (s *MaterialVerifyService) ProcessVideoCallback(ctx context.Context, callba
log, err := dao.MaterialVerifyLog.GetByTaskID(ctx, antispam.TaskID)
if err != nil {
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("查询视频校验日志失败, taskId=%s: %w", antispam.TaskID, err)
}
if log == nil {
g.Log().Warningf(ctx, "未找到校验日志, taskId=%s", antispam.TaskID)
@@ -318,23 +334,47 @@ func (s *MaterialVerifyService) ProcessVideoCallback(ctx context.Context, callba
err = dao.MaterialVerifyLog.UpdateVerifyResult(ctx, log.Id, verifyStatus,
antispam.Suggestion, antispam.Label, antispam.ResultType, callbackData, checkTime)
if err != nil {
return fmt.Errorf("unknown error: %w", err)
return fmt.Errorf("更新视频校验日志结果失败, logId=%d: %w", log.Id, err)
}
if log.SourceTable == consts.SourceTableTencentVideo {
s.updateVideoStatus(ctx, log.SourceID, verifyStatus)
}
// 更新送检审计日志
s.updateCheckLogResult(ctx, antispam.TaskID, antispam.Suggestion, antispam.Label, antispam.ResultType, callbackData)
// 提取风险描述
if antispam.RiskDescription != "" {
_ = dao.MaterialVerifyLog.UpdateRiskDescription(ctx, log.Id, antispam.RiskDescription)
}
g.Log().Infof(ctx, "视频校验回调处理完成, taskId=%s, verifyStatus=%s, suggestion=%d",
antispam.TaskID, verifyStatus, antispam.Suggestion)
return nil
}
// updateCheckLogResult 更新 tencent_content_check_log 的检测结果
func (s *MaterialVerifyService) updateCheckLogResult(ctx context.Context, taskID string, suggestion, label, resultType int, responseData string) {
checkLog, err := dao.TencentContentCheckLog.GetByTaskID(ctx, taskID)
if err != nil {
g.Log().Warningf(ctx, "查询送检审计日志失败, taskId=%s: %v", taskID, err)
return
}
if checkLog == nil {
g.Log().Debugf(ctx, "送检审计日志不存在, taskId=%s(可能是通过 API 直接提交的)", taskID)
return
}
_ = dao.TencentContentCheckLog.UpdateCheckResult(ctx, checkLog.Id, suggestion, label, resultType, time.Now().UnixMilli())
}
// =============================================================================
// 轮询模式处理
// =============================================================================
// ErrResultPending 表示检测结果尚未就绪,非错误状态
var ErrResultPending = fmt.Errorf("检测结果尚未就绪")
// 易盾检测状态常量
const (
YidunStatusNotStart = 0 // 未开始
@@ -344,9 +384,13 @@ const (
)
// ProcessImageResultByTask 根据任务ID处理图片结果(轮询模式)
// 返回 nil 表示结果已处理完成,返回 ErrResultPending 表示仍未就绪
func (s *MaterialVerifyService) ProcessImageResultByTask(ctx context.Context, taskID string) error {
log, err := dao.MaterialVerifyLog.GetByTaskID(ctx, taskID)
if err != nil || log == nil {
if err != nil {
return fmt.Errorf("查询校验日志失败, taskId=%s: %w", taskID, err)
}
if log == nil {
return fmt.Errorf("未找到校验日志, taskId=%s", taskID)
}
@@ -354,23 +398,23 @@ func (s *MaterialVerifyService) ProcessImageResultByTask(ctx context.Context, ta
if err != nil {
if err == yidunService.ErrImageResultNotFound || err == yidunService.ErrImageStillProcessing {
g.Log().Infof(ctx, "图片检测结果未就绪, taskId=%s, 保持pending状态, err=%v", taskID, err)
return nil
return ErrResultPending
}
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending, err.Error())
g.Log().Warningf(ctx, "图片检测查询失败(保持待检验), taskId=%s, error=%v", taskID, err)
return nil
return ErrResultPending
}
if result.Status == YidunStatusProcessing || result.Status == YidunStatusNotStart {
g.Log().Infof(ctx, "图片检测仍在进行中, taskId=%s, status=%d, 保持pending状态", taskID, result.Status)
return nil
return ErrResultPending
}
if result.Status == YidunStatusFailed {
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending,
fmt.Sprintf("易盾检测失败, status=%d", result.Status))
g.Log().Warningf(ctx, "图片检测失败(保持待检验), taskId=%s, status=%d", taskID, result.Status)
return nil
return ErrResultPending
}
verifyStatus := SuggestionToVerifyStatus(result.Suggestion)
@@ -383,6 +427,14 @@ func (s *MaterialVerifyService) ProcessImageResultByTask(ctx context.Context, ta
s.updateImageStatus(ctx, log.SourceID, verifyStatus)
}
// 提取风险描述
if result.Antispam != nil && result.Antispam.RiskDescription != nil {
_ = dao.MaterialVerifyLog.UpdateRiskDescription(ctx, log.Id, *result.Antispam.RiskDescription)
}
// 更新送检审计日志
s.updateCheckLogResult(ctx, taskID, result.Suggestion, result.Label, result.ResultType, string(responseJSON))
g.Log().Infof(ctx, "图片检测结果更新成功, taskId=%s, status=%d, suggestion=%d, verifyStatus=%s",
taskID, result.Status, result.Suggestion, verifyStatus)
return nil
@@ -391,7 +443,10 @@ func (s *MaterialVerifyService) ProcessImageResultByTask(ctx context.Context, ta
// ProcessVideoResultByTask 根据任务ID处理视频结果(轮询模式)
func (s *MaterialVerifyService) ProcessVideoResultByTask(ctx context.Context, taskID string) error {
log, err := dao.MaterialVerifyLog.GetByTaskID(ctx, taskID)
if err != nil || log == nil {
if err != nil {
return fmt.Errorf("查询校验日志失败, taskId=%s: %w", taskID, err)
}
if log == nil {
return fmt.Errorf("未找到校验日志, taskId=%s", taskID)
}
@@ -399,23 +454,23 @@ func (s *MaterialVerifyService) ProcessVideoResultByTask(ctx context.Context, ta
if err != nil {
if err == yidunService.ErrVideoResultNotFound || err == yidunService.ErrVideoStillProcessing {
g.Log().Infof(ctx, "视频检测结果未就绪, taskId=%s, 保持pending状态, err=%v", taskID, err)
return nil
return ErrResultPending
}
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending, err.Error())
g.Log().Warningf(ctx, "视频检测查询失败(保持待检验), taskId=%s, error=%v", taskID, err)
return nil
return ErrResultPending
}
if result.Status == YidunStatusProcessing || result.Status == YidunStatusNotStart {
g.Log().Infof(ctx, "视频检测仍在进行中, taskId=%s, status=%d, 保持pending状态", taskID, result.Status)
return nil
return ErrResultPending
}
if result.Status == YidunStatusFailed {
dao.MaterialVerifyLog.UpdateError(ctx, log.Id, entity.VerifyStatusPending,
fmt.Sprintf("易盾检测失败, status=%d", result.Status))
g.Log().Warningf(ctx, "视频检测失败(保持待检验), taskId=%s, status=%d", taskID, result.Status)
return nil
return ErrResultPending
}
verifyStatus := SuggestionToVerifyStatus(result.Suggestion)
@@ -428,6 +483,14 @@ func (s *MaterialVerifyService) ProcessVideoResultByTask(ctx context.Context, ta
s.updateVideoStatus(ctx, log.SourceID, verifyStatus)
}
// 提取风险描述
if result.Antispam != nil && result.Antispam.RiskDescription != nil {
_ = dao.MaterialVerifyLog.UpdateRiskDescription(ctx, log.Id, *result.Antispam.RiskDescription)
}
// 更新送检审计日志
s.updateCheckLogResult(ctx, taskID, result.Suggestion, result.Label, result.ResultType, string(responseJSON))
g.Log().Infof(ctx, "视频检测结果更新成功, taskId=%s, status=%d, suggestion=%d, verifyStatus=%s",
taskID, result.Status, result.Suggestion, verifyStatus)
return nil
@@ -439,7 +502,6 @@ func (s *MaterialVerifyService) ProcessVideoResultByTask(ctx context.Context, ta
// createVerifyLog 创建校验日志
func (s *MaterialVerifyService) createVerifyLog(ctx context.Context, materialType, materialID, sourceTable string, sourceID, accountID int64) *entity.MaterialVerifyLog {
// 从上下文提取租户ID
var tenantID int64
if user := ctx.Value("user"); user != nil {
if u, ok := user.(*beans.User); ok {
@@ -467,7 +529,7 @@ func (s *MaterialVerifyService) createVerifyLog(ctx context.Context, materialTyp
return log
}
// updateImageStatus 更新图片状态(已记录日志则同步更新,失败仅记录日志不影响主流程)
// updateImageStatus 更新图片状态
func (s *MaterialVerifyService) updateImageStatus(ctx context.Context, imageID int64, verifyStatus string) {
_, err := dao.TencentImage.UpdateStatus(ctx, imageID, verifyStatus)
if err != nil {
@@ -477,7 +539,7 @@ func (s *MaterialVerifyService) updateImageStatus(ctx context.Context, imageID i
}
}
// updateVideoStatus 更新视频状态(已记录日志则同步更新,失败仅记录日志不影响主流程)
// updateVideoStatus 更新视频状态
func (s *MaterialVerifyService) updateVideoStatus(ctx context.Context, videoID int64, verifyStatus string) {
_, err := dao.TencentVideo.UpdateStatus(ctx, videoID, verifyStatus)
if err != nil {
@@ -515,10 +577,8 @@ func (s *MaterialVerifyService) GetStats(ctx context.Context) (map[string]int, e
// 轮询模式 - 批量查询检测结果
// =============================================================================
// PollPendingResults 轮询所有待查询结果的日志(手动触发)
// 返回处理成功的数量和错误信息
// PollPendingResults 轮询所有待查询结果的日志
func (s *MaterialVerifyService) PollPendingResults(ctx context.Context) (int, int, error) {
// 获取待查询的日志
logs, err := dao.MaterialVerifyLog.GetPendingResults(ctx, PollBatchSize)
if err != nil {
return 0, 0, err
@@ -538,7 +598,6 @@ func (s *MaterialVerifyService) PollPendingResults(ctx context.Context) (int, in
for _, log := range logs {
var err error
// 根据来源表判断调用哪个接口
if log.SourceTable == consts.SourceTableTencentImage {
err = s.ProcessImageResultByTask(ctx, log.TaskID)
} else if log.SourceTable == consts.SourceTableTencentVideo {
@@ -548,7 +607,9 @@ func (s *MaterialVerifyService) PollPendingResults(ctx context.Context) (int, in
continue
}
if err != nil {
if err == ErrResultPending {
g.Log().Infof(ctx, "结果未就绪, logId=%d, taskId=%s", log.Id, log.TaskID)
} else if err != nil {
failCount++
lastErr = err
g.Log().Warningf(ctx, "处理结果失败, logId=%d, taskId=%s, error=%v", log.Id, log.TaskID, err)
@@ -557,23 +618,20 @@ func (s *MaterialVerifyService) PollPendingResults(ctx context.Context) (int, in
g.Log().Infof(ctx, "处理结果成功, logId=%d, taskId=%s", log.Id, log.TaskID)
}
// 避免请求过快
time.Sleep(100 * time.Millisecond)
}
g.Log().Infof(ctx, "轮询完成, 成功=%d, 失败=%d", successCount, failCount)
g.Log().Infof(ctx, "轮询完成, 成功=%d, 失败=%d, 未就绪=%d", successCount, failCount, len(logs)-successCount-failCount)
return successCount, failCount, lastErr
}
// PollPendingResultsByType 按类型轮询待查询结果的日志
func (s *MaterialVerifyService) PollPendingResultsByType(ctx context.Context, sourceTable string) (int, int, error) {
// 获取待查询的日志
logs, err := dao.MaterialVerifyLog.GetPendingResults(ctx, PollBatchSize)
if err != nil {
return 0, 0, err
}
// 过滤指定类型
var filteredLogs []entity.MaterialVerifyLog
for _, log := range logs {
if log.SourceTable == sourceTable {
@@ -586,8 +644,6 @@ func (s *MaterialVerifyService) PollPendingResultsByType(ctx context.Context, so
return 0, 0, nil
}
g.Log().Infof(ctx, "开始轮询 %d 条待处理结果, sourceTable=%s", len(filteredLogs), sourceTable)
successCount := 0
failCount := 0
var lastErr error
@@ -611,7 +667,6 @@ func (s *MaterialVerifyService) PollPendingResultsByType(ctx context.Context, so
time.Sleep(100 * time.Millisecond)
}
g.Log().Infof(ctx, "轮询完成, sourceTable=%s, 成功=%d, 失败=%d", sourceTable, successCount, failCount)
return successCount, failCount, lastErr
}
@@ -631,16 +686,16 @@ func (s *MaterialVerifyService) PollPendingVideoResults(ctx context.Context) (in
// ExportRejectedItem 导出的不通过数据项
type ExportRejectedItem struct {
ID int64 `json:"id"` // 素材表主键ID
MaterialID string `json:"materialId"` // 素材IDimageId/videoId
AccountID int64 `json:"accountId"` // 账户ID
CorporationName string `json:"corporationName"` // 公司名称
PreviewURL string `json:"previewUrl"` // 预览URL
Description string `json:"description"` // 描述
ErrorMsg string `json:"errorMsg"` // 失败原因(最后一条失败日志的error_msg)
MaterialType string `json:"materialType"` // 素材类型 IMAGE/VIDEO
ImageUsage string `json:"imageUsage"` // 图片用途(仅图片)
CreatedAt string `json:"createdAt"` // 检测时间(日志创建时间)
ID int64 `json:"id"`
MaterialID string `json:"materialId"`
AccountID int64 `json:"accountId"`
CorporationName string `json:"corporationName"`
PreviewURL string `json:"previewUrl"`
Description string `json:"description"`
ErrorMsg string `json:"errorMsg"`
MaterialType string `json:"materialType"`
ImageUsage string `json:"imageUsage,omitempty"`
CreatedAt string `json:"createdAt"`
}
// getFailureReason 获取失败原因
@@ -648,11 +703,9 @@ func getFailureReason(log *entity.MaterialVerifyLog) string {
if log == nil {
return "无校验日志"
}
// 优先使用 error_msg
if log.ErrorMsg != "" {
return log.ErrorMsg
}
// 根据 suggestion 和 label 生成原因
reasonMap := map[int]string{
0: "内容检测通过",
1: "内容嫌疑(需人工审核)",
@@ -662,7 +715,6 @@ func getFailureReason(log *entity.MaterialVerifyLog) string {
if suggestionText == "" {
suggestionText = fmt.Sprintf("未知(suggestion=%d)", log.Suggestion)
}
// 如果有 response_result,尝试提取更多信息
if log.ResponseResult != "" {
var resultMap map[string]interface{}
if err := json.Unmarshal([]byte(log.ResponseResult), &resultMap); err == nil {
@@ -677,7 +729,7 @@ func getFailureReason(log *entity.MaterialVerifyLog) string {
const exportBatchSize = 1000
// ExportRejectedData 导出不通过数据(分批加载,避免OOM
// ExportRejectedData 导出不通过数据
func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, materialType string) ([]ExportRejectedItem, error) {
var items []ExportRejectedItem
@@ -694,15 +746,13 @@ func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, material
condition := map[string]interface{}{
entity.TencentImageCols.VerifyStatus: entity.VerifyStatusRejected,
}
page := 1
for {
images, total, err := dao.TencentImage.GetByCondition(ctx, condition, page, exportBatchSize)
if err != nil {
g.Log().Errorf(ctx, "查询不通过图片失败: %v", err)
return nil, fmt.Errorf("unknown error: %w", err)
return nil, fmt.Errorf("查询不通过图片失败: %w", err)
}
for _, img := range images {
log, _ := dao.MaterialVerifyLog.GetLastRejectedLogByMaterialID(ctx, img.ImageID, entity.VerifyStatusRejected)
var createdAtStr string
@@ -710,19 +760,13 @@ func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, material
createdAtStr = log.CreatedAt.Format("2006-01-02 15:04:05")
}
items = append(items, ExportRejectedItem{
ID: img.Id,
MaterialID: img.ImageID,
AccountID: img.AccountID,
CorporationName: accountMap[img.AccountID],
PreviewURL: img.PreviewURL,
Description: img.Description,
ErrorMsg: getFailureReason(log),
MaterialType: entity.MaterialTypeImage,
ImageUsage: img.ImageUsage,
CreatedAt: createdAtStr,
ID: img.Id, MaterialID: img.ImageID, AccountID: img.AccountID,
CorporationName: accountMap[img.AccountID], PreviewURL: img.PreviewURL,
Description: img.Description, ErrorMsg: getFailureReason(log),
MaterialType: entity.MaterialTypeImage, ImageUsage: img.ImageUsage,
CreatedAt: createdAtStr,
})
}
if page*exportBatchSize >= total {
break
}
@@ -734,15 +778,13 @@ func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, material
condition := map[string]interface{}{
entity.TencentVideoCols.VerifyStatus: entity.VerifyStatusRejected,
}
page := 1
for {
videos, total, err := dao.TencentVideo.GetByCondition(ctx, condition, page, exportBatchSize)
if err != nil {
g.Log().Errorf(ctx, "查询不通过视频失败: %v", err)
return nil, fmt.Errorf("unknown error: %w", err)
return nil, fmt.Errorf("查询不通过视频失败: %w", err)
}
for _, vid := range videos {
log, _ := dao.MaterialVerifyLog.GetLastRejectedLogByMaterialID(ctx, vid.VideoID, entity.VerifyStatusRejected)
var createdAtStr string
@@ -750,18 +792,12 @@ func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, material
createdAtStr = log.CreatedAt.Format("2006-01-02 15:04:05")
}
items = append(items, ExportRejectedItem{
ID: vid.Id,
MaterialID: vid.VideoID,
AccountID: vid.AccountID,
CorporationName: accountMap[vid.AccountID],
PreviewURL: vid.PreviewURL,
Description: vid.Description,
ErrorMsg: getFailureReason(log),
MaterialType: entity.MaterialTypeVideo,
CreatedAt: createdAtStr,
ID: vid.Id, MaterialID: vid.VideoID, AccountID: vid.AccountID,
CorporationName: accountMap[vid.AccountID], PreviewURL: vid.PreviewURL,
Description: vid.Description, ErrorMsg: getFailureReason(log),
MaterialType: entity.MaterialTypeVideo, CreatedAt: createdAtStr,
})
}
if page*exportBatchSize >= total {
break
}
@@ -776,3 +812,36 @@ func (s *MaterialVerifyService) ExportRejectedData(ctx context.Context, material
func (s *MaterialVerifyService) GetPendingResultsCount(ctx context.Context) (int, error) {
return dao.MaterialVerifyLog.CountPendingResults(ctx)
}
// GetPendingResultsDetail 获取待查询结果的明细列表
type PendingResultItem struct {
LogID int64 `json:"logId"`
MaterialID string `json:"materialId"`
MaterialType string `json:"materialType"`
SourceTable string `json:"sourceTable"`
TaskID string `json:"taskId"`
CreatedAt string `json:"createdAt"`
}
func (s *MaterialVerifyService) GetPendingResultsDetail(ctx context.Context, limit int) ([]PendingResultItem, error) {
logs, err := dao.MaterialVerifyLog.GetPendingResults(ctx, limit)
if err != nil {
return nil, err
}
var items []PendingResultItem
for _, l := range logs {
createdAt := ""
if l.CreatedAt != nil {
createdAt = l.CreatedAt.Format("2006-01-02 15:04:05")
}
items = append(items, PendingResultItem{
LogID: l.Id,
MaterialID: l.MaterialID,
MaterialType: l.MaterialType,
SourceTable: l.SourceTable,
TaskID: l.TaskID,
CreatedAt: createdAt,
})
}
return items, nil
}
@@ -7,10 +7,12 @@ import (
yidunService "cid/service/yidun"
"context"
"encoding/json"
"sync"
"time"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/os/grpool"
)
// ContentCheckConfig 送检配置
@@ -19,6 +21,7 @@ type ContentCheckConfig struct {
ImageEnabled bool `json:"image_enabled"`
VideoEnabled bool `json:"video_enabled"`
IntervalSeconds int `json:"interval_seconds"`
PollInterval int `json:"poll_interval"` // 自动轮询检测结果间隔(秒)
}
// DefaultConfig 默认配置
@@ -27,12 +30,16 @@ var DefaultConfig = ContentCheckConfig{
ImageEnabled: true,
VideoEnabled: true,
IntervalSeconds: 30,
PollInterval: 60,
}
// TencentContentCheckService 腾讯内容送检服务
type TencentContentCheckService struct {
mu sync.RWMutex
config ContentCheckConfig
isRunning bool
pool *grpool.Pool
cancel context.CancelFunc
}
// TencentContentCheck 送检服务单例
@@ -42,47 +49,92 @@ var TencentContentCheck = &TencentContentCheckService{
// SetConfig 设置配置
func (s *TencentContentCheckService) SetConfig(config ContentCheckConfig) {
s.mu.Lock()
s.config = config
s.mu.Unlock()
}
// Start 启动定时任务
func (s *TencentContentCheckService) Start(ctx context.Context) error {
s.mu.Lock()
if s.isRunning {
s.mu.Unlock()
g.Log().Info(ctx, "送检服务已在运行中,跳过启动")
return nil
}
s.isRunning = true
g.Log().Infof(ctx, "启动内容送检服务,配置: batch_size=%d, interval=%ds, image=%v, video=%v",
s.config.BatchSize, s.config.IntervalSeconds, s.config.ImageEnabled, s.config.VideoEnabled)
config := s.config
s.pool = grpool.New(5)
s.mu.Unlock()
schedCtx := context.Background()
if user := ctx.Value("user"); user != nil {
schedCtx = context.WithValue(schedCtx, "user", user)
g.Log().Infof(ctx, "启动内容送检服务,配置: batch_size=%d, interval=%ds, poll=%ds, image=%v, video=%v",
config.BatchSize, config.IntervalSeconds, config.PollInterval, config.ImageEnabled, config.VideoEnabled)
schedCtx, cancel := context.WithCancel(context.Background())
s.cancel = cancel
// 定时送检协程
g.Go(schedCtx, func(ctx context.Context) {
ticker := time.NewTicker(time.Duration(config.IntervalSeconds) * time.Second)
defer ticker.Stop()
// 启动时先执行一次
_ = s.pool.Add(ctx, func(jobCtx context.Context) {
s.processAll(jobCtx)
})
for {
select {
case <-ticker.C:
_ = s.pool.Add(ctx, func(jobCtx context.Context) {
s.processAll(jobCtx)
})
case <-ctx.Done():
s.pool.Close()
return
}
}
}, nil)
// 自动轮询检测结果协程(无论回调/轮询模式,都定期查一次未处理的结果)
pollInterval := config.PollInterval
if pollInterval <= 0 {
pollInterval = 60
}
go s.runScheduler(schedCtx)
g.Go(schedCtx, func(ctx context.Context) {
pollTicker := time.NewTicker(time.Duration(pollInterval) * time.Second)
defer pollTicker.Stop()
g.Log().Infof(ctx, "启动自动轮询检测结果, 间隔=%ds", pollInterval)
for {
select {
case <-pollTicker.C:
_, _, _ = MaterialVerify.PollPendingResults(ctx)
case <-ctx.Done():
return
}
}
}, nil)
return nil
}
// Stop 停止定时任务
func (s *TencentContentCheckService) Stop(ctx context.Context) {
s.isRunning = false
g.Log().Info(ctx, "停止内容送检服务")
}
// runScheduler 定时调度器
func (s *TencentContentCheckService) runScheduler(ctx context.Context) {
ticker := time.NewTicker(time.Duration(s.config.IntervalSeconds) * time.Second)
defer ticker.Stop()
s.processAll(ctx)
for range ticker.C {
if !s.isRunning {
return
}
s.processAll(ctx)
s.mu.Lock()
if !s.isRunning {
s.mu.Unlock()
return
}
s.isRunning = false
s.mu.Unlock()
if s.cancel != nil {
s.cancel()
}
g.Log().Info(ctx, "停止内容送检服务")
}
// processAll 处理所有待送检数据
@@ -94,18 +146,33 @@ func (s *TencentContentCheckService) processAll(ctx context.Context) {
var totalProcessed int
if s.config.ImageEnabled {
imageCount, _ := dao.TencentImage.CountPending(ctx)
if imageCount > 0 {
count, _ := s.processImages(ctx)
s.mu.RLock()
imageEnabled := s.config.ImageEnabled
videoEnabled := s.config.VideoEnabled
s.mu.RUnlock()
if imageEnabled {
imageCount, err := dao.TencentImage.CountPending(ctx)
if err != nil {
g.Log().Errorf(ctx, "统计待送检图片数量失败: %v", err)
} else if imageCount > 0 {
count, procErr := s.processImages(ctx)
if procErr != nil {
g.Log().Errorf(ctx, "图片送检处理失败: %v", procErr)
}
totalProcessed += count
}
}
if s.config.VideoEnabled {
videoCount, _ := dao.TencentVideo.CountPending(ctx)
if videoCount > 0 {
count, _ := s.processVideos(ctx)
if videoEnabled {
videoCount, err := dao.TencentVideo.CountPending(ctx)
if err != nil {
g.Log().Errorf(ctx, "统计待送检视频数量失败: %v", err)
} else if videoCount > 0 {
count, procErr := s.processVideos(ctx)
if procErr != nil {
g.Log().Errorf(ctx, "视频送检处理失败: %v", procErr)
}
totalProcessed += count
}
}
@@ -116,7 +183,11 @@ func (s *TencentContentCheckService) processAll(ctx context.Context) {
// processImages 处理图片送检(统一走 MaterialVerify 系统)
func (s *TencentContentCheckService) processImages(ctx context.Context) (int, error) {
images, err := dao.TencentImage.GetPendingList(ctx, s.config.BatchSize)
s.mu.RLock()
batchSize := s.config.BatchSize
s.mu.RUnlock()
images, err := dao.TencentImage.GetPendingList(ctx, batchSize)
if err != nil {
g.Log().Errorf(ctx, "获取待送检图片失败: %v", err)
return 0, err
@@ -132,15 +203,12 @@ func (s *TencentContentCheckService) processImages(ctx context.Context) (int, er
failedCount := 0
for _, img := range images {
// 统一走 MaterialVerify 系统提交(处理完整校验流程:日志→提交→状态反写)
mLog, err := MaterialVerify.VerifyImageByID(ctx, img.ImageID)
_, err := MaterialVerify.VerifyImageByID(ctx, img.ImageID)
if err != nil {
failedCount++
g.Log().Errorf(ctx, "图片送检失败, imageId=%s, error=%v", img.ImageID, err)
} else {
successCount++
// 审计日志:同步写入 tencent_content_check_log
s.writeAuditLog(ctx, consts.SourceTableTencentImage, img.Id, img.ImageID, img.PreviewURL, mLog.TaskID)
}
time.Sleep(100 * time.Millisecond)
@@ -152,7 +220,11 @@ func (s *TencentContentCheckService) processImages(ctx context.Context) (int, er
// processVideos 处理视频送检(统一走 MaterialVerify 系统)
func (s *TencentContentCheckService) processVideos(ctx context.Context) (int, error) {
videos, err := dao.TencentVideo.GetPendingList(ctx, s.config.BatchSize)
s.mu.RLock()
batchSize := s.config.BatchSize
s.mu.RUnlock()
videos, err := dao.TencentVideo.GetPendingList(ctx, batchSize)
if err != nil {
g.Log().Errorf(ctx, "获取待送检视频失败: %v", err)
return 0, err
@@ -168,13 +240,12 @@ func (s *TencentContentCheckService) processVideos(ctx context.Context) (int, er
failedCount := 0
for _, video := range videos {
mLog, err := MaterialVerify.VerifyVideoByID(ctx, video.VideoID)
_, err := MaterialVerify.VerifyVideoByID(ctx, video.VideoID)
if err != nil {
failedCount++
g.Log().Errorf(ctx, "视频送检失败, videoId=%s, error=%v", video.VideoID, err)
} else {
successCount++
s.writeAuditLog(ctx, consts.SourceTableTencentVideo, video.Id, video.VideoID, video.PreviewURL, mLog.TaskID)
}
time.Sleep(100 * time.Millisecond)
@@ -185,21 +256,32 @@ func (s *TencentContentCheckService) processVideos(ctx context.Context) (int, er
}
// writeAuditLog 写入审计日志(tencent_content_check_log
func (s *TencentContentCheckService) writeAuditLog(ctx context.Context, sourceTable string, sourceID int64, mediaID string, mediaURL string, taskID string) {
// 当 suggestion<0 时记录为 SUBMITTING(已提交等待结果),否则记录为 COMPLETED(检测完成)
func (s *TencentContentCheckService) writeAuditLog(ctx context.Context, sourceTable string, sourceID int64, mediaID string, mediaURL string, taskID string, suggestion, label, resultType int, responseData string, duration int64) {
requestParam := map[string]interface{}{
"media_id": mediaID,
"url": mediaURL,
}
requestParamJSON, _ := json.Marshal(requestParam)
status := consts.CheckStatusSubmitting
if suggestion >= 0 {
status = consts.CheckStatusCompleted
}
log := &entity.TencentContentCheckLog{
SourceTable: sourceTable,
SourceID: sourceID,
RequestURL: "易盾内容安全检测接口",
RequestParam: string(requestParamJSON),
Status: consts.CheckStatusSuccess,
Status: status,
CheckTime: time.Now().UnixMilli(),
TaskID: taskID,
Suggestion: suggestion,
Label: label,
ResultType: resultType,
ResponseData: responseData,
Duration: duration,
}
id, err := dao.TencentContentCheckLog.Create(ctx, log)
@@ -220,7 +302,7 @@ func (s *TencentContentCheckService) SubmitImageByID(ctx context.Context, imageI
image, err := dao.TencentImage.GetByImageID(ctx, imageID)
if err == nil && image != nil {
s.writeAuditLog(ctx, consts.SourceTableTencentImage, image.Id, imageID, image.PreviewURL, mLog.TaskID)
s.writeAuditLog(ctx, consts.SourceTableTencentImage, image.Id, imageID, image.PreviewURL, mLog.TaskID, -1, 0, 0, "", 0)
}
return &yidunService.ImageSubmitResult{
@@ -237,7 +319,7 @@ func (s *TencentContentCheckService) SubmitVideoByID(ctx context.Context, videoI
video, err := dao.TencentVideo.GetByVideoID(ctx, videoID)
if err == nil && video != nil {
s.writeAuditLog(ctx, consts.SourceTableTencentVideo, video.Id, videoID, video.PreviewURL, mLog.TaskID)
s.writeAuditLog(ctx, consts.SourceTableTencentVideo, video.Id, videoID, video.PreviewURL, mLog.TaskID, -1, 0, 0, "", 0)
}
return &yidunService.VideoSubmitResult{
@@ -249,13 +331,24 @@ func (s *TencentContentCheckService) SubmitVideoByID(ctx context.Context, videoI
func (s *TencentContentCheckService) GetPendingStats(ctx context.Context) map[string]int {
stats := make(map[string]int)
if s.config.ImageEnabled {
count, _ := dao.TencentImage.CountPending(ctx)
s.mu.RLock()
imageEnabled := s.config.ImageEnabled
videoEnabled := s.config.VideoEnabled
s.mu.RUnlock()
if imageEnabled {
count, err := dao.TencentImage.CountPending(ctx)
if err != nil {
g.Log().Errorf(ctx, "统计待送检图片数量失败: %v", err)
}
stats["image_pending"] = count
}
if s.config.VideoEnabled {
count, _ := dao.TencentVideo.CountPending(ctx)
if videoEnabled {
count, err := dao.TencentVideo.CountPending(ctx)
if err != nil {
g.Log().Errorf(ctx, "统计待送检视频数量失败: %v", err)
}
stats["video_pending"] = count
}
@@ -264,10 +357,14 @@ func (s *TencentContentCheckService) GetPendingStats(ctx context.Context) map[st
// IsRunning 获取运行状态
func (s *TencentContentCheckService) IsRunning() bool {
s.mu.RLock()
defer s.mu.RUnlock()
return s.isRunning
}
// GetConfig 获取当前配置
func (s *TencentContentCheckService) GetConfig() ContentCheckConfig {
s.mu.RLock()
defer s.mu.RUnlock()
return s.config
}
-26
View File
@@ -2,7 +2,6 @@ package yidun
import (
"context"
"encoding/json"
"errors"
"fmt"
@@ -435,28 +434,3 @@ type StrategyVersionInfo struct {
Label int `json:"label"` // 垃圾类别
Version string `json:"version"` // 版本号
}
// ProcessImageCallback 处理图片检测回调(推送模式)
func (s *ImageDetectionService) ProcessImageCallback(ctx context.Context, callbackData string) error {
if callbackData == "" {
return fmt.Errorf("回调数据不能为空")
}
var data ImageCallbackData
if err := json.Unmarshal([]byte(callbackData), &data); err != nil {
g.Log().Errorf(ctx, "解析回调数据失败: %v", err)
return fmt.Errorf("解析回调数据失败: %w", err)
}
if data.Antispam == nil {
return fmt.Errorf("回调数据格式错误:缺少antispam字段")
}
g.Log().Infof(ctx, "处理图片检测结果 - taskId: %s, suggestion: %d, resultType: %d",
data.Antispam.TaskId, data.Antispam.Suggestion, data.Antispam.ResultType)
// TODO: 业务逻辑,如保存数据库、触发后续流程等
// 可使用完整字段:data.Antispam.Labels, data.Antispam.CensorLabels, data.Antispam.Remark 等
return nil
}
+96 -510
View File
@@ -2,15 +2,14 @@ package yidun
import (
"context"
"encoding/json"
"errors"
"fmt"
"github.com/gogf/gf/v2/frame/g"
audiocallback "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/audio/callback/v4/response"
videocallback "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/video/callback/v4/response"
callbackrequest "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/videosolution/callback/v2/request"
callbackresponse "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/videosolution/callback/v2/response"
queryrequest "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/videosolution/query/v2/request"
vsrequest "github.com/yidun/yidun-golang-sdk/yidun/service/antispam/videosolution/submit/v2/request"
)
@@ -26,53 +25,39 @@ var (
// VideoSubmitResult 视频检测提交结果
type VideoSubmitResult struct {
TaskID string `json:"taskId"` // 任务ID
DataID string `json:"dataId"` // 数据ID
DealingCount int64 `json:"dealingCount"` // 缓冲池排队待处理数据量
TaskID string `json:"taskId"`
DataID string `json:"dataId"`
DealingCount int64 `json:"dealingCount"`
}
// DetectVideo 提交视频检测任务,返回完整响应
// DetectVideo 提交视频检测任务
func (s *VideoDetectionService) DetectVideo(ctx context.Context, videoURL, dataID string, callbackURL string) (*VideoSubmitResult, error) {
if DefaultClients == nil || DefaultClients.VideoClient == nil {
return nil, fmt.Errorf("易盾视频检测客户端未初始化")
}
if videoURL == "" {
return nil, fmt.Errorf("视频URL不能为空")
}
g.Log().Infof(ctx, "视频检测任务提交, url: %s, dataID: %s", videoURL, dataID)
// 创建请求
req := vsrequest.NewVideoSolutionSubmitV2Req()
req.SetURL(videoURL)
req.SetDataID(dataID)
req.SetUniqueKey(dataID)
// 设置回调地址
if callbackURL != "" {
req.SetCallbackURL(callbackURL)
}
// 设置子产品标识(视频解决方案必须)
req.SetSubProduct("videoStream")
// 可选:设置IP(用于风险用户识别)
// req.SetIP("127.0.0.1")
// 调用API
response, err := DefaultClients.VideoClient.Submit(req)
if err != nil {
g.Log().Errorf(ctx, "视频检测提交HTTP错误: %v", err)
return nil, fmt.Errorf("视频检测提交HTTP错误: %w", err)
}
if response.GetCode() != 200 {
g.Log().Errorf(ctx, "视频检测API错误: code=%d, msg=%s", response.GetCode(), response.GetMsg())
// 根据错误码提供更详细的错误信息
errMsg := fmt.Sprintf("视频检测API错误: code=%d, msg=%s", response.GetCode(), response.GetMsg())
// 常见错误码说明
switch response.GetCode() {
case 417:
errMsg += " (可能原因: 视频URL无法访问或业务配置问题)"
@@ -81,7 +66,7 @@ func (s *VideoDetectionService) DetectVideo(ctx context.Context, videoURL, dataI
case 403:
errMsg += " (可能原因: 鉴权失败,检查secretId和secretKey)"
}
return nil, fmt.Errorf(errMsg)
return nil, fmt.Errorf("%s", errMsg)
}
result := &VideoSubmitResult{}
@@ -96,38 +81,64 @@ func (s *VideoDetectionService) DetectVideo(ctx context.Context, videoURL, dataI
result.DealingCount = *response.Result.DealingCount
}
}
g.Log().Infof(ctx, "视频检测任务提交成功, taskID: %s, dealingCount: %d", result.TaskID, result.DealingCount)
return result, nil
}
// VideoResult 视频检测完整结果
// 包含易盾返回的完整检测信息:文本、图片、音频、ASR等
type VideoResult struct {
TaskID string `json:"taskId"` // 任务ID
Status int `json:"status"` // 检测状态:0=未开始1=检测中2=检测
Suggestion int `json:"suggestion"` // 处置建议:0=通过1=嫌疑2=不通过
Label int `json:"label"` // 垃圾类型
ResultType int `json:"resultType"` // 结果类型:1=机器结果2=人审结果
DataID string `json:"dataId"` // 数据ID
CensorTime int64 `json:"censorTime"` // 审核完成时间(毫秒)
Duration int64 `json:"duration"` // 视频时长(毫秒)
TaskID string `json:"taskId"`
Status int `json:"status"` // 0=未开始, 1=检测中, 2=检测成功, 3=检测失败
Suggestion int `json:"suggestion"` // 0=通过, 1=嫌疑, 2=不通过
Label int `json:"label"` // 违规类别
ResultType int `json:"resultType"` // 1=机器结果, 2=人审结果
DataID string `json:"dataId"`
CensorTime int64 `json:"censorTime"`
Duration int64 `json:"duration"`
// 完整证据信息
Antispam *callbackresponse.VideoSolutionAntispamCallbackV2Response `json:"antispam,omitempty"` // 反垃圾检测结果
Language *audiocallback.AudioLanguageCallbackV4Response `json:"language,omitempty"` // 语言检测结果
Voice *audiocallback.AudioVoiceCallbackV4Response `json:"voice,omitempty"` // 语音识别结果
Asr *audiocallback.AudioAsrCallbackV4Response `json:"asr,omitempty"` // ASR语音转文字结果
Ocr *videocallback.VideoCallbackOcrV4Response `json:"ocr,omitempty"` // OCR文字识别结果
Discern *videocallback.VideoCallbackDiscernV4Response `json:"discern,omitempty"` // 画面识别结果
Logo *videocallback.VideoCallbackLogoV4Response `json:"logo,omitempty"` // Logo识别结果
Face *videocallback.VideoCallbackFaceV4Response `json:"face,omitempty"` // 人脸识别结果
Aigc *videocallback.VideoCallbackAigcV4Response `json:"aigc,omitempty"` // AIGC识别结果
Quality *callbackresponse.VideoSolutionQualityCallbackV2Response `json:"quality,omitempty"` // 音视频质量检测结果
Antispam *callbackresponse.VideoSolutionAntispamCallbackV2Response `json:"antispam,omitempty"`
Language *audiocallback.AudioLanguageCallbackV4Response `json:"language,omitempty"`
Voice *audiocallback.AudioVoiceCallbackV4Response `json:"voice,omitempty"`
Asr *audiocallback.AudioAsrCallbackV4Response `json:"asr,omitempty"`
Ocr *videocallback.VideoCallbackOcrV4Response `json:"ocr,omitempty"`
Discern *videocallback.VideoCallbackDiscernV4Response `json:"discern,omitempty"`
Logo *videocallback.VideoCallbackLogoV4Response `json:"logo,omitempty"`
Face *videocallback.VideoCallbackFaceV4Response `json:"face,omitempty"`
Aigc *videocallback.VideoCallbackAigcV4Response `json:"aigc,omitempty"`
Quality *callbackresponse.VideoSolutionQualityCallbackV2Response `json:"quality,omitempty"`
}
// queryCheckStatusToStatus 将易盾 checkStatus 映射为内部 Status
func queryCheckStatusToStatus(checkStatus int) int {
switch checkStatus {
case 0:
return 1 // Processing
case 1:
return 2 // Success
case 2:
return 3 // Failed
default:
return 0 // NotStart
}
}
// queryResultToSuggestion 将易盾 result 映射为内部 suggestion
// result: 1=正常, 2=异常, 3=疑似 → suggestion: 0=通过, 1=嫌疑, 2=不通过
func queryResultToSuggestion(result int) int {
switch result {
case 1:
return 0 // 通过
case 2:
return 2 // 不通过
case 3:
return 1 // 嫌疑
default:
return 0
}
}
// GetVideoResult 获取视频检测结果(轮询模式)
// 返回完整的检测详情,包括文本、图片、音频、ASR等
func (s *VideoDetectionService) GetVideoResult(ctx context.Context, taskID string) (*VideoResult, error) {
if DefaultClients == nil || DefaultClients.VideoClient == nil {
return nil, fmt.Errorf("易盾视频检测客户端未初始化")
@@ -135,41 +146,44 @@ func (s *VideoDetectionService) GetVideoResult(ctx context.Context, taskID strin
g.Log().Infof(ctx, "查询视频检测结果, taskID: %s", taskID)
req := callbackrequest.NewVideoSolutionCallbackV2Request()
req.SetYidunRequestId(taskID)
response, err := DefaultClients.VideoClient.Callback(req)
req := queryrequest.NewVideoSolutionQueryTaskV2Request()
req.SetTaskIds([]string{taskID})
response, err := DefaultClients.VideoClient.QueryTaskV2(req)
if err != nil {
g.Log().Errorf(ctx, "查询视频检测结果失败: %v", err)
return nil, fmt.Errorf("查询视频检测结果失败: %w", err)
}
if response.GetCode() != 200 {
g.Log().Errorf(ctx, "查询视频检测结果API错误: code=%d, msg=%s", response.GetCode(), response.GetMsg())
return nil, fmt.Errorf("查询视频检测结果API错误: code=%d, msg=%s", response.GetCode(), response.GetMsg())
}
if response.Result == nil || len(*response.Result) == 0 {
g.Log().Warningf(ctx, "未找到视频检测结果, taskID: %s", taskID)
return nil, ErrVideoResultNotFound
}
// 查找指定taskID的结果
for _, item := range *response.Result {
if item.Antispam != nil && item.Antispam.TaskID != nil && *item.Antispam.TaskID == taskID {
result := &VideoResult{
Antispam: item.Antispam,
Language: item.Language,
Voice: item.Voice,
Asr: item.Asr,
Ocr: item.Ocr,
Discern: item.Discern,
Logo: item.Logo,
Face: item.Face,
Aigc: item.Aigc,
Quality: item.Quality,
if item.TaskID != nil && *item.TaskID == taskID {
// status: 0=检测完成/失败, 20=非7天内, 30=不存在, 40=检测中
if item.Status != nil {
if *item.Status == 30 {
g.Log().Warningf(ctx, "视频检测结果不存在, taskID: %s", taskID)
return nil, ErrVideoResultNotFound
}
if *item.Status == 40 {
g.Log().Infof(ctx, "视频仍在检测中, taskID: %s", taskID)
return nil, ErrVideoStillProcessing
}
}
result.TaskID = taskID
result := &VideoResult{TaskID: taskID}
if item.DataID != nil {
result.DataID = *item.DataID
}
if item.TaskID != nil {
result.TaskID = *item.TaskID
}
if item.Antispam.Status != nil {
result.Status = *item.Antispam.Status
}
@@ -192,6 +206,11 @@ func (s *VideoDetectionService) GetVideoResult(ctx context.Context, taskID strin
result.Duration = *item.Antispam.Duration
}
// 完整证据信息
result.Antispam = &item.Antispam
g.Log().Infof(ctx, "视频检测结果查询成功, taskID: %s, suggestion=%d, label=%d",
taskID, result.Suggestion, result.Label)
return result, nil
}
}
@@ -200,457 +219,24 @@ func (s *VideoDetectionService) GetVideoResult(ctx context.Context, taskID strin
return nil, ErrVideoResultNotFound
}
// =============================================================================
// 视频检测结果推送模式(易盾主动回调)
// =============================================================================
// VideoCallbackData 推送模式回调数据完整结构
// VideoCallbackData 推送模式回调数据
type VideoCallbackData struct {
Antispam *VideoCallbackAntispam `json:"antispam"` // 反垃圾检测结果
Antispam *VideoCallbackAntispam `json:"antispam"`
}
// VideoCallbackAntispam 视频回调反垃圾结果
type VideoCallbackAntispam struct {
TaskID string `json:"taskId"` // 任务ID
DataID string `json:"dataId"` // 客户数据ID
Callback string `json:"callback"` // 回调参数
Suggestion int `json:"suggestion"` // 处置建议:0=通过,1=嫌疑,2=不通过
Status int `json:"status"` // 检测状态:0=未开始,1=检测中,2=检测成功,3=检测失败
ResultType int `json:"resultType"` // 结果类型:1=机器结果,2=人审结果
CensorRound int `json:"censorRound"` // 人审轮次
Censor string `json:"censor"` // 人审操作人
CensorSource int `json:"censorSource"` // 审核来源:0=易盾人审,1=客户人审,2=易盾机审
CheckTime int64 `json:"checkTime"` // 机器检测结束时间
CensorTime int64 `json:"censorTime"` // 人工审核完成时间
Duration int64 `json:"duration"` // 音视频时长(毫秒)
DurationMs int64 `json:"durationMs"` // 音频时长(毫秒)
Label int `json:"label"` // 一级垃圾类型
SecondLabel string `json:"secondLabel"` // 二级垃圾类型
ThirdLabel string `json:"thirdLabel"` // 三级垃圾类型
RiskDescription string `json:"riskDescription"` // 风险描述
PicCount int64 `json:"picCount"` // 截图数量
CensorLabels []*VideoCensorLabel `json:"censorLabels"` // 审核标签
CensorExtension *VideoCensorExtension `json:"censorExtension"` // 质检扩展结果
Evidences *VideoCallbackEvidence `json:"evidences"` // 机器检测证据信息
SolutionExtra *VideoSolutionExtra `json:"solutionExtra"` // 额外信息
ReviewEvidences *VideoReviewEvidence `json:"reviewEvidences"` // 人审证据信息
}
// VideoCensorLabel 审核标签
type VideoCensorLabel struct {
Code string `json:"code"` // 审核标签编码
Desc string `json:"desc"` // 审核标签描述
Name string `json:"name"` // 审核标签名称
CustomCode string `json:"customCode"` // 自定义标签编码
ParentLabelId string `json:"parentLabelId"` // 父标签ID
Depth int `json:"depth"` // 标签深度
}
// VideoCensorExtension 质检扩展结果
type VideoCensorExtension struct {
QualityInspectionTaskId string `json:"qualityInspectionTaskId"` // 质检任务ID
QualityInspectionType int `json:"qualityInspectionType"` // 质检类型
InspTaskCreateTime int64 `json:"inspTaskCreateTime"` // 质检任务创建时间
}
// VideoCallbackEvidence 机器检测证据信息
type VideoCallbackEvidence struct {
Text *VideoTextEvidence `json:"text,omitempty"` // 文本证据
Images *[]VideoImageEvidence `json:"images,omitempty"` // 图片证据
Audio *VideoAudioEvidence `json:"audio,omitempty"` // 音频证据
Video *VideoVideoEvidence `json:"video,omitempty"` // 视频证据
}
// VideoTextEvidence 文本证据
type VideoTextEvidence struct {
TaskID string `json:"taskId"` // 任务ID
DataID string `json:"dataId"` // 数据ID
Suggestion int `json:"suggestion"` // 处置建议
ResultType int `json:"resultType"` // 结果类型
CensorType int `json:"censorType"` // 审核类型
IsRelatedHit bool `json:"isRelatedHit"` // 是否关联命中
Labels []*VideoTextLabel `json:"labels"` // 分类标签
}
// VideoTextLabel 文本分类标签
type VideoTextLabel struct {
Label int `json:"label"` // 标签类型
Level int `json:"level"` // 级别
SubLabels []*VideoTextSubLabel `json:"subLabels"` // 二级分类
}
// VideoTextSubLabel 文本二级分类
type VideoTextSubLabel struct {
SubLabel string `json:"subLabel"` // 二级分类
SubLabelDepth int `json:"subLabelDepth"` // 细分类层级
SecondLabel string `json:"secondLabel"` // 二级分类
ThirdLabel string `json:"thirdLabel"` // 三级分类
Details *VideoTextSubLabelDetail `json:"details"` // 命中详情
}
// VideoTextSubLabelDetail 文本二级分类命中详情
type VideoTextSubLabelDetail struct {
Keywords *[]VideoTextKeyword `json:"keywords,omitempty"` // 敏感词
LibInfos *[]VideoTextLibInfo `json:"libInfos,omitempty"` // 名单信息
Anticheat *VideoTextAnticheat `json:"anticheat,omitempty"` // 反作弊
HitInfos *[]VideoTextHitInfo `json:"hitInfos,omitempty"` // 其他命中
}
// VideoTextKeyword 敏感词信息
type VideoTextKeyword struct {
Word string `json:"word"` // 敏感词
StrategyGroupId int64 `json:"strategyGroupId"` // 策略组ID
StrategyGroupName string `json:"strategyGroupName"` // 策略组名称
}
// VideoTextLibInfo 名单信息
type VideoTextLibInfo struct {
Type int `json:"type"` // 名单类型
Entity string `json:"entity"` // 名单内容
ReleaseTime int64 `json:"releaseTime"` // 释放时间
}
// VideoTextAnticheat 反作弊信息
type VideoTextAnticheat struct {
HitType int `json:"hitType"` // 命中类型
}
// VideoTextHitInfo 其他命中信息
type VideoTextHitInfo struct {
Value string `json:"value"` // 命中值
Positions *[]VideoTextHitPosition `json:"positions"` // 命中位置
}
// VideoTextHitPosition 命中位置
type VideoTextHitPosition struct {
FieldName string `json:"fieldName"` // 字段名
StartPos int `json:"startPos"` // 开始位置
EndPos int `json:"endPos"` // 结束位置
}
// VideoImageEvidence 图片证据
type VideoImageEvidence struct {
Name string `json:"name"` // 图片名称
DataID string `json:"dataId"` // 数据ID
Suggestion int `json:"suggestion"` // 处置建议
ResultType int `json:"resultType"` // 结果类型
Status int `json:"status"` // 状态
CensorType int `json:"censorType"` // 审核类型
Labels []*VideoImageLabel `json:"labels"` // 分类标签
TaskId string `json:"taskId"` // 任务ID
}
// VideoImageLabel 图片分类标签
type VideoImageLabel struct {
Label int `json:"label"` // 标签类型
Level int `json:"level"` // 级别
Rate float32 `json:"rate"` // 置信度
SubLabels []*VideoImageSubLabel `json:"subLabels"` // 二级分类
}
// VideoImageSubLabel 图片二级分类
type VideoImageSubLabel struct {
SubLabel interface{} `json:"subLabel"` // 二级分类标签
SubLabelDepth int `json:"subLabelDepth"` // 细分类层级
SecondLabel string `json:"secondLabel"` // 二级分类
ThirdLabel string `json:"thirdLabel"` // 三级分类
HitStrategy int `json:"hitStrategy"` // 命中策略
Rate float32 `json:"rate"` // 置信度
Details *VideoImageSubDetail `json:"details"` // 命中详情
Explain string `json:"explain"` // 解释
IsLlmCheck bool `json:"isLlmCheck"` // 是否LLM命中
}
// VideoImageSubDetail 图片二级分类命中详情
type VideoImageSubDetail struct {
Keywords *[]VideoImageHitInfo `json:"keywords,omitempty"` // 敏感词
LibInfos *[]VideoImageLibInfo `json:"libInfos,omitempty"` // 名单信息
HitInfos *[]VideoImageHitInfo `json:"hitInfos,omitempty"` // 其他命中
Anticheat *VideoImageAnticheat `json:"anticheat,omitempty"` // 反作弊
Llm *VideoImageLlm `json:"llm,omitempty"` // 大模型
}
// VideoImageHitInfo 图片命中信息
type VideoImageHitInfo struct {
Type int `json:"type"` // 命中类型
Word string `json:"word"` // 敏感词
Entity string `json:"entity"` // 名单URL
HitCount int `json:"hitCount"` // 命中次数
Value string `json:"value"` // 值
Group string `json:"group"` // 分组
X1 float32 `json:"x1"` // 坐标
Y1 float32 `json:"y1"` // 坐标
X2 float32 `json:"x2"` // 坐标
Y2 float32 `json:"y2"` // 坐标
StrategyGroupName string `json:"strategyGroupName"` // 策略组名称
StrategyGroupId int64 `json:"strategyGroupId"` // 策略组ID
}
// VideoImageLibInfo 图片名单信息
type VideoImageLibInfo struct {
Type int `json:"type"` // 名单类型
Entity string `json:"entity"` // 名单URL
HitCount int `json:"hitCount"` // 命中次数
ReleaseTime int64 `json:"releaseTime"` // 释放时间
StrategyGroupName string `json:"strategyGroupName"` // 策略组名称
StrategyGroupId int64 `json:"strategyGroupId"` // 策略组ID
}
// VideoImageAnticheat 反作弊信息
type VideoImageAnticheat struct {
HitType int `json:"hitType"` // 命中类型
}
// VideoImageLlm 大模型信息
type VideoImageLlm struct {
Keyword string `json:"keyword"` // 关键词
}
// VideoAudioEvidence 音频证据
type VideoAudioEvidence struct {
TaskID string `json:"taskId"` // 任务ID
DataID string `json:"dataId"` // 数据ID
Status int `json:"status"` // 状态
Suggestion int `json:"suggestion"` // 处置建议
Label int `json:"label"` // 标签
ResultType int `json:"resultType"` // 结果类型
Callback string `json:"callback"` // 回调参数
CensorSource int `json:"censorSource"` // 审核来源
CensorTime int64 `json:"censorTime"` // 审核时间
Duration int64 `json:"duration"` // 音频时长
Segments []*VideoAudioSegment `json:"segments"` // 音频片段
}
// VideoAudioSegment 音频片段
type VideoAudioSegment struct {
StartTime int `json:"startTime"` // 开始时间(秒)
EndTime int `json:"endTime"` // 结束时间(秒)
StartTimeMillis int64 `json:"startTimeMillis"` // 开始时间(毫秒)
EndTimeMillis int64 `json:"endTimeMillis"` // 结束时间(毫秒)
Content string `json:"content"` // 语音识别原文
Type int `json:"type"` // 片段类型:0=语音识别,1=声纹检测
LeaderName string `json:"leaderName"` // 声纹检测人名
Labels []*VideoAudioLabel `json:"labels"` // 分类标签
Url string `json:"url"` // 音频URL
}
// VideoAudioLabel 音频分类标签
type VideoAudioLabel struct {
Label int `json:"label"` // 标签类型
Level int `json:"level"` // 级别
SubLabels []*VideoAudioSubLabel `json:"subLabels"` // 二级分类
}
// VideoAudioSubLabel 音频二级分类
type VideoAudioSubLabel struct {
SubLabel string `json:"subLabel"` // 细分类
SubLabelDepth int `json:"subLabelDepth"` // 细分类层级
SecondLabel string `json:"secondLabel"` // 二级分类
ThirdLabel string `json:"thirdLabel"` // 三级分类
SuggestionRiskLevel int `json:"suggestionRiskLevel"` // 嫌疑级别
Rate float64 `json:"rate"` // 置信度
RiskDescription string `json:"riskDescription"` // 风险描述
Details *VideoAudioSubLabelDetail `json:"details"` // 命中详情
}
// VideoAudioSubLabelDetail 音频二级分类命中详情
type VideoAudioSubLabelDetail struct {
HitInfos *[]VideoAudioHitInfo `json:"hitInfos,omitempty"` // 命中内容
Keywords *[]VideoAudioKeyword `json:"keywords,omitempty"` // 自定义敏感词
LibInfos *[]VideoAudioLibInfo `json:"libInfos,omitempty"` // 自定义名单
}
// VideoAudioHitInfo 音频命中内容
type VideoAudioHitInfo struct {
Value string `json:"value"` // 命中敏感词或声纹检测分值
StartTime int `json:"startTime"` // 开始时间
EndTime int `json:"endTime"` // 结束时间
}
// VideoAudioKeyword 自定义敏感词
type VideoAudioKeyword struct {
Word string `json:"word"` // 敏感词
StrategyGroupName string `json:"strategyGroupName"` // 策略组名称
StrategyGroupId int64 `json:"strategyGroupId"` // 策略组ID
}
// VideoAudioLibInfo 自定义名单
type VideoAudioLibInfo struct {
ListType int `json:"listType"` // 名单类型
Entity string `json:"entity"` // 名单内容
}
// VideoVideoEvidence 视频证据
type VideoVideoEvidence struct {
TaskID string `json:"taskId"` // 任务ID
DataID string `json:"dataId"` // 数据ID
Status int `json:"status"` // 状态
Suggestion int `json:"suggestion"` // 处置建议
ResultType int `json:"resultType"` // 结果类型
CensorSource int `json:"censorSource"` // 审核来源
CensorTime int64 `json:"censorTime"` // 审核时间
Pictures []*VideoPicture `json:"pictures"` // 截图列表
}
// VideoPicture 截图信息
type VideoPicture struct {
Type int `json:"type"` // 类型
URL string `json:"url"` // 图片URL
StartTime int64 `json:"startTime"` // 开始时间
EndTime int64 `json:"endTime"` // 结束时间
Labels []*VideoPictureLabel `json:"labels"` // 分类标签
CensorSource int `json:"censorSource"` // 审核来源
FrontPics []*VideoRelatedPic `json:"frontPics"` // 关联前帧
BackPics []*VideoRelatedPic `json:"backPics"` // 关联后帧
PictureID string `json:"pictureId"` // 图片ID
}
// VideoPictureLabel 截图分类标签
type VideoPictureLabel struct {
Label int `json:"label"` // 标签类型
Level int `json:"level"` // 级别
Rate float32 `json:"rate"` // 置信度
SubLabels []*VideoPictureSubLabel `json:"subLabels"` // 二级分类
}
// VideoPictureSubLabel 截图二级分类
type VideoPictureSubLabel struct {
SubLabel interface{} `json:"subLabel"` // 二级分类标签
SubLabelDepth int `json:"subLabelDepth"` // 细分类层级
SecondLabel string `json:"secondLabel"` // 二级分类
ThirdLabel string `json:"thirdLabel"` // 三级分类
HitStrategy int `json:"hitStrategy"` // 命中策略
Rate float32 `json:"rate"` // 置信度
Details *VideoPictureSubDetail `json:"details"` // 命中详情
Explain string `json:"explain"` // 解释
IsLlmCheck bool `json:"isLlmCheck"` // 是否LLM命中
}
// VideoPictureSubDetail 截图二级分类命中详情
type VideoPictureSubDetail struct {
Keywords *[]VideoPictureHitInfo `json:"keywords,omitempty"` // 敏感词
LibInfos *[]VideoPictureLibInfo `json:"libInfos,omitempty"` // 名单信息
HitInfos *[]VideoPictureHitInfo `json:"hitInfos,omitempty"` // 其他命中
Anticheat *VideoPictureAnticheat `json:"anticheat,omitempty"` // 反作弊
Llm *VideoPictureLlm `json:"llm,omitempty"` // 大模型
}
// VideoPictureHitInfo 截图命中信息
type VideoPictureHitInfo struct {
Value string `json:"value"` // 命中值
Group string `json:"group"` // 分组
Type int `json:"type"` // 类型
X1 float32 `json:"x1"` // 坐标
Y1 float32 `json:"y1"` // 坐标
X2 float32 `json:"x2"` // 坐标
Y2 float32 `json:"y2"` // 坐标
}
// VideoPictureLibInfo 截图名单信息
type VideoPictureLibInfo struct {
Type int `json:"type"` // 名单类型
Entity string `json:"entity"` // 名单URL
HitCount int `json:"hitCount"` // 命中次数
ReleaseTime int64 `json:"releaseTime"` // 释放时间
StrategyGroupName string `json:"strategyGroupName"` // 策略组名称
StrategyGroupId int64 `json:"strategyGroupId"` // 策略组ID
}
// VideoPictureAnticheat 反作弊信息
type VideoPictureAnticheat struct {
HitType int `json:"hitType"` // 命中类型
}
// VideoPictureLlm 大模型信息
type VideoPictureLlm struct {
Keyword string `json:"keyword"` // 关键词
}
// VideoRelatedPic 关联截图
type VideoRelatedPic struct {
URL string `json:"url"` // 图片URL
}
// VideoSolutionExtra 额外信息
type VideoSolutionExtra struct {
FailUnit *VideoFailUnit `json:"failUnit,omitempty"` // 失败单元
}
// VideoFailUnit 失败单元
type VideoFailUnit struct {
Images *[]VideoImageFailUnit `json:"images,omitempty"` // 图片失败单元
Audio *VideoTargetFailUnit `json:"audio,omitempty"` // 音频失败单元
Video *VideoTargetFailUnit `json:"video,omitempty"` // 视频失败单元
}
// VideoImageFailUnit 图片失败单元
type VideoImageFailUnit struct {
FailureReason int `json:"failureReason"` // 失败原因
Name string `json:"name"` // 名称
}
// VideoTargetFailUnit 目标失败单元
type VideoTargetFailUnit struct {
FailureReason int `json:"failureReason"` // 失败原因
}
// VideoReviewEvidence 人审证据信息
type VideoReviewEvidence struct {
Description string `json:"description"` // 描述
Detail string `json:"detail"` // 详情
Texts *[]VideoReviewText `json:"texts"` // 文本证据
Images *[]VideoReviewImage `json:"images"` // 图片证据
Audios *[]VideoReviewAudio `json:"audios"` // 音频证据
Videos *[]VideoReviewVideo `json:"videos"` // 视频证据
}
// VideoReviewText 人审文本证据
type VideoReviewText struct {
Snippet string `json:"snippet"` // 片段
Description string `json:"description"` // 描述
}
// VideoReviewImage 人审图片证据
type VideoReviewImage struct {
URL string `json:"url"` // 图片URL
Description string `json:"description"` // 描述
}
// VideoReviewAudio 人审音频证据
type VideoReviewAudio struct {
StartTime int64 `json:"startTime"` // 开始时间
EndTime int64 `json:"endTime"` // 结束时间
Description string `json:"description"` // 描述
}
// VideoReviewVideo 人审视频证据
type VideoReviewVideo struct {
StartTime int64 `json:"startTime"` // 开始时间
EndTime int64 `json:"endTime"` // 结束时间
URL string `json:"url"` // 视频URL
Description string `json:"description"` // 描述
}
// ProcessVideoCallback 处理视频检测回调(推送模式)
func (s *VideoDetectionService) ProcessVideoCallback(ctx context.Context, callbackData string) error {
if callbackData == "" {
return fmt.Errorf("回调数据不能为空")
}
var data VideoCallbackData
if err := json.Unmarshal([]byte(callbackData), &data); err != nil {
g.Log().Errorf(ctx, "解析视频回调数据失败: %v", err)
return fmt.Errorf("解析视频回调数据失败: %w", err)
}
if data.Antispam == nil {
return fmt.Errorf("视频回调数据格式错误:缺少antispam字段")
}
g.Log().Infof(ctx, "处理视频检测结果 - taskId: %s, suggestion: %d, resultType: %d, status: %d",
data.Antispam.TaskID, data.Antispam.Suggestion, data.Antispam.ResultType, data.Antispam.Status)
// TODO: 业务逻辑,如保存数据库、触发后续流程等
// 可使用完整字段:data.Antispam.Evidences, data.Antispam.CensorLabels, data.Antispam.Duration 等
return nil
TaskID string `json:"taskId"`
DataID string `json:"dataId"`
Suggestion int `json:"suggestion"`
Status int `json:"status"`
ResultType int `json:"resultType"`
Label int `json:"label"`
SecondLabel string `json:"secondLabel"`
ThirdLabel string `json:"thirdLabel"`
RiskDescription string `json:"riskDescription"`
CensorSource int `json:"censorSource"`
CensorTime int64 `json:"censorTime"`
CheckTime int64 `json:"checkTime"`
Duration int64 `json:"duration"`
}
@@ -0,0 +1,5 @@
-- 补充 material_verify_log 表的 risk_description 字段
ALTER TABLE material_verify_log
ADD COLUMN IF NOT EXISTS risk_description TEXT DEFAULT '' NOT NULL;
COMMENT ON COLUMN material_verify_log.risk_description IS '风险描述(易盾返回,如:广告法|涉嫌欺诈消费者)';