视频剪辑和处理

This commit is contained in:
lmk
2026-06-22 09:18:45 +08:00
parent 9bc48060dc
commit 7d4ac82334
34 changed files with 3319 additions and 839 deletions
+175 -1
View File
@@ -167,7 +167,181 @@ curl -X POST http://localhost:8900/generate \
--- ---
## 错误响应 ## 5. 字幕叠加任务接口
**服务地址**`http://127.0.0.1:3010`
### 5.1 创建字幕叠加任务
**接口路径**`POST /video/caption`
**功能说明**:将背景视频、字幕、文字/图片元素、背景音频合成输出最终视频。异步执行,返回 taskId。
**请求参数**Content-Type: `application/json`):
| 参数名 | 类型 | 必填 | 说明 |
|--------|------|------|------|
| video_urls | string[] | **是** | 背景视频 URL 列表,多个会自动拼接 |
| audio_url | string | 否 | 背景音频 URL(配音/BGM),原视频会被消音 |
| subtitles | SubtitleSegment[] | 否 | 字幕时间线列表 |
| subtitle_style | SubtitleStyle | 否 | 字幕样式配置 |
| elements | CaptionElement[] | 否 | 文字/图片元素列表 |
| callback_url | string | 否 | 任务完成回调地址 |
**SubtitleSegment(字幕时间线片段):**
| 字段 | 类型 | 必填 | 说明 |
|------|------|------|------|
| start | float | **是** | 开始时间(秒) |
| end | float | **是** | 结束时间(秒) |
| text | string | **是** | 字幕文本 |
**SubtitleStyle(字幕样式):**
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| fontSize | int | 否 | 28 | 字号(px |
| fontColor | string | 否 | `#FFFFFF` | 字体颜色(十六进制) |
| bgColor | string | 否 | `#000000` | 字幕背景色,空=透明 |
| bgOpacity | float | 否 | 0.6 | 背景透明度,0-1 |
| x | string | 否 | `"center"` | 水平位置:`"left"`/`"center"`/`"right"` 或像素值如 `"100"` |
| y | string | 否 | `"bottom"` | 垂直位置:`"top"`/`"center"`/`"bottom"`、像素值如 `"200"`、或 CSS 表达式如 `"calc(100% - 300px)"` |
**CaptionElement(自定义元素):**
| 字段 | 类型 | 必填 | 默认值 | 说明 |
|------|------|------|--------|------|
| type | string | **是** | - | 元素类型:`"text"``"image"` |
| text | string | type=text时必填 | - | 文字内容 |
| imageUrl | string | type=image时必填 | - | 图片下载 URL |
| x | string | 否 | `"center"` | 水平位置:`"left"`/`"center"`/`"right"` 或像素值 |
| y | string | 否 | `"center"` | 垂直位置:`"top"`/`"center"`/`"bottom"`、像素值、或 `calc()` |
| fontSize | int | 否 | 36 | 字号(pxtype=text有效) |
| fontColor | string | 否 | `#FFFFFF` | 字体颜色 |
| bgColor | string | 否 | 透明 | 背景色,如 `"#FFFF00"` |
| bgOpacity | float | 否 | 1.0 | 背景透明度,0-1 |
| width | int | 否 | 原图尺寸 | 图片显示宽度(px,type=image有效) |
| height | int | 否 | 原图尺寸 | 图片显示高度(px,type=image有效) |
| startTime | float | 否 | 0 | 开始显示时间(秒),0=立即显示 |
| duration | float | 否 | 0 | 持续时长(秒),0=一直显示到结束 |
| trackIndex | int | 否 | 1 | 层级(类似 z-index),越大越靠前 |
| animation | string | 否 | 无 | 动画:`"fadeIn"`/`"slideUp"`/`"slideLeft"`/`"scaleIn"` |
**请求示例:**
```bash
curl --location --request POST 'http://127.0.0.1:3010/video/caption' \
--header 'Content-Type: application/json' \
--data-raw '{
"video_urls": [
"http://example.com/video1.mp4",
"http://example.com/video2.mp4"
],
"audio_url": "http://example.com/audio.mp3",
"subtitles": [
{"start": 0.0, "end": 3.0, "text": "第一段字幕"},
{"start": 3.0, "end": 6.0, "text": "第二段字幕"}
],
"subtitle_style": {
"fontSize": 36,
"fontColor": "#FFFF00",
"bgColor": "#000000",
"bgOpacity": 0.7,
"x": "center",
"y": "bottom"
},
"elements": [
{
"type": "text",
"text": "标题文字",
"x": "center",
"y": "60",
"fontSize": 48,
"fontColor": "#FF0000",
"startTime": 0,
"duration": 0,
"trackIndex": 5,
"animation": "fadeIn"
},
{
"type": "image",
"imageUrl": "http://example.com/logo.png",
"x": "right",
"y": "top",
"width": 100,
"height": 100,
"startTime": 0,
"duration": 0,
"trackIndex": 5
}
],
"callback_url": "https://your-server.com/callback"
}'
```
**返回参数:**
| 字段名 | 类型 | 说明 |
|--------|------|------|
| taskId | string | 任务 ID,用于后续查询结果 |
**返回示例:**
```json
{
"taskId": "CAPTION_20260602123456_abc123"
}
```
---
### 5.2 查询字幕任务结果
**接口路径**`GET /video/caption/{taskId}`
**功能说明**:根据 taskId 查询字幕叠加任务的状态和输出文件。
**路径参数:**
| 参数名 | 类型 | 必填 | 说明 |
|--------|------|------|------|
| taskId | string | **是** | 创建任务时返回的 taskId |
**返回参数:**
| 字段名 | 类型 | 说明 |
|--------|------|------|
| taskId | string | 任务 ID |
| status | string | 任务状态:`"running"`(处理中)/ `"completed"`(完成)/ `"failed"`(失败) |
| fileUrl | string | 完成时返回:输出视频下载 URL |
| fileSize | int64 | 完成时返回:输出文件大小(字节) |
| fileName | string | 完成时返回:输出文件名 |
| durationStr | string | 完成时返回:视频时长(如 "0:45" |
| errorMessage | string | 失败时返回:错误信息 |
**返回示例(完成时):**
```json
{
"taskId": "CAPTION_20260602123456_abc123",
"status": "completed",
"fileUrl": "http://127.0.0.1:3010/storage/output.mp4",
"fileSize": 5242880,
"fileName": "output.mp4",
"durationStr": "0:45"
}
```
**返回示例(处理中):**
```json
{
"taskId": "CAPTION_20260602123456_abc123",
"status": "running"
}
```
**返回示例(失败):**
```json
{
"taskId": "CAPTION_20260602123456_abc123",
"status": "failed",
"errorMessage": "视频下载失败"
}
```
当请求失败时,接口返回 HTTP 错误码和错误信息: 当请求失败时,接口返回 HTTP 错误码和错误信息:
+79 -55
View File
@@ -29,58 +29,54 @@ docker run -p 3010:3010 media
## Architecture ## Architecture
这是一个多媒体处理微服务项目,基于 GoFrame 框架开发,提供视频处理、音频提取、语音识别等功能。 这是一个多媒体处理微服务项目,基于 GoFrame 框架开发,提供视频处理、音频提取、语音识别、字幕叠加等功能。
### 目录结构 ### 目录结构
``` ```
main.go # 应用入口 main.go # 应用入口,注册所有 Controller 路由
config.yml # 配置文件 config.yml # 配置文件
consts/ # 常量定义 consts/ # 常量定义
- video/ # 视频相关常量(包括视频分析任务状态)
controller/ # HTTP 控制器层(路由入口) controller/ # HTTP 控制器层(路由入口)
- audio/ # 音频相关接口 - audio/ # 音频提取接口
- video/ # 视频相关接口(拼接、剪切、分析) - video/ # 视频相关接口(拼接、剪切、分析、合并、字幕、转码、场景分割
- common/ # 公共工具 - common/ # 公共工具(文件上传保存)
- scene/ # 场景检测接口
- image/ # 图片处理接口
service/ # 业务逻辑层 service/ # 业务逻辑层
- video/ # 视频服务(拼接、剪切、分析、分析队列 - video/ # 视频服务(拼接、剪切、分析、合并混音、字幕叠加、转码、场景分割
- audio/ # 音频提取服务 - audio/ # 音频提取服务
- asr/ # 语音识别Whisper - asr/ # 语音识别服务
- scene/ # 场景检测 - scene/ # 场景检测服务
- image/ # 图片处理 - image/ # 图片处理服务
- setup/ # 初始化服务 - setup/ # 初始化服务(自动检查/安装依赖)
dao/ # 数据访问层 dao/ # 数据访问层
- audio/ # ASR 任务数据访问
- video/ # 视频分析任务数据访问
- image/
model/ # 数据模型 model/ # 数据模型
- dto/ # 传输对象(请求/响应) - dto/ # 传输对象(请求/响应)
- video/ # 视频相关 DTO(包括视频分析)
- entity/ # 数据库实体 - entity/ # 数据库实体
- video/ # 视频相关实体(包括分析任务)
resource/ # 静态资源(日志、临时文件) resource/ # 静态资源(日志、临时文件)
sql/ # 数据库 SQL sql/ # 数据库建表 SQL
- video_analysis_task.sql - 视频分析任务表建表SQL scripts/ # 外部脚本(scene_detect.py
``` ```
### 核心功能 ### 核心功能
| 功能 | 说明 | 依赖 | | 功能 | 说明 | 依赖 |
|------|------|------| |------|------|------|
| 视频拼接 | 支持多视频拼接,提供 fast(无损 concat demuxer)和 reencode(重编码归一化)两种模式,可上传结果到 MinIO,支持同步和异步任务 | FFmpeg | | 视频拼接 | 支持多视频拼接,提供 fast(无损 concat demuxer)和 reencode(重编码归一化)两种模式,可上传结果到 MinIO,支持同步和异步任务 | FFmpeg |
| 视频分镜剪切 | 根据分镜时间片段列表剪切视频并重新拼接输出,支持同步和异步任务 | FFmpeg | | 视频分镜剪切 | 根据分镜时间片段列表剪切视频并重新拼接输出,支持同步和异步任务 | FFmpeg |
| 视频拼接+混音 | 拼接多段视频后混入音频(支持多段),以视频时长为准自动补静音或截断,异步任务 | FFmpeg |
| 字幕叠加 | 使用 HyperFrames 将字幕/图片/动画元素渲染叠加到视频上,自动检测分辨率、拼接多视频、消音、降噪,异步任务 | FFmpeg + HyperFrames (npm) |
| 场景分割 | 使用 PySceneDetect 自动检测视频场景切分点,按场景分割为独立片段并上传,异步任务 | FFmpeg + Python3 + PySceneDetect |
| 视频转码 | 将视频转码为 H.264 + AAC + MP4 + faststartMOOV前置),同步接口 | FFmpeg |
| 音频提取 | 从视频文件中提取音频,支持 mp3/aac/wav/ogg/flac 多种格式 | FFmpeg | | 音频提取 | 从视频文件中提取音频,支持 mp3/aac/wav/ogg/flac 多种格式 | FFmpeg |
| 语音识别 | 异步语音转文字任务,基于 OpenAI Whisper,支持 whisper.cpp 加速 | FFmpeg + Whisper/whisper.cpp | | 语音识别 | 异步语音转文字任务,基于 OpenAI Whisper | FFmpeg + Whisper |
| 场景检测 | 视频场景切分检测,提取关键帧,输出场景信息 | FFmpeg + ffprobe | | 视频分析 | 调用外部 Marlin-2B VLM 服务对视频进行理解分析,生成场景描述和事件切分,支持 mock 模式,异步串行处理 | FFmpeg + 外部 VLM 服务 |
| 视频分析 | 基于 Marlin-2B Video VLM 大模型进行视频理解,自动生成场景描述、事件切分和向量化,存入 RAG 系统 | FFmpeg + 外部 Marlin-2B VLM 服务 |
### 启动初始化 ### 启动初始化
- `setup` 包在 `init()` 阶段自动执行,启动时会检查 FFmpeg 和 Whisper 依赖是否可用 - `setup` 包在 `init()` 阶段自动执行,启动时自动检查并安装缺失依赖
- 自动检测 whisper-cpp > whisper > python -m whisper 三个优先级 - 检查项:FFmpeg、Python3、PySceneDetectscenedetect)、HyperFramesnpm 全局包)
- 如果依赖缺失会输出警告提示安装 - 每个依赖都按平台自动安装(macOS: brew, Linux: apt/apk/yum, Windows: winget/choco/scoop
- Docker 容器环境自动检测,跳过 sudo
### API 端点 ### API 端点
@@ -96,34 +92,47 @@ sql/ # 数据库 SQL
- `POST /video/cut/async` - 视频分镜剪切(URL 输入,异步) - `POST /video/cut/async` - 视频分镜剪切(URL 输入,异步)
- `GET /video/cut/task/{taskId}` - 查询异步剪切任务结果 - `GET /video/cut/task/{taskId}` - 查询异步剪切任务结果
**视频拼接+混音:**
- `POST /video/merge/async` - 创建拼接混音异步任务
- `GET /video/merge/task/{taskId}` - 查询任务结果
**字幕叠加:**
- `POST /video/caption` - 创建字幕叠加异步任务
- `GET /video/caption/{taskId}` - 查询任务结果
**场景分割:**
- `POST /video/scene-split` - 创建场景分割异步任务
- `GET /video/scene-split/task/{taskId}` - 查询任务结果
**视频转码:**
- `POST /video/transcode` - 上传视频并转码为 H.264+AAC+MP4(同步)
**视频分析:**
- `POST /video/analysis` - 创建视频分析异步任务
- `GET /video/analysis/task/{taskId}` - 查询分析任务结果
**语音识别:** **语音识别:**
- `POST /audio/transcribe` - 创建语音转文字异步任务 - `POST /audio/transcribe` - 创建语音转文字异步任务
- `GET /audio/task/{taskId}` - 获取转写任务详情 - `GET /audio/task/{taskId}` - 获取转写任务详情
- `GET /audio/task/{taskId}/progress` - 获取任务进度 - `GET /audio/task/{taskId}/progress` - 获取任务进度
- `GET /audio/tasks` - 获取任务列表 - `GET /audio/tasks` - 获取任务列表
**视频分析(规划中):**
- `POST /video/analysis` - 创建视频分析异步任务(基于 Marlin-2B VLM
- `GET /video/analysis/task/{taskId}` - 查询分析任务结果
- `GET /video/analysis/task/{taskId}/progress` - 查询分析任务进度
- `POST /video/analysis/retry/{taskId}` - 重试失败的分析事件
> **Note**: `scene` 场景检测和 `image` 图片处理服务目录已创建,但 HTTP 端点尚未实现暴露。音频提取服务已实现但尚未暴露。
### 依赖外部服务 ### 依赖外部服务
- PostgreSQL - 数据存储 - PostgreSQL - 数据存储
- Redis - 缓存 - Redis - 缓存
- Consul - 服务发现 - Consul - 服务发现
- Jaeger - 链路追踪 - Jaeger - 链路追踪
- OSS/MinIO - 文件存储(通过内部 oss 微服务上传) - OSS/MinIO - 文件存储(通过内部 oss 微服务 `oss/file/uploadFile` 接口上传)
- FFmpeg - 多媒体处理 - FFmpeg + ffprobe - 多媒体处理
- Whisper - 语音识别 - Whisper - 语音识别
- Marlin-2B VLM 服务 - 视频理解大模型(提供字幕生成、事件定位功能) - HyperFrames (npm) - HTML/GSAP 视频渲染引擎,用于字幕叠加
- PySceneDetect - Python 场景检测库
- Marlin-2B VLM 服务 - 视频理解大模型(分析功能使用)
### 内部依赖 ### 内部依赖
项目依赖内部私有公共包 `gitea.redpowerfuture.com/red-future/common`,包含 HTTP 路由注册、用户信息解析、Consul、Jaeger 等基础设施封装。Docker 构建过程中已配置访问凭证。 项目依赖内部私有公共包 `gitea.redpowerfuture.com/red-future/common`,包含 HTTP 路由注册`http.RouteRegister`)、用户信息解析(`beans.User`、Consul、Jaeger 等基础设施封装。Docker 构建过程中已配置访问凭证。
### Docker 镜像 ### Docker 镜像
@@ -139,25 +148,37 @@ sql/ # 数据库 SQL
**分层架构:** **分层架构:**
- `controller` - HTTP 入口,参数解析,调用 Service,返回响应 - `controller` - HTTP 入口,参数解析,调用 Service,返回响应
- `service` - 业务逻辑实现,每个功能领域一个子包 - `service` - 业务逻辑实现,每个功能领域一个子包
- `dao` - 数据访问层,数据库操作 - `dao` - 数据访问层,数据库 CRUD 操作
- `model` - 数据模型,`dto` 存放请求/响应传输对象,`entity` 存放数据库实体 - `model` - 数据模型,`dto` 存放请求/响应传输对象,`entity` 存放数据库实体
**设计模式:** **设计模式:**
- 使用 GoFrame 框架的依赖注入模式 - 使用 GoFrame 框架的依赖注入模式
- 所有 Service 和 Controller 都使用**单例模式**`var Xxx = new(XxxStruct)` - 所有 Service 和 Controller 都使用**单例模式**`var Xxx = new(xxxStruct)`
- 遵循标准的 Go 命名约定 - 路由注册在 `main.go` 中通过 `http.RouteRegister` 集中管理
- 临时文件处理完需要**及时清理**(使用 `defer os.Remove()` - 临时文件处理完需要**及时清理**(使用 `defer os.Remove()``defer os.RemoveAll()`
**异步任务处理:** **异步任务处理:**
- 长时任务(视频拼接、视频剪切、语音识别)都支持**异步执行** - 长时任务(拼接、剪切、合并混音、字幕叠加、场景分割、语音识别、视频分析)都支持**异步执行**
- 同步模式直接等待结果返回,异步模式创建任务后立即返回任务 ID - 同步模式直接等待结果返回,异步模式创建任务后立即返回任务 ID
- 异步任务状态持久化到数据库,可通过任务 ID 查询进度和结果 - 异步任务状态持久化到数据库,可通过任务 ID 查询进度和结果
- 支持回调 URL,任务完成后回调通知调用方 - 支持回调 URL,任务完成后 POST 回调通知调用方,携带 `X-User-Info` 头透传用户信息
- 任务执行使用 goroutine 异步处理 - 任务执行使用 goroutine 异步处理,通过 recover 捕获 panic 并更新为失败状态
- 视频拼接+混音使用信号量控制并发(通过 `merge.concurrency` 配置)
**用户身份:** **用户身份:**
- 所有接口优先从请求头 `Authorization` / `X-User-Info` 解析用户信息 - 所有接口优先从请求头 `Authorization` / `X-User-Info` 解析用户信息
- 解析失败使用默认 `admin` / tenantId=1 用于开发和调试 - 解析失败使用默认 `admin` / tenantId=1 用于开发和调试
- 异步任务通过 `context.WithValue` 将用户信息传递给 goroutine
**Service 层共享工具函数(`service/video` 包内):**
- `getUserFromCtx(ctx)` - 从 context 提取用户信息
- `downloadFile(ctx, url, dir)` - 下载远程文件到本地
- `uploadToMinIO(ctx, localPath)` - 通过 OSS 微服务上传文件到 MinIO
- `lookupFFmpegPath()` - 查找 FFmpeg 可执行文件路径(优先配置路径,回退系统 PATH)
- `getVideoResolution(ctx, path)` - 用 ffprobe 获取视频分辨率
- `getVideoRealDuration(ctx, path)` - 用 ffprobe 获取视频时长(float64秒)
- `getVideoDurationStr(ctx, path)` - 获取视频时长可读字符串(m:ss格式)
- `cleanupFiles(paths)` - 批量清理临时文件
### 配置 ### 配置
@@ -175,16 +196,19 @@ sql/ # 数据库 SQL
- `ffmpeg.path` - FFmpeg 可执行文件路径,留空则从 PATH 自动查找 - `ffmpeg.path` - FFmpeg 可执行文件路径,留空则从 PATH 自动查找
- `ffmpeg.temp_dir` - 临时文件目录(存放上传的视频和处理输出) - `ffmpeg.temp_dir` - 临时文件目录(存放上传的视频和处理输出)
**Whisper 语音识别:**
- `whisper.path` - Whisper 可执行文件路径,留空自动查找
- `whisper.model` - 默认模型(tiny(最快)/base/small/medium
- `whisper.language` - 默认语言(zh=中文, en=英文)
- `whisper.model_dir` - 模型缓存目录,留空使用默认 (~/.cache/whisper/)
- `whisper.threads` - CPU 线程数(限制资源占用,建议 2-4)
**视频分析:** **视频分析:**
- `analysis.concurrency` - 并发处理数,控制同时处理的分析任务数量(默认 1,串行处理,建议不超过 CPU 核心数 - `analysis.video_dir` - 视频永久存储目录(按 taskId 子目录组织
- `analysis.maxRetries` - 单事件最大重试次数,失败时自动重试(默认 3) - `analysis.caption_url` - Caption 接口地址
- `analysis.caption_timeout` - 单次调用超时(默认 `30m`
- `analysis.max_new_tokens` - 传递给 Caption 接口的参数
- `analysis.mock_caption` - 是否启用 mock 模式(true: 返回模拟数据,false: 真实调用)
**视频拼接+混音:**
- `merge.concurrency` - 并发数控制(默认 1,串行处理)
**HyperFrames 字幕叠加:**
- `hyperframes.render_timeout` - 渲染超时时间(分钟,默认 30)
- `hyperframes.headless` - 是否启用 headless 模式(默认 true
**外部服务:** **外部服务:**
- `database` - PostgreSQL 数据库配置 - `database` - PostgreSQL 数据库配置
+13 -16
View File
@@ -73,20 +73,17 @@ analysis:
mock_caption: true mock_caption: true
# OSS/MinIO 文件上传配置 # OSS/MinIO 文件上传配置
filePrefix: "http://116.204.74.41:9000" filePrefix: "http://cdn.redpowerfuture.com"
# 视频拼接+混音配置
merge:
# 并发数,控制同时处理的拼接混音任务数量(默认 1,串行)
concurrency: 1
# HyperFrames 字幕叠加配置
hyperframes:
# HyperFrames 渲染超时时间(分钟),复杂项目可能需要更长时间
render_timeout: 30
# 是否启用 headless 模式
headless: true
# Whisper 语音识别配置
whisper:
# whisper 可执行文件路径,留空则自动查找
# 优先检测: whisper-cpp(推荐) > whisper > python -m whisper
# 安装 whisper.cpp: brew install whisper-cpp(速度比 Python 快 3-5 倍)
path: ""
# 默认模型: tiny(75MB/最快) / base(150MB) / small(500MB) / medium(1.5GB)
# CPU 环境建议用 tinyMacBook Air 用 base 即可
model: "medium"
# 默认语言(zh=中文, en=英文, ja=日文 等)
language: "zh"
# 模型缓存目录,留空使用默认 (~/.cache/whisper/)
model_dir: ""
# CPU 线程数(限制资源占用,建议 2-4)
threads: 2
+39
View File
@@ -0,0 +1,39 @@
package video
import (
"context"
"fmt"
dto "media/model/dto/video"
service "media/service/video"
"github.com/gogf/gf/v2/frame/g"
)
type caption struct{}
var Caption = new(caption)
// CreateCaption 创建字幕叠加任务 POST /video/caption
func (c *caption) CreateCaption(ctx context.Context, req *dto.CreateCaptionTaskReq) (res *dto.CreateCaptionTaskRes, err error) {
ctx = withUser(ctx)
g.Log().Infof(ctx, "[字幕叠加] 收到请求 入参: video_count=%d, element_count=%d, callback=%s",
len(req.VideoURLs), len(req.Elements), req.CallbackURL)
if len(req.VideoURLs) < 1 {
return nil, fmt.Errorf("至少需要1个视频")
}
taskID, taskErr := service.Caption.CreateAsyncTask(ctx, req.VideoURLs, req.AudioURL, req.Subtitles, req.SubtitleStyle, req.Elements, req.CallbackURL)
if taskErr != nil {
return nil, taskErr
}
return &dto.CreateCaptionTaskRes{TaskID: taskID}, nil
}
// GetCaptionTask 查询字幕任务结果 GET /video/caption/{taskId}
func (c *caption) GetCaptionTask(ctx context.Context, req *dto.GetCaptionTaskReq) (res *dto.GetCaptionTaskRes, err error) {
ctx = withUser(ctx)
return service.Caption.GetTaskResult(ctx, req.TaskID)
}
+42
View File
@@ -0,0 +1,42 @@
package video
import (
"context"
"fmt"
dto "media/model/dto/video"
service "media/service/video"
"github.com/gogf/gf/v2/frame/g"
)
type merge struct{}
var Merge = new(merge)
// MergeAsync 视频拼接+混音(异步) POST /video/merge/async
func (c *merge) MergeAsync(ctx context.Context, req *dto.VideoAudioMergeAsyncReq) (res *dto.CreateMergeTaskRes, err error) {
ctx = withUser(ctx)
g.Log().Infof(ctx, "[视频拼接+混音-异步] 收到请求 入参: video_count=%d, audio_count=%d, upload=%v, callback=%s",
len(req.VideoURLs), len(req.AudioURLs), req.Upload, req.CallbackURL)
if len(req.VideoURLs) < 1 {
return nil, fmt.Errorf("至少需要1个视频")
}
if len(req.AudioURLs) < 1 {
return nil, fmt.Errorf("至少需要1个音频")
}
taskID, taskErr := service.Merge.CreateAsyncTask(ctx, req.VideoURLs, req.AudioURLs, req.CallbackURL, req.Upload)
if taskErr != nil {
return nil, taskErr
}
return &dto.CreateMergeTaskRes{TaskID: taskID}, nil
}
// GetMergeTask 查询异步拼接+混音任务结果 GET /video/merge/task/{taskId}
func (c *merge) GetMergeTask(ctx context.Context, req *dto.GetMergeTaskReq) (res *dto.GetMergeTaskRes, err error) {
ctx = withUser(ctx)
return service.Merge.GetTaskResult(ctx, req.TaskID)
}
@@ -0,0 +1,39 @@
package video
import (
"context"
"fmt"
dto "media/model/dto/video"
service "media/service/video"
"github.com/gogf/gf/v2/frame/g"
)
type sceneSplit struct{}
var SceneSplit = new(sceneSplit)
// CreateSceneSplit 创建场景分割任务 POST /video/scene-split
func (c *sceneSplit) CreateSceneSplit(ctx context.Context, req *dto.SceneSplitReq) (res *dto.CreateSceneSplitTaskRes, err error) {
ctx = withUser(ctx)
g.Log().Infof(ctx, "[场景分割] 收到请求 入参: videoUrl=%s, threshold=%.1f, callback=%s",
req.VideoURL, req.Threshold, req.CallbackURL)
if req.VideoURL == "" {
return nil, fmt.Errorf("视频URL不能为空")
}
taskID, taskErr := service.SceneSplit.CreateAsyncTask(ctx, req.VideoURL, req.Threshold, req.CallbackURL)
if taskErr != nil {
return nil, taskErr
}
return &dto.CreateSceneSplitTaskRes{TaskID: taskID}, nil
}
// GetSceneSplitTask 查询场景分割任务结果 GET /video/scene-split/task/{taskId}
func (c *sceneSplit) GetSceneSplitTask(ctx context.Context, req *dto.GetSceneSplitTaskReq) (res *dto.GetSceneSplitTaskRes, err error) {
ctx = withUser(ctx)
return service.SceneSplit.GetTaskResult(ctx, req.TaskID)
}
+42
View File
@@ -0,0 +1,42 @@
package video
import (
"context"
"fmt"
dto "media/model/dto/video"
service "media/service/video"
commonController "media/controller/common"
"github.com/gogf/gf/v2/frame/g"
)
type transcode struct{}
var Transcode = new(transcode)
// Transcode 上传视频并转码为 H.264+AAC+MP4(faststart) POST /video/transcode
func (c *transcode) Transcode(ctx context.Context, req *struct {
g.Meta `path:"/video/transcode" method:"post" tags:"视频转码" summary:"上传视频转码为H.264+AAC+MP4(faststart)" dc:"上传视频文件,转码为标准MP4格式"`
OutputDir string `json:"outputDir" dc:"输出目录(可选),不传则输出到resource/temp下"`
}) (res *dto.TranscodeRes, err error) {
ctx = withUser(ctx)
// 保存上传文件
filePaths, saveErr := commonController.SaveUploadedFilesFromCtx(ctx)
if saveErr != nil || len(filePaths) == 0 {
return nil, fmt.Errorf("上传文件失败: %v", saveErr)
}
inputPath := filePaths[0]
g.Log().Infof(ctx, "[转码] 上传文件保存到: %s", inputPath)
// 调用转码服务,outputDir 为空则 service 用默认值
res, err = service.Transcode.TranscodeToMP4(ctx, inputPath, req.OutputDir)
if err != nil {
return nil, fmt.Errorf("转码失败: %v", err)
}
g.Log().Infof(ctx, "[转码] 完成: %s", res.OutputPath)
return res, nil
}
+159
View File
@@ -0,0 +1,159 @@
package video
import (
"context"
"encoding/json"
"sort"
"strconv"
"strings"
"time"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
var SceneSplitTask = new(sceneSplitTaskDao)
type sceneSplitTaskDao struct{}
const sceneSplitTaskTable = "scene_split_task"
// Insert 创建任务(排除 id 字段,让数据库自增)
func (d *sceneSplitTaskDao) Insert(ctx context.Context, data *entity.SceneSplitTask) (id int64, err error) {
r, err := gfdb.DB(ctx).Model(ctx, sceneSplitTaskTable).
Data(data).
FieldsEx(entity.SceneSplitTaskCols.Id).
Insert()
if err != nil {
return 0, err
}
return r.LastInsertId()
}
// GetByTaskID 根据taskId查询任务
func (d *sceneSplitTaskDao) GetByTaskID(ctx context.Context, taskID string) (res *entity.SceneSplitTask, err error) {
r, err := gfdb.DB(ctx).Model(ctx, sceneSplitTaskTable).
Where(entity.SceneSplitTaskCols.TaskID, taskID).
One()
if err != nil {
return nil, err
}
if r == nil {
return nil, nil
}
err = r.Struct(&res)
return
}
// UpdateRunning 更新为运行中
func (d *sceneSplitTaskDao) UpdateRunning(ctx context.Context, taskID string) error {
_, err := gfdb.DB(ctx).Model(ctx, sceneSplitTaskTable).
Data(g.Map{
entity.SceneSplitTaskCols.Status: "running",
}).
Where(entity.SceneSplitTaskCols.TaskID, taskID).
Update()
return err
}
// UpdateSuccess 更新为成功
func (d *sceneSplitTaskDao) UpdateSuccess(ctx context.Context, taskID string, segmentURLs, audioURL string, sceneCount int, audioDuration float64, videoDuration float64) error {
_, err := gfdb.DB(ctx).Model(ctx, sceneSplitTaskTable).
Data(g.Map{
entity.SceneSplitTaskCols.Status: "success",
entity.SceneSplitTaskCols.SegmentURLs: segmentURLs,
entity.SceneSplitTaskCols.AudioURL: audioURL,
entity.SceneSplitTaskCols.SceneCount: sceneCount,
entity.SceneSplitTaskCols.AudioDuration: audioDuration,
entity.SceneSplitTaskCols.VideoDuration: videoDuration,
entity.SceneSplitTaskCols.ErrorMessage: "",
}).
Where(entity.SceneSplitTaskCols.TaskID, taskID).
Update()
return err
}
// UpdateError 更新为失败
func (d *sceneSplitTaskDao) UpdateError(ctx context.Context, taskID string, errMsg string) error {
_, err := gfdb.DB(ctx).Model(ctx, sceneSplitTaskTable).
Data(g.Map{
entity.SceneSplitTaskCols.Status: "failed",
entity.SceneSplitTaskCols.ErrorMessage: errMsg,
}).
Where(entity.SceneSplitTaskCols.TaskID, taskID).
Update()
return err
}
// EntityToSceneSplitTaskRes 实体转DTO
func EntityToSceneSplitTaskRes(e *entity.SceneSplitTask) *dto.GetSceneSplitTaskRes {
res := &dto.GetSceneSplitTaskRes{
TaskID: e.TaskID,
Status: e.Status,
SceneCount: e.SceneCount,
AudioDuration: e.AudioDuration,
VideoDuration: e.VideoDuration,
}
if e.CreatedAt != nil {
res.CreatedAt = gconv.Int64(e.CreatedAt.Timestamp())
} else {
res.CreatedAt = time.Now().UnixMilli()
}
if e.Status == "success" {
res.AudioURL = e.AudioURL
res.Segments = ParseSegmentEntries(e.SegmentURLs)
res.SceneCount = e.SceneCount
res.AudioDuration = e.AudioDuration
res.VideoDuration = e.VideoDuration
}
if e.Status == "failed" {
res.ErrorMessage = e.ErrorMessage
}
return res
}
// ParseSegmentEntries 将 JSON 数组解析为按时间线排序的 SegmentEntry 切片
// JSON 格式: [{"timeline":"0.0-7.2","url":"..."}, ...]
func ParseSegmentEntries(jsonStr string) []dto.SegmentEntry {
if jsonStr == "" {
return nil
}
// 先尝试解析为数组格式
var entries []dto.SegmentEntry
if err := json.Unmarshal([]byte(jsonStr), &entries); err == nil && len(entries) > 0 {
// 按起始时间排序确保有序
sort.Slice(entries, func(i, j int) bool {
return extractStartTime(entries[i].Timeline) < extractStartTime(entries[j].Timeline)
})
return entries
}
// 兼容旧的 map 格式 {"0.0-7.2":"url", ...}
var m map[string]string
if err := json.Unmarshal([]byte(jsonStr), &m); err != nil {
return nil
}
entries = make([]dto.SegmentEntry, 0, len(m))
for timeline, url := range m {
entries = append(entries, dto.SegmentEntry{Timeline: timeline, URL: url})
}
sort.Slice(entries, func(i, j int) bool {
return extractStartTime(entries[i].Timeline) < extractStartTime(entries[j].Timeline)
})
return entries
}
// extractStartTime 从时间线 key(如 "0.0-7.2")提取起始秒数用于排序
func extractStartTime(timeline string) float64 {
idx := strings.Index(timeline, "-")
if idx < 0 {
return 0
}
v, _ := strconv.ParseFloat(timeline[:idx], 64)
return v
}
+112
View File
@@ -0,0 +1,112 @@
package video
import (
"context"
"time"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/gconv"
)
var MergeTask = new(mergeTaskDao)
type mergeTaskDao struct{}
const mergeTaskTable = "video_audio_merge_task"
// Insert 创建任务(排除 id 字段,让数据库自增)
func (d *mergeTaskDao) Insert(ctx context.Context, data *entity.VideoAudioMergeTask) (id int64, err error) {
r, err := gfdb.DB(ctx).Model(ctx, mergeTaskTable).
Data(data).
FieldsEx(entity.VideoAudioMergeTaskCols.Id).
Insert()
if err != nil {
return 0, err
}
return r.LastInsertId()
}
// GetByTaskID 根据taskId查询任务
func (d *mergeTaskDao) GetByTaskID(ctx context.Context, taskID string) (res *entity.VideoAudioMergeTask, err error) {
r, err := gfdb.DB(ctx).Model(ctx, mergeTaskTable).
Where(entity.VideoAudioMergeTaskCols.TaskID, taskID).
One()
if err != nil {
return nil, err
}
if r == nil {
return nil, nil
}
err = r.Struct(&res)
return
}
// UpdateRunning 更新为运行中
func (d *mergeTaskDao) UpdateRunning(ctx context.Context, taskID string) error {
_, err := gfdb.DB(ctx).Model(ctx, mergeTaskTable).
Data(g.Map{
entity.VideoAudioMergeTaskCols.Status: "running",
}).
Where(entity.VideoAudioMergeTaskCols.TaskID, taskID).
Update()
return err
}
// UpdateSuccess 更新为成功
func (d *mergeTaskDao) UpdateSuccess(ctx context.Context, taskID string, fileURL string, fileSize int64, fileName, fileFormat, fileAddrPrefix, durationStr string) error {
_, err := gfdb.DB(ctx).Model(ctx, mergeTaskTable).
Data(g.Map{
entity.VideoAudioMergeTaskCols.Status: "success",
entity.VideoAudioMergeTaskCols.FileURL: fileURL,
entity.VideoAudioMergeTaskCols.FileSize: fileSize,
entity.VideoAudioMergeTaskCols.FileName: fileName,
entity.VideoAudioMergeTaskCols.FileFormat: fileFormat,
entity.VideoAudioMergeTaskCols.FileAddressPrefix: fileAddrPrefix,
entity.VideoAudioMergeTaskCols.DurationStr: durationStr,
entity.VideoAudioMergeTaskCols.ErrorMessage: "",
}).
Where(entity.VideoAudioMergeTaskCols.TaskID, taskID).
Update()
return err
}
// UpdateError 更新为失败
func (d *mergeTaskDao) UpdateError(ctx context.Context, taskID string, errMsg string) error {
_, err := gfdb.DB(ctx).Model(ctx, mergeTaskTable).
Data(g.Map{
entity.VideoAudioMergeTaskCols.Status: "failed",
entity.VideoAudioMergeTaskCols.ErrorMessage: errMsg,
}).
Where(entity.VideoAudioMergeTaskCols.TaskID, taskID).
Update()
return err
}
// EntityToTaskRes 实体转DTO
func EntityToMergeTaskRes(e *entity.VideoAudioMergeTask) *dto.GetMergeTaskRes {
res := &dto.GetMergeTaskRes{
TaskID: e.TaskID,
Status: e.Status,
}
if e.CreatedAt != nil {
res.CreatedAt = gconv.Int64(e.CreatedAt.Timestamp())
} else {
res.CreatedAt = time.Now().UnixMilli()
}
if e.Status == "success" {
res.FileURL = e.FileURL
res.FileSize = e.FileSize
res.FileName = e.FileName
res.FileFormat = e.FileFormat
res.FileAddressPrefix = e.FileAddressPrefix
res.DurationStr = e.DurationStr
}
if e.Status == "failed" {
res.ErrorMessage = e.ErrorMessage
}
return res
}
+98
View File
@@ -0,0 +1,98 @@
package video
import (
"context"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/db/gfdb"
"github.com/gogf/gf/v2/frame/g"
)
// CaptionTask 字幕叠加任务 DAO 单例
var CaptionTask = new(captionTaskDao)
type captionTaskDao struct{}
const captionTaskTable = "video_caption_task"
// Insert 插入字幕任务记录
func (d *captionTaskDao) Insert(ctx context.Context, data *entity.VideoCaptionTask) (id int64, err error) {
r, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).Data(data).Insert()
if err != nil {
return 0, err
}
return r.LastInsertId()
}
// GetByTaskID 按 taskId 查询
func (d *captionTaskDao) GetByTaskID(ctx context.Context, taskID string) (res *entity.VideoCaptionTask, err error) {
r, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).
Where(entity.VideoCaptionTaskCols.TaskID, taskID).One()
if err != nil {
return nil, err
}
if r == nil {
return nil, nil
}
err = r.Struct(&res)
return
}
// UpdateRunning 更新任务为运行中
func (d *captionTaskDao) UpdateRunning(ctx context.Context, taskID string) error {
_, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).
Data(g.Map{entity.VideoCaptionTaskCols.Status: "running"}).
Where(entity.VideoCaptionTaskCols.TaskID, taskID).Update()
return err
}
// UpdateResolution 更新视频分辨率
func (d *captionTaskDao) UpdateResolution(ctx context.Context, taskID string, width, height int) error {
_, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).
Data(g.Map{
entity.VideoCaptionTaskCols.Width: width,
entity.VideoCaptionTaskCols.Height: height,
}).
Where(entity.VideoCaptionTaskCols.TaskID, taskID).Update()
return err
}
// UpdateSuccess 更新任务为成功
func (d *captionTaskDao) UpdateSuccess(ctx context.Context, taskID, fileURL string, fileSize int64, fileName, durationStr string) error {
_, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).
Data(g.Map{
entity.VideoCaptionTaskCols.Status: "success",
entity.VideoCaptionTaskCols.FileURL: fileURL,
entity.VideoCaptionTaskCols.FileSize: fileSize,
entity.VideoCaptionTaskCols.FileName: fileName,
entity.VideoCaptionTaskCols.DurationStr: durationStr,
}).
Where(entity.VideoCaptionTaskCols.TaskID, taskID).Update()
return err
}
// UpdateError 更新任务为失败
func (d *captionTaskDao) UpdateError(ctx context.Context, taskID, errMsg string) error {
_, err := gfdb.DB(ctx).Model(ctx, captionTaskTable).
Data(g.Map{
entity.VideoCaptionTaskCols.Status: "failed",
entity.VideoCaptionTaskCols.ErrorMessage: errMsg,
}).
Where(entity.VideoCaptionTaskCols.TaskID, taskID).Update()
return err
}
// EntityToCaptionTaskRes 实体转查询响应
func EntityToCaptionTaskRes(e *entity.VideoCaptionTask) *dto.GetCaptionTaskRes {
return &dto.GetCaptionTaskRes{
TaskID: e.TaskID,
Status: e.Status,
FileURL: e.FileURL,
FileSize: e.FileSize,
FileName: e.FileName,
DurationStr: e.DurationStr,
ErrorMessage: e.ErrorMessage,
}
}
+7
View File
@@ -8,6 +8,9 @@ import (
_ "gitea.redpowerfuture.com/red-future/common/consul" _ "gitea.redpowerfuture.com/red-future/common/consul"
"gitea.redpowerfuture.com/red-future/common/http" "gitea.redpowerfuture.com/red-future/common/http"
"gitea.redpowerfuture.com/red-future/common/jaeger" "gitea.redpowerfuture.com/red-future/common/jaeger"
_ "media/service/setup"
_ "github.com/gogf/gf/contrib/drivers/pgsql/v2" _ "github.com/gogf/gf/contrib/drivers/pgsql/v2"
) )
@@ -20,6 +23,10 @@ func main() {
controllerVideo.Concat, controllerVideo.Concat,
controllerVideo.Cut, controllerVideo.Cut,
controllerVideo.Analysis, controllerVideo.Analysis,
controllerVideo.Merge,
controllerVideo.Caption,
controllerVideo.Transcode,
controllerVideo.SceneSplit,
}) })
select {} select {}
} }
+43
View File
@@ -0,0 +1,43 @@
package video
import "github.com/gogf/gf/v2/frame/g"
// ---------- 场景检测+视频分割(异步)----------
// SceneSplitReq 场景检测+视频分割请求
type SceneSplitReq struct {
g.Meta `path:"/scene-split" method:"post" tags:"场景分割" summary:"场景检测并分割视频(异步)" dc:"下载视频,基于PySceneDetect检测场景变化并分割成多个视频片段,同时提取音频,全部上传到MinIO后回调通知"`
VideoURL string `json:"videoUrl" v:"required#视频URL不能为空" dc:"视频URL地址"`
Threshold float64 `json:"threshold" dc:"场景检测阈值(默认27),越小越敏感、场景切割越多" d:"27.0"`
CallbackURL string `json:"callbackUrl" dc:"回调地址,处理完成后POST结果到该地址(可选)"`
}
// CreateSceneSplitTaskRes 创建场景分割任务响应
type CreateSceneSplitTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
}
// GetSceneSplitTaskReq 查询场景分割任务请求
type GetSceneSplitTaskReq struct {
g.Meta `path:"/scene-split/task/{taskId}" method:"get" tags:"场景分割" summary:"查询场景分割任务结果" dc:"根据taskId查询异步场景分割任务结果"`
TaskID string `json:"taskId" dc:"任务ID"`
}
// SegmentEntry 单个分片信息(按时间线排列)
type SegmentEntry struct {
Timeline string `json:"timeline" dc:"时间线标识,如 0.0-7.2"`
URL string `json:"url" dc:"分片MinIO地址"`
}
// GetSceneSplitTaskRes 查询场景分割任务响应
type GetSceneSplitTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
Status string `json:"status" dc:"状态: pending/running/success/failed"`
AudioURL string `json:"audioUrl,omitempty" dc:"提取的音频MinIO地址"`
Segments []SegmentEntry `json:"segments,omitempty" dc:"按时间线排列的视频分片列表"`
SceneCount int `json:"sceneCount" dc:"检测到的场景数/分片数"`
AudioDuration float64 `json:"audioDuration,omitempty" dc:"音频时长(秒)"`
VideoDuration float64 `json:"videoDuration,omitempty" dc:"视频总时长(秒)"`
ErrorMessage string `json:"errorMessage,omitempty" dc:"错误信息"`
CreatedAt int64 `json:"createdAt" dc:"创建时间戳"`
}
+9
View File
@@ -0,0 +1,9 @@
package video
// TranscodeRes 视频转码响应
type TranscodeRes struct {
OutputPath string `json:"outputPath" dc:"输出文件完整路径"`
FileSize int64 `json:"fileSize" dc:"文件大小(字节)"`
FileName string `json:"fileName" dc:"输出文件名"`
DurationStr string `json:"durationStr" dc:"视频时长"`
}
+39
View File
@@ -0,0 +1,39 @@
package video
import "github.com/gogf/gf/v2/frame/g"
// ---------- 视频拼接+混音(异步) ----------
// VideoAudioMergeAsyncReq 视频拼接+混音异步请求
type VideoAudioMergeAsyncReq struct {
g.Meta `path:"/merge/async" method:"post" tags:"视频拼接+混音" summary:"视频拼接并混音(异步)" dc:"将多个视频拼接后混入多个音频(按顺序拼接),立即返回taskId,完成后通过callback_url通知结果"`
VideoURLs []string `json:"video_urls" v:"required#视频URL列表不能为空" dc:"视频URL列表(按此顺序拼接)"`
AudioURLs []string `json:"audio_urls" v:"required#音频URL列表不能为空" dc:"音频URL列表(按此顺序拼接)"`
Upload bool `json:"upload" dc:"是否上传到MinIO" d:"false"`
CallbackURL string `json:"callback_url" v:"required#回调地址不能为空" dc:"回调地址,处理完成后POST结果到该地址"`
}
// CreateMergeTaskRes 创建异步拼接+混音任务响应
type CreateMergeTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
}
// GetMergeTaskReq 查询异步拼接+混音任务请求
type GetMergeTaskReq struct {
g.Meta `path:"/merge/task/{taskId}" method:"get" tags:"视频拼接+混音" summary:"查询拼接混音任务结果" dc:"根据taskId查询异步拼接+混音任务的结果"`
TaskID string `json:"taskId" dc:"任务ID"`
}
// GetMergeTaskRes 查询异步拼接+混音任务响应
type GetMergeTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
Status string `json:"status" dc:"状态: pending/running/success/failed"`
FileURL string `json:"fileURL,omitempty" dc:"MinIO文件访问路径"`
FileSize int64 `json:"fileSize,omitempty" dc:"文件大小(字节)"`
FileName string `json:"fileName,omitempty" dc:"文件名"`
FileFormat string `json:"fileFormat,omitempty" dc:"文件格式"`
FileAddressPrefix string `json:"fileAddressPrefix,omitempty" dc:"MinIO地址前缀"`
DurationStr string `json:"durationStr,omitempty" dc:"合并后视频时长"`
ErrorMessage string `json:"errorMessage,omitempty" dc:"错误信息"`
CreatedAt int64 `json:"createdAt" dc:"创建时间戳"`
}
+78
View File
@@ -0,0 +1,78 @@
package video
import "github.com/gogf/gf/v2/frame/g"
// ---------- 字幕元素定义 ----------
// CaptionElement 单个字幕/图片元素
type CaptionElement struct {
Type string `json:"type" v:"required#元素类型必填" dc:"元素类型:text/image"`
Text string `json:"text" dc:"文字内容(type=text时必填)"`
ImageURL string `json:"imageUrl" dc:"图片URL(type=image时必填)"`
StartTime float64 `json:"startTime" d:"0" dc:"开始时间(秒,默认0)"`
Duration float64 `json:"duration" d:"0" dc:"持续时长(秒,0或空=一直显示到视频结束)"`
X string `json:"x" d:"center" dc:"水平位置:left/center/right或像素值"`
Y string `json:"y" d:"center" dc:"垂直位置:top/center/bottom或像素值"`
FontSize int `json:"fontSize" d:"36" dc:"字号(type=text有效)"`
FontColor string `json:"fontColor" d:"#FFFFFF" dc:"字体颜色"`
BgColor string `json:"bgColor" dc:"背景色,如#FFFF00,空=透明"`
BgOpacity float64 `json:"bgOpacity" d:"1.0" dc:"背景透明度0-1"`
Width int `json:"width" dc:"元素宽度(type=image有效)"`
Height int `json:"height" dc:"元素高度(type=image有效)"`
Animation string `json:"animation" dc:"动画效果:fadeIn/slideUp/slideLeft/scaleIn"`
TrackIndex int `json:"trackIndex" d:"1" dc:"轨道层级(数字越大越靠前)"`
}
// SubtitleSegment 外部传入的字幕时间线片段
type SubtitleSegment struct {
Start float64 `json:"start" v:"required#开始时间必填" dc:"开始时间(秒)"`
End float64 `json:"end" v:"required#结束时间必填" dc:"结束时间(秒)"`
Text string `json:"text" v:"required#字幕文本必填" dc:"字幕文本内容"`
}
// SubtitleStyle 字幕样式配置
type SubtitleStyle struct {
FontSize int `json:"fontSize" d:"28" dc:"字号(默认28)"`
FontColor string `json:"fontColor" d:"#FFFFFF" dc:"字体颜色(默认白色)"`
BgColor *string `json:"bgColor" dc:"背景色,空字符串=透明(默认#000000)。传空字符串 '' 可去掉背景色"`
BgOpacity *float64 `json:"bgOpacity" dc:"背景透明度0-1(默认0.6)。传0可使背景完全透明"`
X string `json:"x" d:"center" dc:"水平位置:left/center/right或像素值(默认center)"`
Y string `json:"y" d:"bottom" dc:"垂直位置:top/center/bottom或像素值(默认bottom)"`
}
// ---------- 创建字幕任务 ----------
// CreateCaptionTaskReq 创建字幕叠加任务请求
type CreateCaptionTaskReq struct {
g.Meta `path:"/" method:"post" tags:"字幕叠加" summary:"创建字幕叠加任务(异步)" dc:"创建使用HyperFrames渲染的视频字幕叠加任务,返回taskId"`
VideoURLs []string `json:"video_urls" v:"required#视频URL列表不能为空" dc:"背景视频URL列表"`
AudioURL string `json:"audio_url" dc:"背景音频URL(可选)"`
Subtitles []SubtitleSegment `json:"subtitles" dc:"字幕时间线列表(可选),每段包含起始时间和文本"`
SubtitleStyle *SubtitleStyle `json:"subtitle_style" dc:"字幕样式配置(可选),不传则使用默认值"`
Elements []CaptionElement `json:"elements" dc:"字幕/图片元素列表(可选)"`
CallbackURL string `json:"callback_url" dc:"任务完成后的回调地址(可选)"`
}
// CreateCaptionTaskRes 创建字幕任务响应
type CreateCaptionTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
}
// ---------- 查询字幕任务 ----------
// GetCaptionTaskReq 查询字幕任务请求
type GetCaptionTaskReq struct {
g.Meta `path:"/{taskId}" method:"get" tags:"字幕叠加" summary:"查询字幕任务结果" dc:"根据taskId查询字幕任务详情"`
TaskID string `json:"taskId" dc:"任务ID"`
}
// GetCaptionTaskRes 查询字幕任务响应
type GetCaptionTaskRes struct {
TaskID string `json:"taskId" dc:"任务ID"`
Status string `json:"status" dc:"任务状态"`
FileURL string `json:"fileUrl,omitempty" dc:"输出文件URL"`
FileSize int64 `json:"fileSize,omitempty" dc:"输出文件大小"`
FileName string `json:"fileName,omitempty" dc:"输出文件名"`
DurationStr string `json:"durationStr,omitempty" dc:"视频时长"`
ErrorMessage string `json:"errorMessage,omitempty" dc:"错误信息"`
}
+48
View File
@@ -0,0 +1,48 @@
package video
import "gitea.redpowerfuture.com/red-future/common/beans"
// SceneSplitTask 场景检测+视频分割异步任务实体
type SceneSplitTask struct {
beans.SQLBaseDO `orm:",inherit"`
TaskID string `orm:"task_id" json:"taskId" description:"任务唯一标识"`
VideoURL string `orm:"video_url" json:"videoUrl" description:"原始视频URL"`
Status string `orm:"status" json:"status" description:"任务状态:pending/running/success/failed"`
SegmentURLs string `orm:"segment_urls" json:"segmentURLs" description:"分片列表JSON数组,如 [{\"timeline\":\"0.0-5.2\",\"url\":\"...\"}]"`
AudioURL string `orm:"audio_url" json:"audioUrl" description:"提取的音频URL"`
SceneCount int `orm:"scene_count" json:"sceneCount" description:"场景数/分片数"`
AudioDuration float64 `orm:"audio_duration" json:"audioDuration" description:"音频时长(秒)"`
VideoDuration float64 `orm:"video_duration" json:"videoDuration" description:"视频总时长(秒)"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息"`
CallbackURL string `orm:"callback_url" json:"callbackUrl" description:"回调地址"`
}
// SceneSplitTaskCol 字段定义
type SceneSplitTaskCol struct {
beans.SQLBaseCol
TaskID string
VideoURL string
Status string
SegmentURLs string
AudioURL string
SceneCount string
AudioDuration string
VideoDuration string
ErrorMessage string
CallbackURL string
}
// SceneSplitTaskCols 字段常量
var SceneSplitTaskCols = SceneSplitTaskCol{
SQLBaseCol: beans.DefSQLBaseCol,
TaskID: "task_id",
VideoURL: "video_url",
Status: "status",
SegmentURLs: "segment_urls",
AudioURL: "audio_url",
SceneCount: "scene_count",
AudioDuration: "audio_duration",
VideoDuration: "video_duration",
ErrorMessage: "error_message",
CallbackURL: "callback_url",
}
@@ -0,0 +1,54 @@
package video
import "gitea.redpowerfuture.com/red-future/common/beans"
// VideoAudioMergeTask 视频拼接+混音异步任务实体
type VideoAudioMergeTask struct {
beans.SQLBaseDO `orm:",inherit"`
TaskID string `orm:"task_id" json:"taskId" description:"任务唯一标识"`
VideoURLs string `orm:"video_urls" json:"videoUrls" description:"视频URL列表(JSON数组)"`
AudioURLs string `orm:"audio_urls" json:"audioUrls" description:"音频URL列表(JSON数组)"`
Status string `orm:"status" json:"status" description:"任务状态:pending/running/success/failed"`
FileURL string `orm:"file_url" json:"fileUrl" description:"MinIO文件访问路径"`
FileSize int64 `orm:"file_size" json:"fileSize" description:"文件大小(字节)"`
FileName string `orm:"file_name" json:"fileName" description:"文件名"`
FileFormat string `orm:"file_format" json:"fileFormat" description:"文件格式"`
FileAddressPrefix string `orm:"file_address_prefix" json:"fileAddressPrefix" description:"MinIO地址前缀"`
DurationStr string `orm:"duration_str" json:"durationStr" description:"合并后视频时长"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息"`
CallbackURL string `orm:"callback_url" json:"callbackUrl" description:"回调地址"`
}
// VideoAudioMergeTaskCol 字段定义
type VideoAudioMergeTaskCol struct {
beans.SQLBaseCol
TaskID string
VideoURLs string
AudioURLs string
Status string
FileURL string
FileSize string
FileName string
FileFormat string
FileAddressPrefix string
DurationStr string
ErrorMessage string
CallbackURL string
}
// VideoAudioMergeTaskCols 字段常量
var VideoAudioMergeTaskCols = VideoAudioMergeTaskCol{
SQLBaseCol: beans.DefSQLBaseCol,
TaskID: "task_id",
VideoURLs: "video_urls",
AudioURLs: "audio_urls",
Status: "status",
FileURL: "file_url",
FileSize: "file_size",
FileName: "file_name",
FileFormat: "file_format",
FileAddressPrefix: "file_address_prefix",
DurationStr: "duration_str",
ErrorMessage: "error_message",
CallbackURL: "callback_url",
}
+59
View File
@@ -0,0 +1,59 @@
package video
import (
"gitea.redpowerfuture.com/red-future/common/beans"
)
// VideoCaptionTask 字幕叠加任务实体
type VideoCaptionTask struct {
beans.SQLBaseDO `orm:",inherit"`
TaskID string `orm:"task_id" json:"taskId" description:"任务唯一标识"`
VideoURLs string `orm:"video_urls" json:"videoUrls" description:"视频URL列表JSON"`
AudioURL string `orm:"audio_url" json:"audioUrl" description:"背景音乐URL"`
Elements string `orm:"elements" json:"elements" description:"字幕元素列表JSON"`
Width int `orm:"width" json:"width" description:"视频宽度"`
Height int `orm:"height" json:"height" description:"视频高度"`
Status string `orm:"status" json:"status" description:"任务状态"`
FileURL string `orm:"file_url" json:"fileUrl" description:"输出文件URL"`
FileSize int64 `orm:"file_size" json:"fileSize" description:"输出文件大小"`
FileName string `orm:"file_name" json:"fileName" description:"输出文件名"`
DurationStr string `orm:"duration_str" json:"durationStr" description:"视频时长"`
ErrorMessage string `orm:"error_message" json:"errorMessage" description:"错误信息"`
CallbackURL string `orm:"callback_url" json:"callbackUrl" description:"回调地址"`
}
// VideoCaptionTaskCol 数据库字段名常量
type VideoCaptionTaskCol struct {
beans.SQLBaseCol
TaskID string
VideoURLs string
AudioURL string
Elements string
Width string
Height string
Status string
FileURL string
FileSize string
FileName string
DurationStr string
ErrorMessage string
CallbackURL string
}
// VideoCaptionTaskCols 字段名常量实例
var VideoCaptionTaskCols = VideoCaptionTaskCol{
SQLBaseCol: beans.DefSQLBaseCol,
TaskID: "task_id",
VideoURLs: "video_urls",
AudioURL: "audio_url",
Elements: "elements",
Width: "width",
Height: "height",
Status: "status",
FileURL: "file_url",
FileSize: "file_size",
FileName: "file_name",
DurationStr: "duration_str",
ErrorMessage: "error_message",
CallbackURL: "callback_url",
}
+2 -2
View File
@@ -44,10 +44,10 @@ func (s *audioTaskService) Create(ctx context.Context, params *CreateTaskParams)
taskID := "tsk_" + guid.S() taskID := "tsk_" + guid.S()
if params.Model == "" { if params.Model == "" {
params.Model = g.Cfg().MustGet(ctx, "whisper.model", "medium").String() params.Model = "medium"
} }
if params.Language == "" { if params.Language == "" {
params.Language = g.Cfg().MustGet(ctx, "whisper.language", "zh").String() params.Language = "zh"
} }
if params.Threshold <= 0 { if params.Threshold <= 0 {
params.Threshold = 0.3 params.Threshold = 0.3
+1 -30
View File
@@ -13,7 +13,6 @@ import (
"time" "time"
dto "media/model/dto/audio" dto "media/model/dto/audio"
serviceAudio "media/service/audio"
serviceScene "media/service/scene" serviceScene "media/service/scene"
"github.com/gogf/gf/v2/frame/g" "github.com/gogf/gf/v2/frame/g"
@@ -123,35 +122,7 @@ func (s *transcribeService) processVideos(ctx context.Context, savePaths []strin
// TranscribeVideo 从视频提取音频并转为文字 // TranscribeVideo 从视频提取音频并转为文字
func (s *transcribeService) TranscribeVideo(ctx context.Context, req *VideoTranscribeReq) (res *VideoTranscribeRes, err error) { func (s *transcribeService) TranscribeVideo(ctx context.Context, req *VideoTranscribeReq) (res *VideoTranscribeRes, err error) {
audioReq := &serviceAudio.ExtractAudioReq{VideoPath: req.VideoPath, Format: "mp3"} return nil, fmt.Errorf("语音识别功能已移除(Whisper 已卸载)")
audioRes, err := serviceAudio.AudioExtract.Extract(ctx, audioReq)
if err != nil {
return nil, fmt.Errorf("音频提取失败: %v", err)
}
whisperRes, err := Whisper.Transcribe(ctx, &TranscribeReq{AudioPath: audioRes.AudioPath, Model: req.Model, Language: req.Language})
if err != nil {
os.Remove(audioRes.AudioPath)
return nil, fmt.Errorf("语音识别失败: %v", err)
}
os.Remove(req.VideoPath)
if !req.KeepAudio {
os.Remove(audioRes.AudioPath)
baseName := strings.TrimSuffix(audioRes.AudioPath, filepath.Ext(audioRes.AudioPath))
os.Remove(baseName + ".txt")
os.Remove(baseName + "." + whisperRes.Model + ".txt")
}
res = &VideoTranscribeRes{
Text: whisperRes.Text,
Model: whisperRes.Model,
Language: whisperRes.Language,
AudioPath: audioRes.AudioPath,
AudioSize: audioRes.Size,
AudioDuration: audioRes.Duration,
}
return
} }
func downloadFromURL(ctx context.Context, rawURL, tempDir string) (string, error) { func downloadFromURL(ctx context.Context, rawURL, tempDir string) (string, error) {
-403
View File
@@ -1,403 +0,0 @@
package asr
import (
"context"
"fmt"
"io"
"media/service/setup"
"net/http"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"time"
"github.com/gogf/gf/v2/frame/g"
)
// WhisperBackend 后端类型
type WhisperBackend int
const (
backendPython WhisperBackend = iota // python -m whisper
backendCLI // openai-whisper CLI (whisper 命令)
backendCpp // whisper.cpp (whisper-cpp)
)
type whisperService struct{}
// Whisper 语音识别服务单例
var Whisper = new(whisperService)
// TranscribeReq 语音识别请求
type TranscribeReq struct {
AudioPath string // 音频文件路径
Model string // whisper 模型: tiny/base/small/medium/large
Language string // 语言代码,默认 zh(中文)
}
// TranscribeRes 语音识别响应
type TranscribeRes struct {
Text string // 完整识别文本
Segments []Segment
Model string // 使用的模型
Language string // 识别的语言
OutputPath string // 输出的 txt 文件路径
}
// Segment 识别片段(带时间戳)
type Segment struct {
Start float64 `json:"start"` // 开始时间(秒)
End float64 `json:"end"` // 结束时间(秒)
Text string `json:"text"` // 文本内容
}
// Transcribe 对音频文件进行语音识别(自动检测后端,自动降级)
func (s *whisperService) Transcribe(ctx context.Context, req *TranscribeReq) (res *TranscribeRes, err error) {
// 1. 校验音频文件
if _, err = os.Stat(req.AudioPath); os.IsNotExist(err) {
return nil, fmt.Errorf("音频文件不存在: %s", req.AudioPath)
}
// 2. 设置默认值
model := req.Model
if model == "" {
model = g.Cfg().MustGet(ctx, "whisper.model", "small").String()
}
language := req.Language
if language == "" {
language = g.Cfg().MustGet(ctx, "whisper.language", "zh").String()
}
// 3. 检测后端,C++ 版找不到模型文件时自动降级
backend, whisperPath := s.detectBackend()
if backend == backendCpp {
modelPath := s.resolveCppModelPath(model)
if modelPath == "" {
g.Log().Warningf(ctx, "whisper.cpp 模型文件(%s)未找到,降级到 Python whisper", model)
backend = backendPython
} else {
g.Log().Infof(ctx, "语音识别(whisper.cpp): audio=%s, model=%s", req.AudioPath, modelPath)
return s.transcribeWithCpp(ctx, req, whisperPath, modelPath, language)
}
}
switch backend {
case backendCLI:
g.Log().Infof(ctx, "语音识别(CLI): audio=%s, model=%s, language=%s", req.AudioPath, model, language)
return s.transcribeWithCLI(ctx, req, whisperPath, model, language)
default:
g.Log().Infof(ctx, "语音识别(python): audio=%s, model=%s, language=%s", req.AudioPath, model, language)
return s.transcribeWithPython(ctx, req, model, language)
}
}
// transcribeWithCLI 使用 whisper CLI 命令
func (s *whisperService) transcribeWithCLI(ctx context.Context, req *TranscribeReq, whisperPath, model, language string) (res *TranscribeRes, err error) {
outputDir := filepath.Dir(req.AudioPath)
modelDir := g.Cfg().MustGet(ctx, "whisper.model_dir", "").String()
threads := g.Cfg().MustGet(ctx, "whisper.threads", 2).Int()
args := []string{
req.AudioPath,
"--model", model,
"--language", language,
"--output_dir", outputDir,
"--output_format", "txt",
"--threads", fmt.Sprintf("%d", threads),
}
if modelDir != "" {
args = append(args, "--model_dir", modelDir)
}
cmd := exec.CommandContext(ctx, whisperPath, args...)
output, execErr := cmd.CombinedOutput()
if execErr != nil {
g.Log().Errorf(ctx, "whisper CLI 执行失败: %v\n%s", execErr, string(output))
return nil, fmt.Errorf("语音识别失败: %v", execErr)
}
return s.readTxtResult(outputDir, req.AudioPath, model)
}
// transcribeWithPython 使用 python -m whisper
func (s *whisperService) transcribeWithPython(ctx context.Context, req *TranscribeReq, model, language string) (res *TranscribeRes, err error) {
// 查找 python
pythonPath, err := exec.LookPath("python3")
if err != nil {
pythonPath, err = exec.LookPath("python")
if err != nil {
return nil, fmt.Errorf("未找到 python,请安装: pip3 install openai-whisper")
}
}
outputDir := filepath.Dir(req.AudioPath)
modelDir := g.Cfg().MustGet(ctx, "whisper.model_dir", "").String()
threads := g.Cfg().MustGet(ctx, "whisper.threads", 2).Int()
args := []string{
"-m", "whisper",
req.AudioPath,
"--model", model,
"--language", language,
"--output_dir", outputDir,
"--output_format", "txt",
"--threads", fmt.Sprintf("%d", threads),
}
if modelDir != "" {
args = append(args, "--model_dir", modelDir)
}
cmd := exec.CommandContext(ctx, pythonPath, args...)
output, execErr := cmd.CombinedOutput()
if execErr != nil {
g.Log().Errorf(ctx, "whisper(python) 执行失败: %v\n%s", execErr, string(output))
return nil, fmt.Errorf("语音识别失败: %v", execErr)
}
return s.readTxtResult(outputDir, req.AudioPath, model)
}
// readTxtResult 读取 whisper 输出的 txt 文件
func (s *whisperService) readTxtResult(outputDir, audioPath, model string) (res *TranscribeRes, err error) {
baseName := strings.TrimSuffix(filepath.Base(audioPath), filepath.Ext(audioPath))
txtPaths := []string{
filepath.Join(outputDir, baseName+".txt"),
filepath.Join(outputDir, baseName+"."+model+".txt"),
}
var textBytes []byte
var txtPath string
for _, p := range txtPaths {
if b, e := os.ReadFile(p); e == nil {
textBytes = b
txtPath = p
break
}
}
if textBytes == nil {
return nil, fmt.Errorf("读取识别结果文件失败")
}
res = &TranscribeRes{
Text: cleanTranscript(string(textBytes)),
Model: model,
OutputPath: txtPath,
}
return
}
// cleanTranscript 清理识别结果:去换行、合并空格
func cleanTranscript(text string) string {
text = strings.ReplaceAll(text, "\r\n", " ")
text = strings.ReplaceAll(text, "\n", " ")
text = strings.ReplaceAll(text, "\r", " ")
// 合并多个空格
for strings.Contains(text, " ") {
text = strings.ReplaceAll(text, " ", " ")
}
return strings.TrimSpace(text)
}
// detectBackend 检测可用的 whisper 后端,返回后端类型和可执行路径
func (s *whisperService) detectBackend() (WhisperBackend, string) {
// 1. 优先检测 C++ 版 whisper.cpp(最快,但参数格式不同)
for _, name := range []string{"whisper-cpp", "whisper-cli"} {
if path, err := exec.LookPath(name); err == nil {
return backendCpp, path
}
}
// 2. 检查 setup 检测到的 C++ 路径
if setup.DetectedWhisperPath != "" {
base := filepath.Base(setup.DetectedWhisperPath)
if base == "whisper-cpp" || base == "whisper-cli" {
if _, err := os.Stat(setup.DetectedWhisperPath); err == nil {
return backendCpp, setup.DetectedWhisperPath
}
}
}
// 3. 检测 Python CLIwhisper 命令)
if path, err := exec.LookPath("whisper"); err == nil {
return backendCLI, path
}
// 4. 检查 setup 检测到的 Python CLI 路径
if setup.DetectedWhisperPath != "" {
if _, err := os.Stat(setup.DetectedWhisperPath); err == nil {
return backendCLI, setup.DetectedWhisperPath
}
}
// 5. 检查配置中的路径
if p := g.Cfg().MustGet(context.Background(), "whisper.path", "").String(); p != "" {
if _, err := os.Stat(p); err == nil {
return backendCLI, p
}
}
return backendPython, ""
}
// resolveCppModelPath 查找或下载 whisper.cpp 模型文件
func (s *whisperService) resolveCppModelPath(model string) string {
modelName := strings.TrimPrefix(model, "ggml-")
modelName = strings.TrimSuffix(modelName, ".bin")
cppModelName := "ggml-" + modelName + ".bin"
home, _ := os.UserHomeDir()
// 目标路径:~/.cache/whisper/ggml-{model}.bin
targetDir := filepath.Join(home, ".cache", "whisper")
targetPath := filepath.Join(targetDir, cppModelName)
// 1. 如果已存在,直接返回
if _, err := os.Stat(targetPath); err == nil {
return targetPath
}
// 2. 检查其他常见位置
altPaths := []string{
cppModelName,
filepath.Join(home, ".cache", "whisper", "ggml-"+modelName+"-q5_0.bin"),
}
// macOS: Homebrew 安装的 whisper.cpp 模型路径
if runtime.GOOS == "darwin" {
altPaths = append(altPaths,
"/opt/homebrew/share/whisper-cpp/models/"+cppModelName,
"/usr/local/share/whisper-cpp/models/"+cppModelName,
)
}
// Linux: 常见系统安装路径
if runtime.GOOS == "linux" {
altPaths = append(altPaths,
"/usr/share/whisper-cpp/models/"+cppModelName,
"/usr/local/share/whisper-cpp/models/"+cppModelName,
)
}
for _, p := range altPaths {
if _, err := os.Stat(p); err == nil {
return p
}
}
// 3. 自动下载
modelSize := map[string]string{
"tiny": "75MB",
"base": "150MB",
"small": "500MB",
"medium": "1.5GB",
}
size, _ := modelSize[modelName]
// 下载源:先试 hf-mirror(国内可访问),失败再试官方
modelPath := fmt.Sprintf("ggerganov/whisper.cpp/resolve/main/%s", cppModelName)
urls := []string{
fmt.Sprintf("https://hf-mirror.com/%s", modelPath),
fmt.Sprintf("https://huggingface.co/%s", modelPath),
}
g.Log().Infof(context.TODO(), "[whisper.cpp] 正在下载模型 %s (%s)...", cppModelName, size)
// 创建目录
os.MkdirAll(targetDir, 0755)
// 下载文件(多个源,依次尝试)
var lastErr error
for _, url := range urls {
g.Log().Infof(context.TODO(), "[whisper.cpp] 下载地址: %s", url)
if err := s.downloadFile(url, targetPath, 5*time.Minute); err == nil {
g.Log().Infof(context.TODO(), "[whisper.cpp] 模型下载完成: %s", targetPath)
return targetPath
} else {
lastErr = err
g.Log().Warningf(context.TODO(), "[whisper.cpp] 从 %s 下载失败: %v,尝试下一个源...", url, err)
}
}
g.Log().Errorf(context.TODO(), "[whisper.cpp] 所有下载源均失败: %v", lastErr)
return ""
}
// downloadFile 下载文件到指定路径(支持超时)
func (s *whisperService) downloadFile(url, destPath string, timeout time.Duration) error {
tmpPath := destPath + ".tmp"
out, err := os.Create(tmpPath)
if err != nil {
return fmt.Errorf("创建临时文件失败: %v", err)
}
defer out.Close()
client := &http.Client{Timeout: timeout}
resp, err := client.Get(url)
if err != nil {
os.Remove(tmpPath)
return err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
os.Remove(tmpPath)
return fmt.Errorf("HTTP %d", resp.StatusCode)
}
written, err := io.Copy(out, resp.Body)
if err != nil {
os.Remove(tmpPath)
return err
}
if err := os.Rename(tmpPath, destPath); err != nil {
return fmt.Errorf("文件重命名失败: %v", err)
}
g.Log().Infof(context.TODO(), "[whisper.cpp] 下载完成: %d bytes", written)
return nil
}
// transcribeWithCpp 使用 whisper.cppC++ 版,参数格式不同)
func (s *whisperService) transcribeWithCpp(ctx context.Context, req *TranscribeReq, binaryPath, model, language string) (res *TranscribeRes, err error) {
outputDir := filepath.Dir(req.AudioPath)
baseName := strings.TrimSuffix(filepath.Base(req.AudioPath), filepath.Ext(req.AudioPath))
outputPrefix := filepath.Join(outputDir, baseName)
threads := g.Cfg().MustGet(ctx, "whisper.threads", 2).Int()
// whisper.cpp 参数:
// -f input.mp3 输入文件
// -l zh 语言
// -t 2 线程数
// -otxt 输出 txt
// -of /path/prefix 输出文件前缀(自动加 .txt)
args := []string{
"-f", req.AudioPath,
"-l", language,
"-t", fmt.Sprintf("%d", threads),
"-otxt",
"-of", outputPrefix,
"-m", model,
}
cmd := exec.CommandContext(ctx, binaryPath, args...)
output, execErr := cmd.CombinedOutput()
if execErr != nil {
g.Log().Errorf(ctx, "whisper.cpp 执行失败: %v\n%s", execErr, string(output))
return nil, fmt.Errorf("语音识别失败: %v", execErr)
}
// whisper.cpp 输出: {prefix}.txt
txtPath := outputPrefix + ".txt"
textBytes, readErr := os.ReadFile(txtPath)
if readErr != nil {
return nil, fmt.Errorf("读取识别结果文件失败: %v", readErr)
}
res = &TranscribeRes{
Text: cleanTranscript(string(textBytes)),
Model: model,
Language: language,
OutputPath: txtPath,
}
return
}
+9 -2
View File
@@ -116,21 +116,28 @@ func (s *audioExtractService) Extract(ctx context.Context, req *ExtractAudioReq)
return return
} }
// getFFmpegPath 获取 ffmpeg 可执行路径 // getFFmpegPath 获取 ffmpeg 可执行路径,并打印安装状态日志
func (s *audioExtractService) getFFmpegPath() (string, error) { func (s *audioExtractService) getFFmpegPath() (string, error) {
ctx := context.Background()
// 1. 优先从配置读取 // 1. 优先从配置读取
ffmpegPath := g.Cfg().MustGet(context.Background(), "ffmpeg.path", "").String() ffmpegPath := g.Cfg().MustGet(ctx, "ffmpeg.path", "").String()
if ffmpegPath != "" { if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil { if _, err := os.Stat(ffmpegPath); err == nil {
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 使用配置路径: %s", ffmpegPath)
return ffmpegPath, nil return ffmpegPath, nil
} }
g.Log().Infof(ctx, "[ffmpeg] 配置路径不可用: %s,回退到系统 PATH 查找", ffmpegPath)
} }
// 2. 从 PATH 中查找 // 2. 从 PATH 中查找
path, err := exec.LookPath("ffmpeg") path, err := exec.LookPath("ffmpeg")
if err != nil { if err != nil {
g.Log().Errorf(ctx, "[ffmpeg] ❌ 未找到,请确保已安装 ffmpeg(启动时已自动尝试安装,若仍缺失请手动安装)")
return "", fmt.Errorf("未找到 ffmpeg,请确保已安装 ffmpeg 或在配置中指定路径") return "", fmt.Errorf("未找到 ffmpeg,请确保已安装 ffmpeg 或在配置中指定路径")
} }
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 系统路径: %s", path)
return path, nil return path, nil
} }
+6 -1
View File
@@ -546,16 +546,21 @@ func gcd(a, b int) int {
} }
func getFFmpegPath() (string, error) { func getFFmpegPath() (string, error) {
ffmpegPath := g.Cfg().MustGet(context.Background(), "ffmpeg.path", "").String() ctx := context.Background()
ffmpegPath := g.Cfg().MustGet(ctx, "ffmpeg.path", "").String()
if ffmpegPath != "" { if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil { if _, err := os.Stat(ffmpegPath); err == nil {
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 使用配置路径: %s", ffmpegPath)
return ffmpegPath, nil return ffmpegPath, nil
} }
g.Log().Infof(ctx, "[ffmpeg] 配置路径不可用: %s,回退到系统 PATH 查找", ffmpegPath)
} }
path, err := exec.LookPath("ffmpeg") path, err := exec.LookPath("ffmpeg")
if err != nil { if err != nil {
g.Log().Error(ctx, "[ffmpeg] ❌ 未找到,启动时已自动尝试安装,若仍缺失请手动安装")
return "", fmt.Errorf("未找到 ffmpeg") return "", fmt.Errorf("未找到 ffmpeg")
} }
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 系统路径: %s", path)
return path, nil return path, nil
} }
+214 -325
View File
@@ -1,29 +1,21 @@
package setup package setup
import ( import (
"bytes"
"context" "context"
"fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath"
"runtime" "runtime"
"strings" "strings"
"github.com/gogf/gf/v2/frame/g" "github.com/gogf/gf/v2/frame/g"
) )
var (
envConfigured bool
// DetectedWhisperPath 自动检测到的 whisper 命令行路径(空则使用 python -m whisper
DetectedWhisperPath string
)
func init() { func init() {
ensureDependencies() ensureDependencies()
} }
// ensureDependencies 启动时检查并安装 ffmpeg 和 whisper // ensureDependencies 启动时检查 ffmpeg 依赖
func ensureDependencies() { func ensureDependencies() {
ctx := context.Background() ctx := context.Background()
g.Log().Info(ctx, "========== 检查依赖环境 ==========") g.Log().Info(ctx, "========== 检查依赖环境 ==========")
@@ -32,14 +24,11 @@ func ensureDependencies() {
g.Log().Infof(ctx, "平台: %s/%s, Docker: %v", runtime.GOOS, runtime.GOARCH, isRunningInContainer()) g.Log().Infof(ctx, "平台: %s/%s, Docker: %v", runtime.GOOS, runtime.GOARCH, isRunningInContainer())
ensureFFmpeg(ctx) ensureFFmpeg(ctx)
ensureWhisper(ctx) ensurePython3(ctx)
resolveWhisperPath(ctx) ensureSceneDetect(ctx)
ensureHyperFrames(ctx)
if envConfigured { g.Log().Info(ctx, "依赖检查完成,所有依赖已就绪")
g.Log().Info(ctx, "依赖检查完成,新环境变量已配置,建议重启终端")
} else {
g.Log().Info(ctx, "依赖检查完成,所有依赖已就绪")
}
g.Log().Info(ctx, "===================================") g.Log().Info(ctx, "===================================")
} }
@@ -211,353 +200,253 @@ func installFFmpegOnWindows(ctx context.Context) {
3. 从 https://ffmpeg.org/download.html 下载并加入 PATH`) 3. 从 https://ffmpeg.org/download.html 下载并加入 PATH`)
} }
// ensureWhisper 确保 whisper 可用(优先安装 C++ 版,速度更快) // ensureHyperFrames 检查 HyperFrames 是否可用,不可用时自动安装
func ensureWhisper(ctx context.Context) { func ensureHyperFrames(ctx context.Context) {
// 1. 检查是否已有 whisper-cppC++ 版,最快) // 1. 优先检查全局 hyperframes 命令
// exec.LookPath 在 Windows 上会自动查找 .exe 后缀 if path, err := exec.LookPath("hyperframes"); err == nil {
if path, err := exec.LookPath("whisper-cpp"); err == nil { version := getHyperFramesVersion(path)
g.Log().Infof(ctx, "[whisper] ✔ C++ 版已安装: %s", path) g.Log().Infof(ctx, "[hyperframes] ✔ 已安装, 版本=%s, 路径=%s", version, path)
return
}
if path, err := exec.LookPath("whisper-cli"); err == nil {
g.Log().Infof(ctx, "[whisper] ✔ C++ 版已安装: %s", path)
return return
} }
// 2. 仅在 macOS 上检查 Homebrew 安装目录(即使不在 PATH 也能找到) // 2. 检查 npx hyperframes 是否可用
if runtime.GOOS == "darwin" { var outBuf bytes.Buffer
if p := findHomebrewWhisperCpp(); p != "" { checkCmd := exec.Command("npx", "hyperframes", "--version")
DetectedWhisperPath = p checkCmd.Stdout = &outBuf
if !inContainer { checkCmd.Stderr = nil
addToShellPath(ctx, filepath.Dir(p)) if checkCmd.Run() == nil {
} ver := strings.TrimSpace(outBuf.String())
g.Log().Infof(ctx, "[whisper] ✔ C++ 版已安装(自动检测): %s", p) if ver == "" {
return ver = "未知"
} }
} g.Log().Infof(ctx, "[hyperframes] ✔ 已安装(npx), 版本=%s", ver)
// 3. 仅在 macOS 上尝试使用 Homebrew 安装 C++ 版
if runtime.GOOS == "darwin" {
if _, err := exec.LookPath("brew"); err == nil {
g.Log().Infof(ctx, "[whisper] 安装 C++ 版 (brew install whisper-cpp)...")
cmd := exec.CommandContext(ctx, "brew", "install", "whisper-cpp")
output, err := cmd.CombinedOutput()
if err == nil {
g.Log().Info(ctx, "[whisper] ✔ C++ 版安装成功")
if !inContainer {
addToShellPath(ctx, getHomebrewBinDir())
}
if p := findHomebrewWhisperCpp(); p != "" {
DetectedWhisperPath = p
}
return
}
g.Log().Warningf(ctx, "[whisper] ⚠ brew 安装失败: %v\n%s", err, string(output))
g.Log().Infof(ctx, "[whisper] 降级安装 Python 版...")
}
}
// 4. 降级:检查 python -m whisper 是否可用
if pythonWhisperAvailable() {
g.Log().Info(ctx, "[whisper] ✔ Python 版已安装 (python3 -m whisper)")
return return
} }
// 5. 降级:pip 安装 Python 版 // 3. 未安装,检查 npm/node 可用性再决定是否自动安装
if _, err := exec.LookPath("pip3"); err != nil { if _, err := exec.LookPath("npm"); err != nil {
if _, err2 := exec.LookPath("pip"); err2 != nil { if inContainer {
g.Log().Warningf(ctx, "[whisper] ⚠ 未找到 pip,请手动安装:\n pip3 install openai-whisper") g.Log().Infof(ctx, "[hyperframes] npm 不可用,跳过自动安装(Docker 中请预装 Node.js")
return } else {
g.Log().Infof(ctx, "[hyperframes] npm 不可用,跳过自动安装(请先安装 Node.js: https://nodejs.org")
} }
return
} }
g.Log().Infof(ctx, "[whisper] 安装 Python 版 (pip install openai-whisper)...") g.Log().Infof(ctx, "[hyperframes] 未找到,尝试自动安装 (npm install -g hyperframes)...")
pipCmd := "pip3" installCmd := exec.Command("npm", "install", "-g", "hyperframes")
if _, err := exec.LookPath("pip3"); err != nil { output, err := installCmd.CombinedOutput()
pipCmd = "pip"
}
// pip install --user 可能在某些环境下不兼容,尝试先不加 --user,失败后再加
cmd := exec.CommandContext(ctx, pipCmd, "install", "openai-whisper")
output, err := cmd.CombinedOutput()
if err != nil { if err != nil {
// 尝试 --user 模式 g.Log().Warningf(ctx, "[hyperframes] ⚠ 自动安装失败: %v\n%s", err, string(output))
g.Log().Warningf(ctx, "[whisper] pip 全局安装失败: %v,尝试 --user 模式...", err) g.Log().Infof(ctx, "[hyperframes] 请手动安装: npm install -g hyperframes")
cmd = exec.CommandContext(ctx, pipCmd, "install", "--user", "openai-whisper") return
output, err = cmd.CombinedOutput()
if err != nil {
g.Log().Errorf(ctx, "[whisper] ❌ pip 安装失败: %v\n%s", err, string(output))
return
}
} }
g.Log().Info(ctx, "[whisper] ✔ Python 版安装成功") g.Log().Info(ctx, "[hyperframes] ✔ 自动安装成功")
// 安装后自动配置 PATH(仅在非容器、非 Windows 环境) // 安装后检查路径
if !inContainer && runtime.GOOS != "windows" { if path, err := exec.LookPath("hyperframes"); err == nil {
configureWhisperPath(ctx) version := getHyperFramesVersion(path)
g.Log().Infof(ctx, "[hyperframes] 路径=%s, 版本=%s", path, version)
} }
} }
// resolveWhisperPath 自动找到 whisper 二进制路径并存储 // getHyperFramesVersion 获取 HyperFrames 版本号
func resolveWhisperPath(ctx context.Context) { func getHyperFramesVersion(path string) string {
// 0. 如果已经通过 ensure 检测到了路径,直接使用 var outBuf bytes.Buffer
if DetectedWhisperPath != "" { vCmd := exec.Command(path, "--version")
if _, err := os.Stat(DetectedWhisperPath); err == nil { vCmd.Stdout = &outBuf
g.Log().Infof(ctx, "[whisper] ✔ 路径: %s", DetectedWhisperPath) if vCmd.Run() == nil {
return return strings.TrimSpace(outBuf.String())
}
} }
return "未知"
}
// 1. 优先检测 C++ 版本(快 3-5 倍) // ensurePython3 确保 Python3 可用
// exec.LookPath 在 Windows 上自动查找 .exe 后缀 func ensurePython3(ctx context.Context) {
for _, name := range []string{"whisper-cpp", "whisper-cli"} { // 优先检查 python3
if path, err := exec.LookPath(name); err == nil { if path, err := exec.LookPath("python3"); err == nil {
DetectedWhisperPath = path version := getPythonVersion(ctx, path)
g.Log().Infof(ctx, "[whisper] ✔ C++ 版: %s", path) g.Log().Infof(ctx, "[python3] ✔ 已安装, 版本=%s, 路径=%s", version, path)
return
}
}
// 2. 仅在 macOS 上查找 Homebrew 目录下的 C++ 版本
if runtime.GOOS == "darwin" {
if p := findHomebrewWhisperCpp(); p != "" {
DetectedWhisperPath = p
g.Log().Infof(ctx, "[whisper] ✔ C++ 版(自动检测): %s", p)
return
}
}
// 3. 从 PATH 查找 Python 版 whisper
if path, err := exec.LookPath("whisper"); err == nil {
DetectedWhisperPath = path
g.Log().Infof(ctx, "[whisper] ✔ Python 版: %s", path)
return return
} }
// 4. 尝试常见 pip user bin 路径 // 回退检查 python(部分系统用 python 指向 python3
for _, p := range getWhisperCandidates() { if path, err := exec.LookPath("python"); err == nil {
if info, err := os.Stat(p); err == nil && !info.IsDir() { version := getPythonVersion(ctx, path)
DetectedWhisperPath = p if strings.HasPrefix(version, "3.") {
g.Log().Infof(ctx, "[whisper] ✔ Python 版(自动检测): %s", p) g.Log().Infof(ctx, "[python3] ✔ 已安装(python), 版本=%s, 路径=%s", version, path)
return
}
// Python 2,不满足需求
g.Log().Infof(ctx, "[python3] 检测到 python=%s, 需要 Python 3,尝试自动安装...", version)
} else {
g.Log().Infof(ctx, "[python3] 未找到,尝试自动安装...")
}
installPython3(ctx)
}
// ensureSceneDetect 确保 scenedetect 库可用
func ensureSceneDetect(ctx context.Context) {
// 检查 python3 是否可导入 scenedetect
checkCmd := exec.Command("python3", "-c", "import scenedetect; print(scenedetect.__version__)")
var outBuf bytes.Buffer
checkCmd.Stdout = &outBuf
checkCmd.Stderr = nil
if checkCmd.Run() == nil {
ver := strings.TrimSpace(outBuf.String())
if ver == "" {
ver = "未知"
}
g.Log().Infof(ctx, "[scenedetect] ✔ 已安装, 版本=%s", ver)
return
}
g.Log().Infof(ctx, "[scenedetect] 未找到,尝试自动安装 (pip install scenedetect[opencv,ffmpeg])...")
installCmd := exec.Command("pip3", "install", "scenedetect[opencv,ffmpeg]")
output, err := installCmd.CombinedOutput()
if err != nil {
g.Log().Warningf(ctx, "[scenedetect] ⚠ pip3 安装失败: %v\n%s", err, string(output))
g.Log().Infof(ctx, "[scenedetect] 尝试使用 pip 安装...")
installCmd = exec.Command("pip", "install", "scenedetect[opencv,ffmpeg]")
output, err = installCmd.CombinedOutput()
if err != nil {
g.Log().Warningf(ctx, "[scenedetect] ⚠ pip 安装也失败: %v\n%s", err, string(output))
if inContainer {
g.Log().Infof(ctx, "[scenedetect] Docker 提示: pip 安装失败可能是因为缺少系统级依赖(如 libopencv-dev)。")
g.Log().Infof(ctx, "[scenedetect] 建议在 Dockerfile 中预装: apt-get install -y python3-opencv ffmpeg && pip install scenedetect")
} else {
g.Log().Infof(ctx, "[scenedetect] 请手动安装: pip install 'scenedetect[opencv,ffmpeg]'")
}
return return
} }
} }
g.Log().Info(ctx, "[scenedetect] ✔ 安装成功")
g.Log().Info(ctx, "[whisper] ✔ 使用 python3 -m whisper 方式") // 验证安装
verifyCmd := exec.Command("python3", "-c", "import scenedetect; print(scenedetect.__version__)")
outBuf.Reset()
verifyCmd.Stdout = &outBuf
if verifyCmd.Run() == nil {
g.Log().Infof(ctx, "[scenedetect] 验证通过, 版本=%s", strings.TrimSpace(outBuf.String()))
}
} }
// getWhisperCandidates 返回可能的 whisper 二进制路径 // getPythonVersion 获取 Python 版本号
func getWhisperCandidates() []string { func getPythonVersion(ctx context.Context, pythonPath string) string {
var candidates []string var outBuf bytes.Buffer
vCmd := exec.CommandContext(ctx, pythonPath, "--version")
// 通过 python 探针获取 user-site bin 目录 vCmd.Stdout = &outBuf
if p := getUserPythonBin(); p != "" { vCmd.Stderr = &outBuf
candidates = append(candidates, filepath.Join(p, "whisper")) if vCmd.Run() == nil {
// Windows 上 pip 安装的可执行文件是 .exe return strings.TrimPrefix(strings.TrimSpace(outBuf.String()), "Python ")
if runtime.GOOS == "windows" {
candidates = append(candidates, filepath.Join(p, "whisper.exe"))
}
} }
return "未知"
}
// 常见 pip user base 路径 // installPython3 根据平台自动安装 Python3
userHome, _ := os.UserHomeDir() func installPython3(ctx context.Context) {
switch runtime.GOOS { switch runtime.GOOS {
case "darwin": case "darwin":
// macOS 常见的 Python 版本路径 if _, err := exec.LookPath("brew"); err == nil {
pythonVersions := []string{"3.9", "3.10", "3.11", "3.12", "3.13"} g.Log().Infof(ctx, "[python3] 通过 brew 安装...")
for _, ver := range pythonVersions { cmd := exec.CommandContext(ctx, "brew", "install", "python@3")
candidates = append(candidates, output, err := cmd.CombinedOutput()
filepath.Join(userHome, "Library", "Python", ver, "bin", "whisper"), if err != nil {
) g.Log().Warningf(ctx, "[python3] ⚠ brew 安装失败: %v\n%s", err, string(output))
g.Log().Infof(ctx, "[python3] 请手动安装: brew install python@3")
} else {
g.Log().Info(ctx, "[python3] ✔ 安装成功")
}
} else {
g.Log().Warningf(ctx, "[python3] ⚠ 未检测到 Homebrew,请手动安装 Python3:\n https://www.python.org/downloads/")
} }
case "linux": case "linux":
candidates = append(candidates, sudoPrefix := ""
filepath.Join(userHome, ".local", "bin", "whisper"), if !inContainer {
) if _, err := exec.LookPath("sudo"); err == nil {
sudoPrefix = "sudo"
}
}
// 1. apt (Debian/Ubuntu)
if _, err := exec.LookPath("apt-get"); err == nil {
args := []string{"install", "-y", "python3", "python3-pip"}
if sudoPrefix != "" {
args = append([]string{sudoPrefix}, args...)
}
cmd := exec.CommandContext(ctx, "apt-get", args...)
output, err := cmd.CombinedOutput()
if err != nil {
g.Log().Warningf(ctx, "[python3] ⚠ apt-get 安装失败: %v\n%s", err, string(output))
} else {
g.Log().Info(ctx, "[python3] ✔ 安装成功")
}
return
}
// 2. apk (Alpine)
if _, err := exec.LookPath("apk"); err == nil {
cmd := exec.CommandContext(ctx, "apk", "add", "python3", "py3-pip")
output, err := cmd.CombinedOutput()
if err != nil {
g.Log().Warningf(ctx, "[python3] ⚠ apk 安装失败: %v\n%s", err, string(output))
} else {
g.Log().Info(ctx, "[python3] ✔ 安装成功")
}
return
}
// 3. yum (CentOS/RHEL)
if _, err := exec.LookPath("yum"); err == nil {
args := []string{"install", "-y", "python3", "python3-pip"}
if sudoPrefix != "" {
args = append([]string{sudoPrefix}, args...)
}
cmd := exec.CommandContext(ctx, "yum", args...)
output, err := cmd.CombinedOutput()
if err != nil {
g.Log().Warningf(ctx, "[python3] ⚠ yum 安装失败: %v\n%s", err, string(output))
} else {
g.Log().Info(ctx, "[python3] ✔ 安装成功")
}
return
}
if inContainer {
g.Log().Warningf(ctx, "[python3] ⚠ 容器中未找到包管理器,请将 python3 预装在 Docker 镜像中")
} else {
g.Log().Warningf(ctx, "[python3] ⚠ 请手动安装: sudo apt-get install python3 python3-pip")
}
case "windows": case "windows":
// Windows 上 pip --user 安装的脚本路径 // 1. winget
candidates = append(candidates, if _, err := exec.LookPath("winget"); err == nil {
filepath.Join(userHome, "AppData", "Roaming", "Python", "Scripts", "whisper.exe"), g.Log().Infof(ctx, "[python3] 通过 winget 安装...")
filepath.Join(userHome, "AppData", "Roaming", "Python", "Scripts", "whisper"), cmd := exec.CommandContext(ctx, "winget", "install", "--id", "Python.Python.3", "-e", "--accept-package-agreements")
filepath.Join(userHome, "AppData", "Local", "Programs", "Python", "Scripts", "whisper.exe"), output, err := cmd.CombinedOutput()
filepath.Join(userHome, "AppData", "Local", "Programs", "Python", "Scripts", "whisper"), if err == nil {
) g.Log().Info(ctx, "[python3] ✔ 安装成功")
// Python 版本特定路径 return
pythonVersions := []string{"39", "310", "311", "312", "313"}
for _, ver := range pythonVersions {
candidates = append(candidates,
filepath.Join(userHome, "AppData", "Roaming", "Python", "Python"+ver, "Scripts", "whisper.exe"),
filepath.Join(userHome, "AppData", "Roaming", "Python", "Python"+ver, "Scripts", "whisper"),
)
}
}
return candidates
}
// getUserPythonBin 通过 python 获取 user bin 目录
func getUserPythonBin() string {
pythonCandidates := []string{"python3", "python"}
for _, py := range pythonCandidates {
path, err := exec.LookPath(py)
if err != nil {
continue
}
cmd := exec.Command(path, "-m", "site", "--user-base")
output, err := cmd.Output()
if err != nil {
continue
}
base := strings.TrimSpace(string(output))
if base != "" {
return filepath.Join(base, "bin")
}
}
return ""
}
// configureWhisperPath 将 pip user bin 目录加到 shell 配置
func configureWhisperPath(ctx context.Context) {
binDir := getUserPythonBin()
if binDir == "" {
return
}
// 检查是否已经在 PATH 中
currentPath := os.Getenv("PATH")
if strings.Contains(currentPath, binDir) {
return
}
// 配置到 .zshrc 或 .bashrc
home, _ := os.UserHomeDir()
rcFiles := []string{".zshrc", ".bashrc", ".bash_profile"}
for _, rc := range rcFiles {
rcPath := filepath.Join(home, rc)
// 文件不存在则跳过
if _, err := os.Stat(rcPath); os.IsNotExist(err) {
continue
}
// 检查是否已添加
data, _ := os.ReadFile(rcPath)
if strings.Contains(string(data), binDir) {
continue
}
// 追加
line := fmt.Sprintf("\nexport PATH=\"%s:$PATH\"\n", binDir)
f, err := os.OpenFile(rcPath, os.O_APPEND|os.O_WRONLY, 0644)
if err != nil {
g.Log().Warningf(ctx, "[whisper] 写入 %s 失败: %v", rc, err)
continue
}
f.WriteString(line)
f.Close()
g.Log().Infof(ctx, "[whisper] 已将 %s 添加到 %s,请执行: source ~/%s", binDir, rc, rc)
envConfigured = true
break
}
}
// pythonWhisperAvailable 检查 python -m whisper 是否可用
func pythonWhisperAvailable() bool {
pythonCandidates := []string{"python3", "python"}
for _, py := range pythonCandidates {
if path, err := exec.LookPath(py); err == nil {
cmd := exec.Command(path, "-m", "whisper", "--help")
if cmd.Run() == nil {
return true
} }
g.Log().Warningf(ctx, "[python3] ⚠ winget 安装失败: %v\n%s", err, string(output))
} }
} // 2. choco
return false if _, err := exec.LookPath("choco"); err == nil {
} g.Log().Infof(ctx, "[python3] 通过 choco 安装...")
cmd := exec.CommandContext(ctx, "choco", "install", "python", "-y")
// findHomebrewWhisperCpp 在 Homebrew 安装目录查找 whisper-cpp output, err := cmd.CombinedOutput()
func findHomebrewWhisperCpp() string { if err == nil {
dirs := getHomebrewBinDirs() g.Log().Info(ctx, "[python3] ✔ 安装成功")
for _, dir := range dirs { return
for _, name := range []string{"whisper-cpp", "whisper-cli"} {
p := filepath.Join(dir, name)
if info, err := os.Stat(p); err == nil && !info.IsDir() {
return p
} }
g.Log().Warningf(ctx, "[python3] ⚠ choco 安装失败: %v\n%s", err, string(output))
} }
}
return ""
}
// getHomebrewBinDirs 返回 Homebrew 可能的 bin 目录 g.Log().Warningf(ctx, `[python3] ⚠ 请手动安装 Python3:
func getHomebrewBinDirs() []string { 1. winget install --id Python.Python.3 -e
userHome, _ := os.UserHomeDir() 2. 从 https://www.python.org/downloads/ 下载安装(记得勾选"Add to PATH"`)
return []string{
"/opt/homebrew/bin", // Apple Silicon
"/usr/local/bin", // Intel
filepath.Join(userHome, ".homebrew", "bin"),
}
}
// getHomebrewBinDir 返回当前系统的 Homebrew bin 目录 default:
func getHomebrewBinDir() string { g.Log().Warningf(ctx, "[python3] ⚠ 不支持的平台(%s),请手动安装 Python3", runtime.GOOS)
dirs := getHomebrewBinDirs()
for _, dir := range dirs {
if _, err := os.Stat(filepath.Join(dir, "brew")); err == nil {
return dir
}
// 也检查 brew 命令路径
if path, err := exec.LookPath("brew"); err == nil {
return filepath.Dir(path)
}
}
return "/opt/homebrew/bin" // 默认 Apple Silicon 路径
}
// addToShellPath 将目录添加到 shell rc 文件的 PATH 中
func addToShellPath(ctx context.Context, dir string) {
if dir == "" {
return
}
// 容器环境不修改 shell 配置(无意义)
if inContainer {
return
}
// Windows 环境不修改 shell rc 文件(使用系统环境变量)
if runtime.GOOS == "windows" {
g.Log().Infof(ctx, "[setup] Windows 环境,请手动将 %s 添加到系统 PATH 环境变量", dir)
return
}
// 检查是否已在 PATH 中
currentPath := os.Getenv("PATH")
if strings.Contains(currentPath, dir) {
return
}
home, _ := os.UserHomeDir()
rcFiles := []string{".zshrc", ".bashrc", ".bash_profile"}
for _, rc := range rcFiles {
rcPath := filepath.Join(home, rc)
if _, err := os.Stat(rcPath); os.IsNotExist(err) {
continue
}
data, _ := os.ReadFile(rcPath)
if strings.Contains(string(data), dir) {
continue
}
line := fmt.Sprintf("\nexport PATH=\"%s:$PATH\"\n", dir)
f, err := os.OpenFile(rcPath, os.O_APPEND|os.O_WRONLY, 0644)
if err != nil {
g.Log().Warningf(ctx, "[setup] 写入 %s 失败: %v", rc, err)
continue
}
f.WriteString(line)
f.Close()
g.Log().Infof(ctx, "[setup] 已将 %s 添加到 %s", dir, rc)
envConfigured = true
break
} }
} }
+837
View File
@@ -0,0 +1,837 @@
package video
import (
"bytes"
"context"
"encoding/json"
"fmt"
"html"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"time"
dao "media/dao/video"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/guid"
)
// Caption 字幕叠加服务单例
var Caption = new(captionService)
type captionService struct{}
// ---------- 异步任务管理 ----------
// CreateAsyncTask 创建字幕叠加任务,返回 taskID
func (s *captionService) CreateAsyncTask(ctx context.Context, videoURLs []string, audioURL string, subtitles []dto.SubtitleSegment, subtitleStyle *dto.SubtitleStyle, elements []dto.CaptionElement, callbackURL string) (string, error) {
if len(videoURLs) < 1 {
return "", fmt.Errorf("至少需要1个视频")
}
if len(elements) < 1 && len(subtitles) < 1 {
return "", fmt.Errorf("至少需要字幕时间线或字幕元素")
}
if elements == nil {
elements = []dto.CaptionElement{}
}
if subtitles == nil {
subtitles = []dto.SubtitleSegment{}
}
videoURLsJSON, _ := json.Marshal(videoURLs)
elementsJSON, _ := json.Marshal(elements)
taskID := "cap_" + guid.S()
task := &entity.VideoCaptionTask{
TaskID: taskID,
VideoURLs: string(videoURLsJSON),
AudioURL: audioURL,
Elements: string(elementsJSON),
Status: "pending",
CallbackURL: callbackURL,
}
if _, err := dao.CaptionTask.Insert(ctx, task); err != nil {
return "", fmt.Errorf("创建任务失败: %v", err)
}
user := getUserFromCtx(ctx)
g.Log().Infof(ctx, "[字幕叠加-异步] 创建任务 %s, 视频数=%d, 字幕段数=%d, 元素数=%d, 回调=%s",
taskID, len(videoURLs), len(subtitles), len(elements), callbackURL)
go s.processTask(user, taskID, videoURLs, audioURL, subtitles, subtitleStyle, elements, callbackURL)
return taskID, nil
}
// processTask 后台处理字幕叠加任务
func (s *captionService) processTask(user *beans.User, taskID string, videoURLs []string, audioURL string, subtitles []dto.SubtitleSegment, subtitleStyle *dto.SubtitleStyle, elements []dto.CaptionElement, callbackURL string) {
bgCtx := context.Background()
bgCtx = context.WithValue(bgCtx, "user", user)
dao.CaptionTask.UpdateRunning(bgCtx, taskID)
defer func() {
if r := recover(); r != nil {
errMsg := fmt.Sprintf("字幕叠加异常: %v", r)
g.Log().Errorf(bgCtx, "[字幕 %s] %s", taskID, errMsg)
dao.CaptionTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL, nil)
}
}()
// 1. 创建临时工作目录
tempDir := g.Cfg().MustGet(bgCtx, "ffmpeg.temp_dir", "resource/temp").String()
projectDir := filepath.Join(tempDir, fmt.Sprintf("caption_%s", taskID))
os.RemoveAll(projectDir)
os.MkdirAll(projectDir, 0755)
defer os.RemoveAll(projectDir)
// 2. 下载所有视频
var videoPaths []string
for i, videoURL := range videoURLs {
savePath, dlErr := downloadFile(bgCtx, videoURL, projectDir)
if dlErr != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 视频%d下载失败 %s: %v", taskID, i, videoURL, dlErr)
continue
}
videoPaths = append(videoPaths, savePath)
}
if len(videoPaths) < 1 {
errMsg := fmt.Sprintf("所有视频下载失败(共%d个)", len(videoURLs))
dao.CaptionTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL, nil)
return
}
// 3. 从第一个视频自动检测分辨率
width, height := getVideoResolution(bgCtx, videoPaths[0])
g.Log().Infof(bgCtx, "[字幕 %s] 视频分辨率: %dx%d", taskID, width, height)
// 更新 DB 中的分辨率字段
dao.CaptionTask.UpdateResolution(bgCtx, taskID, width, height)
// 4. 如果有多段视频,先用 FFmpeg 拼接成一段
videoCount := len(videoPaths)
if videoCount > 1 {
concatedPath := filepath.Join(projectDir, "concated.mp4")
if err := s.concatVideos(bgCtx, videoPaths, concatedPath); err != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 视频拼接失败(使用首个视频): %v", taskID, err)
} else {
videoPaths = []string{concatedPath}
g.Log().Infof(bgCtx, "[字幕 %s] 视频拼接完成: %d段 → %s", taskID, videoCount, concatedPath)
}
}
// 4.5 消音处理:去掉原视频音频,避免与新音频混音
mutedPath := filepath.Join(projectDir, "muted.mp4")
if err := s.muteVideo(bgCtx, videoPaths[0], mutedPath); err != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 视频消音失败: %v", taskID, err)
} else {
videoPaths = []string{mutedPath}
g.Log().Infof(bgCtx, "[字幕 %s] 视频消音完成: %s", taskID, mutedPath)
}
// 5. 用 ffprobe 获取视频真实总时长
totalVideoDuration := getVideoRealDuration(bgCtx, videoPaths[0])
if totalVideoDuration <= 0 {
totalVideoDuration = 30
}
g.Log().Infof(bgCtx, "[字幕 %s] 视频总时长: %.2f 秒", taskID, totalVideoDuration)
// 6. 下载元素中的图片资源(type=image)
for i, elem := range elements {
if elem.Type == "image" && elem.ImageURL != "" {
savePath, dlErr := downloadFile(bgCtx, elem.ImageURL, projectDir)
if dlErr != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 图片%d下载失败 %s: %v", taskID, i, elem.ImageURL, dlErr)
continue
}
elements[i].ImageURL = filepath.Base(savePath)
}
}
// 7. 下载背景音乐(可选)
audioPath := ""
if audioURL != "" {
savePath, dlErr := downloadFile(bgCtx, audioURL, projectDir)
if dlErr != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 音频下载失败 %s: %v", taskID, audioURL, dlErr)
} else {
// 7.5 音频降噪:去除气口/口水音/底噪(时长不变)
denoisedPath := filepath.Join(projectDir, "denoised_audio.wav")
if err := s.denoiseAudio(bgCtx, savePath, denoisedPath); err != nil {
g.Log().Warningf(bgCtx, "[字幕 %s] 音频降噪失败(使用原始音频): %v", taskID, err)
audioPath = savePath
} else {
audioPath = denoisedPath
g.Log().Infof(bgCtx, "[字幕 %s] 音频降噪完成: %s", taskID, denoisedPath)
}
}
}
// 8. 将外部字幕时间线转为字幕元素
subtitleElements := subtitlesToElements(subtitles, subtitleStyle)
// 合并所有元素
allElements := append(subtitleElements, elements...)
// 9. 生成 index.html(基于真实视频时长)
htmlContent := s.buildHTML(taskID, videoPaths[0], audioPath, allElements, totalVideoDuration, width, height)
htmlPath := filepath.Join(projectDir, "index.html")
if err := os.WriteFile(htmlPath, []byte(htmlContent), 0644); err != nil {
errMsg := fmt.Sprintf("生成HTML失败: %v", err)
dao.CaptionTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL, nil)
return
}
g.Log().Infof(bgCtx, "[字幕 %s] HTML生成完成: %s", taskID, htmlPath)
// 10. 执行 HyperFrames 渲染
outputPath := filepath.Join(projectDir, "output.mp4")
if err := s.runHyperFramesRender(bgCtx, projectDir, outputPath); err != nil {
errMsg := fmt.Sprintf("视频渲染失败: %v", err)
dao.CaptionTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL, nil)
return
}
// 7. 检查输出文件
stat, statErr := os.Stat(outputPath)
if statErr != nil {
errMsg := fmt.Sprintf("输出文件不存在: %v", statErr)
dao.CaptionTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL, nil)
return
}
durationStr := getVideoDurationStr(bgCtx, outputPath)
// 8. 上传到 MinIO(固定上传)
fileURL := ""
uploadCtx := context.WithValue(context.Background(), "user", user)
uploadRes, uploadErr := uploadToMinIO(uploadCtx, outputPath)
if uploadErr != nil {
dao.CaptionTask.UpdateError(bgCtx, taskID, fmt.Sprintf("上传失败: %v", uploadErr))
s.callback(bgCtx, taskID, callbackURL, nil)
return
}
fileURL = uploadRes.FileURL
g.Log().Infof(bgCtx, "[字幕 %s] MinIO 上传完成: fileUrl=%s, fileName=%s", taskID, uploadRes.FileURL, uploadRes.FileName)
// 9. 更新为成功
fileName := filepath.Base(outputPath)
dao.CaptionTask.UpdateSuccess(bgCtx, taskID, fileURL, stat.Size(), fileName, durationStr)
g.Log().Infof(bgCtx, "[字幕 %s] 完成, 文件=%s, 大小=%d, 时长=%s", taskID, outputPath, stat.Size(), durationStr)
if callbackURL != "" {
extra := map[string]interface{}{
"fileURL": uploadRes.FileURL,
"fileSize": uploadRes.FileSize,
"fileName": uploadRes.FileName,
"fileFormat": uploadRes.FileFormat,
"fileAddressPrefix": uploadRes.FileAddressPrefix,
"durationStr": durationStr,
}
s.callback(bgCtx, taskID, callbackURL, extra)
}
}
// runHyperFramesRender 执行 HyperFrames 渲染
func (s *captionService) runHyperFramesRender(ctx context.Context, projectDir, outputPath string) error {
ctxWithTimeout, cancel := context.WithTimeout(ctx, 30*time.Minute)
defer cancel()
// 使用全局 hyperframes 命令(已在 setup 中安装检查)
hyperframesPath, lookErr := exec.LookPath("hyperframes")
if lookErr != nil {
return fmt.Errorf("hyperframes 未安装, 请运行: npm install -g hyperframes")
}
// 直接在工作目录运行 hyperframes render,输出默认 output.mp4
cmd := exec.CommandContext(ctxWithTimeout, hyperframesPath, "render")
cmd.Dir = projectDir
cmd.Env = append(os.Environ(),
"HYPERFRAMES_HEADLESS=true",
)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("hyperframes render 失败: %v\n%s", err, string(output))
}
g.Log().Infof(ctx, "[HyperFrames] render 完成: %s", strings.TrimSpace(string(output)))
// 查找输出文件:优先 renders/ 目录(HyperFrames 默认输出位置)
rendersDir := filepath.Join(projectDir, "renders")
if entries, readErr := os.ReadDir(rendersDir); readErr == nil {
for _, entry := range entries {
if !entry.IsDir() && strings.HasSuffix(entry.Name(), ".mp4") {
src := filepath.Join(rendersDir, entry.Name())
if err := os.Rename(src, outputPath); err != nil {
return fmt.Errorf("移动输出文件失败: %v", err)
}
g.Log().Infof(ctx, "[HyperFrames] 找到输出文件: %s", entry.Name())
return nil
}
}
}
// 回退:检查 projectDir 下的 output.mp4
fallback := filepath.Join(projectDir, "output.mp4")
if _, statErr := os.Stat(fallback); statErr == nil {
if err := os.Rename(fallback, outputPath); err != nil {
return fmt.Errorf("移动输出文件失败: %v", err)
}
return nil
}
return fmt.Errorf("未找到输出文件(已在 %s 和 %s 中查找)", rendersDir, fallback)
}
// buildHTML 生成 HyperFrames HTML 模板
// videoPath: 已拼接好的单视频文件路径;totalDuration: 视频真实总时长(秒)
func (s *captionService) buildHTML(taskID, videoPath, audioPath string, elements []dto.CaptionElement, totalDuration float64, width, height int) string {
var sb strings.Builder
compID := "main"
sb.WriteString("<!DOCTYPE html>\n<html lang=\"zh-CN\">\n<head>\n")
sb.WriteString("<meta charset=\"UTF-8\">\n")
sb.WriteString("<meta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n")
sb.WriteString("<style>\n")
sb.WriteString("* { margin: 0; padding: 0; box-sizing: border-box; }\n")
sb.WriteString(fmt.Sprintf("body { width: %dpx; height: %dpx; overflow: hidden; background: #000; }\n", width, height))
sb.WriteString(".clip { position: absolute; }\n")
sb.WriteString(".text-element { font-family: Arial, Helvetica, sans-serif; text-align: center; display: flex; align-items: center; justify-content: center; white-space: pre-wrap; word-break: break-word; }\n")
sb.WriteString("@keyframes fadeIn { from { opacity: 0; } to { opacity: 1; } }\n")
sb.WriteString("@keyframes slideUp { from { opacity: 0; transform: translateY(40px); } to { opacity: 1; transform: translateY(0); } }\n")
sb.WriteString("@keyframes slideLeft { from { opacity: 0; transform: translateX(40px); } to { opacity: 1; transform: translateX(0); } }\n")
sb.WriteString("@keyframes scaleIn { from { opacity: 0; transform: scale(0.8); } to { opacity: 1; transform: scale(1); } }\n")
sb.WriteString("@keyframes pulse { 0% { transform: scale(1); } 50% { transform: scale(1.05); } 100% { transform: scale(1); } }\n")
sb.WriteString(".anim-fadeIn { animation: fadeIn 0.5s ease-out; }\n")
sb.WriteString(".anim-slideUp { animation: slideUp 0.6s ease-out; }\n")
sb.WriteString(".anim-slideLeft { animation: slideLeft 0.6s ease-out; }\n")
sb.WriteString(".anim-scaleIn { animation: scaleIn 0.5s ease-out; }\n")
sb.WriteString(".anim-pulse { animation: pulse 1.5s ease-in-out infinite; }\n")
sb.WriteString("</style>\n")
sb.WriteString("<script src=\"https://cdn.jsdelivr.net/npm/gsap@3/dist/gsap.min.js\"></script>\n")
sb.WriteString("</head>\n<body>\n")
// Composition 容器(data-start="0" 必须)
sb.WriteString(fmt.Sprintf("<div id=\"stage\" data-composition-id=\"%s\" data-start=\"0\" data-width=\"%d\" data-height=\"%d\">\n", compID, width, height))
// 背景视频(独占 track=0
relVideoPath := filepath.Base(videoPath)
sb.WriteString(fmt.Sprintf(" <video id=\"bg-video\" class=\"clip\" data-start=\"0\" data-duration=\"%.3f\" data-track-index=\"0\" src=\"%s\" muted playsinline></video>\n",
totalDuration, relVideoPath))
// 字幕/图片元素(从 track=1 开始,避免与视频重叠警告)
clipIndex := 1
for _, elem := range elements {
elemID := fmt.Sprintf("elem-%d", clipIndex)
animClass := s.animationCSS(elem.Animation)
style := s.buildElemStyle(elem, width, height)
// duration<=0 表示一直显示到视频结束
elemDuration := elem.Duration
if elemDuration <= 0 {
elemDuration = totalDuration - elem.StartTime
if elemDuration <= 0 {
elemDuration = totalDuration
}
}
// 确保覆盖层不在 track=0(视频专用)
trackIdx := elem.TrackIndex
if trackIdx <= 0 {
trackIdx = 1
}
if elem.Type == "image" {
imgStyle := style
if elem.Width > 0 {
imgStyle += fmt.Sprintf("width:%dpx;", elem.Width)
}
if elem.Height > 0 {
imgStyle += fmt.Sprintf("height:%dpx;", elem.Height)
}
sb.WriteString(fmt.Sprintf(" <img id=\"%s\" class=\"clip %s\" data-start=\"%.3f\" data-duration=\"%.3f\" data-track-index=\"%d\" src=\"%s\" style=\"%s\" />\n",
elemID, animClass, elem.StartTime, elemDuration, trackIdx, html.EscapeString(elem.ImageURL), imgStyle))
} else {
bgStyle := ""
if elem.BgColor != "" {
if elem.BgOpacity > 0 && elem.BgOpacity < 1 {
bgStyle = fmt.Sprintf("background:%s;padding:12px 24px;border-radius:8px;", hexToRGBA(elem.BgColor, elem.BgOpacity))
} else {
bgStyle = fmt.Sprintf("background:%s;padding:12px 24px;border-radius:8px;", elem.BgColor)
}
}
sb.WriteString(fmt.Sprintf(" <div id=\"%s\" class=\"clip text-element %s\" data-start=\"%.3f\" data-duration=\"%.3f\" data-track-index=\"%d\" style=\"%s%s\">%s</div>\n",
elemID, animClass, elem.StartTime, elemDuration, trackIdx, style, bgStyle, html.EscapeString(elem.Text)))
}
clipIndex++
}
// 背景音乐
if audioPath != "" {
relAudioPath := filepath.Base(audioPath)
sb.WriteString(fmt.Sprintf(" <audio id=\"bg-audio\" class=\"clip\" data-start=\"0\" data-duration=\"%.3f\" data-track-index=\"%d\" data-volume=\"0.5\" src=\"%s\"></audio>\n",
totalDuration, clipIndex+100, relAudioPath))
}
sb.WriteString("</div>\n")
// GSAP 时间线
sb.WriteString("<script>\n")
sb.WriteString("(function() {\n")
sb.WriteString(" var tl = gsap.timeline({ paused: true });\n")
sb.WriteString(" tl.to('#stage', { duration: 0, opacity: 1 }, 0);\n")
sb.WriteString(" window.__timelines = window.__timelines || {};\n")
sb.WriteString(fmt.Sprintf(" window.__timelines['%s'] = tl;\n", compID))
sb.WriteString("})();\n")
sb.WriteString("</script>\n")
sb.WriteString("</body>\n</html>")
return sb.String()
}
// buildElemStyle 构建元素的 CSS 样式(X/Y 独立定位,transform 合并)
func (s *captionService) buildElemStyle(elem dto.CaptionElement, canvasWidth, canvasHeight int) string {
parts := []string{}
// X 轴定位
switch elem.X {
case "left", "":
parts = append(parts, "left:0px;")
case "center":
parts = append(parts, "left:50%;")
case "right":
parts = append(parts, "right:0px;")
default:
// 纯数字自动补 px,否则原样输出(如 calc、百分比等)
if isNumeric(elem.X) {
parts = append(parts, fmt.Sprintf("left:%spx;", elem.X))
} else {
parts = append(parts, fmt.Sprintf("left:%s;", elem.X))
}
}
// Y 轴定位
switch elem.Y {
case "top", "":
parts = append(parts, "top:0px;")
case "center":
parts = append(parts, "top:50%;")
case "bottom":
parts = append(parts, "bottom:0px;")
default:
if isNumeric(elem.Y) {
parts = append(parts, fmt.Sprintf("top:%spx;", elem.Y))
} else {
parts = append(parts, fmt.Sprintf("top:%s;", elem.Y))
}
}
// 合并 transform(避免 translateX 和 translateY 互相覆盖)
var transforms []string
if elem.X == "center" {
transforms = append(transforms, "translateX(-50%)")
}
if elem.Y == "center" {
transforms = append(transforms, "translateY(-50%)")
}
if len(transforms) > 0 {
parts = append(parts, fmt.Sprintf("transform:%s;", strings.Join(transforms, " ")))
}
// 文字样式
if elem.Type == "text" {
// 自动计算 max-width 防止文字溢出画布
// 底部居中字幕: 90% 宽度; 左/右贴边: 45% 宽度
maxWidthPct := 90
switch elem.X {
case "left", "right":
maxWidthPct = 45
case "center", "":
maxWidthPct = 90
}
maxWidth := int(float64(canvasWidth) * float64(maxWidthPct) / 100.0)
parts = append(parts, fmt.Sprintf("max-width:%dpx;", maxWidth))
if elem.FontSize > 0 {
parts = append(parts, fmt.Sprintf("font-size:%dpx;", elem.FontSize))
}
if elem.FontColor != "" {
parts = append(parts, fmt.Sprintf("color:%s;", elem.FontColor))
}
}
return strings.Join(parts, "")
}
// animationCSS 返回动画 CSS class
func (s *captionService) animationCSS(anim string) string {
switch anim {
case "fadeIn":
return "anim-fadeIn"
case "slideUp":
return "anim-slideUp"
case "slideLeft":
return "anim-slideLeft"
case "scaleIn":
return "anim-scaleIn"
case "pulse":
return "anim-pulse"
default:
return ""
}
}
// muteVideo 使用 FFmpeg 去掉视频中的音频轨道(消音)
func (s *captionService) muteVideo(ctx context.Context, inputPath, outputPath string) error {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return err
}
args := []string{
"-i", inputPath,
"-an", // 去掉音频
"-c:v", "libx264", // 重新编码视频
"-crf", "23", // 画质(23=默认,越小越好)
"-preset", "veryfast", // 编码速度优先
"-y", outputPath,
}
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("FFmpeg 消音失败: %v\n%s", err, string(output))
}
return nil
}
// denoiseAudio 使用 FFmpeg afftdn 对音频降噪(去除气口/口水音/底噪,不改变时长)
func (s *captionService) denoiseAudio(ctx context.Context, inputPath, outputPath string) error {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return err
}
args := []string{
"-i", inputPath,
"-af", "afftdn=nr=12:nf=-20", // 频域降噪:nr=降噪强度, nf=噪声底噪
"-y", outputPath,
}
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("FFmpeg 音频降噪失败: %v\n%s", err, string(output))
}
return nil
}
// concatVideos 用 FFmpeg 将多个视频拼接成一个
func (s *captionService) concatVideos(ctx context.Context, videoPaths []string, outputPath string) error {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return err
}
// 使用 concat demuxer 拼接(最快,无需重编码)
fileListPath := outputPath + ".files.txt"
var lines []string
for _, vp := range videoPaths {
absPath, _ := filepath.Abs(vp)
lines = append(lines, "file '"+absPath+"'")
}
if err := os.WriteFile(fileListPath, []byte(strings.Join(lines, "\n")), 0644); err != nil {
return fmt.Errorf("创建文件列表失败: %v", err)
}
defer os.Remove(fileListPath)
args := []string{
"-f", "concat",
"-safe", "0",
"-i", fileListPath,
"-c", "copy",
"-y", outputPath,
}
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("FFmpeg 拼接失败: %v\n%s", err, string(output))
}
return nil
}
// getVideoRealDuration 用 ffprobe 获取视频时长(秒,float)
func getVideoRealDuration(ctx context.Context, videoPath string) float64 {
ffprobePath, err := exec.LookPath("ffprobe")
if err != nil {
g.Log().Warningf(ctx, "[ffprobe] ⚠ 未找到,无法获取视频时长")
return 0
}
args := []string{
"-v", "quiet",
"-print_format", "json",
"-show_format",
videoPath,
}
cmd := exec.CommandContext(ctx, ffprobePath, args...)
output, err := cmd.Output()
if err != nil {
return 0
}
var info struct {
Format struct {
Duration string `json:"duration"`
} `json:"format"`
}
if err := json.Unmarshal(output, &info); err != nil {
return 0
}
var secs float64
fmt.Sscanf(info.Format.Duration, "%f", &secs)
return secs
}
// getVideoDurationStr 获取视频时长可读字符串
func getVideoDurationStr(ctx context.Context, videoPath string) string {
ffprobePath, err := exec.LookPath("ffprobe")
if err != nil {
g.Log().Warningf(ctx, "[ffprobe] ⚠ 未找到,无法获取视频时长字符串")
return ""
}
args := []string{
"-v", "quiet",
"-print_format", "json",
"-show_format",
videoPath,
}
cmd := exec.CommandContext(ctx, ffprobePath, args...)
output, err := cmd.Output()
if err != nil {
return ""
}
var info struct {
Format struct {
Duration string `json:"duration"`
} `json:"format"`
}
if err := json.Unmarshal(output, &info); err != nil {
return ""
}
var secs float64
fmt.Sscanf(info.Format.Duration, "%f", &secs)
if secs <= 0 {
return ""
}
m := int(secs) / 60
s := int(secs) % 60
return fmt.Sprintf("%d:%02d", m, s)
}
// getVideoResolution 使用 ffprobe 获取视频分辨率
func getVideoResolution(ctx context.Context, videoPath string) (width, height int) {
ffprobePath, err := exec.LookPath("ffprobe")
if err != nil {
g.Log().Warningf(ctx, "[ffprobe] ⚠ 未找到,使用默认分辨率 1080x1920")
return 1080, 1920 // 默认竖屏
}
args := []string{
"-v", "quiet",
"-print_format", "json",
"-select_streams", "v:0",
"-show_streams",
videoPath,
}
cmd := exec.CommandContext(ctx, ffprobePath, args...)
output, err := cmd.Output()
if err != nil {
return 1080, 1920
}
var info struct {
Streams []struct {
Width int `json:"width"`
Height int `json:"height"`
} `json:"streams"`
}
if err := json.Unmarshal(output, &info); err != nil || len(info.Streams) == 0 {
return 1080, 1920
}
w := info.Streams[0].Width
h := info.Streams[0].Height
if w <= 0 || h <= 0 {
return 1080, 1920
}
return w, h
}
// hexToRGBA 将十六进制颜色(如 #FF0000)转为 rgba(r,g,b,a) 字符串
func hexToRGBA(hex string, alpha float64) string {
hex = strings.TrimPrefix(hex, "#")
if len(hex) != 6 && len(hex) != 3 {
return hex
}
if len(hex) == 3 {
// 简写 #RGB → #RRGGBB
hex = string([]byte{hex[0], hex[0], hex[1], hex[1], hex[2], hex[2]})
}
r, _ := strconv.ParseInt(hex[0:2], 16, 0)
g, _ := strconv.ParseInt(hex[2:4], 16, 0)
b, _ := strconv.ParseInt(hex[4:6], 16, 0)
return fmt.Sprintf("rgba(%d,%d,%d,%.2f)", r, g, b, alpha)
}
// ---------- 字幕时间线转元素 ----------
// subtitlesToElements 将外部传入的字幕时间线转为底部字幕元素
func subtitlesToElements(subtitles []dto.SubtitleSegment, style *dto.SubtitleStyle) []dto.CaptionElement {
// 应用默认值
fontSize := 28
fontColor := "#FFFFFF"
bgColor := "#000000"
bgOpacity := 0.6
if style != nil {
if style.FontSize > 0 {
fontSize = style.FontSize
}
if style.FontColor != "" {
fontColor = style.FontColor
}
// BgColor/BgOpacity 使用 *string/*float64,精确区分"没传"和"传了零值"
// - 没传字段 → 指针为 nil → 使用默认值
// - 传了 "" 或 0 → 指针非 nil → 按用户意愿设置
if style.BgColor != nil {
bgColor = *style.BgColor
}
if style.BgOpacity != nil {
bgOpacity = *style.BgOpacity
}
}
// 从样式配置中读取字幕位置
subtitleX := "center"
subtitleY := "bottom"
if style != nil {
if style.X != "" {
subtitleX = style.X
}
if style.Y != "" {
subtitleY = style.Y
}
}
g.Log().Infof(context.TODO(), "[字幕定位] subtitleX=%s subtitleY=%s", subtitleX, subtitleY)
var elements []dto.CaptionElement
for _, seg := range subtitles {
text := strings.TrimSpace(seg.Text)
if text == "" {
continue
}
elements = append(elements, dto.CaptionElement{
Type: "text",
Text: text,
StartTime: seg.Start,
Duration: seg.End - seg.Start,
X: subtitleX,
Y: subtitleY,
FontSize: fontSize,
FontColor: fontColor,
BgColor: bgColor,
BgOpacity: bgOpacity,
TrackIndex: 10,
})
}
return elements
}
// ---------- 查询任务 ----------
// GetTaskResult 查询字幕任务结果
func (s *captionService) GetTaskResult(ctx context.Context, taskID string) (*dto.GetCaptionTaskRes, error) {
task, err := dao.CaptionTask.GetByTaskID(ctx, taskID)
if err != nil {
return nil, fmt.Errorf("查询任务失败: %v", err)
}
if task == nil {
return nil, fmt.Errorf("任务不存在: %s", taskID)
}
return dao.EntityToCaptionTaskRes(task), nil
}
// ---------- 回调通知 ----------
func (s *captionService) callback(ctx context.Context, taskID, callbackURL string, extraPayload map[string]interface{}) {
if callbackURL == "" {
return
}
task, err := dao.CaptionTask.GetByTaskID(ctx, taskID)
if err != nil || task == nil {
g.Log().Errorf(ctx, "[字幕回调 %s] 查询任务失败: %v", taskID, err)
return
}
payload := map[string]interface{}{
"taskId": taskID,
"status": task.Status,
}
// 如果传入了额外数据(成功场景的直接上传结果),则合并进去
if extraPayload != nil {
for k, v := range extraPayload {
payload[k] = v
}
} else {
// 否则从 DB 读取(兼容失败场景、旧回调路径)
if task.Status == "success" {
payload["fileURL"] = task.FileURL
payload["fileSize"] = task.FileSize
payload["durationStr"] = task.DurationStr
}
if task.Status == "failed" {
payload["errorMessage"] = task.ErrorMessage
}
}
body, _ := json.Marshal(payload)
g.Log().Infof(ctx, "[字幕回调 %s] 状态=%s, 目标=%s, body=%s", taskID, task.Status, callbackURL, string(body))
req, _ := http.NewRequest("POST", callbackURL, bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
cbUser := getUserFromCtx(ctx)
userJSON, _ := json.Marshal(cbUser)
req.Header.Set("X-User-Info", string(userJSON))
client := &http.Client{Timeout: 2 * time.Minute}
resp, reqErr := client.Do(req)
if reqErr != nil {
g.Log().Errorf(ctx, "[字幕回调 %s] 请求失败: %v", taskID, reqErr)
return
}
defer resp.Body.Close()
respBody, _ := io.ReadAll(resp.Body)
g.Log().Infof(ctx, "[字幕回调 %s] 响应 status=%d, body=%s", taskID, resp.StatusCode, string(respBody))
}
// isNumeric 判断字符串是否为纯数字(含小数)
func isNumeric(s string) bool {
_, err := strconv.ParseFloat(s, 64)
return err == nil
}
+26 -3
View File
@@ -343,16 +343,21 @@ func (s *concatService) hasVideoAudio(ctx context.Context, ffmpegPath, videoPath
} }
func (s *concatService) getFFmpegPath() (string, error) { func (s *concatService) getFFmpegPath() (string, error) {
ffmpegPath := g.Cfg().MustGet(context.Background(), "ffmpeg.path", "").String() ctx := context.Background()
ffmpegPath := g.Cfg().MustGet(ctx, "ffmpeg.path", "").String()
if ffmpegPath != "" { if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil { if _, err := os.Stat(ffmpegPath); err == nil {
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 使用配置路径: %s", ffmpegPath)
return ffmpegPath, nil return ffmpegPath, nil
} }
g.Log().Infof(ctx, "[ffmpeg] 配置路径不可用: %s,回退到系统 PATH 查找", ffmpegPath)
} }
path, err := exec.LookPath("ffmpeg") path, err := exec.LookPath("ffmpeg")
if err != nil { if err != nil {
g.Log().Error(ctx, "[ffmpeg] ❌ 未找到,启动时已自动尝试安装,若仍缺失请手动安装")
return "", fmt.Errorf("未找到 ffmpeg") return "", fmt.Errorf("未找到 ffmpeg")
} }
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 系统路径: %s", path)
return path, nil return path, nil
} }
@@ -716,15 +721,33 @@ func downloadFile(ctx context.Context, rawURL, tempDir string) (string, error) {
} }
savePath := filepath.Join(tempDir, fmt.Sprintf("%d_%s", time.Now().UnixMilli(), fileName)) savePath := filepath.Join(tempDir, fmt.Sprintf("%d_%s", time.Now().UnixMilli(), fileName))
// 用 NewRequestWithContext 代替 client.Get,避免 Go 默认 User-Agent 被 CDN 拒绝(403)
req, err := http.NewRequestWithContext(ctx, "GET", rawURL, nil)
if err != nil {
return "", err
}
// 保留原始 query string,避免签名 URL 被 url.Parse 规范化后签名失效
req.URL.RawQuery = parsedURL.RawQuery
// 模拟浏览器请求头,避免 CDN/WAF 因缺少 Referer/Accept 等头返回 403
req.Header.Set("User-Agent", "Mozilla/5.0 (compatible; media-service/1.0)")
req.Header.Set("Accept", "*/*")
req.Header.Set("Referer", parsedURL.Scheme+"://"+parsedURL.Host+"/")
client := &http.Client{Timeout: 10 * time.Minute} client := &http.Client{Timeout: 10 * time.Minute}
resp, err := client.Get(rawURL) resp, err := client.Do(req)
if err != nil { if err != nil {
return "", err return "", err
} }
defer resp.Body.Close() defer resp.Body.Close()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("HTTP %d", resp.StatusCode) // 读取响应体用于错误诊断
bodyBytes, _ := io.ReadAll(io.LimitReader(resp.Body, 1024))
// 尝试检测签名 URL 是否过期
if parsedURL.Query().Get("x-expires") != "" || parsedURL.Query().Get("expires") != "" {
return "", fmt.Errorf("HTTP %d (签名URL可能已过期,请使用新鲜地址) body=%s", resp.StatusCode, string(bodyBytes))
}
return "", fmt.Errorf("HTTP %d body=%s", resp.StatusCode, string(bodyBytes))
} }
out, err := os.Create(savePath) out, err := os.Create(savePath)
+6 -1
View File
@@ -301,16 +301,21 @@ func (s *cutService) cutByFilterComplex(ctx context.Context, ffmpegPath, inputPa
// getFFmpegPath 获取 FFmpeg 路径 // getFFmpegPath 获取 FFmpeg 路径
func (s *cutService) getFFmpegPath() (string, error) { func (s *cutService) getFFmpegPath() (string, error) {
ffmpegPath := g.Cfg().MustGet(context.Background(), "ffmpeg.path", "").String() ctx := context.Background()
ffmpegPath := g.Cfg().MustGet(ctx, "ffmpeg.path", "").String()
if ffmpegPath != "" { if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil { if _, err := os.Stat(ffmpegPath); err == nil {
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 使用配置路径: %s", ffmpegPath)
return ffmpegPath, nil return ffmpegPath, nil
} }
g.Log().Infof(ctx, "[ffmpeg] 配置路径不可用: %s,回退到系统 PATH 查找", ffmpegPath)
} }
path, err := exec.LookPath("ffmpeg") path, err := exec.LookPath("ffmpeg")
if err != nil { if err != nil {
g.Log().Error(ctx, "[ffmpeg] ❌ 未找到,启动时已自动尝试安装,若仍缺失请手动安装")
return "", fmt.Errorf("未找到 ffmpeg") return "", fmt.Errorf("未找到 ffmpeg")
} }
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 系统路径: %s", path)
return path, nil return path, nil
} }
+436
View File
@@ -0,0 +1,436 @@
package video
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"mime/multipart"
"net/http"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"time"
dao "media/dao/video"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/beans"
commonHttp "gitea.redpowerfuture.com/red-future/common/http"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/guid"
)
type mergeService struct{}
// Merge 视频拼接+混音服务单例
var Merge = new(mergeService)
// mergeSem 并发控制信号量
var mergeSem chan struct{}
var mergeSemOnce sync.Once
// ---------- 异步任务管理 ----------
// CreateAsyncTask 创建异步拼接+混音任务(URL模式),返回 taskId,后台处理
func (s *mergeService) CreateAsyncTask(ctx context.Context, videoURLs, audioURLs []string, callbackURL string, upload bool) (string, error) {
if len(videoURLs) < 1 {
return "", fmt.Errorf("至少需要1个视频")
}
if len(audioURLs) < 1 {
return "", fmt.Errorf("至少需要1个音频")
}
// 将 videoURLs/audioURLs 序列化为 JSON 存入数据库
videoURLsJSON, _ := json.Marshal(videoURLs)
audioURLsJSON, _ := json.Marshal(audioURLs)
taskID := "merge_" + guid.S()
task := &entity.VideoAudioMergeTask{
TaskID: taskID,
VideoURLs: string(videoURLsJSON),
AudioURLs: string(audioURLsJSON),
Status: "pending",
CallbackURL: callbackURL,
}
if _, err := dao.MergeTask.Insert(ctx, task); err != nil {
return "", fmt.Errorf("创建任务失败: %v", err)
}
// 提取调用方用户信息,传给 goroutine
user := getUserFromCtx(ctx)
g.Log().Infof(ctx, "[拼接混音-异步] 创建任务 %s, 视频数=%d, 音频数=%d, 回调=%s",
taskID, len(videoURLs), len(audioURLs), callbackURL)
// 异步处理:先下载再拼接+混音
go s.processAsyncTask(user, taskID, videoURLs, audioURLs, upload, callbackURL)
return taskID, nil
}
// processAsyncTask 后台处理异步拼接+混音任务(URL模式,需要先下载)
func (s *mergeService) processAsyncTask(user *beans.User, taskID string, videoURLs, audioURLs []string, upload bool, callbackURL string) {
bgCtx := context.Background()
bgCtx = context.WithValue(bgCtx, "user", user)
dao.MergeTask.UpdateRunning(bgCtx, taskID)
defer func() {
if r := recover(); r != nil {
errMsg := fmt.Sprintf("异步拼接混音异常: %v", r)
g.Log().Errorf(bgCtx, "[拼接混音 %s] %s", taskID, errMsg)
dao.MergeTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
}
}()
// 1. 下载所有视频
tempDir := g.Cfg().MustGet(bgCtx, "ffmpeg.temp_dir", "resource/temp").String()
os.MkdirAll(tempDir, 0755)
var videoPaths []string
for _, videoURL := range videoURLs {
savePath, dlErr := downloadFile(bgCtx, videoURL, tempDir)
if dlErr != nil {
g.Log().Warningf(bgCtx, "[拼接混音 %s] 视频下载失败 %s: %v", taskID, videoURL, dlErr)
continue
}
videoPaths = append(videoPaths, savePath)
}
if len(videoPaths) < 1 {
errMsg := fmt.Sprintf("所有视频下载失败(共%d个)", len(videoURLs))
dao.MergeTask.UpdateError(bgCtx, taskID, errMsg)
cleanupFiles(videoPaths)
s.callback(bgCtx, taskID, callbackURL)
return
}
// 2. 下载所有音频
var audioPaths []string
for _, audioURL := range audioURLs {
savePath, dlErr := downloadFile(bgCtx, audioURL, tempDir)
if dlErr != nil {
g.Log().Warningf(bgCtx, "[拼接混音 %s] 音频下载失败 %s: %v", taskID, audioURL, dlErr)
continue
}
audioPaths = append(audioPaths, savePath)
}
if len(audioPaths) < 1 {
errMsg := fmt.Sprintf("所有音频下载失败(共%d个)", len(audioURLs))
dao.MergeTask.UpdateError(bgCtx, taskID, errMsg)
cleanupFiles(videoPaths)
cleanupFiles(audioPaths)
s.callback(bgCtx, taskID, callbackURL)
return
}
// 3. 等待并发许可,执行拼接+混音(FFmpeg 密集操作)
acquireMergeSem()
defer releaseMergeSem()
mergeErr := s.executeMerge(bgCtx, taskID, videoPaths, audioPaths, upload)
cleanupFiles(videoPaths)
cleanupFiles(audioPaths)
if mergeErr != nil {
dao.MergeTask.UpdateError(bgCtx, taskID, mergeErr.Error())
s.callback(bgCtx, taskID, callbackURL)
return
}
g.Log().Infof(bgCtx, "[拼接混音 %s] 完成", taskID)
if callbackURL != "" {
s.callback(bgCtx, taskID, callbackURL)
}
}
// executeMerge 执行拼接+混音,输出最终视频并更新任务状态
func (s *mergeService) executeMerge(ctx context.Context, taskID string, videoPaths, audioPaths []string, upload bool) error {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return err
}
tempDir := filepath.Dir(videoPaths[0])
// Step 1: 拼接多视频 → 临时拼接文件
concatPath := filepath.Join(tempDir, fmt.Sprintf("concat_%s.mp4", taskID))
defer os.Remove(concatPath)
concatRes, concatErr := Concat.Concat(ctx, &ConcatReq{
VideoPaths: videoPaths,
OutputPath: concatPath,
Method: "reencode", // 强制重编码以统一分辨率
Upload: false,
})
if concatErr != nil {
return fmt.Errorf("视频拼接失败: %v", concatErr)
}
g.Log().Infof(ctx, "[拼接混音 %s] 视频拼接完成: duration=%s, size=%d", taskID, concatRes.DurationStr, concatRes.FileSize)
// Step 2: 如果有多段音频,先拼接音频
audioPath := audioPaths[0]
if len(audioPaths) > 1 {
concatAudioPath := filepath.Join(tempDir, fmt.Sprintf("concat_audio_%s.wav", taskID))
defer os.Remove(concatAudioPath)
concatAudioArgs := []string{}
for _, ap := range audioPaths {
concatAudioArgs = append(concatAudioArgs, "-i", ap)
}
filterStr := fmt.Sprintf("concat=n=%d:v=0:a=1", len(audioPaths))
concatAudioArgs = append(concatAudioArgs, "-filter_complex", filterStr, "-y", concatAudioPath)
g.Log().Debugf(ctx, "[拼接混音 %s] 音频拼接命令: %s %v", taskID, ffmpegPath, concatAudioArgs)
cmd := exec.CommandContext(ctx, ffmpegPath, concatAudioArgs...)
outputBytes, cmdErr := cmd.CombinedOutput()
if cmdErr != nil {
return fmt.Errorf("音频拼接失败: %v\n%s", cmdErr, string(outputBytes))
}
audioPath = concatAudioPath
g.Log().Infof(ctx, "[拼接混音 %s] 音频拼接完成: %d段 → %s", taskID, len(audioPaths), concatAudioPath)
}
// Step 3: 混入音频(以视频时长为准,音频短则静音补全,音频长则截断)
outputPath := filepath.Join(tempDir, fmt.Sprintf("merge_%s_%s.mp4", taskID, time.Now().Format("150405")))
duration := concatRes.Duration
args := []string{
"-i", concatPath,
"-i", audioPath,
"-c:v", "copy",
"-map", "0:v:0",
"-map", "1:a:0",
"-af", "apad",
"-c:a", "aac",
"-t", fmt.Sprintf("%.3f", duration),
"-y",
outputPath,
}
g.Log().Debugf(ctx, "[拼接混音 %s] 混音命令: %s %v", taskID, ffmpegPath, args)
bgCtx := context.Background()
cmd := exec.CommandContext(bgCtx, ffmpegPath, args...)
outputBytes, err := cmd.CombinedOutput()
if err != nil {
os.Remove(outputPath)
return fmt.Errorf("混音失败: %v\n%s", err, string(outputBytes))
}
// 获取输出文件信息
stat, statErr := os.Stat(outputPath)
if statErr != nil {
os.Remove(outputPath)
return fmt.Errorf("输出文件异常: %v", statErr)
}
durationStr := formatDuration(duration)
// 上传到 MinIO
fileURL := ""
if upload {
uploadCtx := context.WithValue(context.Background(), "user", getUserFromCtx(ctx))
uploadRes, uploadErr := uploadToMinIO(uploadCtx, outputPath)
if uploadErr != nil {
os.Remove(outputPath)
return fmt.Errorf("上传到MinIO失败: %v", uploadErr)
}
fileURL = uploadRes.FileURL
}
// 更新数据库为成功
fileName := filepath.Base(outputPath)
fileFormat := ""
if idx := strings.LastIndex(fileName, "."); idx > 0 {
fileFormat = fileName[idx+1:]
}
dao.MergeTask.UpdateSuccess(ctx, taskID,
fileURL, stat.Size(), fileName, fileFormat,
"", durationStr)
os.Remove(outputPath)
return nil
}
// GetTaskResult 查询异步任务结果
func (s *mergeService) GetTaskResult(ctx context.Context, taskID string) (*dto.GetMergeTaskRes, error) {
task, err := dao.MergeTask.GetByTaskID(ctx, taskID)
if err != nil {
return nil, fmt.Errorf("查询任务失败: %v", err)
}
if task == nil {
return nil, fmt.Errorf("任务不存在: %s", taskID)
}
return dao.EntityToMergeTaskRes(task), nil
}
// callback 回调通知(从数据库读取任务结果发送)
func (s *mergeService) callback(ctx context.Context, taskID, callbackURL string) {
if callbackURL == "" {
return
}
task, err := dao.MergeTask.GetByTaskID(ctx, taskID)
if err != nil || task == nil {
g.Log().Errorf(ctx, "[拼接混音回调 %s] 查询任务失败: %v", taskID, err)
return
}
payload := map[string]interface{}{
"taskId": task.TaskID,
"status": task.Status,
}
if task.Status == "success" {
payload["fileURL"] = task.FileURL
payload["fileSize"] = task.FileSize
payload["durationStr"] = task.DurationStr
}
if task.Status == "failed" {
payload["errorMessage"] = task.ErrorMessage
}
body, _ := json.Marshal(payload)
g.Log().Infof(ctx, "[拼接混音回调 %s] 状态=%s, 目标=%s", taskID, task.Status, callbackURL)
req, _ := http.NewRequest("POST", callbackURL, bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
cbUser := getUserFromCtx(ctx)
userJSON, _ := json.Marshal(cbUser)
req.Header.Set("X-User-Info", string(userJSON))
client := &http.Client{Timeout: 2 * time.Minute}
resp, reqErr := client.Do(req)
if reqErr != nil {
g.Log().Errorf(ctx, "[拼接混音回调 %s] 请求失败: %v", taskID, reqErr)
return
}
defer resp.Body.Close()
respBody, _ := io.ReadAll(resp.Body)
g.Log().Infof(ctx, "[拼接混音回调 %s] 响应 status=%d, body=%s", taskID, resp.StatusCode, string(respBody))
}
// ---------- 上传到 MinIO ----------
// uploadToMinIO 上传文件到 MinIO(复用 concat_service 的逻辑)
func uploadToMinIO(ctx context.Context, localFilePath string) (*uploadFileRes, error) {
var buf bytes.Buffer
mw := multipart.NewWriter(&buf)
file, err := os.Open(localFilePath)
if err != nil {
return nil, fmt.Errorf("打开文件失败: %v", err)
}
defer file.Close()
fw, err := mw.CreateFormFile("file", filepath.Base(localFilePath))
if err != nil {
return nil, fmt.Errorf("创建表单文件字段失败: %v", err)
}
if _, err = io.Copy(fw, file); err != nil {
return nil, fmt.Errorf("写入文件内容失败: %v", err)
}
mw.Close()
client := commonHttp.Httpclient.Clone()
newTransport := http.DefaultTransport.(*http.Transport).Clone()
newTransport.ResponseHeaderTimeout = 5 * time.Minute
client.Transport = newTransport
client.SetTimeout(10 * time.Minute)
hasAuthHeader := false
if r := g.RequestFromCtx(ctx); r != nil {
for k, v := range r.Header {
client.SetHeader(k, v[0])
if strings.EqualFold(k, "Authorization") || strings.EqualFold(k, "X-User-Info") {
hasAuthHeader = true
}
}
}
if !hasAuthHeader {
uploadUser := getUserFromCtx(ctx)
userJSON, _ := json.Marshal(uploadUser)
client.SetHeader("X-User-Info", string(userJSON))
}
contentType := mw.FormDataContentType()
client.SetHeader("Content-Type", contentType)
response, err := client.Post(ctx, "oss/file/uploadFile", buf.Bytes())
if err != nil {
return nil, fmt.Errorf("调用OSS上传服务失败: %v", err)
}
defer response.Close()
body := response.ReadAll()
var apiResp struct {
Code int `json:"code"`
Message string `json:"message"`
Data *uploadFileRes `json:"data"`
}
if err = json.Unmarshal(body, &apiResp); err != nil {
return nil, fmt.Errorf("响应解析失败: %v", err)
}
if apiResp.Code != 200 && apiResp.Code != 0 {
return nil, fmt.Errorf("OSS上传失败: %s", apiResp.Message)
}
return apiResp.Data, nil
}
// acquireMergeSem 获取并发许可(懒初始化信号量)
func acquireMergeSem() {
mergeSemOnce.Do(func() {
concurrency := g.Cfg().MustGet(context.Background(), "merge.concurrency", 1).Int()
if concurrency < 1 {
concurrency = 1
}
mergeSem = make(chan struct{}, concurrency)
})
mergeSem <- struct{}{}
}
// releaseMergeSem 释放并发许可
func releaseMergeSem() {
<-mergeSem
}
// cleanupFiles 清理文件列表
func cleanupFiles(paths []string) {
for _, p := range paths {
os.Remove(p)
}
}
// lookupFFmpegPath 查找 ffmpeg 可执行文件路径
func lookupFFmpegPath() (string, error) {
ctx := context.Background()
ffmpegPath := g.Cfg().MustGet(ctx, "ffmpeg.path", "").String()
if ffmpegPath != "" {
if _, err := os.Stat(ffmpegPath); err == nil {
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 使用配置路径: %s", ffmpegPath)
return ffmpegPath, nil
}
g.Log().Infof(ctx, "[ffmpeg] 配置路径不可用: %s,回退到系统 PATH 查找", ffmpegPath)
}
path, err := exec.LookPath("ffmpeg")
if err != nil {
g.Log().Error(ctx, "[ffmpeg] ❌ 未找到,启动时已自动尝试安装,若仍缺失请手动安装")
return "", fmt.Errorf("未找到 ffmpeg")
}
g.Log().Infof(ctx, "[ffmpeg] ✔ 已安装, 系统路径: %s", path)
return path, nil
}
+433
View File
@@ -0,0 +1,433 @@
package video
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"os/exec"
"path/filepath"
"time"
dao "media/dao/video"
dto "media/model/dto/video"
entity "media/model/entity/video"
"gitea.redpowerfuture.com/red-future/common/beans"
"github.com/gogf/gf/v2/frame/g"
"github.com/gogf/gf/v2/util/guid"
)
// SceneSplit 场景分割服务单例
var SceneSplit = new(sceneSplitService)
type sceneSplitService struct{}
// ---------- 异步任务管理 ----------
// CreateAsyncTask 创建场景分割异步任务,返回 taskID
func (s *sceneSplitService) CreateAsyncTask(ctx context.Context, videoURL string, threshold float64, callbackURL string) (string, error) {
if threshold <= 0 {
threshold = 27.0
}
taskID := "scene_" + guid.S()
task := &entity.SceneSplitTask{
TaskID: taskID,
VideoURL: videoURL,
Status: "pending",
CallbackURL: callbackURL,
}
if _, err := dao.SceneSplitTask.Insert(ctx, task); err != nil {
return "", fmt.Errorf("创建任务失败: %v", err)
}
user := getUserFromCtx(ctx)
g.Log().Infof(ctx, "[场景分割-异步] 创建任务 %s, videoUrl=%s, threshold=%.1f, callback=%s",
taskID, videoURL, threshold, callbackURL)
go s.processTask(user, taskID, videoURL, threshold, callbackURL)
return taskID, nil
}
// processTask 后台处理场景分割任务
func (s *sceneSplitService) processTask(user *beans.User, taskID, videoURL string, threshold float64, callbackURL string) {
bgCtx := context.Background()
bgCtx = context.WithValue(bgCtx, "user", user)
dao.SceneSplitTask.UpdateRunning(bgCtx, taskID)
defer func() {
if r := recover(); r != nil {
errMsg := fmt.Sprintf("场景分割异常: %v", r)
g.Log().Errorf(bgCtx, "[场景分割 %s] %s", taskID, errMsg)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
}
}()
// 1. 创建临时工作目录
tempDir := g.Cfg().MustGet(bgCtx, "ffmpeg.temp_dir", "resource/temp").String()
workDir := filepath.Join(tempDir, fmt.Sprintf("scene_%s", taskID))
os.MkdirAll(workDir, 0755)
defer os.RemoveAll(workDir)
// 2. 下载视频
g.Log().Infof(bgCtx, "[场景分割 %s] 开始下载视频: %s", taskID, videoURL)
videoPath, dlErr := downloadFile(bgCtx, videoURL, workDir)
if dlErr != nil {
errMsg := fmt.Sprintf("视频下载失败: %v", dlErr)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
g.Log().Infof(bgCtx, "[场景分割 %s] 视频下载完成: %s", taskID, videoPath)
// 获取视频总时长
videoDuration := getVideoDurationSeconds(bgCtx, videoPath)
g.Log().Infof(bgCtx, "[场景分割 %s] 视频总时长: %.2f秒", taskID, videoDuration)
// 3. 提取音频(在切片之前,保留完整音频)
g.Log().Infof(bgCtx, "[场景分割 %s] 开始提取音频", taskID)
audioPath := filepath.Join(workDir, fmt.Sprintf("audio_%s.m4a", taskID))
if err := s.extractAudio(bgCtx, videoPath, audioPath); err != nil {
g.Log().Warningf(bgCtx, "[场景分割 %s] 音频提取失败: %v", taskID, err)
audioPath = ""
}
// 4. 调用 PySceneDetect 检测场景
g.Log().Infof(bgCtx, "[场景分割 %s] 开始场景检测, threshold=%.1f", taskID, threshold)
scenesJSONPath := filepath.Join(workDir, "scenes.json")
if err := s.detectScenes(bgCtx, videoPath, scenesJSONPath, threshold); err != nil {
errMsg := fmt.Sprintf("场景检测失败: %v", err)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
// 5. 解析场景时间线
scenes, parseErr := s.parseScenes(scenesJSONPath)
if parseErr != nil {
errMsg := fmt.Sprintf("解析场景结果失败: %v", parseErr)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
g.Log().Infof(bgCtx, "[场景分割 %s] 检测到 %d 个场景", taskID, len(scenes))
if len(scenes) == 0 {
errMsg := "未检测到任何场景"
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
// 6. 按场景分割视频
g.Log().Infof(bgCtx, "[场景分割 %s] 开始分割视频 (%d 个分片)", taskID, len(scenes))
results, splitErr := s.splitVideo(bgCtx, videoPath, scenes, workDir, taskID)
if splitErr != nil {
errMsg := fmt.Sprintf("视频分割失败: %v", splitErr)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
g.Log().Infof(bgCtx, "[场景分割 %s] 视频分割完成, 共 %d 个分片文件", taskID, len(results))
// 7. 上传分片到 MinIO,构建有序分片列表
g.Log().Infof(bgCtx, "[场景分割 %s] 开始上传 %d 个分片到 MinIO", taskID, len(results))
uploadCtx := context.WithValue(context.Background(), "user", user)
segments := make([]dto.SegmentEntry, len(results))
for i, r := range results {
uploadRes, uploadErr := uploadToMinIO(uploadCtx, r.Path)
if uploadErr != nil {
errMsg := fmt.Sprintf("上传分片%d(%s)到MinIO失败: %v", i+1, r.Timeline, uploadErr)
dao.SceneSplitTask.UpdateError(bgCtx, taskID, errMsg)
s.callback(bgCtx, taskID, callbackURL)
return
}
segments[i] = dto.SegmentEntry{
Timeline: r.Timeline,
URL: uploadRes.FileAddressPrefix + uploadRes.FileURL,
}
g.Log().Infof(bgCtx, "[场景分割 %s] 分片 %d/%d [%s] 上传完成: %s", taskID, i+1, len(results), r.Timeline, uploadRes.FileURL)
}
// 8. 上传音频到 MinIO
audioURL := ""
audioDuration := 0.0
if audioPath != "" {
audioDuration = getVideoDurationSeconds(bgCtx, audioPath)
uploadRes, uploadErr := uploadToMinIO(uploadCtx, audioPath)
if uploadErr != nil {
g.Log().Warningf(bgCtx, "[场景分割 %s] 音频上传失败: %v", taskID, uploadErr)
} else {
audioURL = uploadRes.FileAddressPrefix + uploadRes.FileURL
g.Log().Infof(bgCtx, "[场景分割 %s] 音频上传完成: %s", taskID, audioURL)
}
}
// 9. 更新数据库为成功
segmentsJSON, _ := json.Marshal(segments)
dao.SceneSplitTask.UpdateSuccess(bgCtx, taskID, string(segmentsJSON), audioURL, len(segments), audioDuration, videoDuration)
g.Log().Infof(bgCtx, "[场景分割 %s] 完成! 分片数=%d, 音频=%s", taskID, len(segments), audioURL)
if callbackURL != "" {
s.callback(bgCtx, taskID, callbackURL)
}
}
// detectScenes 调用 Python 脚本进行场景检测
func (s *sceneSplitService) detectScenes(ctx context.Context, videoPath, outputJSON string, threshold float64) error {
// 查找 Python 可执行文件
pythonPath, err := exec.LookPath("python3")
if err != nil {
pythonPath, err = exec.LookPath("python")
if err != nil {
return fmt.Errorf("未找到 Python 环境,请安装 Python 3.x 并执行 pip install scenedetect[opencv,ffmpeg]")
}
}
// 脚本路径:相对于服务运行目录的 scripts/scene_detect.py
scriptPath := "scripts/scene_detect.py"
if _, err := os.Stat(scriptPath); os.IsNotExist(err) {
// 尝试绝对路径
if absPath, absErr := filepath.Abs(scriptPath); absErr == nil {
scriptPath = absPath
}
}
ctxWithTimeout, cancel := context.WithTimeout(ctx, 30*time.Minute)
defer cancel()
args := []string{
scriptPath,
"--input", videoPath,
"--output", outputJSON,
"--threshold", fmt.Sprintf("%.1f", threshold),
}
g.Log().Infof(ctx, "[场景检测] 执行命令: %s %v", pythonPath, args)
cmd := exec.CommandContext(ctxWithTimeout, pythonPath, args...)
output, err := cmd.CombinedOutput()
g.Log().Infof(ctx, "[场景检测] 输出: %s", string(output))
if err != nil {
return fmt.Errorf("Python 场景检测失败: %v\n%s", err, string(output))
}
return nil
}
// getVideoDurationSeconds 使用 ffprobe 获取音视频时长(秒)
func getVideoDurationSeconds(ctx context.Context, videoPath string) float64 {
ffprobePath, err := exec.LookPath("ffprobe")
if err != nil {
g.Log().Warningf(ctx, "[ffprobe] ⚠ 未找到,无法获取视频时长")
return 0
}
args := []string{"-v", "quiet", "-print_format", "json", "-show_format", videoPath}
cmd := exec.CommandContext(ctx, ffprobePath, args...)
output, err := cmd.Output()
if err != nil {
return 0
}
var info struct {
Format struct {
Duration string `json:"duration"`
} `json:"format"`
}
if err := json.Unmarshal(output, &info); err != nil {
return 0
}
var secs float64
fmt.Sscanf(info.Format.Duration, "%f", &secs)
return secs
}
// SceneBoundary 场景时间边界
type SceneBoundary struct {
StartTime float64 `json:"start_time"`
EndTime float64 `json:"end_time"`
}
// parseScenes 解析场景检测结果 JSON 文件
func (s *sceneSplitService) parseScenes(jsonPath string) ([]SceneBoundary, error) {
data, err := os.ReadFile(jsonPath)
if err != nil {
return nil, fmt.Errorf("读取场景结果文件失败: %v", err)
}
var scenes []SceneBoundary
if err := json.Unmarshal(data, &scenes); err != nil {
return nil, fmt.Errorf("解析场景 JSON 失败: %v", err)
}
return scenes, nil
}
// segmentResult 单个分片切割结果
type segmentResult struct {
Path string // 本地文件路径
Timeline string // 时间线标识,如 "0.0-5.2"
}
// splitVideo 使用 FFmpeg 按场景时间线分割视频,返回分片文件路径+时间线
func (s *sceneSplitService) splitVideo(ctx context.Context, videoPath string, scenes []SceneBoundary, outputDir, taskID string) ([]segmentResult, error) {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return nil, err
}
var results []segmentResult
for i, scene := range scenes {
startTime := scene.StartTime
duration := scene.EndTime - scene.StartTime
if duration <= 0.1 {
g.Log().Warningf(ctx, "[场景分割] 跳过过短的场景 %d (%.2fs-%.2fs, 时长%.2fs)",
i+1, startTime, scene.EndTime, duration)
continue
}
outputPath := filepath.Join(outputDir, fmt.Sprintf("segment_%03d_%s.mp4", i+1, taskID))
timelineKey := fmt.Sprintf("%.1f-%.1f", startTime, scene.EndTime)
// 重编码模式:帧级精确切割,无重叠无间隙
args := []string{
"-ss", fmt.Sprintf("%.3f", startTime),
"-i", videoPath,
"-to", fmt.Sprintf("%.3f", duration),
"-c:v", "libx264",
"-preset", "fast",
"-crf", "22",
"-c:a", "aac",
"-b:a", "128k",
"-avoid_negative_ts", "make_zero",
"-y", outputPath,
}
g.Log().Infof(ctx, "[场景分割] 切割分片 %d/%d: %.2fs-%.2fs (时长%.2fs)",
i+1, len(scenes), startTime, scene.EndTime, duration)
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, cmdErr := cmd.CombinedOutput()
if cmdErr != nil {
return results, fmt.Errorf("分片%d切割失败: %v\n%s", i+1, cmdErr, string(output))
}
results = append(results, segmentResult{
Path: outputPath,
Timeline: timelineKey,
})
}
if len(results) == 0 {
return nil, fmt.Errorf("所有分片切割后均为空")
}
return results, nil
}
// extractAudio 从视频中提取音频
func (s *sceneSplitService) extractAudio(ctx context.Context, videoPath, audioOutputPath string) error {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return err
}
// 提取音频并编码为 AAC
args := []string{
"-i", videoPath,
"-vn", // 不要视频
"-acodec", "aac", // AAC 编码
"-b:a", "128k", // 音频比特率
"-y", audioOutputPath,
}
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("FFmpeg 音频提取失败: %v\n%s", err, string(output))
}
// 检查输出文件
if stat, statErr := os.Stat(audioOutputPath); statErr != nil || stat.Size() == 0 {
return fmt.Errorf("音频提取失败: 输出文件为空或不存在")
}
return nil
}
// ---------- 查询任务 ----------
// GetTaskResult 查询场景分割任务结果
func (s *sceneSplitService) GetTaskResult(ctx context.Context, taskID string) (*dto.GetSceneSplitTaskRes, error) {
task, err := dao.SceneSplitTask.GetByTaskID(ctx, taskID)
if err != nil {
return nil, fmt.Errorf("查询任务失败: %v", err)
}
if task == nil {
return nil, fmt.Errorf("任务不存在: %s", taskID)
}
return dao.EntityToSceneSplitTaskRes(task), nil
}
// ---------- 回调通知 ----------
func (s *sceneSplitService) callback(ctx context.Context, taskID, callbackURL string) {
if callbackURL == "" {
return
}
task, err := dao.SceneSplitTask.GetByTaskID(ctx, taskID)
if err != nil || task == nil {
g.Log().Errorf(ctx, "[场景分割回调 %s] 查询任务失败: %v", taskID, err)
return
}
payload := map[string]interface{}{
"taskId": taskID,
"status": task.Status,
}
if task.Status == "success" {
payload["audioUrl"] = task.AudioURL
payload["segments"] = dao.ParseSegmentEntries(task.SegmentURLs)
payload["sceneCount"] = task.SceneCount
payload["audioDuration"] = task.AudioDuration
payload["videoDuration"] = task.VideoDuration
}
if task.Status == "failed" {
payload["errorMessage"] = task.ErrorMessage
}
body, _ := json.Marshal(payload)
cbUser := getUserFromCtx(ctx)
userJSON, _ := json.Marshal(cbUser)
g.Log().Infof(ctx, "[场景分割回调 %s] 状态=%s, 目标=%s", taskID, task.Status, callbackURL)
g.Log().Infof(ctx, "[场景分割回调 %s] curl: curl -X POST '%s' -H 'Content-Type: application/json' -H 'X-User-Info: %s' -d '%s'",
taskID, callbackURL, string(userJSON), string(body))
req, _ := http.NewRequest("POST", callbackURL, bytes.NewReader(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-User-Info", string(userJSON))
client := &http.Client{Timeout: 2 * time.Minute}
resp, reqErr := client.Do(req)
if reqErr != nil {
g.Log().Errorf(ctx, "[场景分割回调 %s] 请求失败: %v", taskID, reqErr)
return
}
defer resp.Body.Close()
respBody, _ := io.ReadAll(resp.Body)
g.Log().Infof(ctx, "[场景分割回调 %s] 响应 status=%d, body=%s", taskID, resp.StatusCode, string(respBody))
}
+93
View File
@@ -0,0 +1,93 @@
package video
import (
"context"
"fmt"
"os"
"os/exec"
"path/filepath"
"time"
dto "media/model/dto/video"
"github.com/gogf/gf/v2/frame/g"
)
type transcodeService struct{}
var Transcode = new(transcodeService)
// TranscodeToMP4 将视频转码为 H.264 + AAC + MP4 + faststart
// inputPath: 输入文件路径
// outputDir: 输出目录(可选),为空则使用 resource/temp
// 返回输出文件完整路径、文件信息
func (s *transcodeService) TranscodeToMP4(ctx context.Context, inputPath string, outputDir string) (*dto.TranscodeRes, error) {
ffmpegPath, err := lookupFFmpegPath()
if err != nil {
return nil, fmt.Errorf("ffmpeg 未找到: %v", err)
}
// 输出目录
if outputDir == "" {
outputDir = g.Cfg().MustGet(ctx, "ffmpeg.temp_dir", "resource/temp").String()
if outputDir == "" {
outputDir = "resource/temp"
}
if !filepath.IsAbs(outputDir) {
absDir, _ := filepath.Abs(outputDir)
outputDir = absDir
}
}
os.MkdirAll(outputDir, 0755)
// 输出文件名:原文件名_转码_时间戳.mp4
base := filepath.Base(inputPath)
ext := filepath.Ext(base)
name := base[:len(base)-len(ext)]
outputName := fmt.Sprintf("%s_transcoded_%d.mp4", name, time.Now().UnixMilli())
outputPath := filepath.Join(outputDir, outputName)
g.Log().Infof(ctx, "[转码] 开始: %s → %s", inputPath, outputPath)
// FFmpeg 转码命令
// -c:v libx264 H.264 视频编码
// -preset fast 编码速度/质量平衡
// -crf 22 画质(0-51,越小越好,18-28常用)
// -c:a aac AAC 音频编码
// -b:a 128k 音频码率 128k
// -movflags +faststart MOOV 前置(关键!seed-lite 必需)
args := []string{
"-i", inputPath,
"-c:v", "libx264",
"-preset", "fast",
"-crf", "22",
"-c:a", "aac",
"-b:a", "128k",
"-movflags", "+faststart",
"-y", outputPath,
}
cmd := exec.CommandContext(ctx, ffmpegPath, args...)
output, err := cmd.CombinedOutput()
if err != nil {
return nil, fmt.Errorf("FFmpeg 转码失败: %v\n%s", err, string(output))
}
// 检查输出文件
stat, statErr := os.Stat(outputPath)
if statErr != nil {
return nil, fmt.Errorf("转码输出文件不存在: %v", statErr)
}
// 获取时长
durationStr := getVideoDurationStr(ctx, outputPath)
g.Log().Infof(ctx, "[转码] 完成: %s, 大小=%d, 时长=%s", outputPath, stat.Size(), durationStr)
return &dto.TranscodeRes{
OutputPath: outputPath,
FileSize: stat.Size(),
FileName: outputName,
DurationStr: durationStr,
}, nil
}
+37
View File
@@ -0,0 +1,37 @@
-- scene_split_task 场景检测+视频分割异步任务表
CREATE TABLE IF NOT EXISTS scene_split_task (
id BIGSERIAL NOT NULL,
tenant_id BIGINT NOT NULL DEFAULT 0,
task_id VARCHAR(64) NOT NULL,
video_url TEXT NOT NULL DEFAULT '',
status VARCHAR(20) NOT NULL DEFAULT 'pending',
segment_urls TEXT NOT NULL DEFAULT '',
audio_url TEXT NOT NULL DEFAULT '',
scene_count INT NOT NULL DEFAULT 0,
audio_duration DOUBLE PRECISION NOT NULL DEFAULT 0,
video_duration DOUBLE PRECISION NOT NULL DEFAULT 0,
error_message TEXT,
callback_url VARCHAR(500) NOT NULL DEFAULT '',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
deleted_at TIMESTAMP WITH TIME ZONE,
PRIMARY KEY (id)
);
COMMENT ON TABLE scene_split_task IS '场景检测+视频分割异步任务表';
COMMENT ON COLUMN scene_split_task.task_id IS '任务唯一标识';
COMMENT ON COLUMN scene_split_task.video_url IS '原始视频URL';
COMMENT ON COLUMN scene_split_task.status IS '任务状态:pending/running/success/failed';
COMMENT ON COLUMN scene_split_task.segment_urls IS '分割后的视频分片URL列表(JSON数组)';
COMMENT ON COLUMN scene_split_task.audio_url IS '提取的音频URL';
COMMENT ON COLUMN scene_split_task.scene_count IS '检测到的场景数/分片数';
COMMENT ON COLUMN scene_split_task.audio_duration IS '音频时长(秒)';
COMMENT ON COLUMN scene_split_task.error_message IS '错误信息';
COMMENT ON COLUMN scene_split_task.callback_url IS '回调地址';
COMMENT ON COLUMN scene_split_task.created_at IS '创建时间';
COMMENT ON COLUMN scene_split_task.updated_at IS '更新时间';
COMMENT ON COLUMN scene_split_task.deleted_at IS '删除时间(软删除)';
CREATE UNIQUE INDEX IF NOT EXISTS idx_scene_split_task_id ON scene_split_task(task_id);
CREATE INDEX IF NOT EXISTS idx_scene_split_status ON scene_split_task(status);
CREATE INDEX IF NOT EXISTS idx_scene_split_created_at ON scene_split_task(created_at);
+4
View File
@@ -0,0 +1,4 @@
-- scene_split_task 增加 video_duration 字段
ALTER TABLE scene_split_task ADD COLUMN IF NOT EXISTS video_duration DOUBLE PRECISION NOT NULL DEFAULT 0;
COMMENT ON COLUMN scene_split_task.video_duration IS '视频总时长(秒)';
+42
View File
@@ -0,0 +1,42 @@
-- video_audio_merge_task 视频拼接+混音异步任务表
CREATE TABLE IF NOT EXISTS video_audio_merge_task (
id BIGSERIAL NOT NULL,
tenant_id BIGINT NOT NULL DEFAULT 0,
task_id VARCHAR(64) NOT NULL,
video_urls TEXT NOT NULL DEFAULT '',
audio_urls TEXT NOT NULL DEFAULT '',
status VARCHAR(20) NOT NULL DEFAULT 'pending',
file_url TEXT NOT NULL DEFAULT '',
file_size BIGINT NOT NULL DEFAULT 0,
file_name VARCHAR(255) NOT NULL DEFAULT '',
file_format VARCHAR(32) NOT NULL DEFAULT '',
file_address_prefix TEXT NOT NULL DEFAULT '',
duration_str VARCHAR(32) NOT NULL DEFAULT '',
error_message TEXT,
callback_url VARCHAR(500) NOT NULL DEFAULT '',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
deleted_at TIMESTAMP WITH TIME ZONE,
PRIMARY KEY (id)
);
COMMENT ON TABLE video_audio_merge_task IS '视频拼接+混音异步任务表';
COMMENT ON COLUMN video_audio_merge_task.task_id IS '任务唯一标识';
COMMENT ON COLUMN video_audio_merge_task.video_urls IS '视频URL列表(JSON数组)';
COMMENT ON COLUMN video_audio_merge_task.audio_urls IS '音频URL列表(JSON数组)';
COMMENT ON COLUMN video_audio_merge_task.status IS '任务状态:pending/running/success/failed';
COMMENT ON COLUMN video_audio_merge_task.file_url IS 'MinIO文件访问路径';
COMMENT ON COLUMN video_audio_merge_task.file_size IS '文件大小(字节)';
COMMENT ON COLUMN video_audio_merge_task.file_name IS '文件名';
COMMENT ON COLUMN video_audio_merge_task.file_format IS '文件格式';
COMMENT ON COLUMN video_audio_merge_task.file_address_prefix IS 'MinIO地址前缀';
COMMENT ON COLUMN video_audio_merge_task.duration_str IS '合并后视频时长';
COMMENT ON COLUMN video_audio_merge_task.error_message IS '错误信息';
COMMENT ON COLUMN video_audio_merge_task.callback_url IS '回调地址';
COMMENT ON COLUMN video_audio_merge_task.created_at IS '创建时间';
COMMENT ON COLUMN video_audio_merge_task.updated_at IS '更新时间';
COMMENT ON COLUMN video_audio_merge_task.deleted_at IS '删除时间(软删除)';
CREATE UNIQUE INDEX IF NOT EXISTS idx_merge_task_task_id ON video_audio_merge_task(task_id);
CREATE INDEX IF NOT EXISTS idx_merge_task_status ON video_audio_merge_task(status);
CREATE INDEX IF NOT EXISTS idx_merge_task_created_at ON video_audio_merge_task(created_at);
+38
View File
@@ -0,0 +1,38 @@
-- video_caption_task 字幕叠加任务表
-- 使用 HyperFrames (HTML+CSS+Chrome Headless) 渲染带字幕的营销视频
CREATE TABLE IF NOT EXISTS video_caption_task (
id BIGSERIAL PRIMARY KEY,
tenant_id INT8 NOT NULL DEFAULT 0,
task_id VARCHAR(64) NOT NULL UNIQUE, -- 任务唯一标识,前缀 cap_
video_urls TEXT NOT NULL, -- 视频URL列表,JSON数组
audio_url VARCHAR(1024) DEFAULT '', -- 背景音乐URL(可选)
elements TEXT NOT NULL, -- 字幕元素列表,JSON数组
width INT NOT NULL DEFAULT 1080, -- 视频宽度
height INT NOT NULL DEFAULT 1920, -- 视频高度
status VARCHAR(20) NOT NULL DEFAULT 'pending', -- pending/running/success/failed
file_url VARCHAR(1024) DEFAULT '', -- 输出文件URL
file_size INT8 DEFAULT 0, -- 输出文件大小(字节)
file_name VARCHAR(255) DEFAULT '', -- 输出文件名
duration_str VARCHAR(20) DEFAULT '', -- 视频时长
error_message TEXT, -- 错误信息
callback_url VARCHAR(1024) DEFAULT '', -- 回调地址
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
deleted_at TIMESTAMP WITH TIME ZONE DEFAULT NULL
);
COMMENT ON TABLE video_caption_task IS '字幕叠加任务';
COMMENT ON COLUMN video_caption_task.task_id IS '任务唯一标识';
COMMENT ON COLUMN video_caption_task.video_urls IS '背景视频URL列表';
COMMENT ON COLUMN video_caption_task.audio_url IS '背景音乐URL';
COMMENT ON COLUMN video_caption_task.elements IS '字幕/图片元素列表JSON';
COMMENT ON COLUMN video_caption_task.status IS '任务状态:pending/running/success/failed';
COMMENT ON COLUMN video_caption_task.file_url IS '输出文件URL';
COMMENT ON COLUMN video_caption_task.file_size IS '输出文件大小';
COMMENT ON COLUMN video_caption_task.file_name IS '输出文件名';
COMMENT ON COLUMN video_caption_task.duration_str IS '视频时长';
COMMENT ON COLUMN video_caption_task.error_message IS '错误信息';
COMMENT ON COLUMN video_caption_task.callback_url IS '任务完成后回调地址';
CREATE INDEX idx_caption_task_id ON video_caption_task(task_id);
CREATE INDEX idx_caption_status ON video_caption_task(status);