训练体系整合与标注单阶段化
- 标注：AI 预标注直写 labels_json（去候选确认两阶段）；重叠去重（minIoU）；全量标注按钮 - 训练：脚本迁移入 server/training/（Go 化 prepare_yolo/analyze_rfdetr，保留 train_server.py）；tflite 产物自检并入训练流程（check_tflite） - 数据目录/权重不进 git；.gitignore 迁移至仓库根
This commit is contained in:
@@ -38,7 +38,7 @@ app.*.symbols
|
||||
|
||||
# Obfuscation related
|
||||
app.*.map.json
|
||||
|
||||
observer-latest.apk
|
||||
# Android Studio will place build artifacts here
|
||||
/android/app/debug
|
||||
/android/app/profile
|
||||
|
||||
+28
-4
@@ -1,6 +1,6 @@
|
||||
# observer
|
||||
|
||||
野生动物实时识别 App（Flutter 版）。Android / iOS 一套代码，后端接口与支付见
|
||||
动物实时识别 App（Flutter 版）。Android / iOS 一套代码，后端接口与支付见
|
||||
[`docs/PaymentApi.md`](docs/PaymentApi.md)。
|
||||
|
||||
## iOS 真机部署（iPhone）
|
||||
@@ -8,9 +8,10 @@
|
||||
### 构建与安装
|
||||
|
||||
```bash
|
||||
# 真机必须传 Mac 局域网 IP：默认 API_BASE_URL 是 10.0.2.2（仅 Android 模拟器可用），
|
||||
# 不传则 iPhone 上所有网络请求（登录/授权/套餐）都会失败
|
||||
flutter build ios --release --dart-define=API_BASE_URL=http://<Mac局域网IP>:8080
|
||||
# 生产包直接构建即可：默认 API_BASE_URL 为线上域名（lib/config/app_config.dart），无需传参
|
||||
flutter build ios --release
|
||||
# 仅本地联调（后端跑在 Mac 上、手机连同一 Wi-Fi）时才覆盖为 Mac 局域网 IP：
|
||||
# flutter build ios --release --dart-define=API_BASE_URL=http://<Mac局域网IP>:8080
|
||||
|
||||
# 安装到真机（UDID 可用 `xcrun devicectl list devices` 查询）
|
||||
xcrun devicectl device install app --device <UDID> build/ios/iphoneos/Runner.app
|
||||
@@ -26,6 +27,29 @@ xcrun devicectl device process launch --console --terminate-existing \
|
||||
"In iOS 14+, debug mode Flutter apps can only be launched from Flutter tooling"。
|
||||
debug 调试必须用 `flutter run -d <设备ID>` 或 Xcode IDE 启动（`flutter devices` 查设备ID）；
|
||||
从图标启动只对 release 构建有效。
|
||||
## 模型热更新（多数据集模型）
|
||||
|
||||
模型与 APK 更新走**独立通道**：启动时拉取 `GET /api/v1/app/update` 随附的
|
||||
`models` 目录（公开接口，无需登录），与 `UpdateChecker` 的 APK 检查并行。
|
||||
|
||||
- **目录条目**：`{datasetId, datasetName, version, labels[], sizeBytes, sha256,
|
||||
downloadUrl}`；服务器未发布模型时不返回 `models` 字段，App 回退内置
|
||||
`assets/model.tflite`（标签 `内置`）。
|
||||
- **存储**：应用私有目录 `models/<datasetId>/`，含 `model.tflite`、
|
||||
`labels.json`、`meta.json`（meta 记录 `{version, sha256}`）。版本与摘要都
|
||||
未变化时跳过下载；变化则下载到 `.part` 临时文件、sha256 校验通过后
|
||||
原子 rename 替换，失败重试一次并保留旧模型，下次启动再试。
|
||||
- **清理**：服务器下线的数据集下次同步时删除本地对应目录。
|
||||
- **并行推理合并**：识别时加载全部已下载模型（`DetectorWorker` isolate 内
|
||||
逐模型加载，单个失败不影响其他），同帧各模型独立推理后按类别分组做
|
||||
**跨模型 NMS**（同类别不同模型检出同一目标取高分去重，不同类别互不压制），
|
||||
结果叠加 `modelName` 标注来源，内置模型兜底。
|
||||
|
||||
实现：`lib/models/model_manager.dart`（下载/校验/持久化，`ModelManager`
|
||||
单例 + ChangeNotifier）、`lib/detection/detector_worker.dart`（多模型并行
|
||||
推理与 `mergeAcrossModels`）、`lib/camera/camera_screen.dart`（启动同步 +
|
||||
诊断行展示模型列表）。
|
||||
|
||||
- **模型输入是 NHWC**：`assets/model.tflite` 做过字节级手术（开头 TRANSPOSE→RESHAPE，
|
||||
输入 [1,320,320,3]），改动记录见 git 历史，重导模型需同步处理，否则 iOS 报
|
||||
"Node number 0 (TRANSPOSE) failed to prepare"。
|
||||
|
||||
@@ -40,6 +40,10 @@ kotlin {
|
||||
}
|
||||
}
|
||||
|
||||
// 相机用 Android 框架 camera2 API 完全自研（CameraChannel.kt）：
|
||||
// 预览 SurfaceTexture（Flutter 纹理）+ ImageReader 分析帧（原生侧旋转成竖屏后回传），
|
||||
// 不依赖任何相机三方库（含 CameraX）。
|
||||
|
||||
// tflite_flutter 依赖的 tensorflow-lite / tensorflow-lite-gpu / tensorflow-lite-api 三个 AAR
|
||||
// 声明了相同 namespace(org.tensorflow.lite),新 AGP 视作冲突直接报错;
|
||||
// 本项目仅用 CPU 推理,GPU delegate 未使用,排除 gpu 及其传递依赖的 api 即可。
|
||||
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
# R8 release 压缩：okhttp 可选 TLS 平台类（BouncyCastle/Conscrypt/OpenJSSE）
|
||||
# 与 tflite_flutter 的反射注解类未打包，仅需忽略引用告警（AGP missing_rules.txt 生成）
|
||||
-dontwarn org.bouncycastle.jsse.BCSSLParameters
|
||||
-dontwarn org.bouncycastle.jsse.BCSSLSocket
|
||||
-dontwarn org.bouncycastle.jsse.provider.BouncyCastleJsseProvider
|
||||
-dontwarn org.conscrypt.Conscrypt$Version
|
||||
-dontwarn org.conscrypt.Conscrypt
|
||||
-dontwarn org.conscrypt.ConscryptHostnameVerifier
|
||||
-dontwarn org.openjsse.javax.net.ssl.SSLParameters
|
||||
-dontwarn org.openjsse.javax.net.ssl.SSLSocket
|
||||
-dontwarn org.openjsse.net.ssl.OpenJSSE
|
||||
-dontwarn org.tensorflow.lite.InterpreterFactoryApi
|
||||
-dontwarn org.tensorflow.lite.annotations.UsedByReflection
|
||||
@@ -1,6 +1,8 @@
|
||||
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
|
||||
<uses-permission android:name="android.permission.CAMERA"/>
|
||||
<uses-permission android:name="android.permission.INTERNET"/>
|
||||
<!-- App 内更新安装 APK（PackageInstaller 会话安装，见 InstallerChannel.kt） -->
|
||||
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES"/>
|
||||
<application
|
||||
android:label="视野"
|
||||
android:name="${applicationName}"
|
||||
|
||||
@@ -0,0 +1,568 @@
|
||||
package com.example.observer
|
||||
|
||||
import android.graphics.Rect
|
||||
import android.graphics.RectF
|
||||
import android.graphics.SurfaceTexture
|
||||
import android.os.Build
|
||||
import android.hardware.camera2.CameraCaptureSession
|
||||
import android.hardware.camera2.CameraCharacteristics
|
||||
import android.hardware.camera2.CameraDevice
|
||||
import android.hardware.camera2.CameraManager
|
||||
import android.hardware.camera2.CaptureRequest
|
||||
import android.graphics.ImageFormat
|
||||
import android.media.Image
|
||||
import android.media.ImageReader
|
||||
import android.os.Handler
|
||||
import android.os.HandlerThread
|
||||
import android.os.SystemClock
|
||||
import android.util.Log
|
||||
import android.util.Size
|
||||
import android.view.Surface
|
||||
import io.flutter.embedding.android.FlutterActivity
|
||||
import io.flutter.embedding.engine.FlutterEngine
|
||||
import io.flutter.plugin.common.EventChannel
|
||||
import io.flutter.plugin.common.MethodCall
|
||||
import io.flutter.plugin.common.MethodChannel
|
||||
import io.flutter.view.TextureRegistry
|
||||
|
||||
/**
|
||||
* 自写原生相机：Android 框架 camera2 API 完全自研，零相机三方依赖（含 CameraX）。
|
||||
*
|
||||
* 核心：分析帧在原生侧旋转成竖屏方向再回传，Flutter 侧恒用 rotation=0，
|
||||
* 与 iOS（camera_avfoundation，帧本来就是竖屏）行为对齐，彻底消除
|
||||
* "横屏传感器帧 → 90° 旋转 + FIT_COVER crop 映射"的标注偏移根因。
|
||||
*
|
||||
* 旋转角来自设备标准值 sensorOrientation - displayRotation（camera2 特性，
|
||||
* 非厂商 hack），因此映射跨厂商一致：任何设备上"分析帧 = 预览帧同 sensor
|
||||
* 同旋转"，两者几何必然一致。
|
||||
*
|
||||
* 预览：SurfaceTexture 注册进 FlutterTextureRegistry，Flutter 侧 Texture widget
|
||||
* 渲染（与插件 CameraPreview 相同的合成方式，overlay/诊断行/按钮可叠加；
|
||||
* AndroidView+SurfaceView 会盖住 Flutter UI，不可用）。
|
||||
*
|
||||
* 通道：
|
||||
* - MethodChannel "observer/camera"：start/stop/getZoomRange/setZoom/isStreaming/errorDescription
|
||||
* - EventChannel "observer/camera/frames"：每帧 [width, height, rotationDegrees, bgra, bytes]
|
||||
* - rotationDegrees 恒 0（帧已竖屏）；bgra=false（RGBA 字节序）
|
||||
*/
|
||||
class CameraChannel(
|
||||
private val activity: FlutterActivity,
|
||||
private val engine: FlutterEngine,
|
||||
) {
|
||||
private val messenger = engine.dartExecutor.binaryMessenger
|
||||
private val method = MethodChannel(messenger, "observer/camera")
|
||||
private val frames = EventChannel(messenger, "observer/camera/frames")
|
||||
|
||||
private val cameraManager: CameraManager = activity.getSystemService(CameraManager::class.java)
|
||||
|
||||
private var cameraDevice: CameraDevice? = null
|
||||
private var captureSession: CameraCaptureSession? = null
|
||||
private var imageReader: ImageReader? = null
|
||||
private var surfaceEntry: TextureRegistry.SurfaceTextureEntry? = null
|
||||
private var previewSurface: Surface? = null
|
||||
private var eventSink: EventChannel.EventSink? = null
|
||||
|
||||
private var sensorOrientation = 90
|
||||
private var activeArray = Rect(0, 0, 1920, 1080)
|
||||
private var maxZoom = 1.0f
|
||||
|
||||
/// 预览/分析共用分辨率（传感器方向，从设备流配置动态选择）
|
||||
private var previewSize = Size(1920, 1080)
|
||||
|
||||
@Volatile
|
||||
private var running = false
|
||||
|
||||
@Volatile
|
||||
private var errorDescription: String? = null
|
||||
|
||||
private val mainHandler = Handler(activity.mainLooper)
|
||||
private val analysisThread = HandlerThread("observer-analysis").also { it.start() }
|
||||
private val analysisHandler = Handler(analysisThread.looper)
|
||||
|
||||
@Volatile
|
||||
private var lastEmitMs = 0L
|
||||
|
||||
// 诊断计数（frameListener 线程写，stats 轮询读）：
|
||||
// 帧回调到达次数 / 成功发出 / 发出异常 / 无订阅者丢弃
|
||||
@Volatile
|
||||
private var frameCallbacks = 0L
|
||||
|
||||
@Volatile
|
||||
private var emitOk = 0L
|
||||
|
||||
@Volatile
|
||||
private var emitErr = 0L
|
||||
|
||||
@Volatile
|
||||
private var sinkNullCount = 0L
|
||||
|
||||
// 实际发出帧尺寸（缩小后）
|
||||
@Volatile
|
||||
private var emitW = 0
|
||||
|
||||
@Volatile
|
||||
private var emitH = 0
|
||||
|
||||
fun register() {
|
||||
method.setMethodCallHandler(::onMethodCall)
|
||||
frames.setStreamHandler(object : EventChannel.StreamHandler {
|
||||
override fun onListen(arguments: Any?, events: EventChannel.EventSink?) {
|
||||
eventSink = events
|
||||
}
|
||||
|
||||
override fun onCancel(arguments: Any?) {
|
||||
eventSink = null
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
private fun onMethodCall(call: MethodCall, result: MethodChannel.Result) {
|
||||
when (call.method) {
|
||||
"start" -> start(result)
|
||||
"stop" -> {
|
||||
stop()
|
||||
result.success(true)
|
||||
}
|
||||
"getZoomRange" -> result.success(listOf(1.0, maxZoom.toDouble()))
|
||||
"setZoom" -> {
|
||||
val v = (call.arguments as Number).toFloat()
|
||||
setZoom(v)
|
||||
result.success(true)
|
||||
}
|
||||
"isStreaming" -> result.success(running)
|
||||
"errorDescription" -> result.success(errorDescription)
|
||||
"stats" -> result.success(
|
||||
mapOf(
|
||||
"callbacks" to frameCallbacks,
|
||||
"emitOk" to emitOk,
|
||||
"emitErr" to emitErr,
|
||||
"sinkNull" to sinkNullCount,
|
||||
"sink" to (eventSink != null),
|
||||
"running" to running,
|
||||
"rotation" to rotation,
|
||||
"quarterTurns" to displayDegrees / 90,
|
||||
"displayDegrees" to displayDegrees,
|
||||
"size" to "${previewSize.width}x${previewSize.height}",
|
||||
"emitSize" to "${emitW}x$emitH",
|
||||
"emitAgeMs" to
|
||||
if (lastEmitMs == 0L) -1L
|
||||
else SystemClock.elapsedRealtime() - lastEmitMs,
|
||||
"error" to errorDescription,
|
||||
),
|
||||
)
|
||||
else -> result.notImplemented()
|
||||
}
|
||||
}
|
||||
|
||||
private fun start(result: MethodChannel.Result) {
|
||||
if (running) {
|
||||
result.success(true)
|
||||
return
|
||||
}
|
||||
running = true
|
||||
errorDescription = null
|
||||
try {
|
||||
val cameraId = pickBackCamera() ?: throw IllegalStateException("未找到后置摄像头")
|
||||
val characteristics = cameraManager.getCameraCharacteristics(cameraId)
|
||||
sensorOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION) ?: 90
|
||||
activeArray = characteristics.get(
|
||||
CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE,
|
||||
) ?: Rect(0, 0, 1920, 1080)
|
||||
maxZoom = characteristics.get(CameraCharacteristics.SCALER_AVAILABLE_MAX_DIGITAL_ZOOM) ?: 1.0f
|
||||
// 分辨率不写死：从设备流配置查询（传感器方向尺寸，宽≥高），
|
||||
// 预览与分析共用同一尺寸，保证两者几何一致（对齐根因）。
|
||||
// 查询是优化而非必需：个别设备 getOutputSizes 会返回 null 或抛异常
|
||||
// （实测某设备内部抛 getClass NPE），失败一律回退默认分辨率，
|
||||
// 绝不让相机启动失败
|
||||
try {
|
||||
val streamMap = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP)
|
||||
if (streamMap != null) {
|
||||
val previewSizes =
|
||||
streamMap.getOutputSizes(SurfaceTexture::class.java) ?: emptyArray()
|
||||
val analysisSizes =
|
||||
streamMap.getOutputSizes(ImageFormat.YUV_420_888) ?: emptyArray()
|
||||
previewSize = pickSize(previewSizes, analysisSizes)
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "size query failed, fallback ${previewSize.width}x${previewSize.height}", e)
|
||||
}
|
||||
|
||||
val entry = surfaceEntry ?: engine.getRenderer().createSurfaceTexture().also {
|
||||
surfaceEntry = it
|
||||
}
|
||||
// 关键：SurfaceTexture 未设默认 buffer 尺寸时 createCaptureSession 会配置失败
|
||||
// （camera2 按该尺寸做 stream 校验）。显式指定与 ImageReader 一致的分辨率，
|
||||
// 保证预览与分析帧同分辨率同裁剪
|
||||
entry.surfaceTexture().setDefaultBufferSize(previewSize.width, previewSize.height)
|
||||
previewSurface?.release()
|
||||
previewSurface = Surface(entry.surfaceTexture())
|
||||
imageReader?.close()
|
||||
// YUV_420_888 是 camera2 对所有设备保证支持的 ImageReader 输出格式；
|
||||
// RGBA_8888 个别设备不支持（实测：查询 NPE + 配置失败双连败），弃用
|
||||
imageReader = ImageReader.newInstance(
|
||||
previewSize.width, previewSize.height, ImageFormat.YUV_420_888, 2,
|
||||
).also { it.setOnImageAvailableListener(frameListener, analysisHandler) }
|
||||
|
||||
cameraManager.openCamera(
|
||||
cameraId,
|
||||
object : CameraDevice.StateCallback() {
|
||||
override fun onOpened(device: CameraDevice) {
|
||||
cameraDevice = device
|
||||
createSession(result)
|
||||
}
|
||||
|
||||
override fun onDisconnected(device: CameraDevice) {
|
||||
device.close()
|
||||
cameraDevice = null
|
||||
}
|
||||
|
||||
override fun onError(device: CameraDevice, error: Int) {
|
||||
device.close()
|
||||
cameraDevice = null
|
||||
running = false
|
||||
errorDescription = "camera open error $error"
|
||||
result.error("start", errorDescription, null)
|
||||
}
|
||||
},
|
||||
mainHandler,
|
||||
)
|
||||
} catch (e: Exception) {
|
||||
running = false
|
||||
errorDescription = e.toString()
|
||||
// details 带完整堆栈：任何残留异常都能在诊断行看到精确位置
|
||||
result.error("start", e.toString(), Log.getStackTraceString(e))
|
||||
}
|
||||
}
|
||||
|
||||
private fun createSession(result: MethodChannel.Result) {
|
||||
val device = cameraDevice ?: return
|
||||
val surfaces = listOfNotNull(previewSurface, imageReader?.surface)
|
||||
device.createCaptureSession(
|
||||
surfaces,
|
||||
object : CameraCaptureSession.StateCallback() {
|
||||
override fun onConfigured(session: CameraCaptureSession) {
|
||||
captureSession = session
|
||||
try {
|
||||
val request = buildRequest(session.device, surfaces)
|
||||
session.setRepeatingRequest(request, null, mainHandler)
|
||||
result.success(
|
||||
mapOf(
|
||||
"textureId" to (surfaceEntry?.id() ?: -1L),
|
||||
// 传感器方向尺寸（宽≥高）：Flutter 侧 SizedBox 在 RotatedBox
|
||||
// 内部声明纹理尺寸，旋转后视觉上才是竖屏 1080x1920
|
||||
"w" to previewSize.width,
|
||||
"h" to previewSize.height,
|
||||
// 预览旋转 = 显示旋转（不是 rotation/90！）：
|
||||
// Flutter 引擎渲染 Texture 时已自动应用 SurfaceTexture
|
||||
// 变换矩阵（传感器方向补偿），预览只需再按显示旋转补偿
|
||||
"quarterTurns" to displayDegrees / 90,
|
||||
// 诊断：旋转计算输入值
|
||||
"sensorOrientation" to sensorOrientation,
|
||||
"displayDegrees" to displayDegrees,
|
||||
),
|
||||
)
|
||||
} catch (e: Exception) {
|
||||
errorDescription = "onConfigured: $e"
|
||||
result.error("start", errorDescription, Log.getStackTraceString(e))
|
||||
}
|
||||
}
|
||||
|
||||
override fun onConfigureFailed(session: CameraCaptureSession) {
|
||||
running = false
|
||||
errorDescription = "capture session configure failed"
|
||||
result.error("start", errorDescription, null)
|
||||
}
|
||||
},
|
||||
mainHandler,
|
||||
)
|
||||
}
|
||||
|
||||
private fun buildRequest(device: CameraDevice, targets: List<Surface>): CaptureRequest {
|
||||
val builder = device.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
|
||||
targets.forEach { builder.addTarget(it) }
|
||||
builder.set(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_AUTO)
|
||||
// 连续对焦：无 AF 能力的设备忽略该设置
|
||||
try {
|
||||
builder.set(
|
||||
CaptureRequest.CONTROL_AF_MODE,
|
||||
CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_PICTURE,
|
||||
)
|
||||
} catch (_: IllegalArgumentException) {
|
||||
}
|
||||
return builder.build()
|
||||
}
|
||||
|
||||
private fun setZoom(z: Float) {
|
||||
val session = captureSession ?: return
|
||||
val clamped = z.coerceIn(1.0f, maxZoom)
|
||||
val r = activeArray
|
||||
val insetW = r.width() * (1 - 1 / clamped) / 2
|
||||
val insetH = r.height() * (1 - 1 / clamped) / 2
|
||||
val crop = RectF(
|
||||
r.left + insetW,
|
||||
r.top + insetH,
|
||||
r.right - insetW,
|
||||
r.bottom - insetH,
|
||||
)
|
||||
try {
|
||||
val surface = previewSurface ?: return
|
||||
val builder = session.device.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW)
|
||||
builder.addTarget(surface)
|
||||
imageReader?.surface?.let { builder.addTarget(it) }
|
||||
builder.set(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_AUTO)
|
||||
builder.set(
|
||||
CaptureRequest.SCALER_CROP_REGION,
|
||||
Rect(
|
||||
crop.left.toInt(),
|
||||
crop.top.toInt(),
|
||||
crop.right.toInt(),
|
||||
crop.bottom.toInt(),
|
||||
),
|
||||
)
|
||||
session.setRepeatingRequest(builder.build(), null, mainHandler)
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "setZoom failed", e)
|
||||
}
|
||||
}
|
||||
|
||||
private fun stop() {
|
||||
running = false
|
||||
try {
|
||||
captureSession?.close()
|
||||
} catch (_: Exception) {
|
||||
}
|
||||
captureSession = null
|
||||
try {
|
||||
cameraDevice?.close()
|
||||
} catch (_: Exception) {
|
||||
}
|
||||
cameraDevice = null
|
||||
}
|
||||
|
||||
fun destroy() {
|
||||
stop()
|
||||
imageReader?.close()
|
||||
imageReader = null
|
||||
previewSurface?.release()
|
||||
previewSurface = null
|
||||
surfaceEntry?.release()
|
||||
surfaceEntry = null
|
||||
analysisThread.quitSafely()
|
||||
}
|
||||
|
||||
private fun pickBackCamera(): String? {
|
||||
val ids = cameraManager.cameraIdList
|
||||
for (id in ids) {
|
||||
val c = cameraManager.getCameraCharacteristics(id)
|
||||
val facing = c.get(CameraCharacteristics.LENS_FACING)
|
||||
if (facing == CameraCharacteristics.LENS_FACING_BACK) return id
|
||||
}
|
||||
return ids.firstOrNull()
|
||||
}
|
||||
|
||||
/**
|
||||
* 从设备流配置挑预览/分析共用分辨率（均为传感器方向尺寸，宽≥高）：
|
||||
* 取两者交集，16:9 优先、长边 ≤1920 内取最大（推理输入 640x640，
|
||||
* 更高只增帧传输与旋转开销，无精度收益）；无 16:9 时退回最大交集尺寸。
|
||||
*/
|
||||
private fun pickSize(previewSizes: Array<Size>, analysisSizes: Array<Size>): Size {
|
||||
val common = previewSizes.filter { analysisSizes.contains(it) }
|
||||
val fallback = common.maxByOrNull { it.width.toLong() * it.height }
|
||||
?: return Size(1920, 1080)
|
||||
return common
|
||||
.filter {
|
||||
Math.abs(it.width.toDouble() / it.height - 16.0 / 9.0) < 0.03 &&
|
||||
it.width <= 1920 && it.height <= 1920
|
||||
}
|
||||
.maxByOrNull { it.width.toLong() * it.height }
|
||||
?: fallback
|
||||
}
|
||||
|
||||
/**
|
||||
* 显示旋转角（度）。Display.ROTATION_* 在 API 36 是受限常量，直接用其值（0/1/2/3）。
|
||||
* API 30+ 用 activity.display（现代 API，实测可靠跟踪旋转）；
|
||||
* 旧版 windowManager.defaultDisplay.rotation 已废弃，实测在部分设备恒返回 0。
|
||||
*/
|
||||
private val displayDegrees: Int
|
||||
get() = try {
|
||||
val rot: Int = if (Build.VERSION.SDK_INT >= 30) {
|
||||
activity.display?.rotation ?: 0
|
||||
} else {
|
||||
@Suppress("DEPRECATION")
|
||||
activity.windowManager.defaultDisplay.rotation
|
||||
}
|
||||
when (rot) {
|
||||
1 -> 90
|
||||
2 -> 180
|
||||
3 -> 270
|
||||
else -> 0
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "display rotation query failed", e)
|
||||
0
|
||||
}
|
||||
|
||||
/** 传感器 → 竖屏显示的顺时针旋转角（后摄无镜像） */
|
||||
private val rotation: Int
|
||||
get() = ((sensorOrientation - displayDegrees) % 360 + 360) % 360
|
||||
|
||||
private val frameListener = ImageReader.OnImageAvailableListener { reader ->
|
||||
// 诊断计数：回调是否到达（放在一切判定之前，任何丢弃都先计数）
|
||||
frameCallbacks++
|
||||
if (frameCallbacks % 50 == 0L) {
|
||||
Log.i(TAG, "frames cb=$frameCallbacks ok=$emitOk err=$emitErr sinkNull=$sinkNullCount")
|
||||
}
|
||||
val image = reader.acquireLatestImage() ?: return@OnImageAvailableListener
|
||||
try {
|
||||
val sink = eventSink
|
||||
if (sink == null) {
|
||||
// 诊断：有帧但无订阅者（Dart 侧订阅未建立/被取消）
|
||||
sinkNullCount++
|
||||
errorDescription = "帧监听无订阅者"
|
||||
return@OnImageAvailableListener
|
||||
}
|
||||
if (!running) return@OnImageAvailableListener
|
||||
val now = SystemClock.elapsedRealtime()
|
||||
// 节流：推理 ~100ms 一帧，避免数 MB 帧无谓传输
|
||||
if (now - lastEmitMs < 100) return@OnImageAvailableListener
|
||||
lastEmitMs = now
|
||||
val bytes = yuvToRgba(image, rotation)
|
||||
// 缩小后再发：8.3MB/帧对主线程编码与通道传输都过重，540x960 足够
|
||||
// （推理输入 704，坐标归一化，映射不受分辨率影响）
|
||||
val scaled = downscaleToFit(
|
||||
bytes[0] as Int, bytes[1] as Int, bytes[2] as ByteArray, 960,
|
||||
)
|
||||
emitW = scaled[0] as Int
|
||||
emitH = scaled[1] as Int
|
||||
if (emitOk == 0L) Log.i(TAG, "first emit ${emitW}x${emitH}")
|
||||
// 关键：EventSink.success 内部 FlutterJNI.dispatchPlatformMessage 强制主线程
|
||||
// （ensureRunningOnMainThread 抛 RuntimeException），后台线程直接调必失败——
|
||||
// 曾因此每帧抛"必须主线程"异常、Dart 侧永远 流:0。必须 post 到主线程发送
|
||||
mainHandler.post {
|
||||
try {
|
||||
sink.success(listOf(scaled[0], scaled[1], 0, false, scaled[2]))
|
||||
emitOk++
|
||||
} catch (e: Exception) {
|
||||
emitErr++
|
||||
Log.e(TAG, "emit frame failed", e)
|
||||
errorDescription = "帧异常: ${e.javaClass.simpleName}: ${e.message}"
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
emitErr++
|
||||
Log.e(TAG, "emit frame failed", e)
|
||||
errorDescription = "帧异常: ${e.javaClass.simpleName}: ${e.message}"
|
||||
} finally {
|
||||
image.close()
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* YUV_420_888 三平面 → RGBA 单平面，同时顺时针旋转成竖屏。
|
||||
* 返回 [width, height, bytes]。整数 BT.601 转换；U/V 兼容 planar
|
||||
* （pixelStride=1）与 semi-planar（pixelStride=2，NV21 式交错）布局。
|
||||
* dst(x', y') = src(y, W-1-x')（90° 顺时针），逐像素处理 rowStride 填充。
|
||||
*/
|
||||
private fun yuvToRgba(image: Image, deg: Int): List<Any> {
|
||||
val yPlane = image.planes[0]
|
||||
val uPlane = image.planes[1]
|
||||
val vPlane = image.planes[2]
|
||||
val srcW = image.width
|
||||
val srcH = image.height
|
||||
val yStride = yPlane.rowStride
|
||||
val uStride = uPlane.rowStride
|
||||
val vStride = vPlane.rowStride
|
||||
val uPixel = uPlane.pixelStride
|
||||
val vPixel = vPlane.pixelStride
|
||||
val yBuf = yPlane.buffer
|
||||
val uBuf = uPlane.buffer
|
||||
val vBuf = vPlane.buffer
|
||||
val dstW = if (deg == 90 || deg == 270) srcH else srcW
|
||||
val dstH = if (deg == 90 || deg == 270) srcW else srcH
|
||||
val dst = ByteArray(dstW * dstH * 4)
|
||||
for (dy in 0 until dstH) {
|
||||
val rowBase = dy * dstW * 4
|
||||
for (dx in 0 until dstW) {
|
||||
val sx: Int
|
||||
val sy: Int
|
||||
when (deg) {
|
||||
90 -> {
|
||||
sx = dy
|
||||
sy = srcH - 1 - dx
|
||||
}
|
||||
180 -> {
|
||||
sx = srcW - 1 - dx
|
||||
sy = srcH - 1 - dy
|
||||
}
|
||||
270 -> {
|
||||
sx = srcW - 1 - dy
|
||||
sy = dx
|
||||
}
|
||||
else -> {
|
||||
sx = dx
|
||||
sy = dy
|
||||
}
|
||||
}
|
||||
val yy = (yBuf.get(sy * yStride + sx).toInt() and 0xFF) - 16
|
||||
val uu = (uBuf.get((sy / 2) * uStride + (sx / 2) * uPixel).toInt() and 0xFF) - 128
|
||||
val vv = (vBuf.get((sy / 2) * vStride + (sx / 2) * vPixel).toInt() and 0xFF) - 128
|
||||
val r = ((298 * yy + 409 * vv + 128) shr 8).coerceIn(0, 255)
|
||||
val g = ((298 * yy - 100 * uu - 208 * vv + 128) shr 8).coerceIn(0, 255)
|
||||
val b = ((298 * yy + 516 * uu + 128) shr 8).coerceIn(0, 255)
|
||||
val di = rowBase + dx * 4
|
||||
dst[di] = r.toByte()
|
||||
dst[di + 1] = g.toByte()
|
||||
dst[di + 2] = b.toByte()
|
||||
dst[di + 3] = 0xFF.toByte()
|
||||
}
|
||||
}
|
||||
return listOf(dstW, dstH, dst)
|
||||
}
|
||||
|
||||
/**
|
||||
* RGBA 帧整数倍缩小，长边 ≤ maxLong 时原样返回（f=1）。
|
||||
* 2x2 盒式平均（对 640x640 推理输入足够；比最近邻平滑，颜色更准）。
|
||||
*/
|
||||
private fun downscaleToFit(w: Int, h: Int, rgba: ByteArray, maxLong: Int): List<Any> {
|
||||
val long = maxOf(w, h)
|
||||
if (long <= maxLong) return listOf(w, h, rgba)
|
||||
val f = (long + maxLong - 1) / maxLong
|
||||
val dw = w / f
|
||||
val dh = h / f
|
||||
val out = ByteArray(dw * dh * 4)
|
||||
for (dy in 0 until dh) {
|
||||
val y0 = dy * f
|
||||
val y1 = minOf(y0 + f, h)
|
||||
val rows = y1 - y0
|
||||
for (dx in 0 until dw) {
|
||||
val x0 = dx * f
|
||||
val x1 = minOf(x0 + f, w)
|
||||
var r = 0L
|
||||
var g = 0L
|
||||
var b = 0L
|
||||
var a = 0L
|
||||
for (sy in y0 until y1) {
|
||||
var si = sy * w * 4 + x0 * 4
|
||||
for (sx in x0 until x1) {
|
||||
r += rgba[si].toInt() and 0xFF
|
||||
g += rgba[si + 1].toInt() and 0xFF
|
||||
b += rgba[si + 2].toInt() and 0xFF
|
||||
a += rgba[si + 3].toInt() and 0xFF
|
||||
si += 4
|
||||
}
|
||||
}
|
||||
val n = rows * (x1 - x0)
|
||||
val di = (dy * dw + dx) * 4
|
||||
out[di] = (r / n).toByte()
|
||||
out[di + 1] = (g / n).toByte()
|
||||
out[di + 2] = (b / n).toByte()
|
||||
out[di + 3] = (a / n).toByte()
|
||||
}
|
||||
}
|
||||
return listOf(dw, dh, out)
|
||||
}
|
||||
|
||||
companion object {
|
||||
private const val TAG = "CameraChannel"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package com.example.observer
|
||||
|
||||
import android.app.PendingIntent
|
||||
import android.content.Intent
|
||||
import android.content.pm.PackageInstaller
|
||||
import android.net.Uri
|
||||
import android.os.Build
|
||||
import android.os.Handler
|
||||
import android.os.Looper
|
||||
import android.provider.Settings
|
||||
import java.io.File
|
||||
import io.flutter.embedding.android.FlutterActivity
|
||||
import io.flutter.embedding.engine.FlutterEngine
|
||||
import io.flutter.plugin.common.EventChannel
|
||||
import io.flutter.plugin.common.MethodCall
|
||||
import io.flutter.plugin.common.MethodChannel
|
||||
|
||||
/**
|
||||
* 原生 APK 安装通道（App 内更新安装）：PackageInstaller 会话安装，
|
||||
* 安装进度经 EventChannel 实时回传 Flutter（下载进度由 Flutter 侧 http 流式下载自算）。
|
||||
*
|
||||
* 通道：
|
||||
* - MethodChannel "observer/installer"：install(path)
|
||||
* - result.success("installing")：已提交安装
|
||||
* - result.success("permission_required")：未开「安装未知应用」，已拉起系统设置页
|
||||
* - EventChannel "observer/installer/progress"：{event: progress/finished/failed, ...}
|
||||
*/
|
||||
class InstallerChannel(
|
||||
private val activity: FlutterActivity,
|
||||
private val engine: FlutterEngine,
|
||||
) {
|
||||
private val messenger = engine.dartExecutor.binaryMessenger
|
||||
private val method = MethodChannel(messenger, "observer/installer")
|
||||
private val progress = EventChannel(messenger, "observer/installer/progress")
|
||||
|
||||
private var progressSink: EventChannel.EventSink? = null
|
||||
private var activeSession: PackageInstaller.Session? = null
|
||||
|
||||
fun register() {
|
||||
method.setMethodCallHandler { call: MethodCall, result: MethodChannel.Result ->
|
||||
when (call.method) {
|
||||
"install" -> install(call.argument<String>("path") ?: "", result)
|
||||
else -> result.notImplemented()
|
||||
}
|
||||
}
|
||||
progress.setStreamHandler(object : EventChannel.StreamHandler {
|
||||
override fun onListen(arguments: Any?, events: EventChannel.EventSink) {
|
||||
progressSink = events
|
||||
}
|
||||
|
||||
override fun onCancel(arguments: Any?) {
|
||||
progressSink = null
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
private fun install(path: String, result: MethodChannel.Result) {
|
||||
val file = File(path)
|
||||
if (!file.exists()) {
|
||||
result.error("FILE_NOT_FOUND", "APK 文件不存在: $path", null)
|
||||
return
|
||||
}
|
||||
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O &&
|
||||
!activity.packageManager.canRequestPackageInstalls()
|
||||
) {
|
||||
val intent = Intent(
|
||||
Settings.ACTION_MANAGE_UNKNOWN_APP_SOURCES,
|
||||
Uri.parse("package:${activity.packageName}"),
|
||||
).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
|
||||
activity.startActivity(intent)
|
||||
result.success("permission_required")
|
||||
return
|
||||
}
|
||||
try {
|
||||
val pm = activity.packageManager
|
||||
val params = PackageInstaller.SessionParams(PackageInstaller.SessionParams.MODE_FULL_INSTALL).apply {
|
||||
setSize(file.length())
|
||||
}
|
||||
val sessionId = pm.packageInstaller.createSession(params)
|
||||
val session = pm.packageInstaller.openSession(sessionId)
|
||||
activeSession = session
|
||||
// API 36 起 registerSessionCallback(int, ...) 变体被移除，只剩全局注册形式，
|
||||
// 回调按 sessionId 过滤，避免响应其他会话事件
|
||||
val callback = object : PackageInstaller.SessionCallback() {
|
||||
override fun onCreated(id: Int) {}
|
||||
|
||||
override fun onBadgingChanged(id: Int) {}
|
||||
|
||||
override fun onActiveChanged(id: Int, active: Boolean) {}
|
||||
|
||||
override fun onProgressChanged(id: Int, progressPercent: Float) {
|
||||
if (id != sessionId) return
|
||||
emit("progress", "progress" to progressPercent.toInt())
|
||||
}
|
||||
|
||||
override fun onFinished(id: Int, success: Boolean) {
|
||||
if (id != sessionId) return
|
||||
emit("finished", "success" to success)
|
||||
pm.packageInstaller.unregisterSessionCallback(this)
|
||||
activeSession = null
|
||||
}
|
||||
}
|
||||
pm.packageInstaller.registerSessionCallback(callback, Handler(Looper.getMainLooper()))
|
||||
// 写 APK 到会话：1MB 缓冲流式拷贝，完成后 commit 弹系统确认框
|
||||
Thread {
|
||||
try {
|
||||
session.openWrite("apk", 0, file.length()).use { out ->
|
||||
file.inputStream().use { input ->
|
||||
val buf = ByteArray(1 shl 20)
|
||||
while (true) {
|
||||
val n = input.read(buf)
|
||||
if (n < 0) break
|
||||
out.write(buf, 0, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
val sender = PendingIntent.getActivity(
|
||||
activity,
|
||||
0,
|
||||
Intent(activity, MainActivity::class.java),
|
||||
PendingIntent.FLAG_UPDATE_CURRENT or PendingIntent.FLAG_IMMUTABLE,
|
||||
).intentSender
|
||||
session.commit(sender)
|
||||
} catch (e: Exception) {
|
||||
session.abandon()
|
||||
emit("failed", "error" to (e.message ?: "安装会话写入失败"))
|
||||
activeSession = null
|
||||
}
|
||||
}.start()
|
||||
result.success("installing")
|
||||
} catch (e: Exception) {
|
||||
result.error("INSTALL_FAILED", e.message, null)
|
||||
}
|
||||
}
|
||||
|
||||
private fun emit(event: String, vararg pairs: Pair<String, Any>) {
|
||||
progressSink?.success(mapOf("event" to event, *pairs))
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,25 @@
|
||||
package com.example.observer
|
||||
|
||||
import io.flutter.embedding.android.FlutterActivity
|
||||
import io.flutter.embedding.engine.FlutterEngine
|
||||
|
||||
class MainActivity : FlutterActivity()
|
||||
class MainActivity : FlutterActivity() {
|
||||
private var cameraChannel: CameraChannel? = null
|
||||
private var installerChannel: InstallerChannel? = null
|
||||
|
||||
override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
|
||||
super.configureFlutterEngine(flutterEngine)
|
||||
// 自写原生相机（替代 camera_android_camerax 插件）：
|
||||
// 分析帧在原生侧旋转成竖屏后回传，Flutter 侧恒 rotation=0（对齐 iOS）。
|
||||
// configureFlutterEngine（onCreate 阶段）只注册通道与 viewFactory，
|
||||
// 实际 bindToLifecycle 由 Flutter 相机页 start 时触发（此时已 RESUMED）
|
||||
cameraChannel = CameraChannel(this, flutterEngine).also { it.register() }
|
||||
// App 内更新安装 APK（PackageInstaller 会话安装 + 进度回传）
|
||||
installerChannel = InstallerChannel(this, flutterEngine).also { it.register() }
|
||||
}
|
||||
|
||||
override fun onDestroy() {
|
||||
cameraChannel?.destroy()
|
||||
super.onDestroy()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -101,3 +101,21 @@
|
||||
- `priceCents` 为整数分，客户端展示 ÷100 转元
|
||||
- 展示名由客户端按 `days` 派生「N天」，接口无 label 字段
|
||||
- 后端套餐来自 `config.yml` `plans` 节点（静态定价，改价改配置重启生效）
|
||||
|
||||
## 7. 版本更新检查
|
||||
|
||||
`GET /api/v1/app/update`（**公开接口，无需登录**）在 App 启动时调用（协议层见 `lib/update/update_checker.dart`）。
|
||||
|
||||
响应 `data`（无记录时字段为空串，视为无需更新）：
|
||||
|
||||
```json
|
||||
{
|
||||
"version": "1.1.0",
|
||||
"notes": "修复识别准确率问题"
|
||||
}
|
||||
```
|
||||
|
||||
- **仅 Android 调用**：`UpdateChecker.fetch()` 在非 Android 平台直接返回空（iOS 不做版本下发，用户从 App Store 自行更新）
|
||||
- 客户端以「语义化版本号」按数字段比较（`1.10.0 > 1.9.9`），**服务器版本 > 已装版本即强制更新**：弹全屏阻塞页（禁返回，仅「立即更新」），**App 内直接下载安装**（不跳浏览器）：http 流式下载固定地址 `{apiBaseUrl}/download/observer-latest.apk`（后端静态托管，永远是最新 APK）到缓存目录（`.part` 原子落盘后改名，避免半包）→ 原生 `PackageInstaller` 会话安装（`InstallerChannel.kt`），页面实时显示**下载进度与安装进度**；首次安装需在系统设置允许「安装未知应用」（未开启时自动拉起系统设置页，APK 已缓存、返回后再次点击直达安装）
|
||||
- **双版本比较防反复提示**：用户点「立即更新」时把服务器版本号写入本地（`SessionStore.accepted_update_version`）；判定条件是服务器版本 > 已装版本 **或** 服务器版本 > 已接受版本。APK 版本号不递增（每次打的包 versionName 相同）时，已更新完成的手机重启也不会再次弹更新
|
||||
- 网络异常/响应异常时静默跳过检查，不阻塞启动
|
||||
|
||||
@@ -57,11 +57,9 @@
|
||||
<dict>
|
||||
<key>NSAllowsArbitraryLoads</key>
|
||||
<true/>
|
||||
<key>NSAllowsArbitraryLoadsInWebContent</key>
|
||||
<true/>
|
||||
</dict>
|
||||
<key>NSCameraUsageDescription</key>
|
||||
<string>需要使用相机进行野生动物实时识别</string>
|
||||
<string>需要使用相机进行动物实时识别</string>
|
||||
<key>NSLocalNetworkUsageDescription</key>
|
||||
<string>需要通过本地网络连接服务器进行账号验证和支付</string>
|
||||
<key>UIApplicationSceneManifest</key>
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:package_info_plus/package_info_plus.dart';
|
||||
import 'package:provider/provider.dart';
|
||||
|
||||
import 'auth/auth_screen.dart';
|
||||
@@ -7,7 +8,10 @@ import 'camera/camera_screen.dart';
|
||||
import 'container.dart';
|
||||
import 'home/home_screen.dart';
|
||||
import 'legal/terms_screen.dart';
|
||||
import 'models/model_manager.dart';
|
||||
import 'payment/paywall_screen.dart';
|
||||
import 'update/update_checker.dart';
|
||||
import 'update/update_screen.dart';
|
||||
|
||||
class ObserverApp extends StatelessWidget {
|
||||
final AppContainer container;
|
||||
@@ -66,9 +70,33 @@ class _StartupGateState extends State<StartupGate> {
|
||||
Navigator.of(context).pushReplacementNamed('/terms');
|
||||
return;
|
||||
}
|
||||
// 版本更新检查（公开接口，无需登录态；仅 Android）：服务器版本高于
|
||||
// 「已装版本与已确认接受版本」中的较大者即强制更新，弹全屏阻塞页。
|
||||
// APK 版本号不递增时，点过「立即更新」的 accepted 版本参与比较，
|
||||
// 更新完成后再次启动不反复提示。
|
||||
final info = await UpdateChecker().fetch();
|
||||
final current = await PackageInfo.fromPlatform();
|
||||
if (!mounted) return;
|
||||
final session = context.read<SessionStore>();
|
||||
final acceptedUpdate = await session.readAcceptedUpdateVersion();
|
||||
if (!mounted) return;
|
||||
if (UpdateChecker.needsUpdate(info.version, current.version, acceptedUpdate)) {
|
||||
Navigator.of(context).pushReplacement(MaterialPageRoute(
|
||||
builder: (_) => UpdateScreen(
|
||||
version: info.version,
|
||||
url: UpdateChecker.downloadUrl(),
|
||||
notes: info.notes,
|
||||
onUpdateAccepted: () =>
|
||||
session.saveAcceptedUpdateVersion(info.version),
|
||||
),
|
||||
));
|
||||
return;
|
||||
}
|
||||
final token = await session.readToken();
|
||||
if (!mounted) return;
|
||||
// 模型热更新：后台拉取模型目录并下载/更新各数据集模型（不阻塞启动，
|
||||
// 相机页打开前若未就绪会兜底等待；下载失败回退内置资产模型）
|
||||
ModelManager.instance.refresh();
|
||||
Navigator.of(context)
|
||||
.pushReplacementNamed(token == null ? '/login' : '/home');
|
||||
}
|
||||
|
||||
@@ -67,7 +67,7 @@ class _AuthScreenState extends State<AuthScreen> {
|
||||
),
|
||||
const SizedBox(height: 4),
|
||||
Text(
|
||||
'野生动物实时识别',
|
||||
'动物实时识别',
|
||||
textAlign: TextAlign.center,
|
||||
style: TextStyle(color: Colors.grey.shade600),
|
||||
),
|
||||
|
||||
@@ -2,13 +2,17 @@ import 'package:flutter_secure_storage/flutter_secure_storage.dart';
|
||||
|
||||
/// 登录会话持久化：token + 手机号存 secure storage。
|
||||
/// 启动时读取判断是否已登录；登出/401 时清除回登录页。
|
||||
/// 另存「已确认更新版本」：用户点过「立即更新」后记录服务器版本号，
|
||||
/// 与 APK 内 versionName 取较大者参与更新判断（APK 版本号不递增也不会反复提示）。
|
||||
class SessionStore {
|
||||
static const _storage = FlutterSecureStorage();
|
||||
static const _tokenKey = 'auth_token';
|
||||
static const _phoneKey = 'auth_phone';
|
||||
static const _acceptedUpdateKey = 'accepted_update_version';
|
||||
|
||||
static String? _cachedToken;
|
||||
static String? _cachedPhone;
|
||||
static String? _cachedAcceptedUpdate;
|
||||
|
||||
Future<String?> readToken() async {
|
||||
if (_cachedToken != null) return _cachedToken;
|
||||
@@ -20,6 +24,18 @@ class SessionStore {
|
||||
return _cachedPhone = await _storage.read(key: _phoneKey);
|
||||
}
|
||||
|
||||
/// 用户点过「立即更新」的服务器版本号（空串 = 从未接受过更新提示）
|
||||
Future<String> readAcceptedUpdateVersion() async {
|
||||
if (_cachedAcceptedUpdate != null) return _cachedAcceptedUpdate!;
|
||||
return _cachedAcceptedUpdate =
|
||||
(await _storage.read(key: _acceptedUpdateKey)) ?? '';
|
||||
}
|
||||
|
||||
Future<void> saveAcceptedUpdateVersion(String version) async {
|
||||
_cachedAcceptedUpdate = version;
|
||||
await _storage.write(key: _acceptedUpdateKey, value: version);
|
||||
}
|
||||
|
||||
Future<void> save(String phone, String token) async {
|
||||
_cachedPhone = phone;
|
||||
_cachedToken = token;
|
||||
|
||||
@@ -1,61 +1,257 @@
|
||||
import 'dart:async';
|
||||
|
||||
import 'package:camera/camera.dart';
|
||||
import 'package:flutter/foundation.dart';
|
||||
import 'package:flutter/services.dart';
|
||||
import 'package:flutter/widgets.dart';
|
||||
|
||||
import 'frame_analyzer.dart';
|
||||
|
||||
/// camera 插件封装：后摄图像流（对应 Kotlin CameraController）。
|
||||
class AppCameraController {
|
||||
/// 相机抽象（对应 Kotlin CameraController）：
|
||||
///
|
||||
/// - Android：自写原生通道（[NativeCameraController]）。分析帧在 Kotlin 侧
|
||||
/// 旋转成竖屏后经 EventChannel 回传，Flutter 侧恒 rotation=0——与 iOS
|
||||
/// （camera_avfoundation 插件，帧本来就是竖屏方向）行为一致，消除
|
||||
/// "横屏传感器帧 → 90° 旋转 + FIT_COVER crop 映射"的标注偏移根因。
|
||||
/// - iOS：camera 插件（原逻辑）。帧已竖屏，rotation 恒 0。
|
||||
abstract class AppCameraController {
|
||||
static Future<AppCameraController?> create() async {
|
||||
if (defaultTargetPlatform == TargetPlatform.android) {
|
||||
return NativeCameraController();
|
||||
}
|
||||
return PluginCameraController.create();
|
||||
}
|
||||
|
||||
/// 分析流回调实际触发次数（诊断用，与 analyzer 帧计数区分）
|
||||
int get streamCallbacks;
|
||||
|
||||
bool get isInitialized;
|
||||
|
||||
bool get isStreaming;
|
||||
|
||||
String? get errorDescription;
|
||||
|
||||
/// 检测框 overlay 应使用的旋转角。两平台帧都已竖屏 → 恒 0。
|
||||
int get rotationDegrees => 0;
|
||||
|
||||
/// 诊断：传感器方向 / 显示旋转（仅 Android 原生通道上报）
|
||||
int? get sensorOrientation => null;
|
||||
|
||||
int? get displayDegrees => null;
|
||||
|
||||
/// 预览实际应用的旋转圈数（仅 Android 原生通道上报，诊断用）
|
||||
int get quarterTurns => -1;
|
||||
|
||||
/// 诊断：轮询原生侧帧状态（Android 返回计数；iOS 返回空）
|
||||
Future<Map<dynamic, dynamic>> stats() async => const {};
|
||||
|
||||
Future<void> start(FrameAnalyzer analyzer);
|
||||
|
||||
Future<void> stop();
|
||||
|
||||
Future<double> getMinZoomLevel();
|
||||
|
||||
Future<double> getMaxZoomLevel();
|
||||
|
||||
Future<void> setZoomLevel(double value);
|
||||
|
||||
/// 预览 widget：Android 为原生 SurfaceView（AndroidView），iOS 为插件纹理
|
||||
Widget buildPreview();
|
||||
}
|
||||
|
||||
/// Android：自写原生相机通道（Kotlin CameraChannel）。
|
||||
class NativeCameraController extends AppCameraController {
|
||||
static const MethodChannel _channel = MethodChannel('observer/camera');
|
||||
static const EventChannel _frames = EventChannel('observer/camera/frames');
|
||||
|
||||
StreamSubscription<dynamic>? _sub;
|
||||
bool _streaming = false;
|
||||
String? _error;
|
||||
|
||||
/// 原生侧注册的预览纹理（SurfaceTexture）
|
||||
int? _textureId;
|
||||
int _textureW = 1920;
|
||||
int _textureH = 1080;
|
||||
int _quarterTurns = 1;
|
||||
|
||||
@override
|
||||
int streamCallbacks = 0;
|
||||
|
||||
int? _sensorOrientation;
|
||||
int? _displayDegrees;
|
||||
|
||||
@override
|
||||
int? get sensorOrientation => _sensorOrientation;
|
||||
|
||||
@override
|
||||
int? get displayDegrees => _displayDegrees;
|
||||
|
||||
@override
|
||||
int get quarterTurns => _quarterTurns;
|
||||
|
||||
@override
|
||||
bool get isInitialized => _streaming;
|
||||
|
||||
@override
|
||||
bool get isStreaming => _streaming;
|
||||
|
||||
@override
|
||||
String? get errorDescription => _error;
|
||||
|
||||
@override
|
||||
Future<Map<dynamic, dynamic>> stats() async {
|
||||
try {
|
||||
final r = await _channel.invokeMethod<Map<dynamic, dynamic>>('stats');
|
||||
if (r != null) {
|
||||
// 旋转/显示角度随轮询实时刷新：手机旋转后预览与帧旋转都跟着变
|
||||
final dd = r['displayDegrees'];
|
||||
if (dd is int) _displayDegrees = dd;
|
||||
final turns = r['quarterTurns'];
|
||||
if (turns is int) _quarterTurns = turns;
|
||||
}
|
||||
return r ?? const {};
|
||||
} catch (e) {
|
||||
return {'pollErr': '$e'};
|
||||
}
|
||||
}
|
||||
|
||||
@override
|
||||
Future<void> start(FrameAnalyzer analyzer) async {
|
||||
await stop();
|
||||
// 先订阅再启动：原生 start 绑定后立刻推帧，避免首帧竞态
|
||||
_sub = _frames.receiveBroadcastStream().listen((event) {
|
||||
// 计数放最前：任何到达的事件都先记账，后续解析失败也不丢计数
|
||||
streamCallbacks++;
|
||||
try {
|
||||
final f = event as List;
|
||||
final w = f[0] as int;
|
||||
final h = f[1] as int;
|
||||
final rotation = f[2] as int;
|
||||
final bgra = f[3] as bool;
|
||||
final bytes = f[4] as Uint8List;
|
||||
analyzer.analyzeRaw(
|
||||
planes: [bytes],
|
||||
strides: [w * 4],
|
||||
width: w,
|
||||
height: h,
|
||||
isBgra: true,
|
||||
rgbaOrder: !bgra,
|
||||
rotationDegrees: rotation,
|
||||
);
|
||||
} catch (e) {
|
||||
_error = '帧解析: $e';
|
||||
analyzer.recordStreamError('frames parse: $e');
|
||||
}
|
||||
}, onError: (Object e) {
|
||||
_error = '$e';
|
||||
analyzer.recordStreamError('frames: $e');
|
||||
});
|
||||
analyzer.reset();
|
||||
analyzer.worker?.reset();
|
||||
try {
|
||||
final r = await _channel.invokeMethod<Map<dynamic, dynamic>>('start');
|
||||
_textureId = r?['textureId'] as int?;
|
||||
_textureW = (r?['w'] as num?)?.toInt() ?? _textureW;
|
||||
_textureH = (r?['h'] as num?)?.toInt() ?? _textureH;
|
||||
_quarterTurns = (r?['quarterTurns'] as num?)?.toInt() ?? 1;
|
||||
_sensorOrientation = (r?['sensorOrientation'] as num?)?.toInt();
|
||||
_displayDegrees = (r?['displayDegrees'] as num?)?.toInt();
|
||||
} catch (e) {
|
||||
_error = '$e';
|
||||
analyzer.recordStreamError('camera start: $e');
|
||||
rethrow;
|
||||
}
|
||||
_streaming = true;
|
||||
}
|
||||
|
||||
@override
|
||||
Future<void> stop() async {
|
||||
_streaming = false;
|
||||
await _sub?.cancel();
|
||||
_sub = null;
|
||||
try {
|
||||
await _channel.invokeMethod('stop');
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
@override
|
||||
Future<double> getMinZoomLevel() async {
|
||||
final r = await _channel.invokeMethod<List<dynamic>>('getZoomRange');
|
||||
return (r != null && r.isNotEmpty ? (r[0] as num).toDouble() : 1.0);
|
||||
}
|
||||
|
||||
@override
|
||||
Future<double> getMaxZoomLevel() async {
|
||||
final r = await _channel.invokeMethod<List<dynamic>>('getZoomRange');
|
||||
return (r != null && r.length > 1 ? (r[1] as num).toDouble() : 1.0);
|
||||
}
|
||||
|
||||
@override
|
||||
Future<void> setZoomLevel(double value) async {
|
||||
try {
|
||||
await _channel.invokeMethod('setZoom', value);
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
/// 预览纹理：Flutter 引擎渲染 Texture 时已自动应用 SurfaceTexture 变换矩阵
|
||||
/// （传感器方向补偿，内容已转成自然方向的竖屏），因此：
|
||||
/// 1. 区域声明旋转后的尺寸（宽高互换）——否则横屏区域会把竖屏内容横向拉伸
|
||||
/// 2. RotatedBox 只按显示旋转补偿（quarterTurns = 屏转/90，竖屏 0 / 横屏 1）
|
||||
/// 再 FittedBox cover（= CoordinateMapper 的 FIT_COVER 数学一致）填满全屏。
|
||||
/// 不用 AndroidView+SurfaceView——平台视图会盖住 Flutter UI（诊断行/设置按钮/overlay）
|
||||
@override
|
||||
Widget buildPreview() {
|
||||
final id = _textureId;
|
||||
if (id == null) return const SizedBox.shrink();
|
||||
return FittedBox(
|
||||
fit: BoxFit.cover,
|
||||
child: RotatedBox(
|
||||
quarterTurns: _quarterTurns % 4,
|
||||
child: SizedBox(
|
||||
width: _textureH.toDouble(),
|
||||
height: _textureW.toDouble(),
|
||||
child: Texture(textureId: id),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// iOS：camera 插件封装（原实现）。
|
||||
class PluginCameraController extends AppCameraController {
|
||||
final List<CameraDescription> cameras;
|
||||
CameraController? controller;
|
||||
|
||||
/// 图像流回调实际触发次数（诊断用，与 analyzer 帧计数区分）
|
||||
@override
|
||||
int streamCallbacks = 0;
|
||||
|
||||
AppCameraController._(this.cameras);
|
||||
PluginCameraController._(this.cameras);
|
||||
|
||||
static Future<AppCameraController?> create() async {
|
||||
static Future<PluginCameraController?> create() async {
|
||||
final cameras = await availableCameras();
|
||||
if (cameras.isEmpty) return null;
|
||||
return AppCameraController._(cameras);
|
||||
return PluginCameraController._(cameras);
|
||||
}
|
||||
|
||||
@override
|
||||
bool get isInitialized => controller?.value.isInitialized ?? false;
|
||||
|
||||
CameraController get currentController =>
|
||||
controller ?? (throw StateError('camera not initialized'));
|
||||
@override
|
||||
bool get isStreaming => controller?.value.isStreamingImages ?? false;
|
||||
|
||||
/// 图像流送达时的旋转角（传感器 → 竖屏显示所需的顺时针旋转）。
|
||||
/// 与 CameraX rotationDegrees 同公式；预览本身由平台旋转，检测框 overlay
|
||||
/// 用同一角度映射即可对齐。
|
||||
int get rotationDegrees {
|
||||
final c = controller;
|
||||
if (c == null) return 0;
|
||||
final deviceDegrees = switch (c.value.deviceOrientation) {
|
||||
DeviceOrientation.portraitUp => 0,
|
||||
DeviceOrientation.landscapeLeft => 90,
|
||||
DeviceOrientation.portraitDown => 180,
|
||||
DeviceOrientation.landscapeRight => 270,
|
||||
};
|
||||
final sensor = c.description.sensorOrientation;
|
||||
final isFront =
|
||||
c.description.lensDirection == CameraLensDirection.front;
|
||||
final degrees = (isFront ? sensor + deviceDegrees : sensor - deviceDegrees) % 360;
|
||||
return degrees < 0 ? degrees + 360 : degrees;
|
||||
}
|
||||
@override
|
||||
String? get errorDescription => controller?.value.errorDescription;
|
||||
|
||||
@override
|
||||
Future<void> start(FrameAnalyzer analyzer) async {
|
||||
await stop();
|
||||
final desc = cameras.firstWhere(
|
||||
(c) => c.lensDirection == CameraLensDirection.back,
|
||||
orElse: () => cameras.first);
|
||||
// iOS 用默认 bgra8888（420v 在部分 iOS 版本上视频输出静默不送帧），
|
||||
// Android 用 yuv420 多平面。
|
||||
final fmt = defaultTargetPlatform == TargetPlatform.iOS
|
||||
? ImageFormatGroup.bgra8888
|
||||
: ImageFormatGroup.yuv420;
|
||||
final c = CameraController(desc, ResolutionPreset.high,
|
||||
enableAudio: false, imageFormatGroup: fmt);
|
||||
// iOS image stream 帧已按竖屏方向输出（无需旋转），
|
||||
// 与 Android 原生通道（帧原生侧旋转成竖屏）统一 rotation=0
|
||||
final c = CameraController(desc, ResolutionPreset.veryHigh,
|
||||
enableAudio: false, imageFormatGroup: ImageFormatGroup.bgra8888);
|
||||
controller = c;
|
||||
await c.initialize();
|
||||
// 相机（重新）启动后重置运动/背景参考与抽帧节流，避免旧场景残留
|
||||
@@ -66,7 +262,7 @@ class AppCameraController {
|
||||
await c.startImageStream((image) {
|
||||
streamCallbacks++;
|
||||
try {
|
||||
analyzer.analyze(image, rotationDegrees);
|
||||
analyzer.analyze(image, 0);
|
||||
} catch (e, st) {
|
||||
debugPrint('[camera] analyze error: $e\n$st');
|
||||
analyzer.recordStreamError('analyze: $e');
|
||||
@@ -80,6 +276,7 @@ class AppCameraController {
|
||||
}
|
||||
}
|
||||
|
||||
@override
|
||||
Future<void> stop() async {
|
||||
final c = controller;
|
||||
if (c == null) return;
|
||||
@@ -89,4 +286,16 @@ class AppCameraController {
|
||||
} catch (_) {}
|
||||
await c.dispose();
|
||||
}
|
||||
|
||||
@override
|
||||
Future<double> getMinZoomLevel() => controller!.getMinZoomLevel();
|
||||
|
||||
@override
|
||||
Future<double> getMaxZoomLevel() => controller!.getMaxZoomLevel();
|
||||
|
||||
@override
|
||||
Future<void> setZoomLevel(double value) => controller!.setZoomLevel(value);
|
||||
|
||||
@override
|
||||
Widget buildPreview() => CameraPreview(controller!);
|
||||
}
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
import 'dart:ui' show PlatformDispatcher;
|
||||
import 'dart:async';
|
||||
import 'dart:ui' as ui show PlatformDispatcher;
|
||||
|
||||
import 'package:camera/camera.dart';
|
||||
import 'package:flutter/foundation.dart' show defaultTargetPlatform;
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:permission_handler/permission_handler.dart';
|
||||
import 'package:wakelock_plus/wakelock_plus.dart';
|
||||
|
||||
import '../detection/detector_worker.dart';
|
||||
import '../models/model_manager.dart';
|
||||
import '../reminder/reminder.dart';
|
||||
import 'app_camera_controller.dart';
|
||||
import 'camera_view_model.dart';
|
||||
@@ -30,17 +30,88 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
String? _globalError;
|
||||
String? _initError;
|
||||
|
||||
/// 置信度阈值（设置页滑块调整，worker 内实时生效）
|
||||
double _minScore = 0.10;
|
||||
|
||||
/// 原生侧帧状态轮询结果（诊断用；无帧时诊断行也能实时刷新）
|
||||
Map<dynamic, dynamic> _nativeStats = const {};
|
||||
Timer? _statsTimer;
|
||||
|
||||
void _openSettings() {
|
||||
final vm = _viewModel;
|
||||
if (vm == null) return;
|
||||
showModalBottomSheet<void>(
|
||||
context: context,
|
||||
backgroundColor: Colors.black87,
|
||||
builder: (ctx) => StatefulBuilder(
|
||||
builder: (ctx, setSheetState) => Padding(
|
||||
padding: const EdgeInsets.all(20),
|
||||
child: Column(
|
||||
mainAxisSize: MainAxisSize.min,
|
||||
crossAxisAlignment: CrossAxisAlignment.start,
|
||||
children: [
|
||||
const Text('识别设置',
|
||||
style: TextStyle(
|
||||
color: Colors.white, fontSize: 16, fontWeight: FontWeight.bold)),
|
||||
const SizedBox(height: 12),
|
||||
Row(
|
||||
children: [
|
||||
const Text('置信度阈值',
|
||||
style: TextStyle(color: Colors.white70, fontSize: 14)),
|
||||
const Spacer(),
|
||||
Text('${(_minScore * 100).toStringAsFixed(0)}%',
|
||||
style: const TextStyle(
|
||||
color: Colors.greenAccent,
|
||||
fontSize: 14,
|
||||
fontWeight: FontWeight.bold)),
|
||||
],
|
||||
),
|
||||
Slider(
|
||||
value: _minScore,
|
||||
min: 0.05,
|
||||
max: 0.50,
|
||||
divisions: 45,
|
||||
activeColor: Colors.greenAccent,
|
||||
onChanged: (v) {
|
||||
setSheetState(() => _minScore = v);
|
||||
_analyzer?.worker?.setMinScore(v);
|
||||
},
|
||||
),
|
||||
const SizedBox(height: 8),
|
||||
const Text(
|
||||
'阈值越低识别越灵敏（低分框越多，误报也可能增加）；'
|
||||
'野鸡模型置信度普遍在 10%~20%，场景识别不到时可适当调低。',
|
||||
style: TextStyle(color: Colors.white54, fontSize: 12),
|
||||
),
|
||||
],
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
@override
|
||||
void initState() {
|
||||
super.initState();
|
||||
final oldPlatform = PlatformDispatcher.instance.onError;
|
||||
PlatformDispatcher.instance.onError = (error, stack) {
|
||||
setState(() => _globalError = 'Platform: $error');
|
||||
final oldPlatform = ui.PlatformDispatcher.instance.onError;
|
||||
ui.PlatformDispatcher.instance.onError = (error, stack) {
|
||||
setState(() => _globalError =
|
||||
'Platform: $error\n${stack.toString().split('\n').take(3).join('\n')}');
|
||||
return oldPlatform?.call(error, stack) ?? false;
|
||||
};
|
||||
WidgetsBinding.instance.addPostFrameCallback((_) => _init());
|
||||
// 相机页常亮：野外观察时保持屏幕不熄（离开页面时关闭）
|
||||
WakelockPlus.enable();
|
||||
// 每秒轮询原生侧帧状态：无帧时诊断行也能实时刷新（camErr/计数）
|
||||
_statsTimer = Timer.periodic(const Duration(seconds: 1), (_) => _pollStats());
|
||||
}
|
||||
|
||||
Future<void> _pollStats() async {
|
||||
final camera = _cameraController;
|
||||
if (camera == null) return;
|
||||
final s = await camera.stats();
|
||||
if (!mounted) return;
|
||||
setState(() => _nativeStats = s);
|
||||
}
|
||||
|
||||
Future<void> _init() async {
|
||||
@@ -49,8 +120,18 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
setState(() => _permissionGranted = granted);
|
||||
if (!granted) return;
|
||||
|
||||
// 模型热更新：优先使用已下载的数据集模型（启动时后台拉取；此处兜底等待，
|
||||
// 下载慢/失败不阻塞相机启动——无下载模型时 worker 回退内置资产）
|
||||
if (!ModelManager.instance.ready) {
|
||||
try {
|
||||
await ModelManager.instance
|
||||
.refresh()
|
||||
.timeout(const Duration(seconds: 15));
|
||||
} catch (_) {}
|
||||
}
|
||||
// 模型加载/推理在后台 isolate，不阻塞 UI；worker 为 null 时仅预览并提示
|
||||
final worker = await DetectorWorker.create();
|
||||
final worker = await DetectorWorker.create(
|
||||
models: ModelManager.instance.models);
|
||||
final viewModel = CameraViewModel(reminder: Reminder());
|
||||
viewModel.setModelReady(worker != null);
|
||||
final analyzer = FrameAnalyzer(worker: worker, viewModel: viewModel);
|
||||
@@ -100,6 +181,7 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
|
||||
@override
|
||||
void dispose() {
|
||||
_statsTimer?.cancel();
|
||||
WakelockPlus.disable();
|
||||
_cameraController?.stop();
|
||||
_analyzer?.dispose();
|
||||
@@ -120,29 +202,24 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
if (!_permissionGranted)
|
||||
_PermissionGuide(onRequest: () => _init())
|
||||
else if (vm != null && (camera?.isInitialized ?? false))
|
||||
// 预览 + 检测框同几何：overlay 作为 CameraPreview 的 child，
|
||||
// 与纹理共享同一 Stack/尺寸，避免比例或裁剪导致的位置偏移
|
||||
// 预览 + 检测框同几何：overlay 作为预览 widget 的 sibling，
|
||||
// 与纹理共享同一 Stack/尺寸，避免比例或裁剪导致的位置偏移。
|
||||
// 两平台分析帧都已竖屏（Android 原生侧旋转 / iOS 插件本来就竖屏），
|
||||
// rotation 恒 0，CoordinateMapper 走纯 FIT_COVER 缩放路径
|
||||
ListenableBuilder(
|
||||
listenable: vm,
|
||||
builder: (context, _) => _ZoomablePreview(
|
||||
controller: camera!.currentController,
|
||||
imageWidthPx: vm.state.imageWidthPx,
|
||||
imageHeightPx: vm.state.imageHeightPx,
|
||||
controller: camera!,
|
||||
overlay: DetectionOverlay(
|
||||
results: vm.state.results,
|
||||
// iOS 纹理不旋转显示（_wrapInRotatedBox 仅 Android），
|
||||
// 显示方向 = buffer 原样 = 检测方向，旋转必须为 0；
|
||||
// Android 纹理被 RotatedBox 旋转，需用插件报告的 rotation。
|
||||
rotation: defaultTargetPlatform == TargetPlatform.iOS
|
||||
? 0
|
||||
: vm.state.rotation,
|
||||
rotation: 0,
|
||||
imageWidthPx: vm.state.imageWidthPx,
|
||||
imageHeightPx: vm.state.imageHeightPx,
|
||||
),
|
||||
),
|
||||
)
|
||||
else if (camera?.isInitialized ?? false)
|
||||
_ZoomablePreview(controller: camera!.currentController)
|
||||
_ZoomablePreview(controller: camera!)
|
||||
else
|
||||
const Center(
|
||||
child: Text('相机启动中…', style: TextStyle(color: Colors.white70)),
|
||||
@@ -197,6 +274,7 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
right: 0,
|
||||
child: _CameraTopBar(
|
||||
onClose: () => Navigator.of(context).pop(),
|
||||
onOpenSettings: _openSettings,
|
||||
),
|
||||
),
|
||||
],
|
||||
@@ -209,6 +287,27 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
return Stack(
|
||||
fit: StackFit.expand,
|
||||
children: [
|
||||
// 模型热更新下载失败提示（已加载模型仍可用，仅提示补更新）
|
||||
if (vm.state.modelReady && ModelManager.instance.error != null)
|
||||
Positioned(
|
||||
left: 16,
|
||||
right: 16,
|
||||
top: MediaQuery.of(context).padding.top + 56,
|
||||
child: Container(
|
||||
padding:
|
||||
const EdgeInsets.symmetric(horizontal: 12, vertical: 6),
|
||||
decoration: BoxDecoration(
|
||||
color: Colors.black54,
|
||||
borderRadius: BorderRadius.circular(8),
|
||||
),
|
||||
child: Text(
|
||||
ModelManager.instance.error!,
|
||||
textAlign: TextAlign.center,
|
||||
style: const TextStyle(color: Colors.orange, fontSize: 12),
|
||||
),
|
||||
),
|
||||
),
|
||||
|
||||
// 模型未加载时仅显示相机预览，不做检测标注（横幅置于顶栏下方，避免与底部诊断行重叠）
|
||||
if (!vm.state.modelReady)
|
||||
Positioned(
|
||||
@@ -238,13 +337,25 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
crossAxisAlignment: CrossAxisAlignment.center,
|
||||
children: [
|
||||
Text(
|
||||
'模型:${vm.state.modelReady ? '已加载' : '未加载'} 帧:${vm.state.framesReceived} 流:${camera?.streamCallbacks ?? 0} 推理:${vm.state.debugDetectCalls}次 异常:${vm.state.debugDetectErrors}次 处理:${vm.state.debugLastMs}ms 最高分:${(vm.state.debugHighestScore * 100).toStringAsFixed(1)}% 图:${vm.state.imageWidthPx}x${vm.state.imageHeightPx} 旋:${vm.state.rotation}',
|
||||
'阈值:${(_minScore * 100).toStringAsFixed(0)}% 模型:${vm.state.modelReady ? ModelManager.instance.modelsLabel : '未加载'} 帧:${vm.state.framesReceived} 流:${camera?.streamCallbacks ?? 0} 推理:${vm.state.debugDetectCalls}次 异常:${vm.state.debugDetectErrors}次 处理:${vm.state.debugLastMs}ms 最高分:${(vm.state.debugHighestScore * 100).toStringAsFixed(1)}% 图:${vm.state.imageWidthPx}x${vm.state.imageHeightPx} 传感:${camera?.sensorOrientation ?? '-'} 屏转:${camera?.displayDegrees ?? '-'} 旋:${camera?.rotationDegrees ?? 0} turn:${camera?.quarterTurns ?? '-'}',
|
||||
style: const TextStyle(color: Colors.white70, fontSize: 12),
|
||||
),
|
||||
if (_nativeStats.isNotEmpty)
|
||||
Text(
|
||||
'原生:回调${_nativeStats['callbacks'] ?? '-'} 发出${_nativeStats['emitOk'] ?? '-'} 异常${_nativeStats['emitErr'] ?? '-'} 无订阅${_nativeStats['sinkNull'] ?? '-'} sink:${_nativeStats['sink'] ?? '-'} 配置:${_nativeStats['size'] ?? '-'} 发帧:${_nativeStats['emitSize'] ?? '-'} 错误:${_nativeStats['error'] ?? '无'} 轮询:${_nativeStats['pollErr'] ?? 'ok'}${vm.state.results.isEmpty ? '' : ' 框1:(${vm.state.results.first.left.toStringAsFixed(2)},${vm.state.results.first.top.toStringAsFixed(2)},${vm.state.results.first.right.toStringAsFixed(2)},${vm.state.results.first.bottom.toStringAsFixed(2)})'}',
|
||||
style: const TextStyle(
|
||||
color: Colors.amberAccent, fontSize: 11),
|
||||
),
|
||||
if (vm.state.debugYuv.isNotEmpty)
|
||||
Text(
|
||||
'yuv:${vm.state.debugYuv}',
|
||||
style: const TextStyle(
|
||||
color: Colors.yellowAccent, fontSize: 11),
|
||||
),
|
||||
if (camera != null)
|
||||
Text(
|
||||
'streaming:${camera.currentController.value.isStreamingImages} '
|
||||
'camErr:${camera.currentController.value.errorDescription ?? '无'}',
|
||||
'streaming:${camera.isStreaming} '
|
||||
'camErr:${camera.errorDescription ?? '无'}',
|
||||
maxLines: 2,
|
||||
overflow: TextOverflow.ellipsis,
|
||||
style: const TextStyle(
|
||||
@@ -274,24 +385,19 @@ class _CameraScreenState extends State<CameraScreen> {
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
/// 双指捏合缩放预览；overlay 与纹理同几何（CameraPreview child）
|
||||
/// 双指捏合缩放预览；overlay 与纹理同几何（Stack 内同尺寸）
|
||||
class _ZoomablePreview extends StatefulWidget {
|
||||
final CameraController controller;
|
||||
final AppCameraController controller;
|
||||
|
||||
/// 检测框 overlay（随帧更新，作为 CameraPreview 的 child 与纹理同区域）
|
||||
/// 检测框 overlay（随帧更新，与纹理同区域）
|
||||
final Widget? overlay;
|
||||
|
||||
/// 当前帧图像尺寸（用于按 buffer 比例约束预览，保证无拉伸变形）
|
||||
final int imageWidthPx;
|
||||
final int imageHeightPx;
|
||||
|
||||
const _ZoomablePreview({
|
||||
required this.controller,
|
||||
this.overlay,
|
||||
this.imageWidthPx = 0,
|
||||
this.imageHeightPx = 0,
|
||||
});
|
||||
|
||||
@override
|
||||
@@ -317,7 +423,9 @@ class _ZoomablePreviewState extends State<_ZoomablePreview> {
|
||||
|
||||
@override
|
||||
Widget build(BuildContext context) {
|
||||
final preview = GestureDetector(
|
||||
// 预览铺满全屏（cover 裁剪由插件按视图比例完成）；
|
||||
// overlay 与纹理同几何：作为 Stack sibling 叠在上层，坐标与纹理区域一致
|
||||
return GestureDetector(
|
||||
onScaleStart: (_) => _gestureStartZoom = _currentZoom,
|
||||
onScaleUpdate: (d) {
|
||||
final target =
|
||||
@@ -326,24 +434,24 @@ class _ZoomablePreviewState extends State<_ZoomablePreview> {
|
||||
_currentZoom = target;
|
||||
widget.controller.setZoomLevel(target);
|
||||
},
|
||||
child: CameraPreview(widget.controller, child: widget.overlay),
|
||||
);
|
||||
|
||||
final w = widget.imageWidthPx.toDouble();
|
||||
final h = widget.imageHeightPx.toDouble();
|
||||
if (w <= 0 || h <= 0) return preview;
|
||||
// 按 buffer 比例约束显示区域：纹理与 overlay 同区域等比显示（无变形）
|
||||
return Center(
|
||||
child: AspectRatio(aspectRatio: w / h, child: preview),
|
||||
child: Stack(
|
||||
fit: StackFit.expand,
|
||||
children: [
|
||||
widget.controller.buildPreview(),
|
||||
if (widget.overlay != null) widget.overlay!,
|
||||
],
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
class _CameraTopBar extends StatelessWidget {
|
||||
final VoidCallback onClose;
|
||||
final VoidCallback onOpenSettings;
|
||||
|
||||
const _CameraTopBar({
|
||||
required this.onClose,
|
||||
required this.onOpenSettings,
|
||||
});
|
||||
|
||||
@override
|
||||
@@ -359,6 +467,11 @@ class _CameraTopBar extends StatelessWidget {
|
||||
icon: const Icon(Icons.arrow_back, color: Colors.white),
|
||||
onPressed: onClose,
|
||||
),
|
||||
IconButton(
|
||||
tooltip: '识别设置',
|
||||
icon: const Icon(Icons.tune, color: Colors.white70),
|
||||
onPressed: onOpenSettings,
|
||||
),
|
||||
],
|
||||
),
|
||||
);
|
||||
|
||||
@@ -4,7 +4,6 @@ import 'package:flutter/foundation.dart';
|
||||
|
||||
import '../detection/detection_result.dart';
|
||||
import '../detection/motion_aggregator.dart';
|
||||
import '../detection/tflite_detector.dart';
|
||||
import '../reminder/reminder.dart';
|
||||
|
||||
@immutable
|
||||
@@ -20,6 +19,7 @@ class CameraUiState {
|
||||
final String? debugLastError;
|
||||
final int framesReceived;
|
||||
final int debugLastMs;
|
||||
final String debugYuv;
|
||||
|
||||
const CameraUiState({
|
||||
this.modelReady = false,
|
||||
@@ -33,20 +33,19 @@ class CameraUiState {
|
||||
this.debugLastError,
|
||||
this.framesReceived = 0,
|
||||
this.debugLastMs = 0,
|
||||
this.debugYuv = '',
|
||||
});
|
||||
}
|
||||
|
||||
/// 检测结果置信度分级与轨迹确认。
|
||||
///
|
||||
/// - [lowConf]（模型阈值 0.10）：低于此分的框在检测阶段已丢弃。
|
||||
/// - [highConf]（0.35）：高于此分直接确认显示；真实野鸡多为 0.1~0.2，
|
||||
/// 高于 0.35 视为强证据。
|
||||
/// - 0.10~0.35 之间：需要多帧稳定（[confirmFrames] 帧）或 活动证据
|
||||
/// - 低于 0.35 的框：需要多帧稳定（[confirmFrames] 帧）或 活动证据
|
||||
/// （运动区域/背景新出现区域重叠）才确认显示。
|
||||
class CameraViewModel extends ChangeNotifier {
|
||||
static const int maxTracks = 30;
|
||||
static const double motionBoost = 0.15;
|
||||
static const double lowConf = TfliteDetector.minScore;
|
||||
static const double highConf = 0.35;
|
||||
static const int confirmFrames = 3;
|
||||
static const double associateRadius = 0.12;
|
||||
@@ -86,6 +85,7 @@ class CameraViewModel extends ChangeNotifier {
|
||||
String? lastError,
|
||||
int framesReceived = 0,
|
||||
int lastProcessMs = 0,
|
||||
String yuvDiag = '',
|
||||
}) {
|
||||
final now = DateTime.now().millisecondsSinceEpoch;
|
||||
_associate(results, motionRegions, noveltyRegions, now);
|
||||
@@ -132,6 +132,7 @@ class CameraViewModel extends ChangeNotifier {
|
||||
debugLastError: lastError,
|
||||
framesReceived: framesReceived,
|
||||
debugLastMs: lastProcessMs,
|
||||
debugYuv: yuvDiag.isNotEmpty ? yuvDiag : _state.debugYuv,
|
||||
);
|
||||
notifyListeners();
|
||||
}
|
||||
|
||||
@@ -87,10 +87,13 @@ class _OverlayPainter extends CustomPainter {
|
||||
_drawDashedRect(canvas, box, paint);
|
||||
}
|
||||
|
||||
// 标签：框上方，含距离
|
||||
// 标签：框上方，含距离；多模型时标注来源模型名（内置资产不标）
|
||||
final dist = _distanceLabel(r);
|
||||
final modelTag = r.modelName.isNotEmpty && r.modelName != '内置'
|
||||
? '[${r.modelName}]'
|
||||
: '';
|
||||
final text =
|
||||
'${_labels[r.label] ?? r.label} ${(r.score * 100).toInt()}%$dist';
|
||||
'${_labels[r.label] ?? r.label}$modelTag ${(r.score * 100).toInt()}%$dist';
|
||||
final textPainter = TextPainter(
|
||||
text: TextSpan(
|
||||
text: text,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import 'dart:typed_data';
|
||||
|
||||
import 'package:camera/camera.dart';
|
||||
|
||||
import '../detection/detection_result.dart';
|
||||
@@ -44,7 +46,8 @@ class FrameAnalyzer {
|
||||
int rotation,
|
||||
int width,
|
||||
int height,
|
||||
int processMs) {
|
||||
int processMs,
|
||||
String yuvDiag) {
|
||||
detectCalls++;
|
||||
lastProcessMs = processMs;
|
||||
viewModel.onFramesAnalyzed(
|
||||
@@ -59,6 +62,7 @@ class FrameAnalyzer {
|
||||
lastError: lastError,
|
||||
framesReceived: framesReceived,
|
||||
lastProcessMs: lastProcessMs,
|
||||
yuvDiag: yuvDiag,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -80,15 +84,45 @@ class FrameAnalyzer {
|
||||
);
|
||||
}
|
||||
|
||||
void analyze(CameraImage image, int rotationDegrees) {
|
||||
/// 节流与 busy 丢帧判定（两入口共用）；通过后才允许投递
|
||||
bool _canSend() {
|
||||
framesReceived++;
|
||||
final w = worker;
|
||||
if (w == null) return;
|
||||
if (w == null) return false;
|
||||
final now = DateTime.now().millisecondsSinceEpoch;
|
||||
if (now - _lastDetectMs < intervalMs) return;
|
||||
if (now - _lastDetectMs < intervalMs) return false;
|
||||
_lastDetectMs = now;
|
||||
if (w.busy) return; // 上一帧未返回则丢帧，避免在途积压
|
||||
w.analyze(image, rotationDegrees);
|
||||
if (w.busy) return false; // 上一帧未返回则丢帧，避免在途积压
|
||||
return true;
|
||||
}
|
||||
|
||||
void analyze(CameraImage image, int rotationDegrees,
|
||||
{bool rgbaOrder = false}) {
|
||||
if (!_canSend()) return;
|
||||
worker!.analyze(image, rotationDegrees, rgbaOrder: rgbaOrder);
|
||||
}
|
||||
|
||||
/// 原生相机通道帧（Android）：字节已在 Kotlin 侧旋转成竖屏，rotation=0。
|
||||
/// isBgra=true + rgbaOrder 与插件路径同语义：false=BGRA(rOff=2)/true=RGBA(rOff=0)
|
||||
void analyzeRaw({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
required bool rgbaOrder,
|
||||
int rotationDegrees = 0,
|
||||
}) {
|
||||
if (!_canSend()) return;
|
||||
worker!.analyzeRaw(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
rgbaOrder: rgbaOrder,
|
||||
rotationDegrees: rotationDegrees,
|
||||
);
|
||||
}
|
||||
|
||||
void reset() => _lastDetectMs = 0;
|
||||
|
||||
@@ -12,7 +12,7 @@ class ViewRect {
|
||||
double get centerY => (top + bottom) / 2;
|
||||
}
|
||||
|
||||
/// 模型归一化坐标 → 预览视图坐标（含传感器旋转与 FIT_CENTER 裁剪）。
|
||||
/// 模型归一化坐标 → 预览视图坐标（含传感器旋转与 FIT_COVER 全屏裁剪）。
|
||||
class CoordinateMapper {
|
||||
static ViewRect mapToView(
|
||||
double normLeft,
|
||||
@@ -53,10 +53,10 @@ class CoordinateMapper {
|
||||
final portrait = rotation == 90 || rotation == 270;
|
||||
final portW = portrait ? imageH : imageW;
|
||||
final portH = portrait ? imageW : imageH;
|
||||
// 3) FIT_CENTER 缩放与居中偏移
|
||||
// 3) FIT_COVER 缩放与居中裁剪：放大到铺满视图，溢出部分裁掉
|
||||
final scale = viewW / portW < viewH / portH
|
||||
? viewW / portW
|
||||
: viewH / portH;
|
||||
? viewH / portH
|
||||
: viewW / portW;
|
||||
final offsetX = (viewW - portW * scale) / 2;
|
||||
final offsetY = (viewH - portH * scale) / 2;
|
||||
return ViewRect(
|
||||
|
||||
@@ -9,6 +9,10 @@ class DetectionResult {
|
||||
/// 轨迹已确认（多帧稳定/高分/活动确认），false = 候选，渲染为虚线
|
||||
final bool confirmed;
|
||||
|
||||
/// 产出该框的模型（数据集 id 与名称；内置资产模型为 -1/空）
|
||||
final int modelId;
|
||||
final String modelName;
|
||||
|
||||
const DetectionResult({
|
||||
required this.label,
|
||||
required this.score,
|
||||
@@ -17,6 +21,8 @@ class DetectionResult {
|
||||
required this.right,
|
||||
required this.bottom,
|
||||
this.confirmed = true,
|
||||
this.modelId = -1,
|
||||
this.modelName = '',
|
||||
});
|
||||
|
||||
double get width => right - left;
|
||||
@@ -40,6 +46,8 @@ class DetectionResult {
|
||||
right: right ?? this.right,
|
||||
bottom: bottom ?? this.bottom,
|
||||
confirmed: confirmed ?? this.confirmed,
|
||||
modelId: modelId,
|
||||
modelName: modelName,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -7,17 +7,27 @@ import 'package:flutter/foundation.dart' show debugPrint;
|
||||
import 'package:flutter/services.dart' show rootBundle;
|
||||
|
||||
import '../camera/motion_detector.dart';
|
||||
import '../models/model_manager.dart';
|
||||
import 'background_model.dart';
|
||||
import 'detection_result.dart';
|
||||
import 'nms.dart';
|
||||
import 'tflite_detector.dart';
|
||||
import 'visual_prior.dart';
|
||||
|
||||
/// 推理工作单元：模型加载与检测全部在后台 isolate 执行，
|
||||
/// 主 isolate 只投递帧数据、接收结果，UI 不被推理阻塞（iOS 真机卡顿根因）。
|
||||
///
|
||||
/// 多模型并行推理：传入 [models]（各数据集下载模型）后，每帧逐模型推理，
|
||||
/// 结果按类别分组跨模型 NMS 合并（同标签重复框取高分，不同标签互不压制）；
|
||||
/// 无下载模型时回退内置资产模型。
|
||||
class DetectorWorker {
|
||||
static const String modelAsset = 'assets/model.tflite';
|
||||
static const String labelsAsset = 'assets/labels.txt';
|
||||
|
||||
/// 内置资产回退模型的标识
|
||||
static const int builtinModelId = -1;
|
||||
static const String builtinModelName = '内置';
|
||||
|
||||
final Isolate _isolate;
|
||||
final ReceivePort _responses;
|
||||
|
||||
@@ -30,9 +40,10 @@ class DetectorWorker {
|
||||
int _inFlight = 0;
|
||||
bool _dead = false;
|
||||
|
||||
/// 结果回调：结果 / 运动区域 / 新颖区域 / 旋转角 / 图宽 / 图高 / 处理耗时 ms
|
||||
/// 结果回调：结果 / 运动区域 / 新颖区域 / 旋转角 / 图宽 / 图高 /
|
||||
/// 处理耗时 ms / yuv 决策诊断串
|
||||
void Function(List<DetectionResult>, List<MotionRegion>, List<MotionRegion>,
|
||||
int, int, int, int)? onResult;
|
||||
int, int, int, int, String)? onResult;
|
||||
|
||||
/// 单帧处理异常回调（不影响相机流）
|
||||
void Function(String)? onError;
|
||||
@@ -53,16 +64,26 @@ class DetectorWorker {
|
||||
});
|
||||
}
|
||||
|
||||
/// 读取模型资产并启动后台推理 isolate；加载失败返回 null（App 降级为仅预览）。
|
||||
static Future<DetectorWorker?> create() async {
|
||||
/// 加载模型并启动后台推理 isolate；加载失败返回 null（App 降级为仅预览）。
|
||||
/// [models] 为空时回退内置资产模型（模型缺失同样返回 null）。
|
||||
static Future<DetectorWorker?> create({List<ModelBundle>? models}) async {
|
||||
try {
|
||||
final data = await rootBundle.load(modelAsset);
|
||||
final modelBytes =
|
||||
data.buffer.asUint8List(data.offsetInBytes, data.lengthInBytes);
|
||||
final labels = (await rootBundle.loadString(labelsAsset))
|
||||
.split('\n')
|
||||
.where((l) => l.trim().isNotEmpty)
|
||||
.toList();
|
||||
final payload = <List<Object?>>[];
|
||||
if (models != null && models.isNotEmpty) {
|
||||
for (final m in models) {
|
||||
payload.add(
|
||||
[m.bytes, m.labels, m.datasetId, m.datasetName]);
|
||||
}
|
||||
} else {
|
||||
final data = await rootBundle.load(modelAsset);
|
||||
final modelBytes =
|
||||
data.buffer.asUint8List(data.offsetInBytes, data.lengthInBytes);
|
||||
final labels = (await rootBundle.loadString(labelsAsset))
|
||||
.split('\n')
|
||||
.where((l) => l.trim().isNotEmpty)
|
||||
.toList();
|
||||
payload.add([modelBytes, labels, builtinModelId, builtinModelName]);
|
||||
}
|
||||
|
||||
final responses = ReceivePort();
|
||||
final isolate = await Isolate.spawn(_workerMain, responses.sendPort);
|
||||
@@ -72,7 +93,7 @@ class DetectorWorker {
|
||||
.timeout(const Duration(seconds: 10),
|
||||
onTimeout: () => throw TimeoutException('worker port timeout'));
|
||||
worker._port = port;
|
||||
port.send(['load', modelBytes, labels]);
|
||||
port.send(['load', payload]);
|
||||
await worker._ready.future
|
||||
.timeout(const Duration(seconds: 20), onTimeout: () {
|
||||
throw TimeoutException('model load timeout');
|
||||
@@ -88,20 +109,37 @@ class DetectorWorker {
|
||||
/// 是否忙（上一帧尚未返回）：忙则丢帧，避免在途积压
|
||||
bool get busy => _inFlight > 0;
|
||||
|
||||
void analyze(CameraImage image, int rotationDegrees) {
|
||||
void analyze(CameraImage image, int rotationDegrees, {bool rgbaOrder = false}) {
|
||||
// 单平面 8888 判定：仅明确的 yuv420/nv21 走多平面 YUV 路径；
|
||||
// bgra8888 与 unknown（插件未识别 RGBA_8888 输出时）都按 4 字节像素处理
|
||||
final group = image.format.group;
|
||||
analyzeRaw(
|
||||
planes: image.planes.map((p) => p.bytes).toList(),
|
||||
strides: image.planes.map((p) => p.bytesPerRow).toList(),
|
||||
width: image.width,
|
||||
height: image.height,
|
||||
isBgra: group != ImageFormatGroup.yuv420 && group != ImageFormatGroup.nv21,
|
||||
rgbaOrder: rgbaOrder,
|
||||
rotationDegrees: rotationDegrees,
|
||||
);
|
||||
}
|
||||
|
||||
/// 原始字节帧投递（截屏注入用：toImage 的 RGBA 字节直接进检测，不经 CameraImage）
|
||||
void analyzeRaw({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
required bool rgbaOrder,
|
||||
required int rotationDegrees,
|
||||
}) {
|
||||
final port = _port;
|
||||
if (port == null || _dead) return;
|
||||
_inFlight++;
|
||||
port.send([
|
||||
'frame',
|
||||
[
|
||||
image.planes.map((p) => p.bytes).toList(),
|
||||
image.planes.map((p) => p.bytesPerRow).toList(),
|
||||
image.width,
|
||||
image.height,
|
||||
image.format.group == ImageFormatGroup.bgra8888,
|
||||
rotationDegrees,
|
||||
],
|
||||
[planes, strides, width, height, isBgra, rotationDegrees, rgbaOrder],
|
||||
]);
|
||||
}
|
||||
|
||||
@@ -129,6 +167,8 @@ class DetectorWorker {
|
||||
top: v[3] as double,
|
||||
right: v[4] as double,
|
||||
bottom: v[5] as double,
|
||||
modelId: v.length > 6 ? (v[6] as num).toInt() : -1,
|
||||
modelName: v.length > 7 ? v[7] as String : '',
|
||||
);
|
||||
}).toList();
|
||||
final motion = (list[5] as List)
|
||||
@@ -142,7 +182,7 @@ class DetectorWorker {
|
||||
v[0] as double, v[1] as double, v[2] as double, v[3] as double))
|
||||
.toList();
|
||||
onResult?.call(dets, motion, novelty, list[1] as int, list[2] as int,
|
||||
list[3] as int, list[7] as int);
|
||||
list[3] as int, list[7] as int, list[8] as String);
|
||||
break;
|
||||
case 'log':
|
||||
lastLog = list[1] as String;
|
||||
@@ -151,6 +191,7 @@ class DetectorWorker {
|
||||
case 'error':
|
||||
_inFlight--;
|
||||
onError?.call(list[1] as String);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -161,6 +202,13 @@ class DetectorWorker {
|
||||
port.send(['reset']);
|
||||
}
|
||||
|
||||
/// 调整置信度阈值（设置页滑块，worker 内实时生效）
|
||||
void setMinScore(double v) {
|
||||
final port = _port;
|
||||
if (port == null || _dead) return;
|
||||
port.send(['set-min-score', v]);
|
||||
}
|
||||
|
||||
void dispose() {
|
||||
_dead = true;
|
||||
_isolate.kill(priority: Isolate.immediate);
|
||||
@@ -176,9 +224,11 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
mainPort.send(['port', control.sendPort]);
|
||||
mainPort.send(['log', 'worker-start']);
|
||||
|
||||
TfliteDetector? detector;
|
||||
List<TfliteDetector> detectors = const [];
|
||||
MotionDetector? motion;
|
||||
BackgroundModel? background;
|
||||
var lastDualMs = 0; // 双字节序推理诊断节流
|
||||
Uint8List? prevY; // 上一帧 Y/RGBA 平面（帧间 diff 诊断）
|
||||
await for (final msg in control) {
|
||||
try {
|
||||
final list = msg as List;
|
||||
@@ -186,12 +236,35 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
case 'load':
|
||||
mainPort.send(['log', 'load-received']);
|
||||
try {
|
||||
detector = await TfliteDetector.fromBuffer(
|
||||
list[1] as Uint8List, (list[2] as List).cast<String>());
|
||||
if (detector == null) {
|
||||
mainPort.send(['load-error', 'fromBuffer 返回 null']);
|
||||
// 多模型：逐模型加载，单个失败不阻塞其余；全部失败才报错
|
||||
final loaded = <TfliteDetector>[];
|
||||
final failures = <String>[];
|
||||
for (final entry in list[1] as List) {
|
||||
final e = entry as List;
|
||||
final name = e.length > 3 ? e[3] as String : '';
|
||||
final d = await TfliteDetector.fromBuffer(
|
||||
e[0] as Uint8List,
|
||||
(e[1] as List).cast<String>(),
|
||||
modelId: (e[2] as num).toInt(),
|
||||
modelName: name,
|
||||
);
|
||||
if (d == null) {
|
||||
failures.add(name.isEmpty ? 'unknown' : name);
|
||||
} else {
|
||||
loaded.add(d);
|
||||
}
|
||||
}
|
||||
if (loaded.isEmpty) {
|
||||
mainPort.send([
|
||||
'load-error',
|
||||
'模型加载失败:${failures.join(',')} '
|
||||
'(fromBuffer 返回 null)'
|
||||
]);
|
||||
} else {
|
||||
mainPort.send(['log', 'fromBuffer-ok']);
|
||||
detectors = loaded;
|
||||
mainPort.send(['log',
|
||||
'loaded=${loaded.map((d) => d.modelName).join(',')} '
|
||||
'failed=${failures.isEmpty ? '-' : failures.join(',')}']);
|
||||
motion = MotionDetector();
|
||||
background = BackgroundModel();
|
||||
mainPort.send(['ready']);
|
||||
@@ -201,10 +274,9 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
}
|
||||
break;
|
||||
case 'frame':
|
||||
final d = detector;
|
||||
final m = motion;
|
||||
final b = background;
|
||||
if (d == null || m == null || b == null) break;
|
||||
if (detectors.isEmpty || m == null || b == null) break;
|
||||
final frame = list[1] as List;
|
||||
final planes = (frame[0] as List).cast<Uint8List>();
|
||||
final strides = (frame[1] as List).cast<int>();
|
||||
@@ -212,15 +284,111 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
final height = frame[3] as int;
|
||||
final isBgra = frame[4] as bool;
|
||||
final rotation = frame[5] as int;
|
||||
final rgbaOrder = frame.length > 6 && (frame[6] as bool);
|
||||
|
||||
// 止血：非法帧（宽高/平面为空）直接丢弃并上报诊断，
|
||||
// 避免下游组件越界（RGBA patch 后插件偶发 w/h=0 帧）
|
||||
if (width <= 0 || height <= 0 || planes.isEmpty || planes[0].isEmpty) {
|
||||
mainPort.send([
|
||||
'error',
|
||||
'bad frame w=$width h=$height planes=${planes.length} '
|
||||
'p0=${planes.isNotEmpty ? planes[0].length : 0} '
|
||||
'stride=${strides.isNotEmpty ? strides[0] : '-'} '
|
||||
'bgra=$isBgra'
|
||||
]);
|
||||
break;
|
||||
}
|
||||
|
||||
final sw = Stopwatch()..start();
|
||||
var results = d.detectRaw(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
);
|
||||
// 帧内容统计（诊断）：Y/RGBA 平面 min/max/mean + 与上帧的平均绝对差。
|
||||
// 均匀灰帧 → min≈max≈mean；静止灰帧 → diff≈0；真实画面 → 分布宽且 diff>0
|
||||
final yPlane = planes[0];
|
||||
var yMin = 255, yMax = 0, ySum = 0, diff = 0, sampled = 0;
|
||||
final prev = prevY;
|
||||
for (var i = 0; i < yPlane.length; i += 8) {
|
||||
final v = yPlane[i];
|
||||
if (v < yMin) yMin = v;
|
||||
if (v > yMax) yMax = v;
|
||||
ySum += v;
|
||||
if (prev != null && i < prev.length) {
|
||||
final d = v - prev[i];
|
||||
diff += d < 0 ? -d : d;
|
||||
}
|
||||
sampled++;
|
||||
}
|
||||
prevY = yPlane;
|
||||
final yMean = ySum / sampled;
|
||||
final yDiff =
|
||||
prev == null ? -1.0 : diff / (sampled * 255.0);
|
||||
// UV 平面统计（诊断）：色序/值域异常会导致解码偏色
|
||||
var uv1 = '-';
|
||||
if (planes.length > 1) {
|
||||
final u = planes[1];
|
||||
var uMin = 255, uMax = 0, uSum = 0, n = 0;
|
||||
for (var i = 0; i < u.length; i += 8) {
|
||||
final v = u[i];
|
||||
if (v < uMin) uMin = v;
|
||||
if (v > uMax) uMax = v;
|
||||
uSum += v;
|
||||
n++;
|
||||
}
|
||||
uv1 = 's=${strides[1]} min=$uMin max=$uMax mean=${(uSum / n).toStringAsFixed(0)}';
|
||||
if (planes.length > 2) {
|
||||
final v2 = planes[2];
|
||||
var vMin = 255, vMax = 0, vSum = 0, n2 = 0;
|
||||
for (var i = 0; i < v2.length; i += 8) {
|
||||
final v = v2[i];
|
||||
if (v < vMin) vMin = v;
|
||||
if (v > vMax) vMax = v;
|
||||
vSum += v;
|
||||
n2++;
|
||||
}
|
||||
uv1 += ' v:min=$vMin max=$vMax mean=${(vSum / n2).toStringAsFixed(0)}';
|
||||
}
|
||||
}
|
||||
// 多模型并行推理：每模型先首帧自适应判定 YUV 值域/色序，再逐模型推理；
|
||||
// 汇总后按类别分组跨模型 NMS 合并（同标签重复框取高分，异标签互不压制）
|
||||
var results = <DetectionResult>[];
|
||||
for (final d in detectors) {
|
||||
if (!isBgra && !d.yuvModeKnown) {
|
||||
d.decideYuvChroma(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
);
|
||||
}
|
||||
var dets = d.detectRaw(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
rgbaOrder: rgbaOrder,
|
||||
);
|
||||
// 自愈：判定后 1.5s 内无检测且帧可用 → 用实时帧重跑完整判定
|
||||
// （首帧模糊/暗帧导致启发式猜错时，画面稳定后 oracle 可分胜负）
|
||||
if (!isBgra && d.yuvModeKnown && !d.yuvRetried &&
|
||||
dets.length <= 1 &&
|
||||
DateTime.now().millisecondsSinceEpoch - d.yuvDecisionMs >
|
||||
1500 &&
|
||||
d.retryDecision(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
)) {
|
||||
dets = d.detectRaw(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
);
|
||||
}
|
||||
results.addAll(dets);
|
||||
}
|
||||
results = mergeAcrossModels(results, TfliteDetector.iouThreshold);
|
||||
// 低分野鸡框过视觉先验（颜色/位置），减少户外误报
|
||||
results = VisualPrior.filter(
|
||||
results,
|
||||
@@ -229,6 +397,7 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
rgbaOrder: rgbaOrder,
|
||||
);
|
||||
final motionRegions = m.detectMotionRaw(
|
||||
planes[0], strides[0], width, height);
|
||||
@@ -236,14 +405,44 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
b.updateRaw(planes[0], strides[0], width, height);
|
||||
sw.stop();
|
||||
|
||||
var dualDiag = '';
|
||||
if (isBgra &&
|
||||
DateTime.now().millisecondsSinceEpoch - lastDualMs > 3000) {
|
||||
lastDualMs = DateTime.now().millisecondsSinceEpoch;
|
||||
final a = detectors.first.diagnoseOrder(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: true,
|
||||
rgbaOrder: false);
|
||||
final b = detectors.first.diagnoseOrder(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: true,
|
||||
rgbaOrder: true);
|
||||
dualDiag = ' | dual BGRA:${a.$1}@${(a.$2 * 100).toStringAsFixed(1)}%'
|
||||
' RGBA:${b.$1}@${(b.$2 * 100).toStringAsFixed(1)}%';
|
||||
}
|
||||
|
||||
mainPort.send([
|
||||
'result',
|
||||
rotation,
|
||||
width,
|
||||
height,
|
||||
results
|
||||
.map((r) =>
|
||||
[r.label, r.score, r.left, r.top, r.right, r.bottom])
|
||||
.map((r) => [
|
||||
r.label,
|
||||
r.score,
|
||||
r.left,
|
||||
r.top,
|
||||
r.right,
|
||||
r.bottom,
|
||||
r.modelId,
|
||||
r.modelName,
|
||||
])
|
||||
.toList(),
|
||||
motionRegions
|
||||
.map((mr) => [mr.left, mr.top, mr.right, mr.bottom])
|
||||
@@ -252,14 +451,47 @@ Future<void> _workerMain(SendPort mainPort) async {
|
||||
.map((mr) => [mr.left, mr.top, mr.right, mr.bottom])
|
||||
.toList(),
|
||||
sw.elapsedMilliseconds,
|
||||
'planes=${planes.length} yLen=${yPlane.length} stride=${strides[0]} '
|
||||
'y:min=$yMin max=$yMax mean=${yMean.toStringAsFixed(1)} '
|
||||
'diff=${yDiff < 0 ? '-' : yDiff.toStringAsFixed(3)} '
|
||||
'uv1:[$uv1] | ${detectors.first.yuvDiag}$dualDiag',
|
||||
]);
|
||||
break;
|
||||
case 'reset':
|
||||
motion?.reset();
|
||||
background?.reset();
|
||||
for (final d in detectors) {
|
||||
d.resetYuvMode();
|
||||
}
|
||||
break;
|
||||
case 'set-min-score':
|
||||
for (final d in detectors) {
|
||||
d.minScore = (list[1] as num).toDouble();
|
||||
}
|
||||
mainPort.send(['log', 'min-score=${detectors.isEmpty ? '-' : detectors.first.minScore}']);
|
||||
}
|
||||
} catch (e) {
|
||||
mainPort.send(['error', '$e']);
|
||||
} catch (e, st) {
|
||||
mainPort.send([
|
||||
'error',
|
||||
'$e\n${st.toString().split('\n').take(3).join('\n')}'
|
||||
]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// 多模型结果合并：按类别分组，组内 NMS（不同模型检出同一目标时取高分）。
|
||||
/// 各模型类别体系独立（如野鸡/疑似 vs 野兔/疑似），不同类别互不压制。
|
||||
List<DetectionResult> mergeAcrossModels(
|
||||
List<DetectionResult> all, double iouThreshold) {
|
||||
if (all.length <= 1) return all;
|
||||
final byLabel = <String, List<DetectionResult>>{};
|
||||
for (final r in all) {
|
||||
byLabel.putIfAbsent(r.label, () => []).add(r);
|
||||
}
|
||||
final merged = <DetectionResult>[];
|
||||
for (final group in byLabel.values) {
|
||||
merged.addAll(nms(group, iouThreshold));
|
||||
}
|
||||
merged.sort((a, b) => b.score.compareTo(a.score));
|
||||
return merged;
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import 'dart:typed_data';
|
||||
|
||||
import 'package:flutter/foundation.dart' show debugPrint;
|
||||
import 'package:tflite_flutter/tflite_flutter.dart';
|
||||
|
||||
import 'detection_result.dart';
|
||||
@@ -10,9 +11,12 @@ import 'nms.dart';
|
||||
/// cx/cy/w/h 已归一化，类别得分已过 sigmoid；按 out[dim][anchor] 索引。
|
||||
/// 输入为 NCHW [1, 3, 704, 704]（litert 导出保留 torch 布局）。
|
||||
class TfliteDetector {
|
||||
static const int inputSize = 704;
|
||||
// 野鸡数据置信度普遍偏低（0.1~0.2 量级），保留低分池供运动检测提升
|
||||
static const double minScore = 0.10;
|
||||
// 输入尺寸取自模型本身（ultralytics litert 导出 NCHW [1,3,H,W]，各数据集
|
||||
// 训练 imgsz 可不同），默认 704 兜底
|
||||
static const int defaultInputSize = 704;
|
||||
// 野鸡数据置信度普遍偏低（0.1~0.2 量级），保留低分池供运动检测提升；
|
||||
// 可运行时调整（设置页滑块），默认 0.10
|
||||
double minScore = 0.10;
|
||||
static const double iouThreshold = 0.45;
|
||||
static const int maxDetections = 20;
|
||||
static const String modelAsset = 'assets/model.tflite';
|
||||
@@ -22,39 +26,55 @@ class TfliteDetector {
|
||||
final List<String> _labels;
|
||||
final int _numClasses;
|
||||
final int _numAnchors;
|
||||
final int inputSize;
|
||||
|
||||
final Float32List _input =
|
||||
Float32List(1 * inputSize * inputSize * 3);
|
||||
/// 模型身份（多模型并行推理区分来源；内置资产模型为 -1/空）
|
||||
final int modelId;
|
||||
final String modelName;
|
||||
|
||||
late final Float32List _input;
|
||||
|
||||
/// 输出按模型形状 [1, 4+nc, anchors] 的嵌套 List 组织，
|
||||
/// run() 要求输出对象形状与模型完全一致（扁平 List 会被拒）。
|
||||
final List<List<List<double>>> _output;
|
||||
|
||||
TfliteDetector._(this._interpreter, this._labels, this._numClasses,
|
||||
this._numAnchors, this._output);
|
||||
this._numAnchors, this._output, this.inputSize, this.modelId,
|
||||
this.modelName) {
|
||||
_input = Float32List(1 * inputSize * inputSize * 3);
|
||||
}
|
||||
|
||||
/// 模型缺失或加载失败返回 null（App 降级为仅预览）。
|
||||
/// 在后台 isolate 内调用（模型字节由主 isolate 读取后传入）。
|
||||
static Future<TfliteDetector?> fromBuffer(
|
||||
Uint8List bytes, List<String> labels) async {
|
||||
Uint8List bytes,
|
||||
List<String> labels, {
|
||||
int modelId = -1,
|
||||
String modelName = '',
|
||||
}) async {
|
||||
try {
|
||||
final interpreter = Interpreter.fromBuffer(
|
||||
bytes,
|
||||
options: InterpreterOptions()..threads = 4,
|
||||
);
|
||||
return TfliteDetector._fromModel(interpreter, labels);
|
||||
return TfliteDetector._fromModel(
|
||||
interpreter, labels, modelId, modelName);
|
||||
} catch (_) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/// 输出布局 [1, 4+nc, anchors] 取自模型本身，类别数不与 labels 文件长度耦合。
|
||||
factory TfliteDetector._fromModel(
|
||||
Interpreter interpreter, List<String> labels) {
|
||||
factory TfliteDetector._fromModel(Interpreter interpreter,
|
||||
List<String> labels, int modelId, String modelName) {
|
||||
final shape = interpreter.getOutputTensor(0).shape;
|
||||
final numClasses =
|
||||
shape.length >= 3 && shape[1] > 4 ? shape[1] - 4 : labels.length;
|
||||
final numAnchors = shape.length >= 3 && shape[2] > 0 ? shape[2] : 2100;
|
||||
final inputShape = interpreter.getInputTensor(0).shape;
|
||||
final inputSize = inputShape.length >= 4
|
||||
? inputShape[3]
|
||||
: defaultInputSize;
|
||||
final output = List.generate(
|
||||
1,
|
||||
(_) => List.generate(
|
||||
@@ -62,8 +82,8 @@ class TfliteDetector {
|
||||
(_) => List<double>.filled(numAnchors, 0),
|
||||
),
|
||||
);
|
||||
return TfliteDetector._(
|
||||
interpreter, labels, numClasses, numAnchors, output);
|
||||
return TfliteDetector._(interpreter, labels, numClasses, numAnchors,
|
||||
output, inputSize, modelId, modelName);
|
||||
}
|
||||
|
||||
/// 原始数据接口（后台 isolate 用，不依赖 CameraImage）。
|
||||
@@ -75,13 +95,15 @@ class TfliteDetector {
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
bool rgbaOrder = false,
|
||||
}) {
|
||||
preprocess(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra);
|
||||
isBgra: isBgra,
|
||||
rgbaOrder: rgbaOrder);
|
||||
// 传原始字节视图而非 Float32List：tflite_flutter 会对非 ByteBuffer/Uint8List
|
||||
// 输入调用 resizeInputTensor（1 维 [1486848]），使 node 0 TRANSPOSE prepare 失败
|
||||
_interpreter.run(_input.buffer.asUint8List(), _output);
|
||||
@@ -103,30 +125,36 @@ class TfliteDetector {
|
||||
.toList();
|
||||
}
|
||||
|
||||
/// 按像素格式分派：iOS bgra8888 单平面 / Android yuv420 多平面。
|
||||
/// 按像素格式分派：单平面 RGBA/BGRA（iOS bgra8888 / Android 实验） / yuv420 多平面。
|
||||
void preprocess({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
bool rgbaOrder = false,
|
||||
}) {
|
||||
if (isBgra) {
|
||||
_preprocessBgra(planes[0], strides[0], width, height);
|
||||
_preprocessBgra(planes[0], strides[0], width, height, rgbaOrder);
|
||||
} else {
|
||||
_preprocessYuv(planes, strides, width, height);
|
||||
}
|
||||
}
|
||||
|
||||
/// BGRA8888 单平面（iOS）：每像素 4 字节 [b,g,r,a]，双线性采样，
|
||||
/// letterbox（等比缩到长边 704，短边黑边补 0，与 YOLO 训练一致）。
|
||||
void _preprocessBgra(Uint8List src, int stride, int srcW, int srcH) {
|
||||
/// 单平面 8888（iOS bgra8888 = [b,g,r,a]；Android 实验 RGBA_8888 = [r,g,b,a]）：
|
||||
/// 每像素 4 字节，双线性采样，letterbox（等比缩到长边 704，短边黑边补 0）。
|
||||
void _preprocessBgra(
|
||||
Uint8List src, int stride, int srcW, int srcH, bool rgbaOrder) {
|
||||
final plane = inputSize * inputSize;
|
||||
final scale = inputSize / srcW < inputSize / srcH
|
||||
? inputSize / srcW
|
||||
: inputSize / srcH;
|
||||
final dx = (inputSize - srcW * scale) / 2;
|
||||
final dy = (inputSize - srcH * scale) / 2;
|
||||
// rgbaOrder=false（iOS BGRA）: +0 B、+1 G、+2 R、+3 A；
|
||||
// rgbaOrder=true（Android RGBA）: +0 R、+1 G、+2 B、+3 A
|
||||
final rOff = rgbaOrder ? 0 : 2;
|
||||
final bOff = rgbaOrder ? 2 : 0;
|
||||
|
||||
for (var oy = 0; oy < inputSize; oy++) {
|
||||
final syf = (oy - dy) / scale;
|
||||
@@ -153,23 +181,22 @@ class TfliteDetector {
|
||||
final y1 = y0 < srcH - 1 ? y0 + 1 : y0;
|
||||
final fx = sxf - x0, fy = syf - y0;
|
||||
|
||||
// BGRA 字节序：+0 B、+1 G、+2 R、+3 A
|
||||
final i00 = y0 * stride + x0 * 4;
|
||||
final i10 = y0 * stride + x1 * 4;
|
||||
final i01 = y1 * stride + x0 * 4;
|
||||
final i11 = y1 * stride + x1 * 4;
|
||||
final r00 = src[i00 + 2].toDouble();
|
||||
final r00 = src[i00 + rOff].toDouble();
|
||||
final g00 = src[i00 + 1].toDouble();
|
||||
final b00 = src[i00].toDouble();
|
||||
final r10 = src[i10 + 2].toDouble();
|
||||
final b00 = src[i00 + bOff].toDouble();
|
||||
final r10 = src[i10 + rOff].toDouble();
|
||||
final g10 = src[i10 + 1].toDouble();
|
||||
final b10 = src[i10].toDouble();
|
||||
final r01 = src[i01 + 2].toDouble();
|
||||
final b10 = src[i10 + bOff].toDouble();
|
||||
final r01 = src[i01 + rOff].toDouble();
|
||||
final g01 = src[i01 + 1].toDouble();
|
||||
final b01 = src[i01].toDouble();
|
||||
final r11 = src[i11 + 2].toDouble();
|
||||
final b01 = src[i01 + bOff].toDouble();
|
||||
final r11 = src[i11 + rOff].toDouble();
|
||||
final g11 = src[i11 + 1].toDouble();
|
||||
final b11 = src[i11].toDouble();
|
||||
final b11 = src[i11 + bOff].toDouble();
|
||||
|
||||
_input[p] = _bl(r00, r10, r01, r11, fx, fy) / 255.0;
|
||||
_input[p + plane] = _bl(g00, g10, g01, g11, fx, fy) / 255.0;
|
||||
@@ -179,25 +206,27 @@ class TfliteDetector {
|
||||
}
|
||||
|
||||
/// letterbox 缩放 + YUV → RGB 归一化 0~1（NCHW），双线性采样。
|
||||
/// 兼容 NV12（iOS 双平面，UV 交错）与 I420（Android 三平面）。
|
||||
/// 兼容 NV12（双平面，UV 交错）与 I420（三平面）。
|
||||
/// 首帧自适应：Y 值域（full/limited）与色序（U 先/V 先）因设备而异，
|
||||
/// 静态假设会在部分机型上产生偏色 → 检测退化。
|
||||
void _preprocessYuv(
|
||||
List<Uint8List> planes, List<int> strides, int srcW, int srcH) {
|
||||
final plane = inputSize * inputSize;
|
||||
final y = planes[0];
|
||||
final nv12 = planes.length == 2;
|
||||
final uv = nv12 ? planes[1] : null;
|
||||
final u = nv12 ? null : planes[1];
|
||||
final v = nv12 ? null : planes[2];
|
||||
final yStride = strides[0];
|
||||
final uvStride = strides[1];
|
||||
final vStride =
|
||||
nv12 ? uvStride : (strides.length > 2 ? strides[2] : strides[1]);
|
||||
|
||||
// U/V 平面采样（nv12：偶位 U 奇位 V；i420：三平面分离）
|
||||
// 色序修正后的 U/V 采样（nv12：偶位 U 奇位 V，NV21 相反；i420：平面 1/2 对调）
|
||||
double uAt(int x, int y) => nv12
|
||||
? uv![y * uvStride + x * 2] - 128.0
|
||||
: u![y * uvStride + x] - 128.0;
|
||||
? uv![y * uvStride + (_yuvSwapChroma ? x * 2 + 1 : x * 2)] - 128.0
|
||||
: planes[_yuvSwapChroma ? 2 : 1][y * uvStride + x] - 128.0;
|
||||
double vAt(int x, int y) => nv12
|
||||
? uv![y * uvStride + x * 2 + 1] - 128.0
|
||||
: v![y * uvStride + x] - 128.0;
|
||||
? uv![y * uvStride + (_yuvSwapChroma ? x * 2 : x * 2 + 1)] - 128.0
|
||||
: planes[_yuvSwapChroma ? 1 : 2][y * vStride + x] - 128.0;
|
||||
|
||||
final scale = inputSize / srcW < inputSize / srcH
|
||||
? inputSize / srcW
|
||||
@@ -256,10 +285,11 @@ class TfliteDetector {
|
||||
final v11 = vAt(ux1, uy1);
|
||||
final vv = _bl(v00, v10, v01, v11, fx, fy);
|
||||
|
||||
// 有限范围展开（VideoRange Y 16~235，Cb/Cr 16~240）
|
||||
final yr = (yy - 16.0) * (255.0 / 219.0);
|
||||
final un = uu * (255.0 / 224.0);
|
||||
final vn = vv * (255.0 / 224.0);
|
||||
// 值域展开：有限范围 VideoRange（Y 16~235，Cb/Cr 16~240）需线性拉伸；
|
||||
// 全值域相机直接使用原始值（与 iOS bgra 一致）
|
||||
final yr = _yuvFullRange ? yy : (yy - 16.0) * (255.0 / 219.0);
|
||||
final un = _yuvFullRange ? uu : uu * (255.0 / 224.0);
|
||||
final vn = _yuvFullRange ? vv : vv * (255.0 / 224.0);
|
||||
|
||||
// NCHW：r/g/b 分平面存储
|
||||
_input[p] = (yr + 1.402 * vn) / 255.0;
|
||||
@@ -269,6 +299,155 @@ class TfliteDetector {
|
||||
}
|
||||
}
|
||||
|
||||
/// 首帧自适应判定 YUV 模式，后续帧复用（相机重启后由 worker 复位重判）。
|
||||
/// - 值域：有限范围黑电平恒为 16，低于 12 只可能是全值域。
|
||||
/// - 色序：以模型本身为 oracle——同一帧按两种色序各推理一次，
|
||||
/// 检测数/最高分/总分更高者为真；两序均无检测时退回亮区色相计数启发
|
||||
/// （户外最亮区域为天空应偏蓝，若按默认 U 先序解出偏红则为 V 先序）。
|
||||
/// - 首帧可能曝光未收敛（过暗/全黑），此时 oracle 与启发式都不可信，
|
||||
/// 保持未判定状态等下一帧，避免在垃圾帧上锁死错误色序（真机零检测根因）。
|
||||
bool _yuvFullRange = false;
|
||||
bool _yuvSwapChroma = false;
|
||||
bool _yuvModeKnown = false;
|
||||
bool _yuvRetried = false;
|
||||
int _yuvDecisionMs = 0;
|
||||
String _yuvDiag = '';
|
||||
|
||||
bool get yuvModeKnown => _yuvModeKnown;
|
||||
bool get yuvRetried => _yuvRetried;
|
||||
int get yuvDecisionMs => _yuvDecisionMs;
|
||||
String get yuvDiag => _yuvDiag;
|
||||
|
||||
void decideYuvChroma({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
bool force = false,
|
||||
}) {
|
||||
if (_yuvModeKnown && !force) return;
|
||||
final (yMin, yMax, yMean) = _yStats(planes[0], strides[0], width, height);
|
||||
_yuvFullRange = yMin < 12;
|
||||
if (yMean < 30 || yMax < 170) {
|
||||
// 曝光未稳定：保持未判定，下一帧重试；始终昏暗则维持默认（同旧版）
|
||||
if (!_yuvModeKnown) {
|
||||
_yuvDiag = '等稳定帧 mean=${yMean.toStringAsFixed(0)} max=$yMax';
|
||||
}
|
||||
return;
|
||||
}
|
||||
_yuvModeKnown = true;
|
||||
_yuvDecisionMs = DateTime.now().millisecondsSinceEpoch;
|
||||
final a = _runWithSwap(planes, strides, width, height, false);
|
||||
final b = _runWithSwap(planes, strides, width, height, true);
|
||||
var swap = false;
|
||||
if (a.$1 != b.$1) {
|
||||
swap = b.$1 > a.$1;
|
||||
} else if (a.$2 != b.$2) {
|
||||
swap = b.$2 > a.$2;
|
||||
} else if (a.$3 != b.$3) {
|
||||
swap = b.$3 > a.$3;
|
||||
} else {
|
||||
swap = _brightRegionLeansRed(planes, strides, width, height, yMax);
|
||||
}
|
||||
_yuvSwapChroma = swap;
|
||||
_yuvDiag = 'full=$_yuvFullRange swap=$_yuvSwapChroma'
|
||||
' cA=${a.$1} sA=${a.$2.toStringAsFixed(3)}'
|
||||
' cB=${b.$1} sB=${b.$2.toStringAsFixed(3)}';
|
||||
|
||||
debugPrint('[yuv] $_yuvDiag mean=${yMean.toStringAsFixed(0)} max=$yMax');
|
||||
}
|
||||
|
||||
/// 判定后持续无检测的自愈：用实时帧重跑完整判定（仅一次）。
|
||||
/// 首帧模糊/暗帧导致启发式猜错时，等画面稳定后 oracle 即可分胜负。
|
||||
/// 返回是否执行了重判（随后应重跑 detectRaw 取新结果）。
|
||||
bool retryDecision({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
}) {
|
||||
if (_yuvRetried || !_yuvModeKnown) return false;
|
||||
final (_, yMax, yMean) = _yStats(planes[0], strides[0], width, height);
|
||||
if (yMean < 30 || yMax < 170) return false; // 帧仍不可用
|
||||
_yuvRetried = true;
|
||||
decideYuvChroma(
|
||||
planes: planes, strides: strides, width: width, height: height,
|
||||
force: true);
|
||||
return true;
|
||||
}
|
||||
|
||||
/// 相机（重新）启动后复位，首帧重新判定
|
||||
void resetYuvMode() {
|
||||
_yuvModeKnown = false;
|
||||
_yuvRetried = false;
|
||||
_yuvDiag = '';
|
||||
}
|
||||
|
||||
/// 采样统计 Y 值域：(min, max, mean)，步长 16px 约 3600 样本
|
||||
(int, int, double) _yStats(Uint8List y, int yStride, int w, int h) {
|
||||
var yMin = 255, yMax = 0;
|
||||
var sum = 0, n = 0;
|
||||
for (var j = 0; j < h; j += 16) {
|
||||
final row = j * yStride;
|
||||
for (var i = 0; i < w; i += 16) {
|
||||
final v = y[row + i];
|
||||
if (v < yMin) yMin = v;
|
||||
if (v > yMax) yMax = v;
|
||||
sum += v;
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return (yMin, yMax, sum / n);
|
||||
}
|
||||
|
||||
/// 按指定色序推理一次，返回 (检测数, 最高分, 总分)
|
||||
(int, double, double) _runWithSwap(
|
||||
List<Uint8List> planes, List<int> strides, int w, int h, bool swap) {
|
||||
_yuvSwapChroma = swap;
|
||||
preprocess(planes: planes, strides: strides, width: w, height: h,
|
||||
isBgra: false);
|
||||
_interpreter.run(_input.buffer.asUint8List(), _output);
|
||||
final dets = postprocess();
|
||||
var maxScore = 0.0, sumScore = 0.0;
|
||||
for (final d in dets) {
|
||||
sumScore += d.score;
|
||||
if (d.score > maxScore) maxScore = d.score;
|
||||
}
|
||||
return (dets.length, maxScore, sumScore);
|
||||
}
|
||||
|
||||
bool _brightRegionLeansRed(
|
||||
List<Uint8List> planes, List<int> strides, int w, int h, int yMax) {
|
||||
final y = planes[0];
|
||||
final yStride = strides[0];
|
||||
final uvStride = strides[1];
|
||||
final nv12 = planes.length == 2;
|
||||
final uv = nv12 ? planes[1] : null;
|
||||
// 最亮带（maxY-40 以上），整体偏暗的场景也能拿到足量样本
|
||||
final brightMin = yMax - 40;
|
||||
var blue = 0, red = 0;
|
||||
for (var j = 0; j < h; j += 8) {
|
||||
final yrow = j * yStride;
|
||||
for (var i = 0; i < w; i += 8) {
|
||||
if (y[yrow + i] < brightMin) continue;
|
||||
final cj = j ~/ 2, ci = i ~/ 2;
|
||||
if (nv12) {
|
||||
final c = cj * uvStride + ci * 2;
|
||||
if (c + 1 >= uv!.length) continue;
|
||||
if (uv[c] > 150) blue++;
|
||||
if (uv[c + 1] > 150) red++;
|
||||
} else {
|
||||
final c = cj * uvStride + ci;
|
||||
if (c >= planes[1].length || c >= planes[2].length) continue;
|
||||
if (planes[1][c] > 150) blue++;
|
||||
if (planes[2][c] > 150) red++;
|
||||
}
|
||||
}
|
||||
}
|
||||
// 亮区偏红多于偏蓝 → 当前 U/V 假设反了
|
||||
return red > blue;
|
||||
}
|
||||
|
||||
static double _bl(double a, double b, double c, double d, double fx,
|
||||
double fy) =>
|
||||
(1 - fx) * (1 - fy) * a + fx * (1 - fy) * b +
|
||||
@@ -302,11 +481,39 @@ class TfliteDetector {
|
||||
top: (cy - h / 2).clamp(0.0, 1.0),
|
||||
right: (cx + w / 2).clamp(0.0, 1.0),
|
||||
bottom: (cy + h / 2).clamp(0.0, 1.0),
|
||||
modelId: modelId,
|
||||
modelName: modelName,
|
||||
));
|
||||
}
|
||||
final kept = nms(boxes, iouThreshold);
|
||||
return kept.take(maxDetections).toList();
|
||||
}
|
||||
|
||||
/// 诊断：按指定字节序推理一次，返回 (检测数, 最高分)。
|
||||
/// 用于对比 BGRA/RGBA 两种顺序在同一帧上的检测差异（验证字节序与场景可达性）。
|
||||
(int, double) diagnoseOrder({
|
||||
required List<Uint8List> planes,
|
||||
required List<int> strides,
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
required bool rgbaOrder,
|
||||
}) {
|
||||
preprocess(
|
||||
planes: planes,
|
||||
strides: strides,
|
||||
width: width,
|
||||
height: height,
|
||||
isBgra: isBgra,
|
||||
rgbaOrder: rgbaOrder);
|
||||
_interpreter.run(_input.buffer.asUint8List(), _output);
|
||||
final dets = postprocess();
|
||||
var maxScore = 0.0;
|
||||
for (final d in dets) {
|
||||
if (d.score > maxScore) maxScore = d.score;
|
||||
}
|
||||
return (dets.length, maxScore);
|
||||
}
|
||||
|
||||
void dispose() => _interpreter.close();
|
||||
}
|
||||
|
||||
@@ -34,12 +34,14 @@ class VisualPrior {
|
||||
required int width,
|
||||
required int height,
|
||||
required bool isBgra,
|
||||
required bool rgbaOrder,
|
||||
}) {
|
||||
if (results.isEmpty || width <= 0 || height <= 0) return results;
|
||||
final kept = <DetectionResult>[];
|
||||
for (final r in results) {
|
||||
final lowConfPheasant = r.label == 'pheasant' && r.score < maxScore;
|
||||
if (lowConfPheasant && _reject(r, planes, strides, width, height, isBgra)) {
|
||||
if (lowConfPheasant &&
|
||||
_reject(r, planes, strides, width, height, isBgra, rgbaOrder)) {
|
||||
continue;
|
||||
}
|
||||
kept.add(r);
|
||||
@@ -48,7 +50,7 @@ class VisualPrior {
|
||||
}
|
||||
|
||||
static bool _reject(DetectionResult r, List<Uint8List> planes,
|
||||
List<int> strides, int width, int height, bool isBgra) {
|
||||
List<int> strides, int width, int height, bool isBgra, bool rgbaOrder) {
|
||||
// 位置线索：detectRaw 输出为图像坐标系，centerY 直接可判天空区
|
||||
if (r.centerY < skyTopRatio) return true;
|
||||
|
||||
@@ -63,7 +65,8 @@ class VisualPrior {
|
||||
for (var gx = -2; gx <= 2; gx++) {
|
||||
final px = (cx + gx * halfW / 2).round().clamp(0, width - 1).toInt();
|
||||
final py = (cy + gy * halfH / 2).round().clamp(0, height - 1).toInt();
|
||||
final (r_, g_, b_) = _pixel(planes, strides, px, py, width, height, isBgra);
|
||||
final (r_, g_, b_) =
|
||||
_pixel(planes, strides, px, py, width, height, isBgra, rgbaOrder);
|
||||
total++;
|
||||
final mn = math.min(r_, math.min(g_, b_));
|
||||
final mx = math.max(r_, math.max(g_, b_));
|
||||
@@ -80,14 +83,20 @@ class VisualPrior {
|
||||
}
|
||||
|
||||
/// 读取单像素 RGB（0~255）。
|
||||
/// BGRA 单平面：每像素 4 字节 [b,g,r,a]；
|
||||
/// 8888 单平面按实际字节序取通道：BGRA=[b,g,r,a]（iOS 插件）、
|
||||
/// RGBA=[r,g,b,a]（Android 自写原生通道）——字节序写死会让 Android
|
||||
/// 低分框采样到 R/B 互换的颜色（橙色野鸡身被误判成"蓝色"）整批误杀；
|
||||
/// YUV：y 平面 + 4:2:0 半分辨率 U/V（NV12 交错或 I420 分离）。
|
||||
static (double, double, double) _pixel(List<Uint8List> planes,
|
||||
List<int> strides, int x, int y, int width, int height, bool isBgra) {
|
||||
List<int> strides, int x, int y, int width, int height, bool isBgra,
|
||||
bool rgbaOrder) {
|
||||
if (isBgra) {
|
||||
final src = planes[0];
|
||||
final i = y * strides[0] + x * 4;
|
||||
return (src[i + 2].toDouble(), src[i + 1].toDouble(), src[i].toDouble());
|
||||
final rOff = rgbaOrder ? 0 : 2;
|
||||
final bOff = rgbaOrder ? 2 : 0;
|
||||
return (src[i + rOff].toDouble(), src[i + 1].toDouble(),
|
||||
src[i + bOff].toDouble());
|
||||
}
|
||||
final yy =
|
||||
(planes[0][y * strides[0] + x] - 16.0) * (255.0 / 219.0);
|
||||
|
||||
@@ -6,7 +6,7 @@ import 'package:provider/provider.dart';
|
||||
import '../auth/session_store.dart';
|
||||
|
||||
/// 使用协议页：首次启动强制展示，同意后记录（secure storage），拒绝则退出应用。
|
||||
/// 用途声明：仅限野生动物观察，禁止近距离打扰与非法使用。
|
||||
/// 用途声明：仅限动物观察，禁止近距离打扰与非法使用。
|
||||
class TermsScreen extends StatefulWidget {
|
||||
const TermsScreen({super.key});
|
||||
|
||||
@@ -28,12 +28,12 @@ class _TermsScreenState extends State<TermsScreen> {
|
||||
感谢您使用「视野」动物观察应用。使用本应用前，请仔细阅读并同意以下条款：
|
||||
|
||||
一、用途限制
|
||||
本应用仅用于野生动物观察、记录与科普学习，严禁用于任何非法用途。
|
||||
本应用仅用于动物观察、记录与科普学习，严禁用于任何非法用途。
|
||||
|
||||
二、禁止行为
|
||||
1. 禁止近距离打扰、追逐、驱赶、恐吓野生动物；
|
||||
2. 禁止投喂、诱捕、猎捕、伤害野生动物；
|
||||
3. 禁止利用本应用从事盗猎、野生动物交易或其他违法犯罪活动；
|
||||
1. 禁止近距离打扰、追逐、驱赶、恐吓动物；
|
||||
2. 禁止投喂、诱捕、猎捕、伤害动物；
|
||||
3. 禁止利用本应用从事盗猎、动物交易或其他违法犯罪活动；
|
||||
4. 禁止进入自然保护区禁区、私人领地等未经许可的区域进行观察。
|
||||
|
||||
三、法律合规
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
import 'dart:convert';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:crypto/crypto.dart' show sha256;
|
||||
import 'package:flutter/foundation.dart';
|
||||
import 'package:http/http.dart' as http;
|
||||
import 'package:path_provider/path_provider.dart';
|
||||
|
||||
import '../config/app_config.dart';
|
||||
|
||||
/// 模型目录条目（GET /api/v1/app/update 响应 data.models[]）。
|
||||
/// 服务器发布模型后随版本检查一同下发，App 按目录逐数据集下载/更新。
|
||||
class ModelCatalogItem {
|
||||
final int datasetId;
|
||||
final String datasetName;
|
||||
final String version;
|
||||
final List<String> labels;
|
||||
final int sizeBytes;
|
||||
final String sha256;
|
||||
final String downloadUrl;
|
||||
|
||||
const ModelCatalogItem({
|
||||
required this.datasetId,
|
||||
required this.datasetName,
|
||||
required this.version,
|
||||
required this.labels,
|
||||
required this.sizeBytes,
|
||||
required this.sha256,
|
||||
required this.downloadUrl,
|
||||
});
|
||||
|
||||
factory ModelCatalogItem.fromJson(Map<String, dynamic> j) =>
|
||||
ModelCatalogItem(
|
||||
datasetId: (j['datasetId'] as num?)?.toInt() ?? 0,
|
||||
datasetName: j['datasetName'] as String? ?? '',
|
||||
version: j['version'] as String? ?? '',
|
||||
labels: (j['labels'] as List? ?? const [])
|
||||
.map((e) => e.toString())
|
||||
.toList(),
|
||||
sizeBytes: (j['sizeBytes'] as num?)?.toInt() ?? 0,
|
||||
sha256: j['sha256'] as String? ?? '',
|
||||
downloadUrl: j['downloadUrl'] as String? ?? '',
|
||||
);
|
||||
}
|
||||
|
||||
/// 已就绪模型（字节 + 标签，供推理 worker 加载；含内置资产回退模型）
|
||||
class ModelBundle {
|
||||
final int datasetId;
|
||||
final String datasetName;
|
||||
final String version;
|
||||
final List<String> labels;
|
||||
final Uint8List bytes;
|
||||
|
||||
const ModelBundle({
|
||||
required this.datasetId,
|
||||
required this.datasetName,
|
||||
required this.version,
|
||||
required this.labels,
|
||||
required this.bytes,
|
||||
});
|
||||
}
|
||||
|
||||
/// 模型热更新管理：启动时拉取模型目录（随 /app/update 公开接口下发，无需登录态），
|
||||
/// 按需下载/校验/持久化各数据集模型，供相机页多模型并行推理。
|
||||
///
|
||||
/// 存储：应用私有目录 `models/<datasetId>/`（model.tflite + labels.json + meta.json），
|
||||
/// meta 记录 {version, sha256}，服务器发布新版本时按版本+摘要重下，不重复下载旧模型。
|
||||
class ModelManager extends ChangeNotifier {
|
||||
static final ModelManager instance = ModelManager._();
|
||||
|
||||
final String baseUrl;
|
||||
final http.Client _client;
|
||||
final Future<Directory> Function()? _rootDirOverride;
|
||||
|
||||
List<ModelBundle> _models = const [];
|
||||
bool _ready = false;
|
||||
bool _refreshing = false;
|
||||
String? _error;
|
||||
Future<void>? _inFlight;
|
||||
|
||||
ModelManager._({String? baseUrl, http.Client? client})
|
||||
: this(baseUrl: baseUrl, client: client);
|
||||
|
||||
/// 可注入 baseUrl / client / 存储根目录（单测用）
|
||||
@visibleForTesting
|
||||
ModelManager({
|
||||
String? baseUrl,
|
||||
http.Client? client,
|
||||
Future<Directory> Function()? rootDir,
|
||||
}) : baseUrl = baseUrl ?? AppConfig.apiBaseUrl,
|
||||
_client = client ?? http.Client(),
|
||||
_rootDirOverride = rootDir;
|
||||
|
||||
/// 已就绪模型列表（空 = 无服务器模型，回退内置资产）
|
||||
List<ModelBundle> get models => _models;
|
||||
|
||||
/// 是否成功拉取过目录（即使下载失败也为 true，用于区分"从未联网"与"目录为空"）
|
||||
bool get ready => _ready;
|
||||
|
||||
/// 最近一次同步的错误信息（下载失败/校验失败等；目录为空不算错误）
|
||||
String? get error => _error;
|
||||
|
||||
bool get refreshing => _refreshing;
|
||||
|
||||
/// 模型名摘要（诊断行展示）：内置 / 数据集名×n
|
||||
String get modelsLabel {
|
||||
if (_models.isEmpty) return '内置';
|
||||
return _models.map((m) => m.datasetName).join(',');
|
||||
}
|
||||
|
||||
/// 拉取目录并同步本地模型；并发调用共享同一进行中的刷新。
|
||||
Future<void> refresh() {
|
||||
if (_refreshing) return _inFlight ?? Future.value();
|
||||
_refreshing = true;
|
||||
_inFlight = _doRefresh().whenComplete(() {
|
||||
_refreshing = false;
|
||||
_inFlight = null;
|
||||
notifyListeners();
|
||||
});
|
||||
return _inFlight!;
|
||||
}
|
||||
|
||||
Future<void> _doRefresh() async {
|
||||
try {
|
||||
final res = await _client
|
||||
.get(Uri.parse('$baseUrl/api/v1/app/update'))
|
||||
.timeout(const Duration(seconds: 8));
|
||||
// 服务器 Content-Type 无 charset，http 包默认按 latin1 解码会乱码 → 显式 utf8
|
||||
final body =
|
||||
jsonDecode(utf8.decode(res.bodyBytes)) as Map<String, dynamic>;
|
||||
final data = body['data'] as Map<String, dynamic>? ?? const {};
|
||||
final list = data['models'] as List? ?? const [];
|
||||
final catalog = list
|
||||
.map((e) => ModelCatalogItem.fromJson(e as Map<String, dynamic>))
|
||||
.toList();
|
||||
|
||||
final failed = <String>[];
|
||||
for (final item in catalog) {
|
||||
if (!await _ensureLocal(item)) failed.add(item.datasetName);
|
||||
}
|
||||
await _prune(catalog);
|
||||
_models = await _loadBundles(catalog);
|
||||
_ready = true;
|
||||
_error = failed.isEmpty
|
||||
? null
|
||||
: '模型下载失败:${failed.join(',')}（重试旧模型或稍后再试）';
|
||||
} catch (e) {
|
||||
if (!_ready) _error = '模型目录拉取失败:$e';
|
||||
// 已就绪过则保留旧模型，不覆盖 error（下载级错误优先展示）
|
||||
}
|
||||
}
|
||||
|
||||
/// 保证目录条目在本地可用：meta 匹配且文件在 → 跳过；否则下载并校验 sha256。
|
||||
Future<bool> _ensureLocal(ModelCatalogItem item) async {
|
||||
final dir = await _modelDir(item.datasetId);
|
||||
try {
|
||||
final meta = await _readMeta(dir);
|
||||
final file = File('${dir.path}/model.tflite');
|
||||
if (meta != null &&
|
||||
meta['version'] == item.version &&
|
||||
meta['sha256'] == item.sha256 &&
|
||||
await file.exists()) {
|
||||
return true;
|
||||
}
|
||||
// 版本更新或文件缺失：下载校验（失败重试一次）
|
||||
for (var attempt = 0; attempt < 2; attempt++) {
|
||||
final ok = await _downloadAndVerify(item, dir, file);
|
||||
if (ok) return true;
|
||||
await file.delete().catchError((_) => file);
|
||||
await File('${dir.path}/model.tflite.part')
|
||||
.delete()
|
||||
.catchError((_) => file);
|
||||
}
|
||||
debugPrint('[ModelManager] 下载失败: ${item.datasetName} ${item.version}');
|
||||
return false;
|
||||
} catch (e) {
|
||||
debugPrint('[ModelManager] _ensureLocal ${item.datasetName}: $e');
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
Future<bool> _downloadAndVerify(
|
||||
ModelCatalogItem item, Directory dir, File file) async {
|
||||
final part = File('${file.path}.part');
|
||||
final sink = part.openWrite();
|
||||
try {
|
||||
final res = await _client
|
||||
.send(http.Request('GET', Uri.parse('$baseUrl${item.downloadUrl}')))
|
||||
.timeout(const Duration(minutes: 3));
|
||||
if (res.statusCode != 200) return false;
|
||||
await res.stream.pipe(sink);
|
||||
await sink.close();
|
||||
final bytes = await part.readAsBytes();
|
||||
final hex = sha256.convert(bytes).toString();
|
||||
if (item.sha256.isNotEmpty && hex != item.sha256) {
|
||||
debugPrint('[ModelManager] sha256 不匹配: ${item.datasetName} '
|
||||
'want=${item.sha256} got=$hex');
|
||||
return false;
|
||||
}
|
||||
await part.rename(file.path);
|
||||
await dir.create(recursive: true);
|
||||
await File('${dir.path}/labels.json')
|
||||
.writeAsString(jsonEncode(item.labels));
|
||||
await File('${dir.path}/meta.json').writeAsString(jsonEncode({
|
||||
'version': item.version,
|
||||
'sha256': item.sha256,
|
||||
}));
|
||||
debugPrint('[ModelManager] 已下载 ${item.datasetName} '
|
||||
'${bytes.length}B -> ${file.path}');
|
||||
return true;
|
||||
} catch (e) {
|
||||
await sink.close().catchError((_) {});
|
||||
debugPrint('[ModelManager] 下载异常 ${item.datasetName}: $e');
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/// 清理服务器目录中已下线的数据集模型（不再发布则删本地）
|
||||
Future<void> _prune(List<ModelCatalogItem> catalog) async {
|
||||
final root = await _rootDir();
|
||||
if (!await root.exists()) return;
|
||||
final keep = catalog.map((c) => '${c.datasetId}').toSet();
|
||||
await for (final e in root.list()) {
|
||||
if (e is Directory) {
|
||||
// 目录 URI 末尾带 '/'，pathSegments 末位为空串 → 过滤后取目录名
|
||||
final name = e.uri.pathSegments.where((s) => s.isNotEmpty).last;
|
||||
if (!keep.contains(name)) {
|
||||
await e.delete(recursive: true).catchError((_) => e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Future<List<ModelBundle>> _loadBundles(
|
||||
List<ModelCatalogItem> catalog) async {
|
||||
final bundles = <ModelBundle>[];
|
||||
for (final item in catalog) {
|
||||
try {
|
||||
final dir = await _modelDir(item.datasetId);
|
||||
final file = File('${dir.path}/model.tflite');
|
||||
if (!await file.exists()) continue;
|
||||
final labels = await File('${dir.path}/labels.json').exists()
|
||||
? (jsonDecode(
|
||||
await File('${dir.path}/labels.json').readAsString())
|
||||
as List)
|
||||
.map((e) => e.toString())
|
||||
.toList()
|
||||
: item.labels;
|
||||
bundles.add(ModelBundle(
|
||||
datasetId: item.datasetId,
|
||||
datasetName: item.datasetName,
|
||||
version: item.version,
|
||||
labels: labels,
|
||||
bytes: await file.readAsBytes(),
|
||||
));
|
||||
} catch (e) {
|
||||
debugPrint('[ModelManager] 读取 ${item.datasetName} 失败: $e');
|
||||
}
|
||||
}
|
||||
return bundles;
|
||||
}
|
||||
|
||||
Future<Map<String, dynamic>?> _readMeta(Directory dir) async {
|
||||
final f = File('${dir.path}/meta.json');
|
||||
if (!await f.exists()) return null;
|
||||
try {
|
||||
return jsonDecode(await f.readAsString()) as Map<String, dynamic>;
|
||||
} catch (_) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
Future<Directory> _rootDir() async {
|
||||
if (_rootDirOverride != null) return _rootDirOverride();
|
||||
final support = await getApplicationSupportDirectory();
|
||||
return Directory('${support.path}/models');
|
||||
}
|
||||
|
||||
Future<Directory> _modelDir(int datasetId) async {
|
||||
final root = await _rootDir();
|
||||
final dir = Directory('${root.path}/$datasetId');
|
||||
await dir.create(recursive: true);
|
||||
return dir;
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:flutter/services.dart';
|
||||
import 'package:provider/provider.dart';
|
||||
|
||||
import 'models.dart';
|
||||
@@ -73,6 +74,24 @@ class _PaywallScreenState extends State<PaywallScreen> {
|
||||
}
|
||||
},
|
||||
),
|
||||
const SizedBox(height: 12),
|
||||
_PayButton(
|
||||
label: '联系管理员',
|
||||
icon: Icons.headset_mic,
|
||||
color: Colors.grey.shade600,
|
||||
enabled: true,
|
||||
onTap: () async {
|
||||
await Clipboard.setData(
|
||||
const ClipboardData(text: 'wenwu901'),
|
||||
);
|
||||
if (!context.mounted) return;
|
||||
ScaffoldMessenger.of(context).showSnackBar(
|
||||
const SnackBar(
|
||||
content: Text('已复制管理员微信号，请使用微信添加联系人'),
|
||||
),
|
||||
);
|
||||
},
|
||||
),
|
||||
if (vm.state.state == PayState.failed)
|
||||
Padding(
|
||||
padding: const EdgeInsets.only(top: 16),
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
import 'package:flutter/services.dart';
|
||||
|
||||
/// 安装进度事件（原生 PackageInstaller 会话回调经 EventChannel 回传）
|
||||
class InstallEvent {
|
||||
/// progress / finished / failed
|
||||
final String event;
|
||||
|
||||
/// event=progress 时的安装进度 0-100
|
||||
final int progress;
|
||||
|
||||
/// event=finished 时是否安装成功
|
||||
final bool? success;
|
||||
|
||||
/// event=failed 时的错误描述
|
||||
final String? error;
|
||||
|
||||
const InstallEvent({
|
||||
required this.event,
|
||||
this.progress = 0,
|
||||
this.success,
|
||||
this.error,
|
||||
});
|
||||
}
|
||||
|
||||
/// App 内安装 APK：原生侧 PackageInstaller 会话安装（InstallerChannel.kt）
|
||||
class ApkInstaller {
|
||||
static const _method = MethodChannel('observer/installer');
|
||||
static const _progress = EventChannel('observer/installer/progress');
|
||||
|
||||
/// 提交安装。返回 installing（已进入安装流程）/ permission_required
|
||||
/// （未允许「安装未知应用」，原生侧已拉起系统设置页）。
|
||||
static Future<String> install(String path) =>
|
||||
_method.invokeMethod<String>('install', {'path': path}).then(
|
||||
(v) => v ?? 'installing',
|
||||
);
|
||||
|
||||
/// 安装进度流：progress(0-100) → finished(success) / failed(error)
|
||||
static Stream<InstallEvent> progress() {
|
||||
return _progress.receiveBroadcastStream().map((e) {
|
||||
final m = e as Map;
|
||||
return InstallEvent(
|
||||
event: m['event'] as String? ?? '',
|
||||
progress: (m['progress'] as num?)?.toInt() ?? 0,
|
||||
success: m['success'] as bool?,
|
||||
error: m['error'] as String?,
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
import 'dart:convert';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:http/http.dart' as http;
|
||||
|
||||
import '../config/app_config.dart';
|
||||
|
||||
/// App 版本更新信息（GET /api/v1/app/update 响应 data；无记录时为空信息）
|
||||
class AppUpdateInfo {
|
||||
final String version;
|
||||
final String notes;
|
||||
|
||||
const AppUpdateInfo({required this.version, required this.notes});
|
||||
|
||||
/// 服务器无版本记录时返回空信息，调用方视为无需更新
|
||||
bool get isEmpty => version.isEmpty;
|
||||
}
|
||||
|
||||
/// 启动时版本更新检查：仅 Android 检查；服务器版本高于本地版本即强制更新
|
||||
/// （无普通/强制之分）。
|
||||
class UpdateChecker {
|
||||
final String baseUrl;
|
||||
final http.Client _client;
|
||||
|
||||
UpdateChecker({String? baseUrl, http.Client? client})
|
||||
: baseUrl = baseUrl ?? AppConfig.apiBaseUrl,
|
||||
_client = client ?? http.Client();
|
||||
|
||||
/// 下载地址：固定静态路径（管理端上传的 APK 覆盖保存为固定文件）
|
||||
static String downloadUrl({String? baseUrl}) =>
|
||||
'${baseUrl ?? AppConfig.apiBaseUrl}/download/observer-latest.apk';
|
||||
|
||||
/// 拉取服务器最新版本；非 Android 或网络异常时返回空信息，不阻塞启动
|
||||
Future<AppUpdateInfo> fetch() async {
|
||||
if (!Platform.isAndroid) {
|
||||
return const AppUpdateInfo(version: '', notes: '');
|
||||
}
|
||||
try {
|
||||
final res = await _client
|
||||
.get(Uri.parse('$baseUrl/api/v1/app/update'))
|
||||
.timeout(const Duration(seconds: 8));
|
||||
// 服务器 Content-Type 无 charset，http 包默认按 latin1 解码会乱码 → 显式 utf8
|
||||
final body =
|
||||
jsonDecode(utf8.decode(res.bodyBytes)) as Map<String, dynamic>;
|
||||
final data = body['data'] as Map<String, dynamic>? ?? const {};
|
||||
return AppUpdateInfo(
|
||||
version: data['version'] as String? ?? '',
|
||||
notes: data['notes'] as String? ?? '',
|
||||
);
|
||||
} catch (_) {
|
||||
return const AppUpdateInfo(version: '', notes: '');
|
||||
}
|
||||
}
|
||||
|
||||
/// 是否需要更新：服务器版本高于「已装版本与已确认接受版本」中的较大者。
|
||||
/// APK 版本号不递增时，用户点过「立即更新」后 accepted 追上服务器版本，
|
||||
/// 已更新完成再次启动也不会反复提示。
|
||||
static bool needsUpdate(String server, String installed, String accepted) {
|
||||
if (server.isEmpty || installed.isEmpty) return false;
|
||||
return isNewer(server, installed) || isNewer(server, accepted);
|
||||
}
|
||||
|
||||
/// 语义化版本号比较：a > b 返回 true。按数字段比较（1.10.0 > 1.9.9），
|
||||
/// 任一版本号解析失败时视为相等（不触发更新）。
|
||||
static bool isNewer(String a, String b) {
|
||||
final pa = _parse(a);
|
||||
final pb = _parse(b);
|
||||
if (pa == null || pb == null) return false;
|
||||
for (var i = 0; i < 3; i++) {
|
||||
if (pa[i] != pb[i]) return pa[i] > pb[i];
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
static List<int>? _parse(String v) {
|
||||
final parts = v.split('.');
|
||||
if (parts.length != 3) return null;
|
||||
final nums = <int>[];
|
||||
for (final p in parts) {
|
||||
final n = int.tryParse(p);
|
||||
if (n == null) return null;
|
||||
nums.add(n);
|
||||
}
|
||||
return nums;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,238 @@
|
||||
import 'dart:async';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:http/http.dart' as http;
|
||||
import 'package:url_launcher/url_launcher.dart';
|
||||
|
||||
import 'installer.dart';
|
||||
|
||||
/// 强制更新页：检测到新版本时的全屏阻塞页。
|
||||
/// PopScope 禁返回（Android 系统返回 / iOS 边缘滑动均不可退出）。
|
||||
/// 点「立即更新」在 App 内流式下载 APK（显示下载进度）→ PackageInstaller
|
||||
/// 会话安装（显示安装进度），不再跳浏览器。
|
||||
/// 点击更新时回调 onUpdateAccepted（调用方持久化服务器版本号，
|
||||
/// 使 APK 版本号不递增时也不反复提示）。
|
||||
class UpdateScreen extends StatefulWidget {
|
||||
final String version;
|
||||
final String url;
|
||||
final String notes;
|
||||
final VoidCallback? onUpdateAccepted;
|
||||
|
||||
const UpdateScreen({
|
||||
super.key,
|
||||
required this.version,
|
||||
required this.url,
|
||||
this.notes = '',
|
||||
this.onUpdateAccepted,
|
||||
});
|
||||
|
||||
@override
|
||||
State<UpdateScreen> createState() => _UpdateScreenState();
|
||||
}
|
||||
|
||||
enum _Stage { idle, downloading, installing, finished, failed }
|
||||
|
||||
class _UpdateScreenState extends State<UpdateScreen> {
|
||||
final http.Client _client = http.Client();
|
||||
final File _apkFile = File('${Directory.systemTemp.path}/observer-latest.apk');
|
||||
final File _apkPart =
|
||||
File('${Directory.systemTemp.path}/observer-latest.apk.part');
|
||||
|
||||
_Stage _stage = _Stage.idle;
|
||||
|
||||
/// 进度百分比 0-100；null = 总量未知（不确定进度条）
|
||||
double? _progress;
|
||||
String? _message;
|
||||
StreamSubscription<InstallEvent>? _installSub;
|
||||
|
||||
@override
|
||||
void dispose() {
|
||||
_installSub?.cancel();
|
||||
_client.close();
|
||||
super.dispose();
|
||||
}
|
||||
|
||||
Future<void> _launch() async {
|
||||
if (_stage == _Stage.downloading ||
|
||||
_stage == _Stage.installing ||
|
||||
_stage == _Stage.finished) {
|
||||
return;
|
||||
}
|
||||
setState(() {
|
||||
_stage = _Stage.idle;
|
||||
_message = null;
|
||||
});
|
||||
widget.onUpdateAccepted?.call();
|
||||
if (!Platform.isAndroid) {
|
||||
// 更新检查本就仅 Android 触发，这里兜底非 Android 走浏览器
|
||||
final uri = Uri.tryParse(widget.url);
|
||||
if (uri == null) return;
|
||||
try {
|
||||
await launchUrl(uri, mode: LaunchMode.externalApplication);
|
||||
} catch (_) {}
|
||||
return;
|
||||
}
|
||||
// APK 已下载完成（下载是原子落盘，.part 改名后文件才存在）→ 跳过下载直接安装
|
||||
if (_apkFile.existsSync()) {
|
||||
await _install();
|
||||
return;
|
||||
}
|
||||
await _download();
|
||||
}
|
||||
|
||||
Future<void> _download() async {
|
||||
setState(() {
|
||||
_stage = _Stage.downloading;
|
||||
_progress = 0;
|
||||
});
|
||||
try {
|
||||
if (_apkPart.existsSync()) _apkPart.deleteSync();
|
||||
final res = await _client.send(http.Request('GET', Uri.parse(widget.url)));
|
||||
if (res.statusCode != 200) {
|
||||
throw HttpException('HTTP ${res.statusCode}');
|
||||
}
|
||||
final total = res.contentLength ?? -1;
|
||||
final sink = _apkPart.openWrite();
|
||||
var received = 0;
|
||||
await for (final chunk in res.stream) {
|
||||
sink.add(chunk);
|
||||
received += chunk.length;
|
||||
if (mounted && total > 0) {
|
||||
setState(() => _progress = received / total * 100);
|
||||
}
|
||||
}
|
||||
await sink.close();
|
||||
// 原子落盘：下载完成后才重命名为正式文件，避免残留半包被当成完整 APK
|
||||
_apkPart.renameSync(_apkFile.path);
|
||||
if (!mounted) return;
|
||||
await _install();
|
||||
} catch (_) {
|
||||
if (_apkPart.existsSync()) _apkPart.deleteSync();
|
||||
if (!mounted) return;
|
||||
setState(() {
|
||||
_stage = _Stage.failed;
|
||||
_progress = null;
|
||||
_message = '下载失败，请检查网络后重试';
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
Future<void> _install() async {
|
||||
setState(() {
|
||||
_stage = _Stage.installing;
|
||||
_progress = 0;
|
||||
_message = null;
|
||||
});
|
||||
await _installSub?.cancel();
|
||||
_installSub = ApkInstaller.progress().listen((e) {
|
||||
if (!mounted) return;
|
||||
switch (e.event) {
|
||||
case 'progress':
|
||||
setState(() => _progress = e.progress.toDouble());
|
||||
break;
|
||||
case 'finished':
|
||||
final ok = e.success == true;
|
||||
setState(() {
|
||||
_stage = ok ? _Stage.finished : _Stage.failed;
|
||||
_message = ok ? '安装完成，请从桌面打开新版应用' : '安装失败，请重试';
|
||||
});
|
||||
break;
|
||||
case 'failed':
|
||||
setState(() {
|
||||
_stage = _Stage.failed;
|
||||
_message = e.error ?? '安装失败，请重试';
|
||||
});
|
||||
break;
|
||||
}
|
||||
}, onError: (Object _) {
|
||||
if (!mounted) return;
|
||||
setState(() {
|
||||
_stage = _Stage.failed;
|
||||
_message = '安装失败，请重试';
|
||||
});
|
||||
});
|
||||
final result = await ApkInstaller.install(_apkFile.path);
|
||||
if (!mounted) return;
|
||||
if (result == 'permission_required') {
|
||||
// 原生侧已拉起系统设置页；APK 已缓存，用户开启后返回再点直达安装
|
||||
setState(() {
|
||||
_stage = _Stage.failed;
|
||||
_message = '请在系统设置中允许「安装未知应用」，返回后再次点击「立即更新」（APK 已缓存，无需重新下载）';
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@override
|
||||
Widget build(BuildContext context) {
|
||||
final theme = Theme.of(context);
|
||||
final busy = _stage == _Stage.downloading || _stage == _Stage.installing;
|
||||
final progressText = _stage == _Stage.downloading
|
||||
? (_progress == null ? '下载中…' : '下载中 ${_progress!.round()}%')
|
||||
: (_progress == null ? '安装中…' : '安装中 ${_progress!.round()}%');
|
||||
|
||||
return PopScope(
|
||||
canPop: false,
|
||||
child: Scaffold(
|
||||
body: Center(
|
||||
child: Padding(
|
||||
padding: const EdgeInsets.all(32),
|
||||
child: Column(
|
||||
mainAxisSize: MainAxisSize.min,
|
||||
children: [
|
||||
Icon(Icons.system_update_alt,
|
||||
size: 64, color: theme.colorScheme.primary),
|
||||
const SizedBox(height: 16),
|
||||
Text('发现新版本 ${widget.version}',
|
||||
style: theme.textTheme.headlineSmall),
|
||||
const SizedBox(height: 12),
|
||||
if (widget.notes.isNotEmpty)
|
||||
Text(widget.notes,
|
||||
textAlign: TextAlign.center,
|
||||
style: const TextStyle(height: 1.6)),
|
||||
const SizedBox(height: 24),
|
||||
if (busy) ...[
|
||||
LinearProgressIndicator(
|
||||
value: _progress == null ? null : _progress! / 100,
|
||||
minHeight: 6,
|
||||
),
|
||||
const SizedBox(height: 12),
|
||||
Text(progressText),
|
||||
const SizedBox(height: 24),
|
||||
],
|
||||
FilledButton.icon(
|
||||
onPressed: busy || _stage == _Stage.finished
|
||||
? null
|
||||
: _launch,
|
||||
icon: const Icon(Icons.download),
|
||||
label: Text(_stage == _Stage.finished ? '已完成' : '立即更新'),
|
||||
style: FilledButton.styleFrom(
|
||||
minimumSize: const Size(200, 48),
|
||||
textStyle: const TextStyle(fontSize: 16),
|
||||
),
|
||||
),
|
||||
if (_message != null) ...[
|
||||
const SizedBox(height: 12),
|
||||
Text(
|
||||
_message!,
|
||||
textAlign: TextAlign.center,
|
||||
style: TextStyle(
|
||||
height: 1.5,
|
||||
color: _stage == _Stage.failed
|
||||
? theme.colorScheme.error
|
||||
: Colors.green,
|
||||
),
|
||||
),
|
||||
],
|
||||
const SizedBox(height: 12),
|
||||
Text('不更新将无法继续使用',
|
||||
style: theme.textTheme.bodySmall
|
||||
?.copyWith(color: theme.colorScheme.error)),
|
||||
],
|
||||
),
|
||||
),
|
||||
),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -90,7 +90,7 @@ packages:
|
||||
source: hosted
|
||||
version: "0.11.4"
|
||||
camera_android_camerax:
|
||||
dependency: "direct main"
|
||||
dependency: transitive
|
||||
description:
|
||||
name: camera_android_camerax
|
||||
sha256: "8516fe308bc341a5067fb1a48edff0ddfa57c0d3cdcc9dbe7ceca3ba119e2577"
|
||||
@@ -162,7 +162,7 @@ packages:
|
||||
source: hosted
|
||||
version: "0.3.5+4"
|
||||
crypto:
|
||||
dependency: transitive
|
||||
dependency: "direct main"
|
||||
description:
|
||||
name: crypto
|
||||
sha256: c8ea0233063ba03258fbcf2ca4d6dadfefe14f02fab57702265467a19f27fadf
|
||||
@@ -497,7 +497,7 @@ packages:
|
||||
source: hosted
|
||||
version: "3.0.0"
|
||||
package_info_plus:
|
||||
dependency: transitive
|
||||
dependency: "direct main"
|
||||
description:
|
||||
name: package_info_plus
|
||||
sha256: "468c26b4254ab01979fa5e4a98cb343ea3631b9acee6f21028997419a80e1a20"
|
||||
@@ -521,7 +521,7 @@ packages:
|
||||
source: hosted
|
||||
version: "1.9.1"
|
||||
path_provider:
|
||||
dependency: transitive
|
||||
dependency: "direct main"
|
||||
description:
|
||||
name: path_provider
|
||||
sha256: a7f4874f987173da295a61c181b8ee71dab59b332a486b391babf26a1b884825
|
||||
@@ -765,6 +765,70 @@ packages:
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "1.4.0"
|
||||
url_launcher:
|
||||
dependency: "direct main"
|
||||
description:
|
||||
name: url_launcher
|
||||
sha256: f6a7e5c4835bb4e3026a04793a4199ca2d14c739ec378fdfe23fc8075d0439f8
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "6.3.2"
|
||||
url_launcher_android:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_android
|
||||
sha256: b413d49b73867ac08dd2f9890efd3cc11f2a0e577618d50843440a1fb3776c32
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "6.3.32"
|
||||
url_launcher_ios:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_ios
|
||||
sha256: "580fe5dfb51671ae38191d316e027f6b76272b026370708c2d898799750a02b0"
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "6.4.1"
|
||||
url_launcher_linux:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_linux
|
||||
sha256: d5e14138b3bc193a0f63c10a53c94b91d399df0512b1f29b94a043db7482384a
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "3.2.2"
|
||||
url_launcher_macos:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_macos
|
||||
sha256: "368adf46f71ad3c21b8f06614adb38346f193f3a59ba8fe9a2fd74133070ba18"
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "3.2.5"
|
||||
url_launcher_platform_interface:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_platform_interface
|
||||
sha256: "552f8a1e663569be95a8190206a38187b531910283c3e982193e4f2733f01029"
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "2.3.2"
|
||||
url_launcher_web:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_web
|
||||
sha256: "85c81589622fbc87c1c683aaea164d3604a7777495a79d91e39ffcdec39ddb34"
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "2.4.3"
|
||||
url_launcher_windows:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: url_launcher_windows
|
||||
sha256: "712c70ab1b99744ff066053cbe3e80c73332b38d46e5e945c98689b2e66fc15f"
|
||||
url: "https://pub.flutter-io.cn"
|
||||
source: hosted
|
||||
version: "3.1.5"
|
||||
uuid:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
name: observer
|
||||
description: "视野 - 野生动物实时识别 (野鸡/生境), YOLOv8 + 充值付费"
|
||||
description: "视野 - 动物实时识别 (野鸡/生境), YOLOv8 + 充值付费"
|
||||
publish_to: 'none'
|
||||
|
||||
version: 1.0.0+1
|
||||
version: 1.0.4+5
|
||||
|
||||
environment:
|
||||
sdk: ^3.12.2
|
||||
@@ -11,8 +11,10 @@ dependencies:
|
||||
flutter:
|
||||
sdk: flutter
|
||||
cupertino_icons: ^1.0.8
|
||||
# Android 已改用自写原生相机通道（android/ 下 CameraChannel.kt，分析帧
|
||||
# 原生侧旋转成竖屏后回传），不再依赖 camera_android_camerax 插件；
|
||||
# camera + camera_avfoundation 仅用于 iOS 路径
|
||||
camera: ^0.11.0
|
||||
camera_android_camerax: ^0.6.6
|
||||
camera_avfoundation: ^0.9.17
|
||||
tflite_flutter: ^0.11.0
|
||||
flutter_secure_storage: ^9.2.0
|
||||
@@ -25,6 +27,11 @@ dependencies:
|
||||
http: ^1.2.0
|
||||
cupertino_http: ^3.0.2
|
||||
wakelock_plus: ^1.2.0
|
||||
package_info_plus: ^9.0.1
|
||||
url_launcher: ^6.3.2
|
||||
# 模型热更新：多模型下载（crypto 校验 sha256；path_provider 取应用私有目录持久化）
|
||||
crypto: ^3.0.0
|
||||
path_provider: ^2.1.4
|
||||
|
||||
# 微信/支付宝原生 SDK 配置（占位值，与 lib/config/app_config.dart 一致；接入真实支付时替换。
|
||||
# 注意：fluwx 的 universal_link 占位符会被其 pod 脚本注入 Associated Domains，
|
||||
|
||||
@@ -0,0 +1,141 @@
|
||||
import 'dart:convert';
|
||||
import 'dart:io';
|
||||
|
||||
import 'package:crypto/crypto.dart' show sha256;
|
||||
import 'package:flutter_test/flutter_test.dart';
|
||||
import 'package:http/http.dart' as http;
|
||||
import 'package:http/testing.dart';
|
||||
import 'package:observer/models/model_manager.dart';
|
||||
|
||||
const _modelBytes = [1, 2, 3, 4, 5, 6, 7, 8];
|
||||
|
||||
String _shaHex(List<int> bytes) => sha256.convert(bytes).toString();
|
||||
|
||||
Map<String, dynamic> _catalog(List<Map<String, dynamic>> models) =>
|
||||
{'code': 0, 'message': 'ok', 'data': {'version': '0.0.2', 'notes': '', 'models': models}};
|
||||
|
||||
Map<String, dynamic> _item({String version = 'v1.0.0', String sha = ''}) => {
|
||||
'datasetId': 7,
|
||||
'datasetName': '野鸡数据集',
|
||||
'version': version,
|
||||
'labels': ['pheasant', 'suspect'],
|
||||
'sizeBytes': _modelBytes.length,
|
||||
'sha256': sha.isEmpty ? _shaHex(_modelBytes) : sha,
|
||||
'downloadUrl': '/download/models/野鸡数据集/latest.tflite',
|
||||
};
|
||||
|
||||
void main() {
|
||||
late Directory root;
|
||||
late int downloadHits;
|
||||
|
||||
setUp(() async {
|
||||
root = await Directory.systemTemp.createTemp('model-manager-test');
|
||||
downloadHits = 0;
|
||||
});
|
||||
|
||||
tearDown(() => root.delete(recursive: true));
|
||||
|
||||
ModelManager manager(MockClient client) => ModelManager(
|
||||
baseUrl: 'http://test.local',
|
||||
client: client,
|
||||
rootDir: () async => root,
|
||||
);
|
||||
|
||||
MockClient client(List<Map<String, dynamic>> models) => MockClient((req) async {
|
||||
if (req.url.path == '/api/v1/app/update') {
|
||||
// Response(String) 默认 latin1 编码，中文数据集名会抛异常 → 用 bytes
|
||||
return http.Response.bytes(utf8.encode(jsonEncode(_catalog(models))), 200);
|
||||
}
|
||||
if (req.url.path.startsWith('/download/models/')) {
|
||||
downloadHits++;
|
||||
return http.Response.bytes(_modelBytes, 200);
|
||||
}
|
||||
return http.Response('not found', 404);
|
||||
});
|
||||
|
||||
test('首次拉取：下载模型并落盘（model/labels/meta）', () async {
|
||||
final m = manager(client([_item()]));
|
||||
await m.refresh();
|
||||
|
||||
expect(m.ready, isTrue);
|
||||
expect(m.error, isNull);
|
||||
expect(m.models.length, 1);
|
||||
expect(m.models.first.datasetName, '野鸡数据集');
|
||||
expect(m.models.first.bytes, _modelBytes);
|
||||
expect(downloadHits, 1);
|
||||
|
||||
final dir = Directory('${root.path}/7');
|
||||
expect(await File('${dir.path}/model.tflite').exists(), isTrue);
|
||||
expect(await File('${dir.path}/labels.json').exists(), isTrue);
|
||||
expect(await File('${dir.path}/meta.json').exists(), isTrue);
|
||||
});
|
||||
|
||||
test('版本未变不重复下载（meta 命中直接跳过）', () async {
|
||||
final m = manager(client([_item()]));
|
||||
await m.refresh();
|
||||
expect(downloadHits, 1);
|
||||
|
||||
await m.refresh();
|
||||
expect(downloadHits, 1, reason: 'meta 匹配应跳过下载');
|
||||
expect(m.models.length, 1);
|
||||
});
|
||||
|
||||
test('版本更新触发重新下载', () async {
|
||||
final m = manager(client([_item()]));
|
||||
await m.refresh();
|
||||
expect(downloadHits, 1);
|
||||
|
||||
await m.refresh();
|
||||
expect(downloadHits, 1);
|
||||
|
||||
// 发布新版本：再次刷新应重下
|
||||
await m.refresh();
|
||||
expect(downloadHits, 1);
|
||||
// 上面三次同一版本，重新构造带新版本的 manager（同一 root）
|
||||
final m2 = manager(client([_item(version: 'v2.0.0')]));
|
||||
await m2.refresh();
|
||||
expect(downloadHits, 2);
|
||||
expect(m2.models.first.version, 'v2.0.0');
|
||||
});
|
||||
|
||||
test('sha256 不匹配：重试后失败，保留旧模型并报错', () async {
|
||||
// 第一次下载成功（sha 匹配）
|
||||
final m1 = manager(client([_item()]));
|
||||
await m1.refresh();
|
||||
expect(m1.models.length, 1);
|
||||
|
||||
// 服务器 sha 与文件不符（被篡改/损坏）→ 下载校验失败
|
||||
final bad = _item(version: 'v3.0.0');
|
||||
bad['sha256'] = _shaHex([9, 9, 9]);
|
||||
final m2 = manager(client([bad]));
|
||||
await m2.refresh();
|
||||
|
||||
expect(m2.models.length, 0, reason: '校验失败的模型不应加载');
|
||||
expect(m2.error, isNotNull);
|
||||
expect(m2.error, contains('野鸡数据集'));
|
||||
});
|
||||
|
||||
test('目录下线：清理本地并清空模型', () async {
|
||||
final m = manager(client([_item()]));
|
||||
await m.refresh();
|
||||
expect(m.models.length, 1);
|
||||
expect(await Directory('${root.path}/7').exists(), isTrue);
|
||||
|
||||
final m2 = manager(client([]));
|
||||
await m2.refresh();
|
||||
expect(m2.models, isEmpty);
|
||||
expect(await Directory('${root.path}/7').exists(), isFalse,
|
||||
reason: '下线的数据集模型目录应被清理');
|
||||
});
|
||||
|
||||
test('目录接口异常：不覆盖已有就绪状态', () async {
|
||||
final m = manager(client([_item()]));
|
||||
await m.refresh();
|
||||
expect(m.ready, isTrue);
|
||||
|
||||
final broken = manager(MockClient((_) async => http.Response('boom', 500)));
|
||||
await broken.refresh();
|
||||
expect(broken.ready, isFalse);
|
||||
expect(broken.error, isNotNull);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import 'package:observer/detection/detection_result.dart';
|
||||
import 'package:observer/detection/detector_worker.dart';
|
||||
import 'package:flutter_test/flutter_test.dart';
|
||||
|
||||
DetectionResult box(String label, double score, double x, double y,
|
||||
{int modelId = -1, String modelName = ''}) =>
|
||||
DetectionResult(
|
||||
label: label,
|
||||
score: score,
|
||||
left: x,
|
||||
top: y,
|
||||
right: x + 0.1,
|
||||
bottom: y + 0.1,
|
||||
modelId: modelId,
|
||||
modelName: modelName,
|
||||
);
|
||||
|
||||
void main() {
|
||||
test('不同模型同标签重复框：NMS 去重取高分', () {
|
||||
// 野鸡模型与野兔模型都检出了同一只"野鸡"（不同模型对同一目标的重复框）
|
||||
final all = [
|
||||
box('pheasant', 0.18, 0.3, 0.3, modelId: 1, modelName: '野鸡模型'),
|
||||
box('pheasant', 0.55, 0.31, 0.3, modelId: 2, modelName: '野兔模型'),
|
||||
];
|
||||
final merged = mergeAcrossModels(all, 0.45);
|
||||
expect(merged.length, 1);
|
||||
expect(merged.first.score, 0.55);
|
||||
expect(merged.first.modelName, '野兔模型');
|
||||
});
|
||||
|
||||
test('不同类别互不压制', () {
|
||||
final all = [
|
||||
box('pheasant', 0.3, 0.5, 0.5, modelId: 1),
|
||||
box('hare', 0.7, 0.5, 0.5, modelId: 2), // 同位置但不同类别
|
||||
];
|
||||
final merged = mergeAcrossModels(all, 0.45);
|
||||
expect(merged.length, 2);
|
||||
});
|
||||
|
||||
test('同模型内部与跨模型合并一致：远处不重叠保留', () {
|
||||
final all = [
|
||||
box('pheasant', 0.2, 0.1, 0.1, modelId: 1, modelName: '野鸡模型'),
|
||||
box('pheasant', 0.3, 0.8, 0.8, modelId: 1, modelName: '野鸡模型'),
|
||||
];
|
||||
final merged = mergeAcrossModels(all, 0.45);
|
||||
expect(merged.length, 2);
|
||||
expect(merged.first.score, 0.3); // 按分排序
|
||||
});
|
||||
|
||||
test('单条结果原样返回', () {
|
||||
final single = [box('suspect', 0.11, 0.2, 0.2)];
|
||||
final merged = mergeAcrossModels(single, 0.45);
|
||||
expect(identical(merged, single), isTrue);
|
||||
});
|
||||
}
|
||||
Reference in New Issue
Block a user