feat(ffmpeg): add ffmpeg-core.wasm for video processing capabilities

.github/copilot-instructions.md

@@ -0,0 +1,226 @@
+# XSH Assistant 编码指南
+
+## 项目概览
+
+**xsh-assistant** 是一个基于 Nuxt 3 的 AI 驱动内容生成平台，专注于数字内容创作（微课视频、虚拟讲师、绿幕合成等）。核心特性：
+
+- **前端框架**: Nuxt 3 + Vue 3 + TypeScript + Tailwind CSS + Radix Vue UI
+- **状态管理**: Pinia（带持久化）
+- **媒体处理**: FFmpeg WASM（客户端视频处理）、WebAV（视频剪辑）
+- **API 集成**: 统一的 `useFetchWrapped` 包装器，所有请求都通过 `API_BASE` 代理（`https://service1.fenshenzhike.com/`）
+- **部署模式**: SPA（SSR=false）
+
+## 核心架构模式
+
+### 1. Composables 设计（Pinia + 自定义 Composables）
+
+**状态管理采用分层设计**：
+
+```
+Pinia Stores（持久化状态）
+├── useLoginState: 用户认证、token、个人资料
+├── useHistory: AIGC 会话、聊天历史
+└── useTourState: 新手引导状态
+
+业务 Composables（无状态或短生命周期）
+├── useFetchWrapped: API 请求包装（自动添加 token、user_id）
+├── useLLM: LLM API 调用（Spark 模型集成）
+├── useFFmpeg: FFmpeg WASM 单例管理
+├── useVideoBackgroundCompositing: 视频合成（数字人+背景）
+├── useVideoSubtitleEmbedding: 字幕嵌入
+└── useDownload: 文件下载管理
+```
+
+**关键模式**：
+- **Pinia stores 必须是单一实例**，通过 `storeToRefs()` 获取响应式引用
+- **API 请求必须通过 `useFetchWrapped`** 来自动处理认证头（token/user_id）
+- **FFmpeg 采用单例模式**（`useFFmpeg()` 返回全局加载的实例），避免重复初始化
+
+### 2. API 请求模式
+
+所有 API 请求使用统一的 `useFetchWrapped` 包装器：
+
+```typescript
+// 基础签名
+useFetchWrapped<RequestType, ResponseType>(action: string, payload?: RequestType, options?: FetchOptions)
+
+// 请求格式示例（来自 useHistory）
+useFetchWrapped<AuthedRequest, BaseResponse<resp.xxx>>(
+  'App.User_User.CheckSession',  // action 作为查询参数 ?s=
+  { token: loginState.token, user_id: loginState.user.id, ...payload },
+  { method: 'POST' }  // 默认 POST
+)
+```
+
+**约定**：
+- **每个请求必须包含** `token` 和 `user_id`（来自 `useLoginState`）
+- **响应结构统一**: `BaseResponse<T>` 包含 `ret: number` 状态码和 `data: T` 数据
+- **API_BASE 在 `nuxt.config.ts` 中定义**，所有请求都相对于此 URL
+
+### 3. 媒体处理架构
+
+#### FFmpeg 初始化流程
+- **单例加载**: 首次调用 `useFFmpeg()` 时初始化，后续复用缓存的实例
+- **WASM 资源加载**: 从 CDN（`cdn.jsdelivr.net`）加载 FFmpeg core、wasm、worker
+- **错误恢复**: 调用 `cleanupFFmpeg()` 清理资源并重置单例
+
+#### 视频合成流程（核心用例）
+```
+输入: 透明通道视频 (WebM) + 背景图 (PNG/File)
+  ↓
+1. 获取背景图尺寸
+2. 计算等比缩放到 720P
+3. 加载文件到 FFmpeg vFS
+4. 执行 FFmpeg 滤镜链：
+   - 背景: scale → ${outputWidth}x${outputHeight}
+   - 视频: scale → ${outputWidth}x${outputHeight} (保留 alpha)
+   - overlay: 视频叠加到背景（format=auto）
+5. 使用 VP9 编码（支持 alpha）+ Opus 音频编码
+6. 返回 Blob → 可直接上传或本地预览
+```
+
+### 4. UI 组件架构
+
+#### 内置组件库（`components/uni/`）
+自定义包装组件，提供统一 API：
+- `UniButton`: 按钮 + loading 状态
+- `UniInput`/`UniTextArea`: 表单输入
+- `UniSelect`: 下拉选择
+- `UniMessage`: 全局消息通知（通过 provide/inject）
+- `UniCopyable`: 可复制文本
+
+**消息通知用法**：
+```typescript
+const toast = useToast()  // Radix Vue 的 Toast（顶部通知）
+// 或从 provide 注入
+const messageApi = inject('uni-message')
+messageApi.success('操作成功')
+messageApi.error('操作失败', 5000)
+```
+
+#### Radix Vue + Nuxt UI 集成
+- 使用 Radix Vue for 基础组件（button, dialog, select）
+- Nuxt UI 用于高级组件 + 主题管理
+- **颜色方案**: primary='indigo', gray='neutral'，详见 `app.config.ts`
+
+### 5. 路由与页面结构
+
+**目录映射**：
+```
+pages/
+├── generation.vue (导航枢纽)
+└── aigc/
+    ├── chat/index.vue (聊天页，支持多 LLM 模型)
+    ├── draw/index.vue (绘图生成)
+    └── generation/
+        ├── course.vue (微课生成)
+        ├── green-screen.vue (绿幕视频)
+        ├── avatar-models.vue (数字讲师)
+        ├── materials.vue (片头片尾)
+        ├── ppt-templates.vue (PPT 库)
+        └── admin/ (管理功能)
+```
+
+**导航约定**：
+- `/generation` → 功能导航页面
+- `/aigc/chat` → 聊天/文本生成
+- `/generation/course` → 视频生成工作流
+- 所有生成功能都需要登录（ModalAuthentication 处理）
+
+## 开发工作流
+
+### 启动项目
+```bash
+ni                    # 安装依赖
+nr dev                # 启动 http://localhost:3000
+nr generate           # 生产构建 (生成静态文件)
+```
+
+### 常见任务
+
+**添加新的 API 端点**：
+1. 定义 Request 和 Response 类型（参考 `typings/llm.ts`）
+2. 在 composable 中使用 `useFetchWrapped` 调用
+3. 自动包含 token/user_id（来自 `useLoginState`）
+
+**添加新的视频处理功能**：
+1. 使用 `useFFmpeg()` 获取实例（自动初始化）
+2. 写入文件到 vFS: `ffmpeg.writeFile()`
+3. 执行命令：`ffmpeg.exec([...filterArgs])`
+4. 清理临时文件：`ffmpeg.deleteFile()`
+5. 使用 progress callback 通报处理进度
+
+**添加新的 UI 组件**：
+1. 创建在 `components/` 下（自动注册）
+2. 优先使用 Radix Vue + Nuxt UI（已集成）
+3. 使用 Tailwind CSS utility classes + `@apply` 指令
+4. 通过 `app.config.ts` 自定义 UI 主题
+
+## 项目特定的约定
+
+### 类型定义位置
+- **LLM 相关**: `typings/llm.ts`（ChatMessage, ChatSession, ModelTag, LLMModal）
+- **全局类型**: `typings/types.d.ts`（BaseResponse, AuthedRequest, UserSchema）
+- **组件接口**: 组件目录下的 `index.d.ts`（例 `components/aigc/drawing/index.d.ts`）
+
+### 命名规范
+- **Composables**: `use` 前缀（`useLoginState`, `useLLM`）
+- **Stores**: `use` + 功能名（`useHistory`, `useTourState`）
+- **组件**: PascalCase（`ChatItem.vue`, `ModalAuthentication.vue`）
+- **工具函数**: camelCase，放在 `composables/` 或各功能目录
+
+### 响应式数据模式
+- **Pinia store 返回值**: 必须通过 `storeToRefs()` 才能保持响应式
+- **模板中的 ref**: 直接访问（Vue 自动展开）
+- **跨组件数据**: 优先使用 Pinia store（带持久化）
+
+### 进度反馈与错误处理
+- **长时间操作** (视频处理): 通过 callback 函数报告 progress（0-100）
+- **错误处理**: 返回 Promise reject，上层 catch 处理；可选通过 toast/message 提示
+- **FFmpeg 错误**: 捕获 exitCode 非零，记录详细的 FFmpeg 输出
+
+## 依赖与性能优化
+
+### 关键依赖
+- **@ffmpeg/ffmpeg@0.12.15**: WASM 视频处理（从 CDN 加载）
+- **@webav/av-cliper**: 客户端视频剪辑库
+- **markdown-it + highlight.js**: 内容渲染（支持代码高亮）
+- **date-fns/dayjs**: 时间处理（dayjs-nuxt 提供全局实例）
+- **idb-keyval**: IndexedDB 简化操作（缓存大文件）
+
+### Vite 优化设置
+```typescript
+// nuxt.config.ts 中排除以下包进行优化，避免 bundling WASM
+optimizeDeps.exclude: ['@ffmpeg/ffmpeg', 'idb-keyval', '@webav/av-cliper', 'gsap', 'markdown-it']
+```
+
+### 构建排除项
+Worker 格式设置为 ES Module，避免 Vite 默认处理：
+```typescript
+vite.worker.format = 'es'
+```
+
+## 测试与调试
+
+- **开发服务器日志**: 浏览器控制台查看 FFmpeg、API、业务日志
+- **FFmpeg 调试**: `[FFmpeg]` 前缀的日志输出包含加载进度、命令执行信息
+- **状态调试**: Pinia DevTools（启用 `devtools: true`）
+- **样式调试**: Tailwind 配置在 `tailwind.config.ts`，按需自定义
+
+## 常见陷阱与解决方案
+
+| 问题 | 原因 | 解决方案 |
+|------|------|--------|
+| API 请求 401 | 缺少 token 或已过期 | 检查 `useLoginState().token`，通过 ModalAuthentication 重新登录 |
+| FFmpeg 加载超时 | CDN 资源加载慢 | 检查网络，可切换到本地 `/public/assets/ffmpeg` |
+| 视频输出无声音 | 滤镜链未映射音频 | 确保 FFmpeg 命令包含 `-map '1:a?'` 映射音频轨道 |
+| 组件未注册 | 文件位置错误 | 确保在 `components/` 目录下，子目录自动扁平化注册 |
+| Pinia 状态未持久化 | 未配置 persist 选项 | 在 store 返回语句后添加 persist 配置（参考 `useLoginState`） |
+
+## 资源链接
+
+- [Nuxt 3 文档](https://nuxt.com/docs)
+- [Pinia 文档](https://pinia.vuejs.org)
+- [FFmpeg.wasm 文档](https://ffmpegwasm.netlify.app/)
+- [Radix Vue](https://www.radix-vue.com/)
+- [Nuxt UI 组件库](https://ui.nuxt.com/)：可使用 nuxt-ui MCP 工具

components/SlideCreateCourseGreen.vue

@@ -15,8 +15,16 @@ const creationPending = ref(false)
 const isDigitalSelectorOpen = ref(false)
 
 const createCourseSchema = object({
-  title: string().trim().min(4, '标题必须大于4个字符').max(20, '标题不能超过20个字符').required('请输入视频标题'),
+  title: string()
-  content: string().trim().min(4, '内容必须大于4个字符').max(1000, '内容不能超过1000个字符').required('请输入驱动文本内容'),
+    .trim()
+    .min(4, '标题必须大于4个字符')
+    .max(20, '标题不能超过20个字符')
+    .required('请输入视频标题'),
+  content: string()
+    .trim()
+    .min(4, '内容必须大于4个字符')
+    .max(1000, '内容不能超过1000个字符')
+    .required('请输入驱动文本内容'),
   digital_human_id: number().not([0], '请选择数字人'),
   source_type: number().default(0).required(),
   speed: number().default(1.0).min(0.5).max(1.5).required(),
@@ -31,40 +39,38 @@ const createCourseState = reactive({
   digital_human_id: 0,
   source_type: 0,
   speed: 1.0,
-  bg_img: undefined,
+  bg_img: 'https://service1.fenshenzhike.com/default_background.png',
 })
 
 const selected_digital_human = ref<DigitalHumanItem | null>(null)
-const selected_bg_img = ref<File | undefined>();
+const selected_bg_img = ref<File | undefined>()
 
 watchEffect(() => {
   if (selected_digital_human.value) {
     // 2025.02.26 使用内部数字人 ID
     createCourseState.digital_human_id =
-      selected_digital_human.value.digital_human_id ?? selected_digital_human.value.id ?? 0
+      selected_digital_human.value.digital_human_id ??
+      selected_digital_human.value.id ??
+      0
     createCourseState.source_type = selected_digital_human.value.type!
   }
 })
 
-const onCreateCourseGreenSubmit = async (event: FormSubmitEvent<CreateCourseSchema>) => {  
+const onCreateCourseGreenSubmit = async (
+  event: FormSubmitEvent<CreateCourseSchema>
+) => {
   creationPending.value = true
 
-  let bgImgUrl = undefined
-
-  if (selected_bg_img.value) {
-    bgImgUrl = await useFileGo(selected_bg_img.value, 'tmp')
-  }
-
   let payload: {
-    token: string;
+    token: string
-    user_id: number;
+    user_id: number
-    title: string;
+    title: string
-    content: string;
+    content: string
-    digital_human_id: any;
+    digital_human_id: any
-    speed: number;
+    speed: number
-    device_id: string;
+    device_id: string
-    source_type: 1 | 2 | undefined;
+    source_type: 1 | 2 | undefined
-    bg_img?: string;
+    bg_img?: string
   } = {
     token: loginState.token!,
     user_id: loginState.user.id,
@@ -74,66 +80,60 @@ const onCreateCourseGreenSubmit = async (event: FormSubmitEvent<CreateCourseSche
     speed: 2 - event.data.speed,
     device_id: 'XSHAssistant Web',
     source_type: event.data.source_type as 1 | 2 | undefined,
+    bg_img: event.data.bg_img,
   }
 
-  if (selected_bg_img.value) {
+  useFetchWrapped<
-    if (!bgImgUrl) {
+    req.gen.GBVideoCreate & AuthedRequest,
-      toast.add({
+    BaseResponse<resp.gen.GBVideoCreate>
-        title: '上传失败',
+  >('App.Digital_VideoTask.Create', payload)
-        description: '背景图片上传失败，请重试',
+    .then((res) => {
-        color: 'red',
+      if (!!res.data.task_id) {
-        icon: 'i-tabler-alert-triangle',
+        toast.add({
-      })
+          title: '创建成功',
-      selected_bg_img.value = undefined
+          description: '视频已加入生成队列',
+          color: 'green',
+          icon: 'i-tabler-check',
+        })
+        emit('success')
+        slide.close()
+      } else {
+        toast.add({
+          title: '创建失败',
+          description: res.msg || '未知错误',
+          color: 'red',
+          icon: 'i-tabler-alert-triangle',
+        })
+      }
+      creationPending.value = false
+    })
+    .catch((e) => {
       creationPending.value = false
-      return
-    }
-    payload = {
-      ...payload,
-      bg_img: bgImgUrl,
-    }
-  }
-
-  useFetchWrapped<req.gen.GBVideoCreate & AuthedRequest, BaseResponse<resp.gen.GBVideoCreate>>('App.Digital_VideoTask.Create', payload).then(res => {
-    if (!!res.data.task_id) {
-      toast.add({
-        title: '创建成功',
-        description: '视频已加入生成队列',
-        color: 'green',
-        icon: 'i-tabler-check',
-      })
-      emit('success')
-      slide.close()
-    } else {
       toast.add({
         title: '创建失败',
-        description: res.msg || '未知错误',
+        description: e.message || '未知错误',
         color: 'red',
         icon: 'i-tabler-alert-triangle',
       })
-    }
-    creationPending.value = false
-  }).catch(e => {
-    creationPending.value = false
-    toast.add({
-      title: '创建失败',
-      description: e.message || '未知错误',
-      color: 'red',
-      icon: 'i-tabler-alert-triangle',
     })
-  })
 }
 </script>
 
 <template>
   <USlideover prevent-close>
     <UCard
-      :ui="{ body: { base: 'flex-1' }, ring: '', divide: 'divide-y divide-gray-100 dark:divide-gray-800' }"
+      :ui="{
+        body: { base: 'flex-1' },
+        ring: '',
+        divide: 'divide-y divide-gray-100 dark:divide-gray-800',
+      }"
       class="flex flex-col flex-1"
     >
       <template #header>
         <div class="flex items-center justify-between">
-          <h3 class="text-base font-semibold leading-6 text-gray-900 dark:text-white">
+          <h3
+            class="text-base font-semibold leading-6 text-gray-900 dark:text-white"
+          >
             新建绿幕视频
           </h3>
           <UButton
@@ -154,28 +154,52 @@ const onCreateCourseGreenSubmit = async (event: FormSubmitEvent<CreateCourseSche
         @submit="onCreateCourseGreenSubmit"
       >
         <div class="flex justify-between gap-2 *:flex-1">
-          <UFormGroup label="视频标题" name="title" required>
+          <UFormGroup
-            <UInput v-model="createCourseState.title" placeholder="请输入视频标题"/>
+            label="视频标题"
+            name="title"
+            required
+          >
+            <UInput
+              v-model="createCourseState.title"
+              placeholder="请输入视频标题"
+            />
           </UFormGroup>
         </div>
 
         <div class="grid grid-cols-1 sm:grid-cols-2 gap-2">
-          <UFormGroup label="数字人" name="digital_human_id" required>
+          <UFormGroup
+            label="数字人"
+            name="digital_human_id"
+            required
+          >
             <div
-              :class="{'shadow-inner': !!selected_digital_human}"
+              :class="{ 'shadow-inner': !!selected_digital_human }"
               class="flex items-center gap-2 bg-neutral-100 dark:bg-neutral-800 p-2 rounded-md cursor-pointer select-none transition-all"
               @click="isDigitalSelectorOpen = true"
             >
               <div
-                class="w-12 aspect-square border dark:border-neutral-700 rounded-md flex justify-center items-center overflow-hidden">
+                class="w-12 aspect-square border dark:border-neutral-700 rounded-md flex justify-center items-center overflow-hidden"
-                <UIcon v-if="!selected_digital_human" class="text-2xl opacity-50" name="i-tabler-user-screen"/>
+              >
-                <NuxtImg v-else :src="selected_digital_human?.avatar"/>
+                <UIcon
+                  v-if="!selected_digital_human"
+                  class="text-2xl opacity-50"
+                  name="i-tabler-user-screen"
+                />
+                <NuxtImg
+                  v-else
+                  :src="selected_digital_human?.avatar"
+                />
               </div>
               <div class="flex flex-col text-neutral-400 text-sm font-medium">
-                <span :class="!!selected_digital_human ? 'text-neutral-600' : ''">{{
+                <span
-                    selected_digital_human?.name || '点击选择数字人'
+                  :class="!!selected_digital_human ? 'text-neutral-600' : ''"
-                  }}</span>
+                >
-                <span v-if="selected_digital_human?.description" class="text-2xs">
+                  {{ selected_digital_human?.name || '点击选择数字人' }}
+                </span>
+                <span
+                  v-if="selected_digital_human?.description"
+                  class="text-2xs"
+                >
                   {{ selected_digital_human?.description }}
                 </span>
               </div>
@@ -183,23 +207,44 @@ const onCreateCourseGreenSubmit = async (event: FormSubmitEvent<CreateCourseSche
           </UFormGroup>
         </div>
 
-        <UFormGroup label="背景图片" name="bg_img" help="可以上传图片作为视频背景，留空则为绿幕背景">
+        <!-- <UFormGroup label="背景图片" name="bg_img" help="可以上传图片作为视频背景，留空则为绿幕背景">
           <UInput type="file" accept="image/jpg,image/png" placeholder="选择背景图片" @change="selected_bg_img = $event?.[0] || undefined"/>
+        </UFormGroup> -->
+
+        <UFormGroup
+          label="驱动内容"
+          name="content"
+          required
+        >
+          <UTextarea
+            v-model="createCourseState.content"
+            :rows="6"
+            autoresize
+            placeholder="请输入驱动文本内容"
+          />
         </UFormGroup>
 
-        <UFormGroup label="驱动内容" name="content" required>
+        <UAlert
-          <!--          <template #help>-->
+          icon="tabler:background"
-          <!--            <p class="text-xs text-neutral-400">-->
+          color="sky"
-          <!--              仅支持 .pptx 格式-->
+          variant="subtle"
-          <!--            </p>-->
+          title="自定义背景图片"
-          <!--          </template>-->
+          description="背景图片可在视频生成完毕后，在下载选单中选择合成。默认为绿幕背景。"
-          <UTextarea v-model="createCourseState.content" :rows="6" autoresize placeholder="请输入驱动文本内容"/>
+        />
-        </UFormGroup>
 
-        <UAccordion :items="[{label: '高级选项'}]" color="gray" size="lg">
+        <UAccordion
+          :items="[{ label: '高级选项' }]"
+          color="gray"
+          size="lg"
+        >
           <template #item>
-            <div class="border dark:border-neutral-700 rounded-lg space-y-4 p-4 pb-6">
+            <div
-              <UFormGroup :label="`视频倍速：${createCourseState.speed}`" name="speed">
+              class="border dark:border-neutral-700 rounded-lg space-y-4 p-4 pb-6"
+            >
+              <UFormGroup
+                :label="`视频倍速：${createCourseState.speed}`"
+                name="speed"
+              >
                 <URange
                   v-model="createCourseState.speed"
                   :max="1.5"
@@ -244,6 +289,4 @@ const onCreateCourseGreenSubmit = async (event: FormSubmitEvent<CreateCourseSche
   </USlideover>
 </template>
 
-<style scoped>
+<style scoped></style>
-
-</style>

components/aigc/generation/GBTaskCard.vue

@@ -15,13 +15,132 @@ const emit = defineEmits({
 const dayjs = useDayjs()
 const toast = useToast()
 
+const isFailed = computed(() => {
+  return props.video.progress === -1
+})
 const isPreviewModalOpen = ref(false)
+const isVideoBackgroundPreviewOpen = ref(false)
 const isFullContentOpen = ref(false)
 const downloadingState = reactive({
   subtitle: 0,
   video: 0,
 })
 
+// 背景选择相关状态
+const selectedBackgroundFile = ref<File | null>(null)
+const selectedBackgroundPreview = ref<string>('')
+const isCombinatorLoading = ref(false)
+const compositingProgress = ref(0)
+const compositingPhase = ref<'loading' | 'analyzing' | 'preparing' | 'executing' | 'finalizing'>('loading')
+const combinatorError = ref<string>('')
+const fileInputRef = ref<HTMLInputElement | null>(null)
+const compositedVideoBlob = ref<Blob | null>(null)
+
+// 阶段显示文本
+const phaseText = computed(() => {
+  const phaseMap: Record<typeof compositingPhase.value, string> = {
+    'loading': '加载资源...',
+    'analyzing': '分析图片...',
+    'preparing': '准备合成...',
+    'executing': '合成中...',
+    'finalizing': '完成处理...',
+  }
+  return phaseMap[compositingPhase.value]
+})
+
+const handleBackgroundFileSelect = (event: Event) => {
+  const target = event.target as HTMLInputElement
+  const file = target.files?.[0]
+  
+  if (!file) return
+  
+  // 验证文件类型
+  if (!file.type.startsWith('image/')) {
+    toast.add({
+      title: '文件类型错误',
+      description: '请选择一个图片文件',
+      color: 'red',
+      icon: 'i-tabler-alert-triangle',
+    })
+    return
+  }
+  
+  selectedBackgroundFile.value = file
+  const reader = new FileReader()
+  reader.onload = (e) => {
+    selectedBackgroundPreview.value = e.target?.result as string
+  }
+  reader.readAsDataURL(file)
+  combinatorError.value = ''
+  compositedVideoBlob.value = null
+}
+
+const composeBackgroundVideo = async () => {
+  if (!selectedBackgroundFile.value) {
+    toast.add({
+      title: '未选择图片',
+      description: '请先选择一个背景图片',
+      color: 'orange',
+      icon: 'i-tabler-alert-circle',
+    })
+    return
+  }
+  
+  try {
+    isCombinatorLoading.value = true
+    compositingProgress.value = 0
+    combinatorError.value = ''
+    
+    // 使用 FFmpeg WASM 进行视频背景合成
+    const resultBlob = await useVideoBackgroundCompositing(
+      props.video.video_alpha_url!,
+      selectedBackgroundFile.value,
+      {
+        onProgress: (info) => {
+          compositingProgress.value = info.progress
+          compositingPhase.value = info.phase
+        }
+      }
+    )
+    
+    compositedVideoBlob.value = resultBlob
+    
+    toast.add({
+      title: '合成成功',
+      description: '背景已成功合成，可预览或下载',
+      color: 'green',
+      icon: 'i-tabler-check',
+    })
+  } catch (err: any) {
+    combinatorError.value = err.message || '合成失败，请重试'
+    toast.add({
+      title: '合成失败',
+      description: combinatorError.value,
+      color: 'red',
+      icon: 'i-tabler-alert-triangle',
+    })
+  } finally {
+    isCombinatorLoading.value = false
+  }
+}
+
+const downloadCompositedVideo = () => {
+  if (!compositedVideoBlob.value) return
+  
+  const url = URL.createObjectURL(compositedVideoBlob.value)
+  const link = document.createElement('a')
+  link.href = url
+  link.download = `${props.video.title || props.video.task_id}_合成.webm`
+  document.body.appendChild(link)
+  link.click()
+  document.body.removeChild(link)
+  URL.revokeObjectURL(url)
+}
+
+const compositedVideoUrl = computed(() => {
+  return compositedVideoBlob.value ? URL.createObjectURL(compositedVideoBlob.value) : ''
+})
+
 const startDownload = (url: string, filename: string) => {
   if (url.endsWith('.ass')) {
     downloadingState.subtitle = 0
@@ -73,10 +192,6 @@ const startDownload = (url: string, filename: string) => {
 
   download()
 }
-
-const onClick = () => {
-  console.log('click delete')
-}
 </script>
 
 <template>
@@ -84,11 +199,14 @@ const onClick = () => {
     class="w-full flex gap-2 rounded-xl border border-neutral-200 dark:border-neutral-700 hover:shadow transition overflow-hidden p-3"
   >
     <div class="flex-0 h-48 aspect-[10/16] flex flex-col items-center justify-center rounded-lg shadow overflow-hidden relative group">
-      <div v-if="!video.video_cover" class="w-full h-full bg-primary flex flex-col justify-center items-center gap-2">
+      <div v-if="!video.video_cover" class="w-full h-full flex flex-col justify-center items-center gap-2" :class="!isFailed ? 'bg-primary' : 'bg-rose-400'">
-        <UIcon class="animate-spin text-4xl text-white" name="tabler:loader"/>
+        <UIcon v-if="!isFailed" class="animate-spin text-4xl text-white" name="tabler:loader"/>
+        <UIcon v-else class="text-4xl text-white" name="tabler:alert-triangle"/>
         <div class="flex flex-col items-center gap-0.5">
-          <span class="text-sm font-bold text-white/90">火速生成中</span>
+          <span class="text-sm font-bold text-white/90">
-          <span class="text-xs font-medium text-white/50">{{ video.progress }}%</span>
+            {{ isFailed ? '生成失败' : '火速生成中...' }}
+          </span>
+          <span v-if="!isFailed" class="text-xs font-medium text-white/50">{{ video.progress }}%</span>
         </div>
       </div>
       <NuxtImg v-else :src="video.video_cover" class="w-full h-full brightness-90 object-cover"/>
@@ -148,15 +266,36 @@ const onClick = () => {
               variant="soft"
               @click="startDownload(video.subtitle!, (video.title || video.task_id) + '.ass')"
             />
-            <UButton
+            <UDropdown
-              :label="downloadingState.video > 0 && downloadingState.video < 100 ? `${downloadingState.video.toFixed(0)}%` : '视频'"
+              :items="[
-              :loading="downloadingState.video > 0 && downloadingState.video < 100"
+                [
-              :disabled="!video.video_url"
+                  {
-              color="primary"
+                    label: '绿幕视频下载',
-              leading-icon="i-tabler-download"
+                    icon: 'tabler:download',
-              variant="soft"
+                    click: () => {
-              @click="startDownload(video.video_url!, (video.title || video.task_id) + '.mp4')"
+                      startDownload(video.video_url!, (video.title || video.task_id) + '.mp4')
-            />
+                    }
+                  },
+                  {
+                    label: '合成背景图片',
+                    icon: 'tabler:background',
+                    click: () => {
+                      isVideoBackgroundPreviewOpen = true
+                    },
+                    disabled: !video.video_alpha_url
+                  },
+                ],
+              ]"
+            >
+              <UButton
+                :label="downloadingState.video > 0 && downloadingState.video < 100 ? `${downloadingState.video.toFixed(0)}%` : '视频'"
+                :loading="downloadingState.video > 0 && downloadingState.video < 100"
+                :disabled="!video.video_url"
+                color="primary"
+                leading-icon="i-tabler-download"
+                variant="soft"
+              />
+            </UDropdown>
           </UButtonGroup>
         </div>
       </div>
@@ -209,6 +348,132 @@ const onClick = () => {
         <video class="w-full rounded shadow" controls autoplay :src="video.video_url" />
       </UCard>
     </UModal>
+    <UModal v-model="isVideoBackgroundPreviewOpen">
+      <UCard :ui="{ ring: '', divide: 'divide-y divide-gray-100 dark:divide-gray-800' }">
+        <template #header>
+          <div class="flex items-center justify-between">
+            <div class="text-base font-semibold leading-6 text-gray-900 dark:text-white overflow-hidden">
+              <p>视频背景合成</p>
+              <p class="text-xs text-blue-500 w-full overflow-hidden text-nowrap text-ellipsis">
+                {{ video.title }}
+              </p>
+            </div>
+            <UButton class="-my-1" color="gray" icon="i-tabler-x" variant="ghost" @click="isVideoBackgroundPreviewOpen = false" />
+          </div>
+        </template>
+
+        <div class="space-y-4">
+          <!-- 背景图片选择区域 -->
+          <div v-if="!compositedVideoBlob && !isCombinatorLoading" class="border-2 border-dashed border-neutral-200 dark:border-neutral-700 rounded-lg p-4">
+            <div class="space-y-3">
+              <div class="text-sm font-medium text-gray-900 dark:text-white">选择背景图片</div>
+              
+              <!-- 预览区域 -->
+              <!-- <div v-if="selectedBackgroundPreview" class="relative w-full aspect-video rounded-lg overflow-hidden bg-neutral-100 dark:bg-neutral-800">
+                <img :src="selectedBackgroundPreview" alt="背景预览" class="w-full h-full object-cover" />
+              </div>
+              <div v-else class="w-full aspect-video rounded-lg overflow-hidden bg-neutral-100 dark:bg-neutral-800 flex flex-col items-center justify-center gap-2">
+                <UIcon class="text-3xl text-neutral-400" name="tabler:photo" />
+                <span class="text-xs text-neutral-400">点击选择图片</span>
+              </div> -->
+              
+              <!-- 文件输入 -->
+              <input
+                ref="fileInputRef"
+                type="file"
+                accept="image/*"
+                class="hidden"
+                @change="handleBackgroundFileSelect"
+              />
+              
+              <!-- 选择按钮 -->
+              <UButton
+                block
+                color="primary"
+                icon="i-tabler-photo-plus"
+                label="选择图片"
+                variant="soft"
+                @click="fileInputRef?.click()"
+              />
+              
+              <!-- 选中的文件名 -->
+              <div v-if="selectedBackgroundFile" class="text-xs text-neutral-500 dark:text-neutral-400">
+                已选择: {{ selectedBackgroundFile.name }}
+              </div>
+            </div>
+          </div>
+
+          <!-- 错误提示 -->
+          <UAlert
+            v-if="combinatorError"
+            color="red"
+            icon="i-tabler-alert-triangle"
+            title="合成失败"
+            :description="combinatorError"
+          />
+
+          <!-- 合成进度 -->
+          <div v-if="isCombinatorLoading" class="space-y-2">
+            <div class="flex justify-between items-center">
+              <span class="text-sm font-medium text-gray-900 dark:text-white">{{ phaseText }}</span>
+              <span class="text-xs text-neutral-500">{{ compositingProgress }}%</span>
+            </div>
+            <UProgress :value="compositingProgress" />
+          </div>
+
+          <!-- 合成预览 -->
+          <div v-if="compositedVideoBlob" class="space-y-2">
+            <div class="text-sm font-medium text-gray-900 dark:text-white">视频预览</div>
+            <video 
+              class="w-full rounded-lg shadow bg-black"
+              controls 
+              autoplay
+              muted
+              :src="compositedVideoUrl"
+            />
+          </div>
+        </div>
+
+        <template #footer>
+          <div class="flex justify-end gap-2">
+            <UButton
+              color="gray"
+              label="取消"
+              :disabled="isCombinatorLoading"
+              @click="isVideoBackgroundPreviewOpen = false"
+            />
+            <UButton
+              v-if="compositedVideoBlob"
+              color="gray"
+              label="重新选择"
+              @click="() => {
+                selectedBackgroundFile = null
+                selectedBackgroundPreview = ''
+                compositedVideoBlob = null
+                combinatorError = ''
+                isCombinatorLoading = false
+              }"
+            />
+            <UButton
+              v-if="compositedVideoBlob"
+              color="green"
+              icon="i-tabler-download"
+              label="下载合成视频"
+              @click="downloadCompositedVideo"
+            />
+            <UButton
+              v-else
+              :disabled="!selectedBackgroundFile || isCombinatorLoading"
+              :loading="isCombinatorLoading"
+              color="primary"
+              icon="i-tabler-wand"
+              :label="isCombinatorLoading ? '合成中' : '开始合成'"
+              @click="composeBackgroundVideo"
+            />
+          </div>
+        </template>
+      </UCard>
+    </UModal>
   </div>
 </template>
 

composables/useFFmpeg.ts

@@ -0,0 +1,103 @@
+import { FFmpeg } from '@ffmpeg/ffmpeg'
+import { toBlobURL } from '@ffmpeg/util'
+
+let ffmpegInstance: FFmpeg | null = null
+let loadPromise: Promise<FFmpeg> | null = null
+
+/**
+ * 获取或初始化 FFmpeg 实例（单例模式）
+ */
+export const useFFmpeg = async () => {
+  // 如果已经加载过，直接返回
+  if (ffmpegInstance && ffmpegInstance.loaded) {
+    return ffmpegInstance
+  }
+
+  // 如果正在加载中，等待加载完成
+  if (loadPromise) {
+    return loadPromise
+  }
+
+  loadPromise = initializeFFmpeg()
+  return loadPromise
+}
+
+async function initializeFFmpeg(enableMT: boolean = false): Promise<FFmpeg> {
+  try {
+    const ffmpeg = new FFmpeg()
+
+    ffmpeg.on('log', ({ message, type }) => {
+      console.log(`[ffmpeg - ${type}]`, message)
+    })
+
+    ffmpeg.on('progress', ({ progress, time }) => {
+      console.log(`[ffmpeg] P: ${(progress * 100).toFixed(2)}%, T: ${time}ms`)
+    })
+
+    const baseURL = enableMT
+      ? 'https://cdn.jsdelivr.net/npm/@ffmpeg/core-mt@0.12.10/dist/esm'
+      : 'https://cdn.jsdelivr.net/npm/@ffmpeg/core@0.12.10/dist/esm'
+
+    const coreURL = await toBlobURL(
+      `${baseURL}/ffmpeg-core.js`,
+      'text/javascript'
+    )
+    const wasmURL = await toBlobURL(
+      `${baseURL}/ffmpeg-core.wasm`,
+      'application/wasm'
+    )
+
+    let loadPayload = {
+      coreURL,
+      wasmURL,
+    }
+
+    if (enableMT) {
+      const workerURL = await toBlobURL(
+        `${baseURL}/ffmpeg-core.worker.js`,
+        'text/javascript'
+      )
+      Object.assign(loadPayload, { workerURL })
+    }
+
+    const isLoaded = await ffmpeg.load(loadPayload)
+    console.log('[FFmpeg] FFmpeg 加载完成，isLoaded:', isLoaded)
+
+    ffmpegInstance = ffmpeg
+    loadPromise = null
+    return ffmpeg
+  } catch (error) {
+    console.error('[FFmpeg] 初始化失败:', error)
+    loadPromise = null
+    throw error
+  }
+}
+
+/**
+ * 清理 FFmpeg 资源
+ */
+export const cleanupFFmpeg = () => {
+  if (ffmpegInstance && ffmpegInstance.loaded) {
+    console.log('[FFmpeg] 清理 FFmpeg 资源...')
+    ffmpegInstance.terminate()
+    ffmpegInstance = null
+    loadPromise = null
+  }
+}
+
+/**
+ * 将 Blob/File 转换为 Uint8Array
+ */
+export const fileToUint8Array = async (
+  file: File | Blob
+): Promise<Uint8Array> => {
+  return new Promise((resolve, reject) => {
+    const reader = new FileReader()
+    reader.onload = (e) => {
+      const arrayBuffer = e.target?.result as ArrayBuffer
+      resolve(new Uint8Array(arrayBuffer))
+    }
+    reader.onerror = reject
+    reader.readAsArrayBuffer(file)
+  })
+}

composables/useVideoBackgroundCombinator.ts

@@ -0,0 +1,6 @@
+/**
+ * 已废弃：使用 useVideoBackgroundCompositing 替代
+ * 该文件保留用于向后兼容
+ */
+
+export { useVideoBackgroundCompositing as useVideoBackgroundCombinator } from './useVideoBackgroundCompositing'

composables/useVideoBackgroundCompositing.ts

@@ -0,0 +1,166 @@
+import { fetchFile } from '@ffmpeg/util'
+import { useFFmpeg, fileToUint8Array } from './useFFmpeg'
+
+/**
+ * 获取图片的宽高信息
+ */
+const getImageDimensions = async (
+  imageData: Uint8Array
+): Promise<{ width: number; height: number }> => {
+  return new Promise((resolve, reject) => {
+    const blob = new Blob([imageData], { type: 'image/png' })
+    const url = URL.createObjectURL(blob)
+    const img = new Image()
+
+    img.onload = () => {
+      URL.revokeObjectURL(url)
+      resolve({ width: img.width, height: img.height })
+    }
+
+    img.onerror = () => {
+      URL.revokeObjectURL(url)
+      reject(new Error('Failed to load image'))
+    }
+
+    img.src = url
+  })
+}
+
+/**
+ * 计算等比缩放到720P的尺寸
+ * 720P 指高度为720，宽度按原宽高比计算
+ */
+const calculateScaledDimensions = (
+  width: number,
+  height: number
+): { width: number; height: number } => {
+  const targetHeight = 720
+
+  // 如果原始高度小于等于720，保持原始尺寸
+  if (height <= targetHeight) {
+    return { width, height }
+  }
+
+  // 计算缩放比例
+  const scale = targetHeight / height
+  const scaledWidth = Math.round(width * scale)
+
+  // 确保宽度为偶数（视频编码要求）
+  const finalWidth = scaledWidth % 2 === 0 ? scaledWidth : scaledWidth - 1
+
+  return { width: finalWidth, height: targetHeight }
+}
+
+export type CompositingPhase = 
+  | 'loading'
+  | 'analyzing'
+  | 'preparing'
+  | 'executing'
+  | 'finalizing'
+
+export type CompositingProgressCallback = (info: {
+  progress: number
+  phase: CompositingPhase
+}) => void
+
+/**
+ * 使用 FFmpeg WASM 将透明通道的视频与背景图片进行合成
+ * @param videoUrl - WebM 视频 URL（带透明通道的数字人视频）
+ * @param backgroundImage - 背景图片（File 对象或 URL 字符串）
+ * @param options - 额外选项
+ * @returns 合成后的视频 Blob
+ */
+export const useVideoBackgroundCompositing = async (
+  videoUrl: string,
+  backgroundImage: File | string,
+  options?: {
+    onProgress?: CompositingProgressCallback
+  }
+) => {
+  const ffmpeg = await useFFmpeg()
+  const progressCallback = options?.onProgress
+
+  const videoFileName = 'input_video.webm'
+  const backgroundFileName = 'background.png'
+  const outputFileName = 'output.mp4'
+
+  try {
+    progressCallback?.({ progress: 10, phase: 'loading' })
+    const videoData = await fetchFile(videoUrl)
+    const backgroundData = await fetchFile(backgroundImage)
+
+    progressCallback?.({ progress: 15, phase: 'analyzing' })
+    const { width: bgWidth, height: bgHeight } = await getImageDimensions(
+      backgroundData
+    )
+    console.log(
+      `[Compositing] Background image dimensions: ${bgWidth}x${bgHeight}`
+    )
+
+    const { width: outputWidth, height: outputHeight } =
+      calculateScaledDimensions(bgWidth, bgHeight)
+    console.log(
+      `[Compositing] Output dimensions: ${outputWidth}x${outputHeight}`
+    )
+
+    progressCallback?.({ progress: 20, phase: 'preparing' })
+
+    await ffmpeg.writeFile(videoFileName, videoData)
+    await ffmpeg.writeFile(backgroundFileName, backgroundData)
+
+    progressCallback?.({ progress: 25, phase: 'preparing' })
+
+    // HACK: 不明原因导致首次执行合成时会报 memory access out of bounds 错误，先执行一次空命令能够规避
+    await ffmpeg.exec(['-i', 'not-found'])
+
+    // 设置 progress 事件监听，映射 FFmpeg 进度到 30-95% 范围
+    const executingProgressHandler = ({ progress }: { progress: number }) => {
+      // progress 范围是 0-1，映射到 30-95
+      const mappedProgress = Math.round(30 + progress * 65)
+      progressCallback?.({ progress: mappedProgress, phase: 'executing' })
+    }
+    ffmpeg.on('progress', executingProgressHandler)
+
+    progressCallback?.({ progress: 30, phase: 'executing' })
+
+    // prettier-ignore
+    const exitCode = await ffmpeg.exec([
+      '-i', backgroundFileName,
+      '-c:v', 'libvpx-vp9',
+      '-i', videoFileName,
+      '-filter_complex', 'overlay=(W-w)/2:H-h',
+      '-c:v', 'libx264',
+      outputFileName
+    ])
+
+    ffmpeg.off('progress', executingProgressHandler)
+
+    if (exitCode !== 0) {
+      throw new Error(`FFmpeg command failed with exit code ${exitCode}`)
+    }
+
+    progressCallback?.({ progress: 95, phase: 'finalizing' })
+
+    const outputData = await ffmpeg.readFile(outputFileName)
+    let outputArray: Uint8Array
+    if (outputData instanceof Uint8Array) {
+      outputArray = outputData
+    } else if (typeof outputData === 'string') {
+      outputArray = new TextEncoder().encode(outputData)
+    } else {
+      outputArray = new Uint8Array(outputData as ArrayBufferLike)
+    }
+    const outputBlob = new Blob([outputArray], { type: 'video/mp4' })
+
+    progressCallback?.({ progress: 100, phase: 'finalizing' })
+
+    return outputBlob
+  } catch (error) {
+    console.error('Video compositing failed:', error)
+    throw error
+  } finally {
+    await ffmpeg.deleteFile(videoFileName)
+    await ffmpeg.deleteFile(backgroundFileName)
+    await ffmpeg.deleteFile(outputFileName)
+  }
+}

nuxt.config.ts

@@ -58,4 +58,23 @@ export default defineNuxtConfig({
   },
 
   compatibilityDate: '2024-07-28',
+
+  vite: {
+    worker: {
+      format: 'es',
+    },
+    optimizeDeps: {
+      exclude: [
+        '@ffmpeg/ffmpeg',
+        'idb-keyval',
+        '@uniiem/uuid',
+        '@uniiem/object-trim',
+        'gsap',
+        '@monosky/base64',
+        'markdown-it',
+        'highlight.js',
+        'driver.js',
+      ],
+    },
+  },
 })

package.json

@@ -10,8 +10,10 @@
     "preview": "nuxt preview",
     "postinstall": "nuxt prepare"
   },
-  "packageManager": "pnpm@10.19.0",
+  "packageManager": "pnpm@10.22.0",
   "dependencies": {
+    "@ffmpeg/ffmpeg": "^0.12.15",
+    "@ffmpeg/util": "^0.12.2",
     "@iconify-json/line-md": "^1.1.38",
     "@iconify-json/solar": "^1.1.9",
     "@iconify-json/svg-spinners": "^1.1.2",
@@ -20,7 +22,7 @@
     "@nuxt/image": "^1.7.0",
     "@uniiem/object-trim": "^0.2.0",
     "@uniiem/uuid": "^0.2.1",
-    "@webav/av-cliper": "^1.0.10",
+    "@webav/av-cliper": "^1.2.7",
     "date-fns": "^4.1.0",
     "events": "^3.3.0",
     "gsap": "^3.12.5",

typings/types.d.ts

@@ -113,6 +113,7 @@ interface GBVideoItem {
   title: string
   content: string
   bg_img: string
+  video_alpha_url?: string
   video_url?: string
   video_cover?: string
   subtitle?: string