---
id: 20261118-T0-06
title: "让AI从视频里学用电脑：音视频理解提升computer-use智能体表现"
title_en: "Video-as-Experience: Audio-Visual Understanding Improves Computer-Use Agents"
url: https://ai.daily.yangsir.net/daily/20261118-T0-06
issue_date: 2026-11-18
publish_date: 2026-11-17T05:00:00.000Z
category: research
source_name: "arXiv cs.AI"
source_url: https://arxiv.org/abs/2611.18747
---

# 让AI从视频里学用电脑：音视频理解提升computer-use智能体表现

arXiv新论文提出Video-as-Experience方法，让多模态大模型通过音视频演示学习计算机操作。现有从视频示范学习的方法通常只利用视觉画面，忽略了音频中的操作线索。该研究把音视频理解结合起来训练computer-use智能体，使其能感知图形界面、理解指令并执行操作。这为用日常录屏和配音训练操作类智能体提供了一条新路径。

## English Version

**Video-as-Experience: Audio-Visual Understanding Improves Computer-Use Agents**

A new arXiv paper introduces Video-as-Experience, teaching multimodal LLMs computer use from audio-visual demonstrations. Existing video-based learning approaches typically rely on visuals alone and discard audio cues. This work trains computer-use agents that perceive GUIs, reason over instructions and execute actions using combined audio-visual understanding. It points to a new route for training operation agents from everyday screen recordings with narration.

---

**来源**：[arXiv cs.AI](https://arxiv.org/abs/2611.18747)

**详情页**：https://ai.daily.yangsir.net/daily/20261118-T0-06

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*