---
id: 20260702-T0-06
title: "离线模仿学习存在缺陷：专家数据有噪声时，同策略蒸馏才最优"
title_en: "Study Reveals On-Policy Distillation is Optimal for Noisy Expert Imitation Learning"
url: https://ai.daily.yangsir.net/daily/20260702-T0-06
issue_date: 2026-07-02
publish_date: 2026-07-01T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2606.30923
---

# 离线模仿学习存在缺陷：专家数据有噪声时，同策略蒸馏才最优

一项新研究揭示了离线模仿学习（IL）在理论上的局限性，特别是在处理含有噪声的专家反馈时。论文证明了标准的离线行为克隆（Behavior Cloning）并非最佳方案，当专家数据存在噪声时，同策略蒸馏（On-Policy Distillation）才是最优策略。这一发现对当前的序列决策系统训练具有重要意义，尤其是对于依赖模仿学习范式的大语言模型（LLM）训练流程，提示开发者需关注数据质量对模型收敛性的影响。

## English Version

**Study Reveals On-Policy Distillation is Optimal for Noisy Expert Imitation Learning**

A new study highlights the theoretical limitations of offline Imitation Learning (IL), particularly with noisy expert feedback. The paper proves that standard offline Behavior Cloning is suboptimal, establishing On-Policy Distillation as the superior strategy when expert data is noisy. This finding has significant implications for sequential decision-making systems, especially for LLM training pipelines relying on IL, suggesting that data quality and distillation methods are critical for model convergence.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2606.30923)

**详情页**：https://ai.daily.yangsir.net/daily/20260702-T0-06

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*