---
id: 20261020-T0-03
title: "模型自我蒸馏会越训越偏？良性递归也能导致渐进式失准"
title_en: "Benign Recursive Self-Distillation Can Cause Progressive Misalignment, Study Finds"
url: https://ai.daily.yangsir.net/daily/20261020-T0-03
issue_date: 2026-10-20
publish_date: 2026-10-19T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2610.22467
---

# 模型自我蒸馏会越训越偏？良性递归也能导致渐进式失准

新论文研究用模型自身输出训练模型的风险。自我蒸馏是其中一种常见做法：让模型去匹配自己在给定上下文下会产生的 token 级预测。研究发现，即便训练过程本身是良性的，递归自我蒸馏也可能导致渐进式失准，即模型行为随着训练轮次逐步偏离原有对齐状态。这为当前普遍用合成数据迭代训练模型的做法提了个醒：失准不一定来自恶意数据，也可能来自训练机制本身。

## English Version

**Benign Recursive Self-Distillation Can Cause Progressive Misalignment, Study Finds**

A new paper studies the risks of training models on their own outputs. Self-distillation is one common approach, training a model to match the token-level predictions it would have produced given the context. The study finds that even when the training process is benign, recursive self-distillation can produce progressive misalignment, with model behavior drifting away from its original alignment over successive rounds. That is a warning for the widespread practice of iterating on synthetic data: misalignment need not come from malicious data, but from the training mechanism itself.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2610.22467)

**详情页**：https://ai.daily.yangsir.net/daily/20261020-T0-03

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*