---
id: 20260717-T0-16
title: "ShortOPD：短序列蒸馏恢复剪枝LLM的生成能力"
title_en: "ShortOPD Recovers Pruned LLMs' Generation Ability via Short-to-Long Distillation"
url: https://ai.daily.yangsir.net/daily/20260717-T0-16
issue_date: 2026-07-17
publish_date: 2026-07-16T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2607.13124
---

# ShortOPD：短序列蒸馏恢复剪枝LLM的生成能力

一项新研究针对结构化剪枝后大语言模型在自由生成任务上崩溃的问题，提出ShortOPD（Short-to-Long On-Policy Distillation）方法。该方法利用从短到长的在线蒸馏策略，仅用短序列训练即可恢复剪枝模型在长文本生成上的表现。实验表明，在多种生成基准上，被剪枝的LLM经ShortOPD恢复后性能接近甚至超过原模型，而模型体积显著减小。

## English Version

**ShortOPD Recovers Pruned LLMs' Generation Ability via Short-to-Long Distillation**

A new study proposes ShortOPD (Short-to-Long On-Policy Distillation) to address the collapse of pruned LLMs on free-form generation tasks. Using short-to-long distillation, it recovers generation quality from compressed checkpoints with only short-sequence training. Experiments show that pruned LLMs restored via ShortOPD achieve performance close to or exceeding the original model across multiple generation benchmarks, while being significantly smaller.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2607.13124)

**详情页**：https://ai.daily.yangsir.net/daily/20260717-T0-16

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*