---
id: 20260509-T0-08
title: "LLM训练数据筛选：在线加权比离线挑选泛化能力更强"
title_en: "Online Reweighting Outperforms Offline Data Curation for LLM Generalization"
url: https://ai.daily.yangsir.net/daily/20260509-T0-08
issue_date: 2026-05-09
publish_date: 2026-05-08T04:00:00.000Z
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2605.05227
---

# LLM训练数据筛选：在线加权比离线挑选泛化能力更强

当前LLM训练中的数据筛选（数据选择、混合比例调整）几乎都离线进行，与训练过程脱节。这种脱节带来工程开销且效果有限。研究者证明，在线动态调整样本权重（online reweighting）能持续根据模型训练状态调整数据分布，泛化能力优于任何离线方法。实践意义明确：与其花大量时间在训练前精调数据集，不如在训练中实时调整。

## English Version

**Online Reweighting Outperforms Offline Data Curation for LLM Generalization**

Current LLM data curation operates offline, detached from training. Researchers show online reweighting—dynamically adjusting sample weights during training—delivers better generalization than any offline method. The practical takeaway: instead of spending heavy compute pre-curating datasets, adapt data distribution in real-time during training.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2605.05227)

**详情页**：https://ai.daily.yangsir.net/daily/20260509-T0-08

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*