---
id: 20261023-T0-06
title: "ReAPO：让 LLM 推理训练少走重复失败的老路"
title_en: "ReAPO Cuts Repeated Failed Strategies in LLM Reasoning Training"
url: https://ai.daily.yangsir.net/daily/20261023-T0-06
issue_date: 2026-10-23
publish_date: 2026-10-22T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2610.28770
---

# ReAPO：让 LLM 推理训练少走重复失败的老路

针对 RLVR 训练中独立随机采样容易反复采到相似失败策略的问题，论文提出 ReAPO（重试感知策略优化）。方法核心是让探索过程感知此前的重试结果，避免在已失败的策略上浪费采样预算，从而提升 LLM 推理训练的探索效率。这属于强化学习训练侧的优化，直接影响推理模型的训练成本与收敛速度，对自建 RL 训练流程的团队有参考价值。

## English Version

**ReAPO Cuts Repeated Failed Strategies in LLM Reasoning Training**

ReAPO (Retry-Aware Policy Optimization) targets a known inefficiency in RLVR training for LLM reasoning: independent stochastic rollouts often resample similar failed strategies. The method makes exploration aware of prior retries, so sampling budget is not wasted on strategies that already failed, improving exploration efficiency. This is a training-side optimization for reinforcement learning, directly affecting the cost and convergence speed of reasoning model training. It is relevant to teams running their own RL pipelines.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2610.28770)

**详情页**：https://ai.daily.yangsir.net/daily/20261023-T0-06

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*