---
id: 20261001-T0-10
title: "MaD-RL：用分布匹配校准LLM的强化学习新方法"
title_en: "MaD-RL Matches Distributions to Calibrate LLMs via Reinforcement Learning"
url: https://ai.daily.yangsir.net/daily/20261001-T0-10
issue_date: 2026-10-01
publish_date: 2026-09-30T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2609.31644
---

# MaD-RL：用分布匹配校准LLM的强化学习新方法

针对强化学习后训练中奖励最大化容易导致模型校准失准的问题，研究者提出MaD-RL方法，通过匹配分布来校准语言模型。传统RL后训练只关注单个输出获得的奖励分数（如二元验证器或人类反馈训练的奖励模型），忽略了输出整体的分布特性，导致模型在合成数据等场景下出现过拟合或校准偏差。MaD-RL的核心思路是在优化奖励的同时，让模型输出分布与目标分布保持一致，从而在提升任务表现的同时保持校准性。该方法适用于需要可靠概率输出的场景，如合成数据生成和验证器训练。

## English Version

**MaD-RL Matches Distributions to Calibrate LLMs via Reinforcement Learning**

Researchers propose MaD-RL, a method that calibrates language models by matching distributions during reinforcement learning post-training. Standard RL post-training maximizes rewards assigned to individual outputs—such as scores from binary verifiers or reward models trained on human feedback—but this can distort the model's overall output distribution and hurt calibration. MaD-RL addresses this by aligning the model's output distribution with a target distribution while optimizing rewards. The approach is relevant for applications requiring reliable probability estimates, including synthetic data generation and verifier training.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2609.31644)

**详情页**：https://ai.daily.yangsir.net/daily/20261001-T0-10

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*