---
id: 20260814-T0-08
title: "词表大小不是拍脑袋：部署场景决定LLM成本最优配置"
title_en: "Lifecycle-Optimal Tokenization: Vocabulary Size Tied to Deployment"
url: https://ai.daily.yangsir.net/daily/20260814-T0-08
issue_date: 2026-08-14
publish_date: 2026-08-13T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2608.11361
---

# 词表大小不是拍脑袋：部署场景决定LLM成本最优配置

arXiv新论文指出，LLM分词器词表大小通常按惯例在训练时固定，而非根据部署需求分析确定。研究发现，成本最优的词表大小并非固定值，而是取决于部署场景——如推理硬件、批处理规模和生成长度。调整词表大小可在不影响质量的情况下显著降低推理成本，为模型部署提供新的优化维度。

## English Version

**Lifecycle-Optimal Tokenization: Vocabulary Size Tied to Deployment**

A new arXiv paper argues that tokenizer vocabulary size is typically set at training time by convention, not deployment analysis. The authors show the cost-optimal vocabulary varies with deployment context—such as inference hardware, batch size, and generation length—and that tuning it can significantly cut inference costs without quality loss, opening a new optimization lever for model operators.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2608.11361)

**详情页**：https://ai.daily.yangsir.net/daily/20260814-T0-08

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*