---
id: 20260811-T0-02
title: "EntropyMoE：免分词器大模型的新稀疏路由方案"
title_en: "EntropyMoE: Entropy-Aware Sparse Routing for Tokenizer-Free LLMs"
url: https://ai.daily.yangsir.net/daily/20260811-T0-02
issue_date: 2026-08-11
publish_date: 2026-08-10T04:00:00.000Z
category: research
source_name: "arXiv cs.AI"
source_url: https://arxiv.org/abs/2608.06398
---

# EntropyMoE：免分词器大模型的新稀疏路由方案

EntropyMoE 论文提出一种适用于免分词器大模型的熵感知稀疏专家路由方法。现有字节级 LLM 通过动态分组的字节补丁替代了传统分词器，但在前馈计算中仍采用密集计算。EntropyMoE 根据熵（信息不确定性）动态路由，只激活必要的专家（MoE 模块），减少计算量。实验表明，该方法在保持性能的同时显著降低了推理成本，为构建更高效的免分词器大模型提供了新思路。

## English Version

**EntropyMoE: Entropy-Aware Sparse Routing for Tokenizer-Free LLMs**

The EntropyMoE paper introduces an entropy-aware sparse expert routing method for tokenizer-free LLMs. While byte-level LLMs replace traditional tokenizers with dynamically sized byte patches, they still apply dense feed-forward computation. EntropyMoE routes based on entropy (information uncertainty), activating only necessary experts (MoE modules) to cut computation. Experiments show that the method significantly reduces inference cost while maintaining performance, offering a new direction for building more efficient tokenizer-free LLMs.

---

**来源**：[arXiv cs.AI](https://arxiv.org/abs/2608.06398)

**详情页**：https://ai.daily.yangsir.net/daily/20260811-T0-02

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*