---
id: 20260320-T0-11
title: "HoloByte：无需词组的连续超球蒸馏建模方案"
title_en: "HoloByte: Tokenizer-Free Modeling via Continuous Hyperspherical Distillation"
url: https://ai.daily.yangsir.net/daily/20260320-T0-11
issue_date: 2026-03-20
publish_date: 2026-03-19T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2603.16917
---

# HoloByte：无需词组的连续超球蒸馏建模方案

HoloByte研究提出了一种无需词组的序列建模方法。传统方法依赖子词分词处理字节级注意力，计算复杂度为O(N²)。该方案通过连续超球蒸馏技术，避免了人工形态边界，直接在字节级进行建模。实验显示，该方法在保持模型性能的同时，显著降低了计算复杂度，为无分词建模提供了新思路。

## English Version

**HoloByte: Tokenizer-Free Modeling via Continuous Hyperspherical Distillation**

HoloByte introduces a tokenizer-free sequence modeling method that avoids the O(N²) computational complexity of byte-level attention. By using continuous hyperspherical distillation, the approach eliminates artificial morphological boundaries, directly modeling at the byte level. This method maintains performance while reducing computational overhead, offering a novel solution for token-free modeling.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2603.16917)

**详情页**：https://ai.daily.yangsir.net/daily/20260320-T0-11

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*