---
id: 20260327-T0-10
title: "MSA：内存稀疏注意力让长上下文模型扩展至1亿token"
title_en: "MSA Achieves 100M Token Context via Memory Sparse Attention"
url: https://ai.daily.yangsir.net/daily/20260327-T0-10
issue_date: 2026-03-27
publish_date: 2026-03-26T04:00:00.000Z
category: research
source_name: "arXiv cs.CL (NLP)"
source_url: https://arxiv.org/abs/2603.23516
---

# MSA：内存稀疏注意力让长上下文模型扩展至1亿token

论文提出MSA（Memory Sparse Attention）方法，通过稀疏注意力机制解决长上下文模型扩展难题，将有效上下文长度提升至1亿token。传统全注意力架构在处理超长文本时计算效率低下，而MSA通过动态选择关键token进行计算，在保持准确率的同时降低90%以上计算量。该方法为构建具备“终身记忆”能力的AI系统提供了新路径，未来可应用于大规模知识库检索和长文档分析。

## English Version

**MSA Achieves 100M Token Context via Memory Sparse Attention**

Researchers introduce MSA (Memory Sparse Attention), a method that enables large language models to scale to 100M token contexts by solving computational bottlenecks in long-text processing. By dynamically selecting key tokens for computation, MSA reduces computational overhead by over 90% while maintaining accuracy. This breakthrough paves the way for AI systems with 'lifetime memory' capabilities, applicable to large-scale knowledge retrieval and long-document analysis.

---

**来源**：[arXiv cs.CL (NLP)](https://arxiv.org/abs/2603.23516)

**详情页**：https://ai.daily.yangsir.net/daily/20260327-T0-10

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*