---
id: 20260619-T0-07
title: "Gaussian Mixture Attention：线性时间复杂度的注意力机制新方案"
title_en: "Gaussian Mixture Attention: Achieving Linear-Time Sequence Mixing"
url: https://ai.daily.yangsir.net/daily/20260619-T0-07
issue_date: 2026-06-19
publish_date: 2026-06-18T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2606.18283
---

# Gaussian Mixture Attention：线性时间复杂度的注意力机制新方案

针对Transformer处理长上下文时的算力瓶颈，研究者提出了Gaussian Mixture Attention (GMA)。该方法通过概率隐路由机制，将密集的Token交互转化为稀疏的高斯混合模式。GMA实现了线性时间复杂度的序列混合，在不牺牲性能的前提下大幅降低了计算开销，为扩展长上下文模型架构提供了新的技术路径。

## English Version

**Gaussian Mixture Attention: Achieving Linear-Time Sequence Mixing**

To address the computational bottleneck in Transformers, researchers proposed Gaussian Mixture Attention (GMA). This method uses probabilistic latent routing to convert dense token interactions into sparse patterns, achieving linear-time sequence mixing. GMA significantly reduces computational costs for long contexts without sacrificing performance.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2606.18283)

**详情页**：https://ai.daily.yangsir.net/daily/20260619-T0-07

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*