---
id: 20260711-T0-09
title: "注意力机制新优化：不确定性门控选择，解决块稀疏注意力阈值判断问题"
title_en: "Uncertainty-Gated Selection Fixes Myopic Threshold Cutoff in Block-Sparse Attention"
url: https://ai.daily.yangsir.net/daily/20260711-T0-09
issue_date: 2026-07-11
publish_date: 2026-07-10T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2607.07724
---

# 注意力机制新优化：不确定性门控选择，解决块稀疏注意力阈值判断问题

块稀疏注意力机制通过只选择 top-k 个关键块来降低长上下文模型的计算成本。但研究发现，当第 k 个和第 k+1 个块的分数几乎相同时，简单切断会导致信息丢失。新提出的“不确定性门控选择”方法能在分数接近时引入不确定性度量并动态调整选择，从而在保持稀疏化效率的同时保留重要信息。

## English Version

**Uncertainty-Gated Selection Fixes Myopic Threshold Cutoff in Block-Sparse Attention**

Block-sparse attention reduces computation in long-context models by selecting only the top-k key blocks. However, research found that when the k-th and (k+1)-th blocks have nearly identical scores, a simple cutoff leads to information loss. The proposed 'uncertainty-gated selection' method introduces an uncertainty metric and dynamically adjusts the selection when scores are close, preserving important information while maintaining sparsity efficiency.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2607.07724)

**详情页**：https://ai.daily.yangsir.net/daily/20260711-T0-09

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*