---
id: 20261021-T0-05
title: "PeekKV：第一个内容token就能判断哪些注意力头重要"
title_en: "PeekKV: The First Content Token Already Reveals Which Attention Heads Matter"
url: https://ai.daily.yangsir.net/daily/20261021-T0-05
issue_date: 2026-10-21
publish_date: 2026-10-20T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2610.23939
---

# PeekKV：第一个内容token就能判断哪些注意力头重要

KV cache随上下文长度线性增长，是LLM推理服务的主要显存瓶颈，近期不少工作利用注意力头的功能分工来做非均匀压缩。PeekKV的核心观察是：序列中第一个内容token的信息就足以判断哪些注意力头对当前请求重要，从而在生成开始前就定好压缩策略。这让KV cache压缩可以更早生效，减少长上下文场景下的显存占用，而不必等生成过程中再动态调整。

## English Version

**PeekKV: The First Content Token Already Reveals Which Attention Heads Matter**

KV cache grows linearly with context length and is a major memory bottleneck in LLM serving, and recent work exploits the functional roles of attention heads to compress it non-uniformly. PeekKV's key observation: the first content token in a sequence already carries enough signal to determine which attention heads matter for the current request, so the compression strategy can be fixed before generation begins. That lets KV cache compression kick in earlier and cuts memory use in long-context scenarios without waiting to adapt mid-generation.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2610.23939)

**详情页**：https://ai.daily.yangsir.net/daily/20261021-T0-05

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*