---
id: 20260807-T0-11
title: "KV缓存向量量化新方法：按查询注意力分布分配位数，提速长文本推理"
title_en: "Attention-Aware KV Cache Quantization Boosts Long-Context LLM Speed"
url: https://ai.daily.yangsir.net/daily/20260807-T0-11
issue_date: 2026-08-07
publish_date: 2026-08-06T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2608.04074
---

# KV缓存向量量化新方法：按查询注意力分布分配位数，提速长文本推理

arXiv 新论文提出一种针对 KV 缓存的向量量化方法，核心思路是根据查询对缓存不同部分的注意力权重来分配量化位数。长文本模型解码时，读取 KV 缓存耗时超过注意力计算，成为吞吐瓶颈。该方法的改进点是让量化误差集中在注意力权重低的地方，从而在不明显影响生成质量的前提下减小缓存体积，提升解码速度和服务器承载量。

## English Version

**Attention-Aware KV Cache Quantization Boosts Long-Context LLM Speed**

A new arXiv paper proposes an attention-aware vector quantization method for KV caches, allocating more bits to cache regions that receive higher attention weights. Since long-context decoding is bandwidth-bound due to KV cache loading, this approach reduces cache size without significant quality loss, boosting decoding speed and serving capacity.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2608.04074)

**详情页**：https://ai.daily.yangsir.net/daily/20260807-T0-11

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*