---
id: 20260704-T0-02
title: "Kara：滑动窗口KV缓存压缩，让推理模型推理更快、吞吐更高"
title_en: "Kara Uses Sliding-Window KV Cache to Speed Up Reasoning LLMs by 50%"
url: https://ai.daily.yangsir.net/daily/20260704-T0-02
issue_date: 2026-07-04
publish_date: 2026-07-03T04:00:00.000Z
source_name: "arXiv cs.CL (NLP)"
source_url: https://arxiv.org/abs/2607.01237
---

# Kara：滑动窗口KV缓存压缩，让推理模型推理更快、吞吐更高

针对推理模型长链思维(CoT)导致KV缓存过大、解码延迟高的问题，Kara提出了一种滑动窗口KV缓存压缩方案。在LLaMA-3.1-8B-Instruct上，该方案能将延迟降低50%、吞吐提升3倍。开发者可直接将其应用于现有推理模型部署。

## English Version

**Kara Uses Sliding-Window KV Cache to Speed Up Reasoning LLMs by 50%**

Reasoning LLMs suffer from long chain-of-thought generation, causing massive KV cache overhead. Kara introduces a sliding-window KV cache compression method that reduces decoding latency by 50% and triples throughput on LLaMA-3.1-8B-Instruct. This approach can be directly applied to existing reasoning model deployments.

---

**来源**：[arXiv cs.CL (NLP)](https://arxiv.org/abs/2607.01237)

**详情页**：https://ai.daily.yangsir.net/daily/20260704-T0-02

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*