---
id: 20261023-T0-05
title: "分离式LLM推理中KV缓存该留还是该丢？新论文研究缓存保留策略"
title_en: "New Paper Studies KV Cache Retention Policies in Disaggregated LLM Serving"
url: https://ai.daily.yangsir.net/daily/20261023-T0-05
issue_date: 2026-10-23
publish_date: 2026-10-22T04:00:00.000Z
category: research
source_name: "arXiv cs.LG (ML)"
source_url: https://arxiv.org/abs/2610.28637
---

# 分离式LLM推理中KV缓存该留还是该丢？新论文研究缓存保留策略

arXiv新论文指出，分离式LLM推理将prefill和decode分置于不同节点池，KV缓存的计算与消费之间隔着网络，这打破了传统缓存策略"未命中只带来本地计算开销"的假设。论文研究如何在分离架构下决定KV缓存的保留与淘汰，以平衡网络传输成本与重算成本。对部署分离式推理系统的团队来说，这套策略直接影响延迟和带宽开销。

## English Version

**New Paper Studies KV Cache Retention Policies in Disaggregated LLM Serving**

A new arXiv paper examines KV cache retention in disaggregated LLM serving, where prefill and decode run on separate node pools and a network fabric sits between cache computation and consumption. This breaks the classical cache assumption that a miss only incurs local recomputation cost. The work studies when to keep versus evict KV cache to balance network transfer against recomputation. The findings are directly relevant to teams running disaggregated inference, affecting latency and bandwidth costs.

---

**来源**：[arXiv cs.LG (ML)](https://arxiv.org/abs/2610.28637)

**详情页**：https://ai.daily.yangsir.net/daily/20261023-T0-05

---

*智语观潮 · Daily — https://ai.daily.yangsir.net/llms.txt*