{"ID":6267006,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-12T01:50:31.311443228Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.08057","arxiv_id":"2607.08057","title":"Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization","abstract":"Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware KV infrastructure for serving LLMs (abbreviated as sKis). We revisit recent work from a system behavior perspective, organizing existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural). Furthermore, we analyze cross-behavior co-design affinity and behavior-objective links, highlighting future opportunities. Our work systematizes a rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.","short_abstract":"Despite the rapid advancements of large language models (LLMs), LLM serving systems remain memory-intensive and costly. The key-value (KV) cache, which stores KV tensors during autoregressive decoding, is crucial for enabling low-latency, high-throughput LLM inference serving. In this survey, we focus on system-aware K...","url_abs":"https://arxiv.org/abs/2607.08057","url_pdf":"https://arxiv.org/pdf/2607.08057v1","authors":"[\"Jiantong Jiang\",\"Peiyu Yang\",\"Rui Zhang\",\"Feng Liu\"]","published":"2026-07-09T02:11:18Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\",\"Generative Adversarial Network\"]","has_code":false}