{"ID":2827039,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.17452","arxiv_id":"2512.17452","title":"KV Admission: Learning What to Write for Efficient Long-Context Inference","abstract":"Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear KV cache growth. Prior approaches mitigate this via post-hoc selection or eviction but overlook the root inefficiency: indiscriminate writing to memory. In this paper, we formalize KV cache management as a causal system of three primitives: KV Admission, Selection, and Eviction. We instantiate KV Admission via Write-Gated KV (WG-KV), a lightweight mechanism that learns to predict token utility before cache entry. By filtering out low-utility states early to maintain a compact global cache alongside a sliding local cache, WG-KV reduces memory usage by 46-68% and delivers 3.03-3.70x prefill and 1.85-2.56x decode speedups on Llama and Qwen models, while maintaining compatibility with FlashAttention and Paged-KV systems. These results demonstrate that learning what to write is a principled and practical recipe for efficient long-context inference. Code is available at https://github.com/EMCLab-Sinica/WG-KV.","short_abstract":"Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear KV cache growth. Prior approaches mitigate this via post-hoc selection or eviction but overlook the root inefficiency: indiscriminate writing to memory. In this paper, we formalize KV cache management as a causal system of three...","url_abs":"https://arxiv.org/abs/2512.17452","url_pdf":"https://arxiv.org/pdf/2512.17452v3","authors":"[\"Yen-Chieh Huang\",\"Pi-Cheng Hsiu\",\"Rui Fang\",\"Ming-Syan Chen\"]","published":"2025-12-19T11:08:58Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Large Language Model\"]","has_code":false,"code_links":[{"ID":605780,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2827039,"paper_url":"https://arxiv.org/abs/2512.17452","paper_title":"KV Admission: Learning What to Write for Efficient Long-Context Inference","repo_url":"https://github.com/EMCLab-Sinica/WG-KV","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}