{"ID":5675786,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-04T13:53:05.627525587Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01299","arxiv_id":"2607.01299","title":"HYPIC: Accelerating Hybrid-Attention LLM Serving with Position-Independent Caching","abstract":"In retrieval augmented generation (RAG) and agentic LLM serving, prompts are assembled from independent segments into long contexts, making the prefill stage dominate the per-request computation cost. To this cost, two directions have emerged in parallel: position-independent caching (PIC) admits KV reuse for non-contiguous segments shared across different requests, while hybrid-attention models reduce computation complexity by replacing most full-attention layers with linear attention. However, they cannot coexist: applying PIC to hybrid-attention models breaks down because per-token KV-cache reuse primitives do not transfer to the per-request recurrent state. In this work, we present Hypic, the first serving system for hybrid-attention LLMs with position-independent caching. For linear-attention layers, we identify the segment-cumulative transition operator as the missing algebraic primitive, and cache it alongside each segment's zero-start end-state, enabling near-exact and constant-time state composition of independently cached segments. For the remaining full-attention layers, existing PIC methods also fail as linear layers do not expose the per-token hidden states for selective recomputation. We show that the most significant attention deviation concentrates at segment boundaries, so recomputing only a small seam window at each boundary suffices to restore cross-segment lookback. Finally, Hypic exploits segment-level self-containment to parallelize cache-miss prefill across instances, turning long cold requests -- a major tail-latency contributor under both prefix caching and prior PIC -- into an accelerable workload. Evaluated across four hybrid-attention models and five workloads, Hypic reduces time-to-first-token (TTFT) by 2.45x on average and improves peak throughput by up to 2.0x over existing systems, while staying within 3.3 points of full-recompute accuracy.","short_abstract":"In retrieval augmented generation (RAG) and agentic LLM serving, prompts are assembled from independent segments into long contexts, making the prefill stage dominate the per-request computation cost. To this cost, two directions have emerged in parallel: position-independent caching (PIC) admits KV reuse for non-conti...","url_abs":"https://arxiv.org/abs/2607.01299","url_pdf":"https://arxiv.org/pdf/2607.01299v1","authors":"[\"Yifei Liu\",\"Juntong Wu\",\"Yang Liu\",\"Junhao Hu\",\"Minghao Li\",\"Xiaoxu Chen\",\"Weihang Chen\"]","published":"2026-07-01T14:03:56Z","proceeding":"cs.DC","tasks":"[\"cs.DC\"]","methods":"[\"Large Language Model\"]","has_code":false}