{"ID":3004915,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-05T10:21:46.366257699Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.03458","arxiv_id":"2606.03458","title":"KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks","abstract":"Test-time scaling is a powerful approach to obtain better reasoning in large language models, but it becomes memory-bottlenecked during long-horizon decoding, as the KV-cache grows. KV-cache quantization can help improve this, but current methods are evaluated under prefill-like settings and errors behave differently under autoregressive decoding. We show that in the latter regime, quantization errors accumulate across timesteps, driven primarily by incorrect token scales. We introduce KVarN, a calibration-free KV-cache quantizer that applies a Hadamard rotation followed by a dual-scaling variance normalization across both axes of the K and V matrices. We find that this combination fixes outlying token-scale errors and substantially reduces error accumulation over existing baselines. KVarN establishes a new state-of-theart for KV-cache quantization on generative benchmarks, including MATH500, AIME24 and HumanEval, at 2-bit precision. A vLLM implementation of the KVarN method is available at https://github.com/huawei-csl/KVarN","short_abstract":"Test-time scaling is a powerful approach to obtain better reasoning in large language models, but it becomes memory-bottlenecked during long-horizon decoding, as the KV-cache grows. KV-cache quantization can help improve this, but current methods are evaluated under prefill-like settings and errors behave differently u...","url_abs":"https://arxiv.org/abs/2606.03458","url_pdf":"https://arxiv.org/pdf/2606.03458v1","authors":"[\"Lorenz K. Muller\",\"Philippe Bich\",\"Chiara Boretti\",\"Hyun-Min Chang\",\"Jiawei Zhuang\",\"Lukas Cavigelli\"]","published":"2026-06-02T10:34:56Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":612718,"CreatedAt":"2026-06-03T03:09:48.883664427Z","UpdatedAt":"2026-06-03T03:09:48.883664427Z","DeletedAt":null,"paper_id":3004915,"paper_url":"https://arxiv.org/abs/2606.03458","paper_title":"KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks","repo_url":"https://github.com/huawei-csl/KVarN","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}