{"ID":2921841,"CreatedAt":"2026-06-02T02:42:49.606572591Z","UpdatedAt":"2026-06-03T19:17:13.289046021Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.01412","arxiv_id":"2606.01412","title":"GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation","abstract":"Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality. A common remedy is to augment the quantized weights with a low-rank correction, leading to approximations of the form $W\\approx Q+LR$. In this paper, we study this low-precision plus low-rank representation through the layer-wise reconstruction objective $\\|XW-X(Q+LR)\\|_F^2$, where $X$ is a calibration matrix. We establish, to our knowledge, the first information-theoretic lower bounds for this problem under finite-alphabet and bounded low-rank compensation constraints. We then propose GPTQ-intrinsic LoRA, a training-free algorithm that incorporates the low-rank correction directly into a GPTQ-style quantization pass by appropriately augmenting the calibration Hessian. For the choice $L=V_r$, where $V_r$ contains the top right singular vectors of $X$, we prove layer-wise reconstruction error bounds in which the usual GPTQ dependence on $\\|X\\|_F^2$ is replaced by the rank-$r$ residual $\\|X-X_r\\|_F^2$, up to regularization terms. Under natural structural assumptions, these bounds match the information-theoretic lower bounds in their dominant scaling, up to constants and mild factors. We also introduce Bid-Up, a fixed-grid quantization refinement step that can be alternated with optimal low-rank compensation with guaranteed non-increasing layer-wise reconstruction error. Experiments on Qwen3 language models and DeiT vision transformers show that GPTQ-intrinsic LoRA improves over GPTQ and GPTQ followed by low-rank compensation, with additional gains from refinement loops.","short_abstract":"Post-training quantization is widely used for compressing large neural networks, but aggressive low-bit quantization can significantly degrade model quality. A common remedy is to augment the quantized weights with a low-rank correction, leading to approximations of the form $W\\approx Q+LR$. In this paper, we study thi...","url_abs":"https://arxiv.org/abs/2606.01412","url_pdf":"https://arxiv.org/pdf/2606.01412v1","authors":"[\"Shihao Zhang\",\"Rayan Saab\"]","published":"2026-05-31T19:17:39Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.IT\"]","methods":"[\"Vision Transformer\",\"Transformer\",\"Language Model\",\"LoRA\"]","has_code":false}