{"ID":2867724,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.17786","arxiv_id":"2509.17786","title":"Accurate and Efficient Low-Rank Model Merging in Core Space","abstract":"In this paper, we address the challenges associated with merging low-rank adaptations of large neural networks. With the rise of parameter-efficient adaptation techniques, such as Low-Rank Adaptation (LoRA), model fine-tuning has become more accessible. While fine-tuning models with LoRA is highly efficient, existing merging methods often sacrifice this efficiency by merging fully-sized weight matrices. We propose the Core Space merging framework, which enables the merging of LoRA-adapted models within a common alignment basis, thereby preserving the efficiency of low-rank adaptation while substantially improving accuracy across tasks. We further provide a formal proof that projection into Core Space ensures no loss of information and provide a complexity analysis showing the efficiency gains. Extensive empirical results demonstrate that Core Space significantly improves existing merging techniques and achieves state-of-the-art results on both vision and language tasks while utilizing a fraction of the computational resources. Codebase is available at https://github.com/apanariello4/core-space-merging.","short_abstract":"In this paper, we address the challenges associated with merging low-rank adaptations of large neural networks. With the rise of parameter-efficient adaptation techniques, such as Low-Rank Adaptation (LoRA), model fine-tuning has become more accessible. While fine-tuning models with LoRA is highly efficient, existing m...","url_abs":"https://arxiv.org/abs/2509.17786","url_pdf":"https://arxiv.org/pdf/2509.17786v4","authors":"[\"Aniello Panariello\",\"Daniel Marczak\",\"Simone Magistri\",\"Angelo Porrello\",\"Bartłomiej Twardowski\",\"Andrew D. Bagdanov\",\"Simone Calderara\",\"Joost van de Weijer\"]","published":"2025-09-22T13:48:15Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[\"LoRA\"]","has_code":false,"code_links":[{"ID":609508,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2867724,"paper_url":"https://arxiv.org/abs/2509.17786","paper_title":"Accurate and Efficient Low-Rank Model Merging in Core Space","repo_url":"https://github.com/apanariello4/core-space-merging","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}