{"ID":2885605,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.04153","arxiv_id":"2508.04153","title":"ICM-Fusion: In-Context Meta-Optimized LoRA Fusion for Multi-Task Adaptation","abstract":"Enabling multi-task adaptation in pre-trained Low-Rank Adaptation (LoRA) models is crucial for enhancing their generalization capabilities. Most existing pre-trained LoRA fusion methods decompose weight matrices, sharing similar parameters while merging divergent ones. However, this paradigm inevitably induces inter-weight conflicts and leads to catastrophic domain forgetting. While incremental learning enables adaptation to multiple tasks, it struggles to achieve generalization in few-shot scenarios. Consequently, when the weight data follows a long-tailed distribution, it can lead to forgetting in the fused weights. To address this issue, we propose In-Context Meta LoRA Fusion (ICM-Fusion), a novel framework that synergizes meta-learning with in-context adaptation. The key innovation lies in our task vector arithmetic, which dynamically balances conflicting optimization directions across domains through learned manifold projections. ICM-Fusion obtains the optimal task vector orientation for the fused model in the latent space by adjusting the orientation of the task vectors. Subsequently, the fused LoRA is reconstructed by a self-designed Fusion VAE (F-VAE) to realize multi-task LoRA generation. We have conducted extensive experiments on visual and linguistic tasks, and the experimental results demonstrate that ICM-Fusion can be adapted to a wide range of architectural models and applied to various tasks. Compared to the current pre-trained LoRA fusion method, ICM-Fusion fused LoRA can significantly reduce the multi-tasking loss and can even achieve task enhancement in few-shot scenarios.","short_abstract":"Enabling multi-task adaptation in pre-trained Low-Rank Adaptation (LoRA) models is crucial for enhancing their generalization capabilities. Most existing pre-trained LoRA fusion methods decompose weight matrices, sharing similar parameters while merging divergent ones. However, this paradigm inevitably induces inter-we...","url_abs":"https://arxiv.org/abs/2508.04153","url_pdf":"https://arxiv.org/pdf/2508.04153v1","authors":"[\"Yihua Shao\",\"Xiaofeng Lin\",\"Xinwei Long\",\"Siyu Chen\",\"Minxi Yan\",\"Yang Liu\",\"Ziyang Yan\",\"Ao Ma\",\"Hao Tang\",\"Jingcai Guo\"]","published":"2025-08-06T07:28:25Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"LoRA\",\"Variational Autoencoder\"]","has_code":false}