{"ID":2865739,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.21413","arxiv_id":"2509.21413","title":"Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity","abstract":"Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data. This paper revisits two fundamental desiderata for DFCMM: stability, avoiding interference with earlier tasks, and plasticity, adapting faithfully to each new task. This poses a challenge that existing approaches fail to address: how to bridge data-level desiderata with parameter-space optimization to ensure stability and plasticity in the absence of task data. To this end, we propose NUFILT (NUll-space FILTering), a data-free framework that directly links these desiderata into parameter-space optimization. Our key observation is that task vectors approximately align with representation subspaces, providing structural surrogates for enforcing stability and plasticity. Accordingly, we design a null-space projector that preserves prior responses by filtering overlapping components of new task vectors, ensuring stability. We further introduce a lightweight LoRA adapter that injects complementary task-specific signals to enable plasticity. The adapter is trained with a projection-based surrogate loss that preserves consistency with prior knowledge while introducing novel directions. This joint filtering-adaptation process enables the backbone to absorb new knowledge while retaining existing behaviors, with updates fused back in a layer-wise linear fashion without extra parameters or inference cost. Theoretically, we establish approximate subspace alignment guarantees that justify null-space filtering. Empirically, NUFILT achieves state-of-the-art performance with minimal forgetting on both vision and NLP benchmarks, improving average accuracy by 4-7% over OPCM and WUDI-Merging, while narrowing the gap to fine-tuning and reducing computation overhead. The code is available at: https://github.com/zihuanqiu/NUFILT","short_abstract":"Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data. This paper revisits two fundamental desiderata for DFCMM: stability, avoiding interference with earlier tasks, and plasticity, adapting faithfully t...","url_abs":"https://arxiv.org/abs/2509.21413","url_pdf":"https://arxiv.org/pdf/2509.21413v2","authors":"[\"Zihuan Qiu\",\"Lei Wang\",\"Yang Cao\",\"Runtong Zhang\",\"Bing Su\",\"Yi Xu\",\"Fanman Meng\",\"Linfeng Xu\",\"Qingbo Wu\",\"Hongliang Li\"]","published":"2025-09-25T03:33:02Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"LoRA\"]","has_code":false,"code_links":[{"ID":609299,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2865739,"paper_url":"https://arxiv.org/abs/2509.21413","paper_title":"Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity","repo_url":"https://github.com/zihuanqiu/NUFILT","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}