{"ID":2923663,"CreatedAt":"2026-06-02T04:05:25.881865328Z","UpdatedAt":"2026-06-04T13:12:39.622923895Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.02252","arxiv_id":"2606.02252","title":"ResMerge: Residual-based Spectral Merging of Large Language Models","abstract":"Model merging offers a training-free way to combine multiple post-trained expert models, but merging experts obtained through reinforcement learning (RL) remains challenging. Existing spectral merging methods often assume that leading singular directions contain the main task signal, while lower-energy residual components can be compressed, selected, or attenuated to reduce interference. We find that this assumption does not hold for RL task vectors: after decomposing each task vector into a leading spectral head and a residual component, both parts can independently recover substantial behavior knowledge, while exhibiting different merging properties. The head is highly concentrated and informative but more prone to sharp cross-expert conflicts, whereas the residual component is more dispersed and provides a more stable basis for aggregation. Based on this observation, we propose ResMerge, a residual-based spectral merging framework for RL experts. ResMerge first constructs a stable residual backbone with Spherical Residual Consensus Adaptation, which estimates a reliability-weighted consensus direction on the Frobenius sphere. It then reintroduces leading-head information through a Lightweight Head Correction module gated by positive cross-expert agreement. Experiments across multiple RL expert groups and capability domains show that ResMerge better preserves expert capabilities than representative task-vector and spectral merging baselines. The implementation of ResMerge is publicly available at https://github.com/sunyd0303-cpu/ResMerge-release.","short_abstract":"Model merging offers a training-free way to combine multiple post-trained expert models, but merging experts obtained through reinforcement learning (RL) remains challenging. Existing spectral merging methods often assume that leading singular directions contain the main task signal, while lower-energy residual compone...","url_abs":"https://arxiv.org/abs/2606.02252","url_pdf":"https://arxiv.org/pdf/2606.02252v1","authors":"[\"Yandu Sun\",\"Zhiyan Hou\",\"Haokai Ma\",\"Yuheng Jia\",\"Junfeng Fang\",\"Haiyun Guo\",\"Hongyan An\",\"weizhen wang\",\"Jinqiao Wang\"]","published":"2026-06-01T13:42:45Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Reinforcement Learning\",\"Language Model\"]","has_code":false,"code_links":[{"ID":612676,"CreatedAt":"2026-06-02T04:05:25.881865328Z","UpdatedAt":"2026-06-02T04:05:25.881865328Z","DeletedAt":null,"paper_id":2923663,"paper_url":"https://arxiv.org/abs/2606.02252","paper_title":"ResMerge: Residual-based Spectral Merging of Large Language Models","repo_url":"https://github.com/sunyd0303-cpu/ResMerge-release","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}