{"ID":2852601,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.17234","arxiv_id":"2510.17234","title":"Taming Modality Entanglement in Continual Audio-Visual Segmentation","abstract":"Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previously learned ones. However, existing methods mainly focus on coarse-grained tasks, with limitations in addressing modality entanglement in fine-grained continual learning settings. To bridge this gap, we introduce a novel Continual Audio-Visual Segmentation (CAVS) task, aiming to continuously segment new classes guided by audio. Through comprehensive analysis, two critical challenges are identified: 1) multi-modal semantic drift, where a sounding objects is labeled as background in sequential tasks; 2) co-occurrence confusion, where frequent co-occurring classes tend to be confused. In this work, a Collision-based Multi-modal Rehearsal (CMR) framework is designed to address these challenges. Specifically, for multi-modal semantic drift, a Multi-modal Sample Selection (MSS) strategy is proposed to select samples with high modal consistency for rehearsal. Meanwhile, for co-occurence confusion, a Collision-based Sample Rehearsal (CSR) mechanism is designed, allowing for the increase of rehearsal sample frequency of those confusable classes during training process. Moreover, we construct three audio-visual incremental scenarios to verify effectiveness of our method. Comprehensive experiments demonstrate that our method significantly outperforms single-modal continual learning methods.","short_abstract":"Recently, significant progress has been made in multi-modal continual learning, aiming to learn new tasks sequentially in multi-modal settings while preserving performance on previously learned ones. However, existing methods mainly focus on coarse-grained tasks, with limitations in addressing modality entanglement in...","url_abs":"https://arxiv.org/abs/2510.17234","url_pdf":"https://arxiv.org/pdf/2510.17234v2","authors":"[\"Yuyang Hong\",\"Qi Yang\",\"Tao Zhang\",\"Zili Wang\",\"Zhaojin Fu\",\"Kun Ding\",\"Bin Fan\",\"Shiming Xiang\"]","published":"2025-10-20T07:23:36Z","proceeding":"cs.MM","tasks":"[\"cs.MM\",\"cs.AI\",\"cs.CV\"]","methods":"[]","has_code":false}