{"ID":2868947,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.16170","arxiv_id":"2509.16170","title":"UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation","abstract":"Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance. While existing methods address training-inference modality gaps via specialized per-combination models, they introduce high deployment costs by requiring exhaustive model subsets and model-modality matching. In this work, we propose a unified modality-relax segmentation network (UniMRSeg) through hierarchical self-supervised compensation (HSSC). Our approach hierarchically bridges representation gaps between complete and incomplete modalities across input, feature and output levels. % First, we adopt modality reconstruction with the hybrid shuffled-masking augmentation, encouraging the model to learn the intrinsic modality characteristics and generate meaningful representations for missing modalities through cross-modal fusion. % Next, modality-invariant contrastive learning implicitly compensates the feature space distance among incomplete-complete modality pairs. Furthermore, the proposed lightweight reverse attention adapter explicitly compensates for the weak perceptual semantics in the frozen encoder. Last, UniMRSeg is fine-tuned under the hybrid consistency constraint to ensure stable prediction under all modality combinations without large performance fluctuations. Without bells and whistles, UniMRSeg significantly outperforms the state-of-the-art methods under diverse missing modality scenarios on MRI-based brain tumor segmentation, RGB-D semantic segmentation, RGB-D/T salient object segmentation. The code will be released at https://github.com/Xiaoqi-Zhao-DLUT/UniMRSeg.","short_abstract":"Multi-modal image segmentation faces real-world deployment challenges from incomplete/corrupted modalities degrading performance. While existing methods address training-inference modality gaps via specialized per-combination models, they introduce high deployment costs by requiring exhaustive model subsets and model-m...","url_abs":"https://arxiv.org/abs/2509.16170","url_pdf":"https://arxiv.org/pdf/2509.16170v1","authors":"[\"Xiaoqi Zhao\",\"Youwei Pang\",\"Chenyang Yu\",\"Lihe Zhang\",\"Huchuan Lu\",\"Shijian Lu\",\"Georges El Fakhri\",\"Xiaofeng Liu\"]","published":"2025-09-19T17:29:25Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":609635,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2868947,"paper_url":"https://arxiv.org/abs/2509.16170","paper_title":"UniMRSeg: Unified Modality-Relax Segmentation via Hierarchical Self-Supervised Compensation","repo_url":"https://github.com/Xiaoqi-Zhao-DLUT/UniMRSeg","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}