{"ID":5346745,"CreatedAt":"2026-06-30T04:09:55.830587294Z","UpdatedAt":"2026-07-02T14:12:34.668891255Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.30355","arxiv_id":"2606.30355","title":"Residual-Guided Expert Specialization for Incomplete Multimodal Learning","abstract":"As real-world prediction systems often face missing modalities at inference, incomplete multimodal learning (IML) remains a practical challenge. While prior methods aim to learn representations robust to missing inputs, representations from incomplete modalities inevitably deviate from their full-modality counterparts due to missing evidence. To explicitly leverage these deviations, we propose MARS (Missingness-Aware Residual-guided Specialization), a mixture-of-experts framework that guides expert specialization based on how representations are reshaped by missingness. By contrasting task representations derived from incomplete inputs with their complete counterparts during training, we derive a privileged residual signal that captures this representational gap. The residual signal guides a residual router to assign samples to experts specialized for the corresponding deviation patterns. In parallel, a feature router learns to imitate this routing behavior using only incomplete inputs, enabling deployment without access to full modalities. To mitigate this train-test router gap, we develop a discrepancy-aware noise regularization that adaptively perturbs the residual router's decisions when the feature router deviates, enhancing expert robustness under imperfect imitation. Experiments on multimodal classification (CASIA-SURF, CREMA-D, UPMC Food-101) and segmentation (MCubeS) under missing scenarios show that MARS consistently surpasses baselines while remaining efficient and extensible to diverse backbones and tasks.","short_abstract":"As real-world prediction systems often face missing modalities at inference, incomplete multimodal learning (IML) remains a practical challenge. While prior methods aim to learn representations robust to missing inputs, representations from incomplete modalities inevitably deviate from their full-modality counterparts...","url_abs":"https://arxiv.org/abs/2606.30355","url_pdf":"https://arxiv.org/pdf/2606.30355v1","authors":"[\"Seunghun Baek\",\"Jihwan Park\",\"Jaeyoon Sim\",\"Minjae Jeong\",\"Hoseok Lee\",\"Won Hwa Kim\"]","published":"2026-06-29T14:24:05Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[]","has_code":false}