{"ID":2846986,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.00859","arxiv_id":"2511.00859","title":"Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion","abstract":"In autonomous driving, transparency in the decision-making of perception models is critical, as even a single misperception can be catastrophic. Yet with multi-sensor inputs, it is difficult to determine how each modality contributes to a prediction because sensor information becomes entangled within the fusion network. We introduce Layer-Wise Modality Decomposition (LMD), a post-hoc, model-agnostic interpretability method that disentangles modality-specific information across all layers of a pretrained fusion model. To our knowledge, LMD is the first approach to attribute the predictions of a perception model to individual input modalities in a sensor-fusion system for autonomous driving. We evaluate LMD on pretrained fusion models under camera-radar, camera-LiDAR, and camera-radar-LiDAR settings for autonomous driving. Its effectiveness is validated using structured perturbation-based metrics and modality-wise visual decompositions, demonstrating practical applicability to interpreting high-capacity multimodal architectures. Code is available at https://github.com/detxter-jvb/Layer-Wise-Modality-Decomposition.","short_abstract":"In autonomous driving, transparency in the decision-making of perception models is critical, as even a single misperception can be catastrophic. Yet with multi-sensor inputs, it is difficult to determine how each modality contributes to a prediction because sensor information becomes entangled within the fusion network...","url_abs":"https://arxiv.org/abs/2511.00859","url_pdf":"https://arxiv.org/pdf/2511.00859v1","authors":"[\"Jaehyun Park\",\"Konyul Park\",\"Daehun Kim\",\"Junseo Park\",\"Jun Won Choi\"]","published":"2025-11-02T08:52:24Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":607477,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2846986,"paper_url":"https://arxiv.org/abs/2511.00859","paper_title":"Layer-Wise Modality Decomposition for Interpretable Multimodal Sensor Fusion","repo_url":"https://github.com/detxter-jvb/Layer-Wise-Modality-Decomposition","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}