{"ID":2836605,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.21893","arxiv_id":"2511.21893","title":"Breaking the Illusion: Consensus-Based Generative Mitigation of Adversarial Illusions in Multi-Modal Embeddings","abstract":"Multi-modal foundation models align images, text, and other modalities in a shared embedding space but remain vulnerable to adversarial illusions [35], where imperceptible perturbations disrupt cross-modal alignment and mislead downstream tasks. To counteract the effects of adversarial illusions, we propose a task-agnostic mitigation mechanism that purifies the attacker's perturbed input using generative models, e.g., Variational Autoencoders (VAEs), to restore natural alignment. To further enhance the defense mechanism, we adopt a generative sampling strategy combined with a consensus-based aggregation scheme over the outcomes of the generated samples. Our experiments on ImageBind, a state-of-the-art multi-modal encoder, show that our approach substantially reduces the illusion attack success rates to near-zero and improves cross-modal alignment in unperturbed and perturbed input settings, providing an effective and task-agnostic defense against adversarial illusions. The code is available at https://github.com/fatemehakb/adversarial-illusions-mitigation.","short_abstract":"Multi-modal foundation models align images, text, and other modalities in a shared embedding space but remain vulnerable to adversarial illusions [35], where imperceptible perturbations disrupt cross-modal alignment and mislead downstream tasks. To counteract the effects of adversarial illusions, we propose a task-agno...","url_abs":"https://arxiv.org/abs/2511.21893","url_pdf":"https://arxiv.org/pdf/2511.21893v2","authors":"[\"Fatemeh Akbarian\",\"Anahita Baninajjar\",\"Yingyi Zhang\",\"Ananth Balashankar\",\"Amir Aminifar\"]","published":"2025-11-26T20:18:20Z","proceeding":"cs.LG","tasks":"[\"cs.LG\"]","methods":"[\"Variational Autoencoder\"]","has_code":false,"code_links":[{"ID":606615,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2836605,"paper_url":"https://arxiv.org/abs/2511.21893","paper_title":"Breaking the Illusion: Consensus-Based Generative Mitigation of Adversarial Illusions in Multi-Modal Embeddings","repo_url":"https://github.com/fatemehakb/adversarial-illusions-mitigation","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}