{"ID":2847206,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.00446","arxiv_id":"2511.00446","title":"ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training","abstract":"The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from large-scale web data through self-supervised contrastive learning. Yet, its reliance on uncurated Internet-sourced data exposes it to data poisoning and backdoor risks. While existing studies primarily investigate image-based attacks, the text modality, which is equally central to CLIP's training, remains underexplored. In this work, we introduce ToxicTextCLIP, a framework for generating high-quality adversarial texts that target CLIP during the pre-training phase. The framework addresses two key challenges: semantic misalignment caused by background inconsistency with the target class, and the scarcity of background-consistent texts. To this end, ToxicTextCLIP iteratively applies: 1) a background-aware selector that prioritizes texts with background content aligned to the target class, and 2) a background-driven augmenter that generates semantically coherent and diverse poisoned samples. Extensive experiments on classification and retrieval tasks show that ToxicTextCLIP achieves up to 95.83% poisoning success and 98.68% backdoor Hit@1, while bypassing RoCLIP, CleanCLIP and SafeCLIP defenses. The source code can be accessed via https://github.com/xinyaocse/ToxicTextCLIP/.","short_abstract":"The Contrastive Language-Image Pretraining (CLIP) model has significantly advanced vision-language modeling by aligning image-text pairs from large-scale web data through self-supervised contrastive learning. Yet, its reliance on uncurated Internet-sourced data exposes it to data poisoning and backdoor risks. While exi...","url_abs":"https://arxiv.org/abs/2511.00446","url_pdf":"https://arxiv.org/pdf/2511.00446v1","authors":"[\"Xin Yao\",\"Haiyang Zhao\",\"Yimin Chen\",\"Jiawei Guo\",\"Kecheng Huang\",\"Ming Zhao\"]","published":"2025-11-01T08:25:49Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.CR\",\"cs.LG\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":607496,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2847206,"paper_url":"https://arxiv.org/abs/2511.00446","paper_title":"ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training","repo_url":"https://github.com/xinyaocse/ToxicTextCLIP","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}