{"ID":2843477,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.08367","arxiv_id":"2511.08367","title":"Why does weak-OOD help? A Further Step Towards Understanding Jailbreaking VLMs","abstract":"Large Vision-Language Models (VLMs) are susceptible to jailbreak attacks: researchers have developed a variety of attack strategies that can successfully bypass the safety mechanisms of VLMs. Among these approaches, jailbreak methods based on the Out-of-Distribution (OOD) strategy have garnered widespread attention due to their simplicity and effectiveness. This paper further advances the in-depth understanding of OOD-based VLM jailbreak methods. Experimental results demonstrate that jailbreak samples generated via mild OOD strategies exhibit superior performance in circumventing the safety constraints of VLMs--a phenomenon we define as ''weak-OOD''. To unravel the underlying causes of this phenomenon, this study takes SI-Attack, a typical OOD-based jailbreak method, as the research object. We attribute this phenomenon to a trade-off between two dominant factors: input intent perception and model refusal triggering. The inconsistency in how these two factors respond to OOD manipulations gives rise to this phenomenon. Furthermore, we provide a theoretical argument for the inevitability of such inconsistency from the perspective of discrepancies between model pre-training and alignment processes. Building on the above insights, we draw inspiration from optical character recognition (OCR) capability enhancement--a core task in the pre-training phase of mainstream VLMs. Leveraging this capability, we design a simple yet highly effective VLM jailbreak method, whose performance outperforms that of SOTA baselines.","short_abstract":"Large Vision-Language Models (VLMs) are susceptible to jailbreak attacks: researchers have developed a variety of attack strategies that can successfully bypass the safety mechanisms of VLMs. Among these approaches, jailbreak methods based on the Out-of-Distribution (OOD) strategy have garnered widespread attention due...","url_abs":"https://arxiv.org/abs/2511.08367","url_pdf":"https://arxiv.org/pdf/2511.08367v1","authors":"[\"Yuxuan Zhou\",\"Yuzhao Peng\",\"Yang Bai\",\"Kuofeng Gao\",\"Yihao Zhang\",\"Yechao Zhang\",\"Xun Chen\",\"Tao Yu\",\"Tao Dai\",\"Shu-Tao Xia\"]","published":"2025-11-11T15:46:44Z","proceeding":"cs.CR","tasks":"[\"cs.CR\"]","methods":"[\"Language Model\"]","has_code":false}