{"ID":2857022,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.10073","arxiv_id":"2510.10073","title":"SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents","abstract":"Large vision-language model (LVLM)-based web agents are emerging as powerful tools for automating complex online tasks. However, when deployed in real-world environments, they face serious security risks, motivating the design of security evaluation benchmarks. Existing benchmarks provide only partial coverage, typically restricted to narrow scenarios such as user-level prompt manipulation, and thus fail to capture the broad range of agent vulnerabilities. To address this gap, we present \\tool{}, the first holistic benchmark for evaluating the security of LVLM-based web agents. \\tool{} first introduces a unified evaluation suite comprising six simulated but realistic web environments (\\eg, e-commerce platforms, community forums) and includes 2,970 high-quality trajectories spanning diverse tasks and attack settings. The suite defines a structured taxonomy of six attack vectors spanning both user-level and environment-level manipulations. In addition, we introduce a multi-layered evaluation protocol that analyzes agent failures across three critical dimensions: internal reasoning, behavioral trajectory, and task outcome, facilitating a fine-grained risk analysis that goes far beyond simple success metrics. Using this benchmark, we conduct large-scale experiments on 9 representative LVLMs, which fall into three categories: general-purpose, agent-specialized, and GUI-grounded. Our results show that all tested agents are consistently vulnerable to subtle adversarial manipulations and reveal critical trade-offs between model specialization and security. By providing (1) a comprehensive benchmark suite with diverse environments and a multi-layered evaluation pipeline, and (2) empirical insights into the security challenges of modern LVLM-based web agents, \\tool{} establishes a foundation for advancing trustworthy web agent deployment.","short_abstract":"Large vision-language model (LVLM)-based web agents are emerging as powerful tools for automating complex online tasks. However, when deployed in real-world environments, they face serious security risks, motivating the design of security evaluation benchmarks. Existing benchmarks provide only partial coverage, typical...","url_abs":"https://arxiv.org/abs/2510.10073","url_pdf":"https://arxiv.org/pdf/2510.10073v2","authors":"[\"Zonghao Ying\",\"Yangguang Shao\",\"Jianle Gan\",\"Gan Xu\",\"Wenxin Zhang\",\"Quanchen Zou\",\"Junzheng Shi\",\"Zhenfei Yin\",\"Mingchuan Zhang\",\"Aishan Liu\",\"Xianglong Liu\"]","published":"2025-10-11T07:18:12Z","proceeding":"cs.CR","tasks":"[\"cs.CR\",\"cs.CV\"]","methods":"[\"Language Model\"]","has_code":false}