{"ID":5937132,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-09T11:06:26.522512815Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04953","arxiv_id":"2607.04953","title":"Real-World Perturbation Testing of Autonomous Driving Systems","abstract":"Autonomous Driving Systems (ADS) must operate reliably under diverse conditions, yet representative data for rare or adverse scenarios is difficult to obtain. Perturbation-based testing is widely used to assess robustness, but most studies focus on offline datasets or simulation, leaving open questions about how such results translate to real-world driving. We present a large-scale study of 72 camera and LiDAR perturbations, evaluated across three testing modalities: offline model-level analysis, hardware-in-the-loop execution, and closed-loop system-level testing on a full-scale autonomous vehicle. The study covers both an end-to-end vision-based driving model and a modular LiDAR-based perception and planning stack. Our results reveal a clear gap between testing levels. For camera-based systems, perturbations with limited offline impact can still induce unstable control and failures in real-world driving. For LiDAR-based systems, degradation is more consistent at the perception level but weakly predictive of system-level failures. Across both modalities, model-level metrics alone are insufficient to identify the most harmful perturbations. We further show that real-time feasibility is a key constraint in real-world testing, and that robustness observations obtained from recorded data do not consistently transfer to closed-loop behavior on a physical vehicle, highlighting the importance of complementary real-world, system-level evaluation.","short_abstract":"Autonomous Driving Systems (ADS) must operate reliably under diverse conditions, yet representative data for rare or adverse scenarios is difficult to obtain. Perturbation-based testing is widely used to assess robustness, but most studies focus on offline datasets or simulation, leaving open questions about how such r...","url_abs":"https://arxiv.org/abs/2607.04953","url_pdf":"https://arxiv.org/pdf/2607.04953v1","authors":"[\"Stefano Carlo Lambertenghi\",\"Matthias Weil\",\"Andrea Stocco\"]","published":"2026-07-06T11:34:31Z","proceeding":"cs.SE","tasks":"[\"cs.SE\",\"cs.RO\"]","methods":"[]","has_code":false}