{"ID":2839997,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.14391","arxiv_id":"2511.14391","title":"Enhancing LLM-based Autonomous Driving with Modular Traffic Light and Sign Recognition","abstract":"Large Language Models (LLMs) are increasingly used for decision-making and planning in autonomous driving, showing promising reasoning capabilities and potential to generalize across diverse traffic situations. However, current LLM-based driving agents lack explicit mechanisms to enforce traffic rules and often struggle to reliably detect small, safety-critical objects such as traffic lights and signs. To address this limitation, we introduce TLS-Assist, a modular redundancy layer that augments LLM-based autonomous driving agents with explicit traffic light and sign recognition. TLS-Assist converts detections into structured natural language messages that are injected into the LLM input, enforcing explicit attention to safety-critical cues. The framework is plug-and-play, model-agnostic, and supports both single-view and multi-view camera setups. We evaluate TLS-Assist in a closed-loop setup on the LangAuto benchmark in CARLA. The results demonstrate relative driving performance improvements of up to 14% over LMDrive and 7% over BEVDriver, while consistently reducing traffic light and sign infractions. We publicly release the code and models on https://github.com/iis-esslingen/TLS-Assist.","short_abstract":"Large Language Models (LLMs) are increasingly used for decision-making and planning in autonomous driving, showing promising reasoning capabilities and potential to generalize across diverse traffic situations. However, current LLM-based driving agents lack explicit mechanisms to enforce traffic rules and often struggl...","url_abs":"https://arxiv.org/abs/2511.14391","url_pdf":"https://arxiv.org/pdf/2511.14391v1","authors":"[\"Fabian Schmidt\",\"Noushiq Mohammed Kayilan Abdul Nazar\",\"Markus Enzweiler\",\"Abhinav Valada\"]","published":"2025-11-18T11:52:52Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":606935,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2839997,"paper_url":"https://arxiv.org/abs/2511.14391","paper_title":"Enhancing LLM-based Autonomous Driving with Modular Traffic Light and Sign Recognition","repo_url":"https://github.com/iis-esslingen/TLS-Assist","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}