{"ID":5675104,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-05T03:20:12.707043107Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01658","arxiv_id":"2607.01658","title":"Teaching Vision-Language-Action Models What to See and Where to Look","abstract":"Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual question answering and chain-of-thought reasoning data, which emphasizes linguistic reasoning rather than action-grounded planning. As a result, the learned representations capture semantic knowledge but lack spatial dependencies crucial for reliable trajectory prediction. We propose DriveTeach-VLA, a framework that explicitly teaches VLAs what to see and where to look. Driving-aware Vision Distillation (DVD) injects driving-specific perceptual priors into the vision encoder, while 2D Trajectory-Guided Prompts (2D-TGP) provide spatial conditioning aligned with feasible driving trajectories. Together, they form a vision-guided learning pipeline: what to see (DVD pretraining) - where to look (TGP-guided SFT) - how to act (TGP-guided GRPO). DriveTeach-VLA achieves the state-of-the-art performance on NAVSIM and nuScenes. Our code is available at: https://github.com/ShivaTeam/DriveTeach-VLA.","short_abstract":"Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing VLAs' training relies heavily on text-centric visual question answering and chain-of-thought reasoning data, which emphasizes linguistic reasoning rather than action-grounded planning. As a resu...","url_abs":"https://arxiv.org/abs/2607.01658","url_pdf":"https://arxiv.org/pdf/2607.01658v1","authors":"[\"Yuguang Yang\",\"Canyu Chen\",\"Zhewen Tan\",\"Yizhi Wang\",\"Zichao Feng\",\"Chunyang Liu\",\"Kehua Sheng\",\"Juan Zhang\",\"Linlin Yang\",\"Baochang Zhang\",\"Yan Wang\",\"Bo Zhang\",\"Xianbin Cao\"]","published":"2026-07-02T03:34:32Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":613874,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-03T01:40:09.565152011Z","DeletedAt":null,"paper_id":5675104,"paper_url":"https://arxiv.org/abs/2607.01658","paper_title":"Teaching Vision-Language-Action Models What to See and Where to Look","repo_url":"https://github.com/ShivaTeam/DriveTeach-VLA","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}