{"ID":2829665,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.11218","arxiv_id":"2512.11218","title":"Seeing to Act, Prompting to Specify: A Bayesian Factorization of Vision Language Action Policy","abstract":"The pursuit of out-of-distribution generalization in Vision-Language-Action (VLA) models is often hindered by catastrophic forgetting of the Vision-Language Model (VLM) backbone during fine-tuning. While co-training with external reasoning data helps, it requires experienced tuning and data-related overhead. Beyond such external dependencies, we identify an intrinsic cause within VLA datasets: modality imbalance, where language diversity is much lower than visual and action diversity. This imbalance biases the model toward visual shortcuts and language forgetting. To address this, we introduce BayesVLA, a Bayesian factorization that decomposes the policy into a visual-action prior, supporting seeing-to-act, and a language-conditioned likelihood, enabling prompt-to-specify. This inherently preserves generalization and promotes instruction following. We further incorporate pre- and post-contact phases to better leverage pre-trained foundation models. Information-theoretic analysis formally validates our effectiveness in mitigating shortcut learning. Extensive experiments show superior generalization to unseen instructions, objects, and environments compared to existing methods. Project page is available at: https://xukechun.github.io/papers/BayesVLA.","short_abstract":"The pursuit of out-of-distribution generalization in Vision-Language-Action (VLA) models is often hindered by catastrophic forgetting of the Vision-Language Model (VLM) backbone during fine-tuning. While co-training with external reasoning data helps, it requires experienced tuning and data-related overhead. Beyond suc...","url_abs":"https://arxiv.org/abs/2512.11218","url_pdf":"https://arxiv.org/pdf/2512.11218v1","authors":"[\"Kechun Xu\",\"Zhenjie Zhu\",\"Anzhe Chen\",\"Shuqi Zhao\",\"Qing Huang\",\"Yifei Yang\",\"Haojian Lu\",\"Rong Xiong\",\"Masayoshi Tomizuka\",\"Yue Wang\"]","published":"2025-12-12T01:59:23Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.CV\"]","methods":"[\"Language Model\"]","has_code":false}