{"ID":5439552,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-02T23:45:32.241992796Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.30988","arxiv_id":"2606.30988","title":"Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force","abstract":"Robot manipulation often relies on sensory feedback beyond vision, particularly in contact-rich settings where force, tactile, or audio signals reveal interaction states that are not directly observable from images. However, these modalities are often hardware- and task-specific, and large-scale multisensory robot datasets remain scarce. As a result, it is impractical to pretrain policies with every sensor they may encounter. We study multisensory continual learning: adapting a pretrained robot policy to new tasks with newly introduced modalities while preserving performance under the original sensor suite. We propose MuSe, which incorporates limited multisensory data into pretrained vision-only policies through multi-stage fusion, multisensory future prediction, and experience replay over pretraining data. We instantiate MuSe by augmenting a pretrained vision-only policy with force-torque sensing and evaluate it on real-world manipulation tasks. Our experiments show that MuSe performs strongly on contact-rich finetuning tasks while preserving, and in some cases improving, performance on the original pretraining tasks. These results suggest that a modest multisensory dataset can improve general robot capabilities beyond the finetuning distribution. Project website: https://jadenvc.github.io/multisensory-continual-learning/","short_abstract":"Robot manipulation often relies on sensory feedback beyond vision, particularly in contact-rich settings where force, tactile, or audio signals reveal interaction states that are not directly observable from images. However, these modalities are often hardware- and task-specific, and large-scale multisensory robot data...","url_abs":"https://arxiv.org/abs/2606.30988","url_pdf":"https://arxiv.org/pdf/2606.30988v1","authors":"[\"Jaden Clark\",\"Changhao Wang\",\"Yihuai Gao\",\"Seongheon Hong\",\"Hojung Choi\",\"Mark Cutkosky\",\"Yifan Hou\",\"Shuran Song\"]","published":"2026-06-29T23:58:07Z","proceeding":"cs.RO","tasks":"[\"cs.RO\"]","methods":"[]","has_code":false}