{"ID":2860057,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.06277","arxiv_id":"2510.06277","title":"General and Efficient Visual Goal-Conditioned Reinforcement Learning using Object-Agnostic Masks","abstract":"Goal-conditioned reinforcement learning (GCRL) allows agents to learn diverse objectives using a unified policy. The success of GCRL, however, is contingent on the choice of goal representation. In this work, we propose a mask-based goal representation system that provides object-agnostic visual cues to the agent, enabling efficient learning and superior generalization. In contrast, existing goal representation methods, such as target state images, 3D coordinates, and one-hot vectors, face issues of poor generalization to unseen objects, slow convergence, and the need for special cameras. Masks can be processed to generate dense rewards without requiring error-prone distance calculations. Learning with ground truth masks in simulation, we achieved 99.9% reaching accuracy on training and unseen test objects. Our proposed method can be utilized to perform pick-up tasks with high accuracy, without using any positional information of the target. Moreover, we demonstrate learning from scratch and sim-to-real transfer applications using two different physical robots, utilizing pretrained open vocabulary object detection models for mask generation.","short_abstract":"Goal-conditioned reinforcement learning (GCRL) allows agents to learn diverse objectives using a unified policy. The success of GCRL, however, is contingent on the choice of goal representation. In this work, we propose a mask-based goal representation system that provides object-agnostic visual cues to the agent, enab...","url_abs":"https://arxiv.org/abs/2510.06277","url_pdf":"https://arxiv.org/pdf/2510.06277v1","authors":"[\"Fahim Shahriar\",\"Cheryl Wang\",\"Alireza Azimi\",\"Gautham Vasan\",\"Hany Hamed Elanwar\",\"A. Rupam Mahmood\",\"Colin Bellinger\"]","published":"2025-10-06T18:20:35Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.LG\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}