{"ID":6537717,"CreatedAt":"2026-07-14T02:54:43.516908796Z","UpdatedAt":"2026-07-15T03:28:55.185153975Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.11167","arxiv_id":"2607.11167","title":"Pix2Act: Image-Space Manipulation Policies with Equivariant Augmentation","abstract":"Representing manipulation actions as 2D trajectories in the camera plane provides a compact and interpretable basis for learning complex 3D manipulation policies. However, it also creates challenges from out-of-frame trajectories and limited precision. We propose Pix2Act, an imitation learning method that addresses these challenges by generating continuous image-space keypoint trajectories in each camera plane and losslessly recovering end-effector poses via triangulation. This reformulates high-dimensional 3D control as a simpler, more learnable 2D prediction problem. Crucially, it aligns observations and actions in the same coordinate space, enabling equivariant transformations to jointly rotate individual camera images together with their image-space actions. We analyze the symmetry properties of this augmentation and design a network architecture that can fuse multiple camera views while respecting their per-view rotations. As a result, Pix2Act implicitly enlarges the support of the data distribution and learns invariant action structures across transformations, yielding improved generalization and overall performance. Across diverse simulated and real-world manipulation tasks, Pix2Act outperforms state-of-the-art baselines and remains robust under camera perturbations.","short_abstract":"Representing manipulation actions as 2D trajectories in the camera plane provides a compact and interpretable basis for learning complex 3D manipulation policies. However, it also creates challenges from out-of-frame trajectories and limited precision. We propose Pix2Act, an imitation learning method that addresses the...","url_abs":"https://arxiv.org/abs/2607.11167","url_pdf":"https://arxiv.org/pdf/2607.11167v1","authors":"[\"Haojie Huang\",\"Linfeng Zhao\",\"Haotian Liu\",\"Zhang Ye\",\"Si-Yuan Huang\",\"Mingxi Jia\",\"Boce Hu\",\"Fangzhou Lin\",\"Yu Qi\",\"Dian Wang\",\"Robin Walters\",\"Robert Platt\"]","published":"2026-07-13T07:03:29Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.AI\",\"cs.LG\"]","methods":"[]","has_code":false}