{"ID":6029823,"CreatedAt":"2026-07-08T02:57:47.77373338Z","UpdatedAt":"2026-07-10T17:41:27.792927618Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.06555","arxiv_id":"2607.06555","title":"ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation","abstract":"Tracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself-such as 3D models, depth maps, object masks, or task-specific learned features-and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6-DoF pose tracking as video-to-video translation. Given only a video and a single marked pixel in the first frame, a fine-tuned video diffusion model translates the input into a proxy video-a synthetic video depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel. Because the proxy's geometry and appearance are known by construction, recovering its full 6-DoF trajectory reduces to classical pose estimation with off-the-shelf solvers. This formulation leverages large-scale video pre-training to absorb the hardest aspects of pose tracking-handling challenging materials, occlusions, and deformations-into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state-of-the-art 6-DoF pose tracking accuracy without the additional inputs required by competing methods and after fine-tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in-the-wild scenes that are beyond the reach of existing approaches. Project page: https://ruihangzhang97.github.io/proxypose/.","short_abstract":"Tracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself-such as 3D models, depth maps, object masks, or task-specific learned features-and they struggle wit...","url_abs":"https://arxiv.org/abs/2607.06555","url_pdf":"https://arxiv.org/pdf/2607.06555v1","authors":"[\"Ruihang Zhang\",\"Felix Taubner\",\"Pooja Ravi\",\"Kiriakos N. Kutulakos\",\"David B. Lindell\"]","published":"2026-07-07T17:56:07Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Diffusion Model\"]","has_code":false}