{"ID":2853919,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.15352","arxiv_id":"2510.15352","title":"GaussGym: An open-source real-to-sim framework for learning locomotion from pixels","abstract":"We present a novel approach for photorealistic robot simulation that integrates 3D Gaussian Splatting as a drop-in renderer within vectorized physics simulators such as IsaacGym. This enables unprecedented speed -- exceeding 100,000 steps per second on consumer GPUs -- while maintaining high visual fidelity, which we showcase across diverse tasks. We additionally demonstrate its applicability in a sim-to-real robotics setting. Beyond depth-based sensing, our results highlight how rich visual semantics improve navigation and decision-making, such as avoiding undesirable regions. We further showcase the ease of incorporating thousands of environments from iPhone scans, large-scale scene datasets (e.g., GrandTour, ARKit), and outputs from generative video models like Veo, enabling rapid creation of realistic training worlds. This work bridges high-throughput simulation and high-fidelity perception, advancing scalable and generalizable robot learning. All code and data will be open-sourced for the community to build upon. Videos, code, and data available at https://escontrela.me/gauss_gym/.","short_abstract":"We present a novel approach for photorealistic robot simulation that integrates 3D Gaussian Splatting as a drop-in renderer within vectorized physics simulators such as IsaacGym. This enables unprecedented speed -- exceeding 100,000 steps per second on consumer GPUs -- while maintaining high visual fidelity, which we s...","url_abs":"https://arxiv.org/abs/2510.15352","url_pdf":"https://arxiv.org/pdf/2510.15352v1","authors":"[\"Alejandro Escontrela\",\"Justin Kerr\",\"Arthur Allshire\",\"Jonas Frey\",\"Rocky Duan\",\"Carmelo Sferrazza\",\"Pieter Abbeel\"]","published":"2025-10-17T06:34:52Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.AI\",\"cs.GR\"]","methods":"[]","project_urls":"[\"https://escontrela.me/gauss_gym/\"]","has_code":false,"code_links":[{"ID":608096,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2853919,"paper_url":"https://arxiv.org/abs/2510.15352","paper_title":"GaussGym: An open-source real-to-sim framework for learning locomotion from pixels","repo_url":"https://github.com/escontra/gauss_gym","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0},{"ID":608097,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2853919,"paper_url":"https://arxiv.org/abs/2510.15352","paper_title":"GaussGym: An open-source real-to-sim framework for learning locomotion from pixels","repo_url":"https://github.com/nv-tlabs/NKSR","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0},{"ID":608098,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2853919,"paper_url":"https://arxiv.org/abs/2510.15352","paper_title":"GaussGym: An open-source real-to-sim framework for learning locomotion from pixels","repo_url":"https://github.com/nerfstudio-project/gsplat","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}