{"ID":2886690,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.02068","arxiv_id":"2508.02068","title":"\"Set It Up\": Functional Object Arrangement with Compositional Generative Models (Journal Version)","abstract":"Functional object arrangement (FORM) is the task of arranging objects to fulfill a function, e.g., \"set up a dining table for two\". One key challenge here is that the instructions for FORM are often under-specified and do not explicitly specify the desired object goal poses. This paper presents SetItUp, a neuro-symbolic framework that learns to specify the goal poses of objects from a few training examples and a structured natural-language task specification. SetItUp uses a grounding graph, which is composed of abstract spatial relations among objects (e.g., left-of), as its intermediate representation. This decomposes the FORM problem into two stages: (i) predicting this graph among objects and (ii) predicting object poses given the grounding graph. For (i), SetItUp leverages large language models (LLMs) to induce Python programs from a task specification and a few training examples. This program can be executed to generate grounding graphs in novel scenarios. For (ii), SetItUp pre-trains a collection of diffusion models to capture primitive spatial relations and online composes these models to predict object poses based on the grounding graph. We evaluated SetItUp on a dataset spanning three distinct task families: arranging tableware on a dining table, organizing items on a bookshelf, and laying out furniture in a bedroom. Experiments show that SetItUp outperforms existing models in generating functional, physically feasible, and aesthetically pleasing object arrangements. This article extends our conference paper published at Robotics: Science and Systems (RSS) 2024.","short_abstract":"Functional object arrangement (FORM) is the task of arranging objects to fulfill a function, e.g., \"set up a dining table for two\". One key challenge here is that the instructions for FORM are often under-specified and do not explicitly specify the desired object goal poses. This paper presents SetItUp, a neuro-symboli...","url_abs":"https://arxiv.org/abs/2508.02068","url_pdf":"https://arxiv.org/pdf/2508.02068v2","authors":"[\"Yiqing Xu\",\"Jiayuan Mao\",\"Linfeng Li\",\"Yilun Du\",\"Tomas Lozáno-Pérez\",\"Leslie Pack Kaelbling\",\"David Hsu\"]","published":"2025-08-04T05:16:09Z","proceeding":"cs.RO","tasks":"[\"cs.RO\"]","methods":"[\"Diffusion Model\",\"Large Language Model\",\"Language Model\",\"Generative Adversarial Network\"]","has_code":false}