{"ID":2831965,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.06721","arxiv_id":"2512.06721","title":"ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wild","abstract":"Recent studies have begun to explore proactive large language model (LLM) agents that provide unobtrusive assistance by automatically leveraging contextual information, such as in code editing and in-app suggestions. However, most focus on short, task-specific episodes or on-screen contexts, rather than continuously perceiving and assisting users throughout daily life. Enabling such in-the-wild assistance requires continuous sensing of users' surroundings, which can incur substantial system overhead. In this work, we propose ProAgent, an end-to-end proactive agent system that harnesses on-demand sensory contexts to provide in-the-wild assistance. ProAgent first employs on-demand tiered perception to continuously sense users' surroundings by integrating low-cost contextual cues with richer perception on demand, and uses proactive-oriented context extraction to derive hierarchical contexts integrating both sensory contexts and human preferences. ProAgent then employs a context-aware proactive reasoner to infer user needs and invokes external tools to deliver proactive assistance. We implement ProAgent on AR glasses and evaluate it on a public dataset and a real-world dataset. Results demonstrate that ProAgent achieves up to 27.7% higher proactive prediction accuracy and 20.5% lower false detection than state-of-the-art baselines. A user study with 20 participants shows that 85% were satisfied with ProAgent and willing to use it in daily life.","short_abstract":"Recent studies have begun to explore proactive large language model (LLM) agents that provide unobtrusive assistance by automatically leveraging contextual information, such as in code editing and in-app suggestions. However, most focus on short, task-specific episodes or on-screen contexts, rather than continuously pe...","url_abs":"https://arxiv.org/abs/2512.06721","url_pdf":"https://arxiv.org/pdf/2512.06721v2","authors":"[\"Bufang Yang\",\"Lilin Xu\",\"Liekang Zeng\",\"Yunqi Guo\",\"Siyang Jiang\",\"Wenrui Lu\",\"Kaiwei Liu\",\"Yixuan Li\",\"Xiaofan Jiang\",\"Guoliang Xing\",\"Zhenyu Yan\"]","published":"2025-12-07T08:21:07Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cs.CL\",\"cs.HC\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}