{"ID":23475102,"CreatedAt":"2026-09-18T01:09:05.407443952Z","UpdatedAt":"2026-09-20T18:11:56.143995915Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2609.19870","arxiv_id":"2609.19870","title":"Penquiry: A Pen-based Interactive In-situ Q\u0026A System Leveraging LLMs","abstract":"Pen-based digital devices remain a preferred medium for active, cognitively engaging study. Concurrently, Large Language Models (LLMs) have become indispensable for self-directed learning, enabling students to clarify concepts. However, a fundamental interaction gap exists between the fluid, spatial nature of pen-based workflows and the discrete, keyboard-heavy requirements of LLMs. We present Penquiry, an in-situ question-and-answer system that bridges this gap by enabling learners to pose questions directly on digital study materials via a pen. We characterize two primary interaction challenges in this multimodal transition: a Referential Barrier, which hinders grounding fine-grained visual elements into the query context, and an Expressive Barrier, which forces learners to translate diverse, non-textual intents--such as equations and diagrams--into rigid, typed sentences. To resolve these, Penquiry introduces a mediation layer featuring Content Snapping for unambiguous referencing and Question Autocompletion to expand sparse ink keywords into rich semantic queries. Through two iterative user studies (N = 16 per study), we demonstrate that Penquiry significantly reduces the cognitive and physical overhead of inquiry compared to traditional interfaces, providing a new blueprint for pen-based, in-situ AI interaction","short_abstract":"Pen-based digital devices remain a preferred medium for active, cognitively engaging study. Concurrently, Large Language Models (LLMs) have become indispensable for self-directed learning, enabling students to clarify concepts. However, a fundamental interaction gap exists between the fluid, spatial nature of pen-based...","url_abs":"https://arxiv.org/abs/2609.19870","url_pdf":"https://arxiv.org/pdf/2609.19870v1","authors":"[\"Jeongmin Rhee\",\"Changhee Lee\",\"Hyunwoo Kim\",\"Kiroong Choe\",\"Bohyoung Kim\",\"Sungahn Ko\",\"Jinwook Seo\"]","published":"2026-09-17T08:22:54Z","proceeding":"cs.HC","tasks":"[\"cs.HC\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}