{"ID":2876417,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.00366","arxiv_id":"2509.00366","title":"KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation","abstract":"Despite recent progress, Graphic User Interface (GUI) agents powered by Large Language Models (LLMs) struggle with complex mobile tasks due to limited app-specific knowledge. While UI Transition Graphs (UTGs) offer structured navigation representations, they are underutilized due to poor extraction and inefficient integration. We introduce KG-RAG, a Knowledge Graph-driven Retrieval-Augmented Generation framework that transforms fragmented UTGs into structured vector databases for efficient real-time retrieval. By leveraging an intent-guided LLM search method, KG-RAG generates actionable navigation paths, enhancing agent decision-making. Experiments across diverse mobile apps show that KG-RAG outperforms existing methods, achieving a 75.8% success rate (8.9% improvement over AutoDroid), 84.6% decision accuracy (8.1% improvement), and reducing average task steps from 4.5 to 4.1. Additionally, we present KG-Android-Bench and KG-Harmony-Bench, two benchmarks tailored to the Chinese mobile ecosystem for future research. Finally, KG-RAG transfers to web/desktop (+40% SR on Weibo-web; +20% on QQ Music-desktop), and a UTG cost ablation shows accuracy saturates at ~4h per complex app, enabling practical deployment trade-offs.","short_abstract":"Despite recent progress, Graphic User Interface (GUI) agents powered by Large Language Models (LLMs) struggle with complex mobile tasks due to limited app-specific knowledge. While UI Transition Graphs (UTGs) offer structured navigation representations, they are underutilized due to poor extraction and inefficient inte...","url_abs":"https://arxiv.org/abs/2509.00366","url_pdf":"https://arxiv.org/pdf/2509.00366v1","authors":"[\"Ziyi Guan\",\"Jason Chun Lok Li\",\"Zhijian Hou\",\"Pingping Zhang\",\"Donglai Xu\",\"Yuzhi Zhao\",\"Mengyang Wu\",\"Jinpeng Chen\",\"Thanh-Toan Nguyen\",\"Pengfei Xian\",\"Wenao Ma\",\"Shengchao Qin\",\"Graziano Chesi\",\"Ngai Wong\"]","published":"2025-08-30T05:32:32Z","proceeding":"cs.MA","tasks":"[\"cs.MA\",\"cs.CL\",\"cs.MM\"]","methods":"[\"RAG\",\"Large Language Model\",\"Language Model\"]","has_code":false}