{"ID":2865007,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.21848","arxiv_id":"2509.21848","title":"Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration","abstract":"As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering, which limit generalizability. In this work, we introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem, yielding an information-theoretic compression objective. Building on this framework, we propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective. For Llama 3.1 8B and Qwen3 8B across six document question answering benchmarks, GoA improves the average $F_1$ score of retrieval-augmented generation by 5.7\\% and a strong multi-agent baseline using a fixed collaboration structure by 16.35\\%, respectively. Even with only a 2K context window, GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length. Our source code is available at https://github.com/tjoo512/graph-of-agents.","short_abstract":"As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering...","url_abs":"https://arxiv.org/abs/2509.21848","url_pdf":"https://arxiv.org/pdf/2509.21848v1","authors":"[\"Taejong Joo\",\"Shu Ishida\",\"Ivan Sosnovik\",\"Bryan Lim\",\"Sahand Rezaei-Shoshtari\",\"Adam Gaier\",\"Robert Giaquinto\"]","published":"2025-09-26T04:15:40Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"RAG\",\"Language Model\"]","has_code":false,"code_links":[{"ID":609228,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2865007,"paper_url":"https://arxiv.org/abs/2509.21848","paper_title":"Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration","repo_url":"https://github.com/tjoo512/graph-of-agents","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}