{"ID":2849114,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.24428","arxiv_id":"2510.24428","title":"CodeWiki: Evaluating AI's Ability to Generate Holistic Documentation for Large-Scale Codebases","abstract":"Given a large and evolving codebase, the ability to automatically generate holistic, architecture-aware documentation that captures not only individual functions but also cross-file, cross-module, and system-level interactions remains an open challenge. Comprehensive documentation is essential for long-term software maintenance and collaboration, yet current automated approaches still fail to model the rich semantic dependencies and architectural structures that define real-world software systems. We present \\textbf{CodeWiki}, a unified framework for automated repository-level documentation across seven programming languages. CodeWiki introduces three key innovations: (i) hierarchical decomposition that preserves architectural context across multiple levels of granularity, (ii) recursive multi-agent processing with dynamic task delegation for scalable generation, and (iii) multi-modal synthesis that integrates textual descriptions with visual artifacts such as architecture diagrams and data-flow representations. To enable rigorous evaluation, we introduce \\textbf{CodeWikiBench}, a comprehensive benchmark featuring multi-dimensional rubrics and LLM-based assessment protocols. Experimental results show that CodeWiki achieves a 68.79\\% quality score with proprietary models, outperforming the closed-source DeepWiki baseline (64.06\\%) by 4.73\\%, with particularly strong improvements on high-level scripting languages (+10.47\\%). We open-source CodeWiki to foster future research and community adoption.","short_abstract":"Given a large and evolving codebase, the ability to automatically generate holistic, architecture-aware documentation that captures not only individual functions but also cross-file, cross-module, and system-level interactions remains an open challenge. Comprehensive documentation is essential for long-term software ma...","url_abs":"https://arxiv.org/abs/2510.24428","url_pdf":"https://arxiv.org/pdf/2510.24428v6","authors":"[\"Anh Nguyen Hoang\",\"Minh Le-Anh\",\"Bach Le\",\"Nghi D. Q. Bui\"]","published":"2025-10-28T13:52:46Z","proceeding":"cs.SE","tasks":"[\"cs.SE\"]","methods":"[\"Large Language Model\"]","has_code":false}