{"ID":2869884,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.14030","arxiv_id":"2509.14030","title":"CrowdAgent: Multi-Agent Managed Multi-Source Annotation System","abstract":"High-quality annotated data is a cornerstone of modern Natural Language Processing (NLP). While recent methods begin to leverage diverse annotation sources-including Large Language Models (LLMs), Small Language Models (SLMs), and human experts-they often focus narrowly on the labeling step itself. A critical gap remains in the holistic process control required to manage these sources dynamically, addressing complex scheduling and quality-cost trade-offs in a unified manner. Inspired by real-world crowdsourcing companies, we introduce CrowdAgent, a multi-agent system that provides end-to-end process control by integrating task assignment, data annotation, and quality/cost management. It implements a novel methodology that rationally assigns tasks, enabling LLMs, SLMs, and human experts to advance synergistically in a collaborative annotation workflow. We demonstrate the effectiveness of CrowdAgent through extensive experiments on six diverse multimodal classification tasks. The source code and video demo are available at https://github.com/QMMMS/CrowdAgent.","short_abstract":"High-quality annotated data is a cornerstone of modern Natural Language Processing (NLP). While recent methods begin to leverage diverse annotation sources-including Large Language Models (LLMs), Small Language Models (SLMs), and human experts-they often focus narrowly on the labeling step itself. A critical gap remain...","url_abs":"https://arxiv.org/abs/2509.14030","url_pdf":"https://arxiv.org/pdf/2509.14030v1","authors":"[\"Maosheng Qin\",\"Renyu Zhu\",\"Mingxuan Xia\",\"Chenkai Chen\",\"Zhen Zhu\",\"Minmin Lin\",\"Junbo Zhao\",\"Lu Xu\",\"Changjie Fan\",\"Runze Wu\",\"Haobo Wang\"]","published":"2025-09-17T14:31:18Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":609727,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2869884,"paper_url":"https://arxiv.org/abs/2509.14030","paper_title":"CrowdAgent: Multi-Agent Managed Multi-Source Annotation System","repo_url":"https://github.com/QMMMS/CrowdAgent","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}