{"ID":2885177,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.05283","arxiv_id":"2508.05283","title":"Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue","abstract":"Meta-reviewing is a pivotal stage in the peer-review process, serving as the final step in determining whether a paper is recommended for acceptance. Prior research on meta-reviewing has treated this as a summarization problem over review reports. However, complementary to this perspective, meta-reviewing is a decision-making process that requires weighing reviewer arguments and placing them within a broader context. Prior research has demonstrated that decision-makers can be effectively assisted in such scenarios via dialogue agents. In line with this framing, we explore the practical challenges for realizing dialog agents that can effectively assist meta-reviewers. Concretely, we first address the issue of data scarcity for training dialogue agents by generating synthetic data using Large Language Models (LLMs) based on a self-refinement strategy to improve the relevance of these dialogues to expert domains. Our experiments demonstrate that this method produces higher-quality synthetic data and can serve as a valuable resource towards training meta-reviewing assistants. Subsequently, we utilize this data to train dialogue agents tailored for meta-reviewing and find that these agents outperform \\emph{off-the-shelf} LLM-based assistants for this task. Finally, we apply our agents in real-world meta-reviewing scenarios and confirm their effectiveness in enhancing the efficiency of meta-reviewing.\\footnote{Code available at: https://github.com/UKPLab/eacl2026-meta-review-as-dialog","short_abstract":"Meta-reviewing is a pivotal stage in the peer-review process, serving as the final step in determining whether a paper is recommended for acceptance. Prior research on meta-reviewing has treated this as a summarization problem over review reports. However, complementary to this perspective, meta-reviewing is a decision...","url_abs":"https://arxiv.org/abs/2508.05283","url_pdf":"https://arxiv.org/pdf/2508.05283v2","authors":"[\"Sukannya Purkayastha\",\"Nils Dycke\",\"Anne Lauscher\",\"Iryna Gurevych\"]","published":"2025-08-07T11:27:43Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":611160,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2885177,"paper_url":"https://arxiv.org/abs/2508.05283","paper_title":"Decision-Making with Deliberation: Meta-reviewing as a Document-grounded Dialogue","repo_url":"https://github.com/UKPLab/eacl2026-meta-review-as-dialog","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}