{"ID":2846425,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.03048","arxiv_id":"2511.03048","title":"ROBoto2: An Interactive System and Dataset for LLM-assisted Clinical Trial Risk of Bias Assessment","abstract":"We present ROBOTO2, an open-source, web-based platform for large language model (LLM)-assisted risk of bias (ROB) assessment of clinical trials. ROBOTO2 streamlines the traditionally labor-intensive ROB v2 (ROB2) annotation process via an interactive interface that combines PDF parsing, retrieval-augmented LLM prompting, and human-in-the-loop review. Users can upload clinical trial reports, receive preliminary answers and supporting evidence for ROB2 signaling questions, and provide real-time feedback or corrections to system suggestions. ROBOTO2 is publicly available at https://roboto2.vercel.app/, with code and data released to foster reproducibility and adoption. We construct and release a dataset of 521 pediatric clinical trial reports (8954 signaling questions with 1202 evidence passages), annotated using both manually and LLM-assisted methods, serving as a benchmark and enabling future research. Using this dataset, we benchmark ROB2 performance for 4 LLMs and provide an analysis into current model capabilities and ongoing challenges in automating this critical aspect of systematic review.","short_abstract":"We present ROBOTO2, an open-source, web-based platform for large language model (LLM)-assisted risk of bias (ROB) assessment of clinical trials. ROBOTO2 streamlines the traditionally labor-intensive ROB v2 (ROB2) annotation process via an interactive interface that combines PDF parsing, retrieval-augmented LLM promptin...","url_abs":"https://arxiv.org/abs/2511.03048","url_pdf":"https://arxiv.org/pdf/2511.03048v1","authors":"[\"Anthony Hevia\",\"Sanjana Chintalapati\",\"Veronica Ka Wai Lai\",\"Thanh Tam Nguyen\",\"Wai-Tat Wong\",\"Terry Klassen\",\"Lucy Lu Wang\"]","published":"2025-11-04T22:45:06Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\"]","project_urls":"[\"https://roboto2.vercel.app/\"]","has_code":false}