{"ID":2867962,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.02839","arxiv_id":"2511.02839","title":"Evaluating Generative AI as an Educational Tool for Radiology Resident Report Drafting","abstract":"Objective: Radiology residents require timely, personalized feedback to develop accurate image analysis and reporting skills. Increasing clinical workload often limits attendings' ability to provide guidance. This study evaluates a HIPAA-compliant GPT-4o system that delivers automated feedback on breast imaging reports drafted by residents in real clinical settings. Methods: We analyzed 5,000 resident-attending report pairs from routine practice at a multi-site U.S. health system. GPT-4o was prompted with clinical instructions to identify common errors and provide feedback. A reader study using 100 report pairs was conducted. Four attending radiologists and four residents independently reviewed each pair, determined whether predefined error types were present, and rated GPT-4o's feedback as helpful or not. Agreement between GPT and readers was assessed using percent match. Inter-reader reliability was measured with Krippendorff's alpha. Educational value was measured as the proportion of cases rated helpful. Results: Three common error types were identified: (1) omission or addition of key findings, (2) incorrect use or omission of technical descriptors, and (3) final assessment inconsistent with findings. GPT-4o showed strong agreement with attending consensus: 90.5%, 78.3%, and 90.4% across error types. Inter-reader reliability showed moderate variability (α = 0.767, 0.595, 0.567), and replacing a human reader with GPT-4o did not significantly affect agreement (Δ = -0.004 to 0.002). GPT's feedback was rated helpful in most cases: 89.8%, 83.0%, and 92.0%. Discussion: ChatGPT-4o can reliably identify key educational errors. It may serve as a scalable tool to support radiology education.","short_abstract":"Objective: Radiology residents require timely, personalized feedback to develop accurate image analysis and reporting skills. Increasing clinical workload often limits attendings' ability to provide guidance. This study evaluates a HIPAA-compliant GPT-4o system that delivers automated feedback on breast imaging reports...","url_abs":"https://arxiv.org/abs/2511.02839","url_pdf":"https://arxiv.org/pdf/2511.02839v1","authors":"[\"Antonio Verdone\",\"Aidan Cardall\",\"Fardeen Siddiqui\",\"Motaz Nashawaty\",\"Danielle Rigau\",\"Youngjoon Kwon\",\"Mira Yousef\",\"Shalin Patel\",\"Alex Kieturakis\",\"Eric Kim\",\"Laura Heacock\",\"Beatriu Reig\",\"Yiqiu Shen\"]","published":"2025-09-22T20:51:09Z","proceeding":"cs.HC","tasks":"[\"cs.HC\",\"cs.AI\",\"cs.CY\"]","methods":"[]","has_code":false}