{"ID":5553605,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-02T05:03:02.375444803Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31292","arxiv_id":"2606.31292","title":"AtomiMed: Hierarchical Atomic Fact-Checking for Universal Clinical-Aware Medical Report Evaluation","abstract":"Traditional metrics for Medical Report Generation (MRG) predominantly rely on surface-level n-gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing \\textbf{AtomiMed}, a universal, modality-agnostic evaluation framework that decomposes complex medical narratives into a standardized, multi-level hierarchy of Atomic Clinical Facts, encompassing Disease-level entities and Attribute-level descriptors, including location, morphology, and severity. By implementing an Agentic Cross-Verification loop between ground-truth and predicted reports, AtomiMed simulates a multi-radiologist peer-review process to verify clinical consistency, thus enabling the decoupled assessment of diagnostic detection and descriptive accuracy. To facilitate standardized evaluation, we introduce \\textbf{MRGEvalKit}, an open-source toolkit for automated hierarchical extraction, and curate \\textbf{OmniMRG-Bench}, a comprehensive multi-modal benchmark covering X-ray, CT, MRI, and Ultrasound. Extensive experiments on multiple expert-annotated reader studies demonstrate that AtomiMed achieves significantly higher correlation with human radiologist judgment compared to traditional and model-based metrics. Our code are release at https://github.com/Venn2336/MRGEvalkit","short_abstract":"Traditional metrics for Medical Report Generation (MRG) predominantly rely on surface-level n-gram overlap, which fails to capture clinical factual accuracy and often overlooks catastrophic diagnostic errors. We address this fundamental limitation by proposing \\textbf{AtomiMed}, a universal, modality-agnostic evaluatio...","url_abs":"https://arxiv.org/abs/2606.31292","url_pdf":"https://arxiv.org/pdf/2606.31292v1","authors":"[\"Yuan Wang\",\"Wanxing Chang\",\"Songtao Jiang\",\"Shujian Gao\",\"Xiaotian Zhang\",\"Ruifeng Yuan\",\"Weiwei Cao\",\"Bowen Shi\",\"Ling Zhang\",\"Zuozhu Liu\",\"Jianpeng Zhang\"]","published":"2026-06-30T08:07:45Z","proceeding":"cs.CE","tasks":"[\"cs.CE\"]","methods":"[]","has_code":false,"code_links":[{"ID":613868,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-02T01:54:51.863792489Z","DeletedAt":null,"paper_id":5553605,"paper_url":"https://arxiv.org/abs/2606.31292","paper_title":"AtomiMed: Hierarchical Atomic Fact-Checking for Universal Clinical-Aware Medical Report Evaluation","repo_url":"https://github.com/Venn2336/MRGEvalkit","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}