{"ID":6536352,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-14T06:08:37.952498173Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.10147","arxiv_id":"2607.10147","title":"REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation","abstract":"Automated chest X-ray report generation has recently benefited from reinforcement learning (RL) and large language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback-Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We propose Response-Weighted and Validation-Anchored Policy Optimization (REVA-PO), a RL framework that stabilizes long-term training via Response-Weighted Regularization (RER) and Validation-Anchored Policy Reset (VAPR). RER dynamically adjusts per-response KL weights based on advantage and reference-policy entropy, relaxing constraints for high-quality responses while tightening them for low-quality ones. Complementarily, VAPR periodically synchronizes the reference and current policies to the best validation checkpoint, resetting accumulated regularization pressure to expand the viable exploration space. To ensure a robust starting point, we employ a three-stage pipeline consisting of warm-up training, classifier-guided supervised fine-tuning, and RL. Extensive evaluations on MIMIC-CXR and IU-Xray demonstrate that REVA-PO sets new state-of-the-art benchmarks in both linguistic quality and clinical accuracy. Notably, BLEU-4 improves by 5.1% on MIMIC-CXR and 3.6% on IU-Xray, while CheXpert F1 and RadGraph F1 scores increase by 4.5% and 12.8%, respectively, over prior leading methods. The code is publicly available at https://github.com/LiGuo12/REVA_PO/.","short_abstract":"Automated chest X-ray report generation has recently benefited from reinforcement learning (RL) and large language models. However, RL training often suffers from instability or limited exploration due to fixed Kullback-Leibler (KL) regularization and a static reference policy that accumulates KL pressure over time. We...","url_abs":"https://arxiv.org/abs/2607.10147","url_pdf":"https://arxiv.org/pdf/2607.10147v1","authors":"[\"Li Guo\",\"Anas M. Tahir\",\"Z. Jane Wang\"]","published":"2026-07-11T06:15:03Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Reinforcement Learning\",\"Language Model\",\"LoRA\"]","has_code":false,"code_links":[{"ID":614161,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-14T01:21:01.169441415Z","DeletedAt":null,"paper_id":6536352,"paper_url":"https://arxiv.org/abs/2607.10147","paper_title":"REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation","repo_url":"https://github.com/LiGuo12/REVA_PO","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}