{"ID":23507398,"CreatedAt":"2026-09-18T02:21:44.056544415Z","UpdatedAt":"2026-09-20T18:11:56.143995915Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2609.20684","arxiv_id":"2609.20684","title":"HerHealthEval: Evaluating Multilingual and Register-Sensitive Understanding of Women's Health Communication","abstract":"Large language models are increasingly used in healthcare communication, yet most evaluations emphasize response quality while assuming that the user's concern has been interpreted correctly. We introduce HerHealthEval, a controlled evaluation framework for multilingual understanding of women's-health communication. For each clinical case, HerHealthEval provides matched versions in English, French, and Modern Standard Arabic using six communicative forms: canonical, clinical, layperson, indirect or hedged, emotionally concerned, and deliberately under-specified. The first five express the same underlying concern and retain the same clinical information, whereas the under-specified form intentionally omits relevant details to test whether the model recognizes that clarification is needed. We evaluate a multilingual instruction model and QLoRA-adapted variants on concern classification, risk calibration, clarification behavior, parse compliance, and cross-form consistency. Results reveal that aggregate accuracy and consistency can conceal safety-relevant failures. A multilingual adaptation model reaches 0.994 under-triage in French and Arabic under language-asymmetric risk supervision. A controlled re-adaptation using source-derived, language-invariant risk labels reduces under-triage to 0.572 and 0.558, respectively. These findings show that robust multilingual healthcare evaluation requires explicit testing of register variation, uncertainty handling, and the provenance and invariance of adaptation labels.","short_abstract":"Large language models are increasingly used in healthcare communication, yet most evaluations emphasize response quality while assuming that the user's concern has been interpreted correctly. We introduce HerHealthEval, a controlled evaluation framework for multilingual understanding of women's-health communication. Fo...","url_abs":"https://arxiv.org/abs/2609.20684","url_pdf":"https://arxiv.org/pdf/2609.20684v1","authors":"[\"Hassan Saeed Hassan Albattra\",\"Mazen Mohammed Bahgat\",\"Rahatara Ferdousi\",\"Hana Essam Sayed Ahmed Amrya\",\"Mariam Mousa\"]","published":"2026-09-17T16:53:47Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Language Model\",\"LoRA\"]","has_code":false}