{"ID":2885297,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.05492","arxiv_id":"2508.05492","title":"MoMA: A Mixture-of-Multimodal-Agents Architecture for Enhancing Clinical Prediction Modelling","abstract":"Multimodal electronic health record (EHR) data provide richer, complementary insights into patient health compared to single-modality data. However, effectively integrating diverse data modalities for clinical prediction modeling remains challenging due to the substantial data requirements. We introduce a novel architecture, Mixture-of-Multimodal-Agents (MoMA), designed to leverage multiple large language model (LLM) agents for clinical prediction tasks using multimodal EHR data. MoMA employs specialized LLM agents (\"specialist agents\") to convert non-textual modalities, such as medical images and laboratory results, into structured textual summaries. These summaries, together with clinical notes, are combined by another LLM (\"aggregator agent\") to generate a unified multimodal summary, which is then used by a third LLM (\"predictor agent\") to produce clinical predictions. Evaluating MoMA on three prediction tasks using real-world datasets with different modality combinations and prediction settings, MoMA outperforms current state-of-the-art methods, highlighting its enhanced accuracy and flexibility across various tasks.","short_abstract":"Multimodal electronic health record (EHR) data provide richer, complementary insights into patient health compared to single-modality data. However, effectively integrating diverse data modalities for clinical prediction modeling remains challenging due to the substantial data requirements. We introduce a novel archite...","url_abs":"https://arxiv.org/abs/2508.05492","url_pdf":"https://arxiv.org/pdf/2508.05492v1","authors":"[\"Jifan Gao\",\"Mahmudur Rahman\",\"John Caskey\",\"Madeline Oguss\",\"Ann O'Rourke\",\"Randy Brown\",\"Anne Stey\",\"Anoop Mayampurath\",\"Matthew M. Churpek\",\"Guanhua Chen\",\"Majid Afshar\"]","published":"2025-08-07T15:28:34Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"cs.MA\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}