{"ID":2866261,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.21670","arxiv_id":"2509.21670","title":"MORPH: PDE Foundation Models with Arbitrary Data Modality","abstract":"We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields with mixed scalar and vector components. The architecture combines (i) component-wise convolution, which jointly processes scalar and vector channels to capture local interactions, (ii) inter-field cross-attention, which models and selectively propagates information between different physical fields, (iii) axial attentions, which factorize full spatiotemporal self-attention along individual spatial and temporal axes to reduce computational burden while retaining expressivity. We pretrain multiple model variants on a diverse collection of heterogeneous PDE datasets and evaluate transfer to a range of downstream prediction tasks. Using both full-model fine-tuning and parameter-efficient low-rank adapters, MORPH outperforms models trained from scratch. Across extensive evaluations, MORPH matches or surpasses strong baselines and recent state-of-the-art models. Collectively, these capabilities present a flexible and powerful backbone for learning from the heterogeneous and multimodal nature of scientific observations, charting a path toward scalable and data-efficient scientific machine learning. The source code, datasets, and models are publicly available at https://github.com/lanl/MORPH.","short_abstract":"We introduce MORPH, a modality-agnostic, autoregressive foundation model for partial differential equations (PDEs). MORPH is built on a convolutional vision transformer backbone that seamlessly handles heterogeneous spatiotemporal datasets of varying data modality (1D--3D) at different resolutions, and multiple fields...","url_abs":"https://arxiv.org/abs/2509.21670","url_pdf":"https://arxiv.org/pdf/2509.21670v4","authors":"[\"Mahindra Singh Rautela\",\"Alexander Most\",\"Siddharth Mansingh\",\"Bradley C. Love\",\"Alexander Scheinker\",\"Diane Oyen\",\"Nathan Debardeleben\",\"Earl Lawrence\",\"Ayan Biswas\"]","published":"2025-09-25T22:38:36Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\",\"cs.LG\",\"physics.comp-ph\"]","methods":"[\"Vision Transformer\",\"Transformer\"]","has_code":false,"code_links":[{"ID":609356,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2866261,"paper_url":"https://arxiv.org/abs/2509.21670","paper_title":"MORPH: PDE Foundation Models with Arbitrary Data Modality","repo_url":"https://github.com/lanl/MORPH","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}