{"ID":6621301,"CreatedAt":"2026-07-15T01:01:48.440468303Z","UpdatedAt":"2026-07-15T03:28:55.185153975Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.12211","arxiv_id":"2607.12211","title":"Overcoming Orchestration Bottlenecks at Exascale: A Decentralized, Policy-Driven Approach for Sim-AI Ensembles","abstract":"Scientific computing is increasingly shifting from monolithic applications to coupled simulation-AI workflows composed of highly heterogeneous tasks with diverse hardware, scale, and runtime requirements. As these workflows scale to leadership-class systems, the resulting extreme ensemble sizes and task variability can create orchestration bottlenecks. System-level schedulers are often configured for limited throughput, while workflow tools face scalability issues due to rigid control-plane topologies and static scheduling heuristics. We introduce EnsembleLauncher, a recursively hierarchical workflow orchestrator for exascale systems, featuring a fully decentralized control plane and a programmable scheduling policy interface. On the Aurora supercomputer, EnsembleLauncher successfully scales to the entire machine with up to eight million serial tasks, outperforming state-of-the-art tools by more than four times. Additionally, we implement a programmable scheduling interface and demonstrate a significant impact of scheduling policies on resource utilization for high-variance ensembles and active learning pipelines representative of modern coupled simulation-AI workflows.","short_abstract":"Scientific computing is increasingly shifting from monolithic applications to coupled simulation-AI workflows composed of highly heterogeneous tasks with diverse hardware, scale, and runtime requirements. As these workflows scale to leadership-class systems, the resulting extreme ensemble sizes and task variability can...","url_abs":"https://arxiv.org/abs/2607.12211","url_pdf":"https://arxiv.org/pdf/2607.12211v1","authors":"[\"Harikrishna Tummalapalli\",\"Christine M. Simpson\",\"Riccardo Balin\",\"Vitali A. Morozov\",\"Thang D. Pham\",\"Murat Keceli\",\"Thomas D. Uram\"]","published":"2026-07-13T23:19:22Z","proceeding":"cs.DC","tasks":"[\"cs.DC\",\"cs.CE\"]","methods":"[]","has_code":false}