{"ID":5439486,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-02T19:06:01.127452785Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.30848","arxiv_id":"2606.30848","title":"StreamGuard: Low-Overhead Resilience for Real-time HPC Data Streams","abstract":"Real-time scientific workflows operate on continuous data streams and must produce timely, high-quality results despite executing on complex, failure-prone infrastructure. Hardware faults, network disruptions, and performance anomalies caused by resource contention or system heterogeneity can severely degrade performance and violate real-time constraints. We focus on strengthening the resilience of the producer-consumer streaming pattern, a fundamental building block of scientific streaming workflows. We present two complementary techniques: (i) a dynamic, asynchronous, non-blocking checkpointing mechanism that preserves progress without interrupting computation, and (ii) a progress-aware load redistribution strategy that detects slow workers and proactively rebalances tasks. Together, these mechanisms maintain forward progress and balanced execution even in highly error-prone environments. Experimental results show that our approach reduces the impact of failures and performance anomalies by up to 6x, while introducing less than 1% overhead in failure-free execution.","short_abstract":"Real-time scientific workflows operate on continuous data streams and must produce timely, high-quality results despite executing on complex, failure-prone infrastructure. Hardware faults, network disruptions, and performance anomalies caused by resource contention or system heterogeneity can severely degrade performan...","url_abs":"https://arxiv.org/abs/2606.30848","url_pdf":"https://arxiv.org/pdf/2606.30848v1","authors":"[\"Hai Duc Nguyen\",\"Bogdan Nicolae\",\"Tekin Bicer\",\"Amal Gueroudji\",\"Matthieu Dorier\",\"Kyle Chard\",\"Ian Foster\"]","published":"2026-06-29T19:24:49Z","proceeding":"cs.DC","tasks":"[\"cs.DC\"]","methods":"[]","has_code":false}