{"ID":2876017,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.01626","arxiv_id":"2509.01626","title":"STZ: A High Quality and High Speed Streaming Lossy Compression Framework for Scientific Data","abstract":"Error-bounded lossy compression is one of the most efficient solutions to reduce the volume of scientific data. For lossy compression, progressive decompression and random-access decompression are critical features that enable on-demand data access and flexible analysis workflows. However, these features can severely degrade compression quality and speed. To address these limitations, we propose a novel streaming compression framework that supports both progressive decompression and random-access decompression while maintaining high compression quality and speed. Our contributions are three-fold: (1) we design the first compression framework that simultaneously enables both progressive decompression and random-access decompression; (2) we introduce a hierarchical partitioning strategy to enable both streaming features, along with a hierarchical prediction mechanism that mitigates the impact of partitioning and achieves high compression quality -- even comparable to state-of-the-art (SOTA) non-streaming compressor SZ3; and (3) our framework delivers high compression and decompression speed, up to 6.7$\\times$ faster than SZ3.","short_abstract":"Error-bounded lossy compression is one of the most efficient solutions to reduce the volume of scientific data. For lossy compression, progressive decompression and random-access decompression are critical features that enable on-demand data access and flexible analysis workflows. However, these features can severely d...","url_abs":"https://arxiv.org/abs/2509.01626","url_pdf":"https://arxiv.org/pdf/2509.01626v1","authors":"[\"Daoce Wang\",\"Pascal Grosset\",\"Jesus Pulido\",\"Jiannan Tian\",\"Tushar M. Athawale\",\"Jinda Jia\",\"Baixi Sun\",\"Boyuan Zhang\",\"Sian Jin\",\"Kai Zhao\",\"James Ahrens\",\"Fengguang Song\"]","published":"2025-09-01T17:14:31Z","proceeding":"cs.DC","tasks":"[\"cs.DC\",\"cs.MM\"]","methods":"[]","has_code":false}