TY - JOUR
T1 - A Survey on Error-Bounded Lossy Compression for Scientific Datasets
AU - Di, Sheng
AU - Liu, Jinyang
AU - Zhao, Kai
AU - Liang, Xin
AU - Underwood, Robert
AU - Zhang, Zhaorui
AU - Shah, Milan
AU - Huang, Yafan
AU - Huang, Jiajun
AU - Yu, Xiaodong
AU - Ren, Congrong
AU - Guo, Hanqi
AU - Wilkins, Grant
AU - Tao, Dingwen
AU - Tian, Jiannan
AU - Jin, Sian
AU - Jian, Zizhe
AU - Wang, Daoce
AU - Rahman, Md Hasanur
AU - Zhang, Boyuan
AU - Song, Shihui
AU - Calhoun, Jon
AU - Li, Guanpeng
AU - Yoshii, Kazutomo
AU - Alharthi, Khalid
AU - Cappello, Franck
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/6/11
Y1 - 2025/6/11
N2 - Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.
AB - Error-bounded lossy compression has been effective in significantly reducing the data storage/transfer burden while preserving the reconstructed data fidelity very well. Many error-bounded lossy compressors have been developed for a wide range of parallel and distributed use cases for years. They are designed with distinct compression models and principles, such that each of them features particular pros and cons. In this article, we provide a comprehensive survey of emerging error-bounded lossy compression techniques. The key contribution is fourfold. (1) We summarize a novel taxonomy of lossy compression into six classic models. (2) We provide a comprehensive survey of 10 commonly used compression components/modules. (3) We summarized pros and cons of 47 state-of-the-art lossy compressors and present how state-of-the-art compressors are designed based on different compression techniques. (4) We discuss how customized compressors are designed for specific scientific applications and use-cases. We believe this survey is useful to multiple communities including scientific applications, high-performance computing, lossy compression, and big data.
KW - Error-bounded lossy compression
KW - scientific applications
UR - https://www.scopus.com/pages/publications/105011277188
UR - https://www.scopus.com/pages/publications/105011277188#tab=citedBy
U2 - 10.1145/3733104
DO - 10.1145/3733104
M3 - Article
AN - SCOPUS:105011277188
SN - 0360-0300
VL - 57
JO - ACM Computing Surveys
JF - ACM Computing Surveys
IS - 11
M1 - 287
ER -