An Optimized Error-controlled MPI Collective Framework Integrated with Lossy Compression

Jiajun Huang, Sheng Di, Xiaodong Yu, Yujia Zhai, Zhaorui Zhang, Jinyang Liu, Xiaoyi Lu, Ken Raffenetti, Hui Zhou, Kai Zhao, Zizhong Chen, Franck Cappello, Yanfei Guo, Rajeev Thakur

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

With the ever-increasing computing power of supercomputers and the growing scale of scientific applications, the efficiency of MPI collective communications turns out to be a critical bottleneck in large-scale distributed and parallel processing. The large message size in MPI collectives is particularly concerning because it can significantly degrade the overall parallel performance. To address this issue, prior research simply applies the off-the-shelf fix-rate lossy compressors in the MPI collectives, leading to suboptimal performance, limited generalizability, and unbounded errors. In this paper, we propose a novel solution, called C-Coll, which leverages error-bounded lossy compression to significantly reduce the message size, resulting in a substantial reduction in communication cost. The key contributions are three-fold. (1) We develop two general, optimized lossy-compression-based frameworks for both types of MPI collectives (collective data movement as well as collective computation), based on their particular characteristics. Our framework not only reduces communication cost but also preserves data accuracy. (2) We customize SZx, an ultra-fast error-bounded lossy compressor, to meet the specific needs of collective communication. (3) We integrate C-Coll into multiple collectives, such as MPI Allreduce, MPI Scatter, and MPI Bcast, and perform a comprehensive evaluation based on real-world scientific datasets. Experiments show that our solution outperforms the original MPI collectives as well as multiple baselines and related efforts by 1.8-2.7×.

Original languageEnglish
Title of host publicationProceedings - 2024 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2024
Pages752-764
Number of pages13
ISBN (Electronic)9798350337662
DOIs
StatePublished - 2024
Event38th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2024 - San Francisco, United States
Duration: 27 May 202431 May 2024

Publication series

NameProceedings - 2024 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2024

Conference

Conference38th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2024
Country/TerritoryUnited States
CitySan Francisco
Period27/05/2431/05/24

Keywords

  • Distributed Systems
  • Lossy Compression
  • MPI Collective

Fingerprint

Dive into the research topics of 'An Optimized Error-controlled MPI Collective Framework Integrated with Lossy Compression'. Together they form a unique fingerprint.

Cite this