Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing

被引：1

作者：

Cappello, Franck ^{[1
]}

Acosta, Mario ^{[10
]}

Agullo, Emmanuel ^{[9
]}

Anzt, Hartwig ^{[12
]}

Calhoun, Jon ^{[7
]}

Di, Sheng ^{[2
]}

Giraud, Luc ^{[9
]}

Gruetzmacher, Thomas ^{[12
]}

Jin, Sian ^{[4
]}

Sano, Kentaro ^{[8
]}

Sato, Kento ^{[8
]}

Singh, Amarjit ^{[8
]}

Tao, Dingwen ^{[5
]}

Tian, Jiannan ^{[6
]}

Ueno, Tomohiro ^{[8
]}

Underwood, Robert ^{[3
]}

Vivien, Frederic ^{[9
]}

Yepes, Xavier ^{[11
]}

Kazutomo, Yoshii ^{[1
]}

Zhang, Boyuan ^{[6
]}

机构：

[1] Argonne Natl Lab, Lemont, IL 60439 USA

[2] Argonne Natl Lab, Math & Comp Sci MCS Div, Lemont, IL USA

[3] Argonne Natl Lab, Math & Comp Sci Div, Lemont, IL USA

[4] Indiana Univ, Bloomington, IN USA

[5] Indiana Univ, HighPerformance Data Analyt & Comp Lab, Bloomington, IN USA

[6] Indiana Univ, Intelligent Syst Engn, Bloomington, IN USA

[7] Clemson Univ, Holcombe Dept Elect & Comp Engn, Clemson, SC USA

[8] RIKEN, Ctr Computat Sci, Kobe, Japan

[9] Natl Res Inst Comp & Automat, Lyon, France

[10] Barcelona Supercomp Ctr, Earth Sci Dept, Computat Grp, Barcelona, Spain

[11] Barcelona Supercomp Ctr, Barcelona, Spain

[12] Karlsruhe Inst Technol, Karlsruhe, Germany

来源：

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE | 2025年 / 163卷

基金：

美国国家科学基金会;

关键词：

Lossy compression; Scientific data; Compression for AI; GPU acceleration; I/O scheduling; EARTH SYSTEM MODEL; GMRES; SIMULATION; I/O;

D O I：

10.1016/j.future.2024.05.022

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

The Joint Laboratory on Extreme-Scale Computing (JLESC) was initiated at the same time lossy compression for scientific data became an important topic for the scientific communities. The teams involved in the JLESC played and are still playing an important role in developing the research, techniques, methods, and technologies making lossy compression for scientific data a key tool for scientists and engineers. In this paper, we present the evolution of lossy compression for scientific data from 2015, describing the situation before the JLESC started, the evolution of this discipline in the past 8 years (until 2023) through the prism of the JLESC collaborations on this topic and some of the remaining open research questions.

引用

页数：27

共 50 条

[21] Performance Optimization for Relative-Error-Bounded Lossy Compression on Scientific Data
Zou, Xiangyu
Lu, Tao
Xia, Wen
Wang, Xuan
Zhang, Weizhe
Zhang, Haijun
Di, Sheng
Tao, Dingwen
Cappello, Franck
IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2020, 31 (07) : 1665 - 1680
[22] Optimizing Scientific Data Transfer on Globus with Error-bounded Lossy Compression
Liu, Yuanjian
Di, Sheng
Chard, Kyle
Foster, Ian
Cappello, Franck
2023 IEEE 43RD INTERNATIONAL CONFERENCE ON DISTRIBUTED COMPUTING SYSTEMS, ICDCS, 2023, : 703 - 713
[23] High-Ratio Lossy Compression: Exploring the Autoencoder to Compress Scientific Data
Liu, Tong
Wang, Jinzhen
Liu, Qing
Alibhai, Shakeel
Lu, Tao
He, Xubin
IEEE TRANSACTIONS ON BIG DATA, 2023, 9 (01) : 22 - 36
[24] Black-box statistical prediction of lossy compression ratios for scientific data
Underwood, Robert
Bessac, Julie
Krasowska, David
Calhoun, Jon C.
Di, Sheng
Cappello, Franck
INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS, 2023, 37 (3-4): : 412 - 433
[25] Efficient Lossy Compression for Scientific Data Based on Pointwise Relative Error Bound
Di, Sheng
Tao, Dingwen
Liang, Xin
Cappello, Franck
IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS, 2019, 30 (02) : 331 - 345
[26] Optimizing Error-Bounded Lossy Compression for Scientific Data by Dynamic Spline Interpolation
Zhao, Kai
Di, Sheng
Dmitriev, Maxim
Tonellot, Thierry-Laurent D.
Chen, Zizhong
Cappello, Franck
2021 IEEE 37TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE 2021), 2021, : 1643 - 1654
[27] A Data-driven Approach to Harvesting Latent Reduced Models to Precondition Lossy Compression for Scientific Data
Luo, Huizhang
Wang, Junqi
Qin, Zhenlu
Huang, Dan
Liu, Qing
Zhou, Mengchu
Jiang, Hong
IEEE TRANSACTIONS ON BIG DATA, 2023, 9 (03) : 949 - 963
[28] Amdahl's Laws and Extreme Data-Intensive Scientific Computing
Szalay, Alexander S.
ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XX, 2011, 442 : 405 - 414
[29] LZW Data Compression on Large Scale and Extreme Distributed Systems
De Agostino, Sergio
PROCEEDINGS OF THE PRAGUE STRINGOLOGY CONFERENCE 2012, 2012, : 18 - 27
[30] SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data
Qiao, Zhenbo
Lu, Tao
Luo, Huizhang
Liu, Qing
Klasky, Scott
Podhorszki, Norbert
Wang, Jinzhen
IEEE TRANSACTIONS ON MULTI-SCALE COMPUTING SYSTEMS, 2018, 4 (04): : 900 - 913

← 1 2 3 4 5 →