Submitted as: development and technical paper 02 Jun 2020
Submitted as: development and technical paper | 02 Jun 2020
Adaptive lossy compression of climate model data based on hierarchical tensor with Adaptive-HGFDR (v1.0)
- 1Key Laboratory of Virtual Geographic Environment, Ministry of Education, Nanjing Normal University, Nanjing, China
- 2Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, China
- 3Jiangsu Key Laboratory of Crop Genetics and Physiology/Jiangsu Key Laboratory of Crop Cultivation and Physiology, Agricultural College of Yangzhou University, Yangzhou, China
- 4Jiangsu Co Innovation Center for Modern Production Technology of Grain Crops, Yangzhou University, Yangzhou, China
- 1Key Laboratory of Virtual Geographic Environment, Ministry of Education, Nanjing Normal University, Nanjing, China
- 2Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, China
- 3Jiangsu Key Laboratory of Crop Genetics and Physiology/Jiangsu Key Laboratory of Crop Cultivation and Physiology, Agricultural College of Yangzhou University, Yangzhou, China
- 4Jiangsu Co Innovation Center for Modern Production Technology of Grain Crops, Yangzhou University, Yangzhou, China
Abstract. Lossy compression has been applied to large-scale experimental model data compression due to its advantages of a high compression ratio. However, few methods consider the uneven distribution of compression errors affecting compression quality. Here we develop an adaptive lossy compression method with the stable compression error for earth system model data based on Hierarchical Geospatial Field Data Representation (HGFDR). We extended the original HGFDR by firstly dividing the original data into a series of the local block according to the exploratory experiment to maximize the local correlations of the data. After that, from the mathematical model of the HGFDR, the relationship between the compression parameter and compression error in HGFDR for each block is analyzed and calculated. Using optimal compression parameter selection rule and an adaptive compression algorithm, our method, the Adaptive-HGFDR, achieved the data compression under the constraints that the compression error is as stable as possible through each dimension. Experiments concerning model data compression are carried out based on the Community Earth System Model (CESM) data. The results show that our method has higher compression ratio and more uniform error distributions, compared with other commonly used lossy compression methods, such as the Fixed-Rate Compressed Floating-Point Arrays method.
Zhaoyuan Yu et al.


-
RC1: 'review of "Adaptive lossy compression of climate model data based on hierarchical tensor with Adaptive-HGFDR (v1.0)"', Anonymous Referee #1, 01 Jul 2020
-
RC2: 'Adaptive lossy compression of climate model data based on hierarchical tensor', Anonymous Referee #2, 27 Jul 2020
-
AC1: 'Response to reviewers', Zhaoyuan Yu, 26 Aug 2020


-
RC1: 'review of "Adaptive lossy compression of climate model data based on hierarchical tensor with Adaptive-HGFDR (v1.0)"', Anonymous Referee #1, 01 Jul 2020
-
RC2: 'Adaptive lossy compression of climate model data based on hierarchical tensor', Anonymous Referee #2, 27 Jul 2020
-
AC1: 'Response to reviewers', Zhaoyuan Yu, 26 Aug 2020
Zhaoyuan Yu et al.
Data sets
AndyWZJ/Adaptive-lossy-compression- v1.0 W. Z. J. Andy https://doi.org/10.5281/zenodo.3862130
Model code and software
AndyWZJ/Adaptive-lossy-compression- v1.0 W. Z. J. Andy https://doi.org/10.5281/zenodo.3862130
Executable research compendia (ERC)
AndyWZJ/Adaptive-lossy-compression- v1.0 W. Z. J. Andy https://doi.org/10.5281/zenodo.3862130
Zhaoyuan Yu et al.
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
222 | 92 | 36 | 350 | 35 | 37 |
- HTML: 222
- PDF: 92
- XML: 36
- Total: 350
- BibTeX: 35
- EndNote: 37
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1