Articles | Volume 14, issue 2
https://doi.org/10.5194/gmd-14-875-2021
https://doi.org/10.5194/gmd-14-875-2021
Development and technical paper
 | 
11 Feb 2021
Development and technical paper |  | 11 Feb 2021

Lossy compression of Earth system model data based on a hierarchical tensor with Adaptive-HGFDR (v1.0)

Zhaoyuan Yu, Dongshuang Li, Zhengfang Zhang, Wen Luo, Yuan Liu, Zengjie Wang, and Linwang Yuan

Related authors

An improved method of the Globally Resolved Energy Balance model by the Bayesian networks
Zhenxia Liu, Zengjie Wang, Jian Wang, Zhengfang Zhang, Dongshuang Li, Zhaoyuan Yu, Linwang Yuan, and Wen Luo
Geosci. Model Dev., 16, 2939–2955, https://doi.org/10.5194/gmd-16-2939-2023,https://doi.org/10.5194/gmd-16-2939-2023, 2023
Short summary
Clifford algebra-based structure filtering analysis for geophysical vector fields
Z. Yu, W. Luo, L. Yi, Y. Hu, and L. Yuan
Nonlin. Processes Geophys., 20, 563–570, https://doi.org/10.5194/npg-20-563-2013,https://doi.org/10.5194/npg-20-563-2013, 2013

Related subject area

Climate and Earth system modeling
A diffusion-based kernel density estimator (diffKDE, version 1) with optimal bandwidth approximation for the analysis of data in geoscience and ecological research
Maria-Theresia Pelz, Markus Schartau, Christopher J. Somes, Vanessa Lampe, and Thomas Slawig
Geosci. Model Dev., 16, 6609–6634, https://doi.org/10.5194/gmd-16-6609-2023,https://doi.org/10.5194/gmd-16-6609-2023, 2023
Short summary
Monte Carlo drift correction – quantifying the drift uncertainty of global climate models
Benjamin S. Grandey, Zhi Yang Koh, Dhrubajyoti Samanta, Benjamin P. Horton, Justin Dauwels, and Lock Yue Chew
Geosci. Model Dev., 16, 6593–6608, https://doi.org/10.5194/gmd-16-6593-2023,https://doi.org/10.5194/gmd-16-6593-2023, 2023
Short summary
Improvements in the Canadian Earth System Model (CanESM) through systematic model analysis: CanESM5.0 and CanESM5.1
Michael Sigmond, James Anstey, Vivek Arora, Ruth Digby, Nathan Gillett, Viatcheslav Kharin, William Merryfield, Catherine Reader, John Scinocca, Neil Swart, John Virgin, Carsten Abraham, Jason Cole, Nicolas Lambert, Woo-Sung Lee, Yongxiao Liang, Elizaveta Malinina, Landon Rieger, Knut von Salzen, Christian Seiler, Clint Seinen, Andrew Shao, Reinel Sospedra-Alfonso, Libo Wang, and Duo Yang
Geosci. Model Dev., 16, 6553–6591, https://doi.org/10.5194/gmd-16-6553-2023,https://doi.org/10.5194/gmd-16-6553-2023, 2023
Short summary
Earth System Model Aerosol–Cloud Diagnostics (ESMAC Diags) package, version 2: assessing aerosols, clouds, and aerosol–cloud interactions via field campaign and long-term observations
Shuaiqi Tang, Adam C. Varble, Jerome D. Fast, Kai Zhang, Peng Wu, Xiquan Dong, Fan Mei, Mikhail Pekour, Joseph C. Hardin, and Po-Lun Ma
Geosci. Model Dev., 16, 6355–6376, https://doi.org/10.5194/gmd-16-6355-2023,https://doi.org/10.5194/gmd-16-6355-2023, 2023
Short summary
CIOFC1.0: a common parallel input/output framework based on C-Coupler2.0
Xinzhu Yu, Li Liu, Chao Sun, Qingu Jiang, Biao Zhao, Zhiyuan Zhang, Hao Yu, and Bin Wang
Geosci. Model Dev., 16, 6285–6308, https://doi.org/10.5194/gmd-16-6285-2023,https://doi.org/10.5194/gmd-16-6285-2023, 2023
Short summary

Cited articles

Andrew, P., Joseph, N., Noah, Feldman., Allison, H. B., Alexander, P., and Dorit, M. H.: A statistical analysis of lossily compressed climate model data, Comput. Geosci., 145, 104599, https://doi.org/10.1016/j.cageo.2020.104599, 2020. 
Baker, A. H., Xu, H., Dennis, J. M., Levy, M. N., Nychka, D., Mickelson, S. A., Edwards, J., Vertenstein, M., and Wegener, A.: A methodology for evaluating the impact of data compression on climate simulation data, in: Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing, Vancouver, Canada, 23–27 June 2014. 
Baker, A. H., Hammerling, D. M., Mickelson, S. A., Xu, H., Stolpe, M. B., Naveau, P., Sanderson, B., Ebert-Uphoff, I., Samarasinghe, S., De Simone, F., Carbone, F., Gencarelli, C. N., Dennis, J. M., Kay, J. E., and Lindstrom, P.: Evaluating lossy data compression on climate simulation data within a large ensemble, Geosci. Model Dev., 9, 4381–4403, https://doi.org/10.5194/gmd-9-4381-2016, 2016. 
Bengua, J. A., Phien, H. N., Tuan, H. D., and Do, M. N.: Matrix product state for higher-order tensor compression and classification, IEEE Trans. Signal Process., 65, 4019–4030, https://doi.org/10.1109/TSP.2017.2703882, 2016. 
Cai, J. Y., Chen, X., and Lu, P.: Non-negative weighted #csps: an effective complexity dichotomy, Comput. Sci., 6, 45–54, https://doi.org/10.1109/CCC.2011.32, 2012. 
Download
Short summary
Few lossy compression methods consider both the global and local multidimensional coupling correlations, which could lead to information loss in data compression. Here we develop an adaptive lossy compression method, Adaptive-HGFDR, to capture both the global and local variation of multidimensional coupling correlations and improve approximation accuracy. The method can achieve good compression performances for most flux variables with significant spatiotemporal heterogeneity.