Articles | Volume 9, issue 12
https://doi.org/10.5194/gmd-9-4381-2016
https://doi.org/10.5194/gmd-9-4381-2016
Development and technical paper
 | 
07 Dec 2016
Development and technical paper |  | 07 Dec 2016

Evaluating lossy data compression on climate simulation data within a large ensemble

Allison H. Baker, Dorit M. Hammerling, Sheri A. Mickelson, Haiying Xu, Martin B. Stolpe, Phillipe Naveau, Ben Sanderson, Imme Ebert-Uphoff, Savini Samarasinghe, Francesco De Simone, Francesco Carbone, Christian N. Gencarelli, John M. Dennis, Jennifer E. Kay, and Peter Lindstrom

Related authors

Optimizing high-resolution Community Earth System Model on a heterogeneous many-core supercomputing platform
Shaoqing Zhang, Haohuan Fu, Lixin Wu, Yuxuan Li, Hong Wang, Yunhui Zeng, Xiaohui Duan, Wubing Wan, Li Wang, Yuan Zhuang, Hongsong Meng, Kai Xu, Ping Xu, Lin Gan, Zhao Liu, Sihai Wu, Yuhu Chen, Haining Yu, Shupeng Shi, Lanning Wang, Shiming Xu, Wei Xue, Weiguo Liu, Qiang Guo, Jie Zhang, Guanghui Zhu, Yang Tu, Jim Edwards, Allison Baker, Jianlin Yong, Man Yuan, Yangyang Yu, Qiuying Zhang, Zedong Liu, Mingkui Li, Dongning Jia, Guangwen Yang, Zhiqiang Wei, Jingshan Pan, Ping Chang, Gokhan Danabasoglu, Stephen Yeager, Nan Rosenbloom, and Ying Guo
Geosci. Model Dev., 13, 4809–4829, https://doi.org/10.5194/gmd-13-4809-2020,https://doi.org/10.5194/gmd-13-4809-2020, 2020
Short summary
Nine time steps: ultra-fast statistical consistency testing of the Community Earth System Model (pyCECT v3.0)
Daniel J. Milroy, Allison H. Baker, Dorit M. Hammerling, and Elizabeth R. Jessup
Geosci. Model Dev., 11, 697–711, https://doi.org/10.5194/gmd-11-697-2018,https://doi.org/10.5194/gmd-11-697-2018, 2018
Short summary
P-CSI v1.0, an accelerated barotropic solver for the high-resolution ocean model component in the Community Earth System Model v2.0
Xiaomeng Huang, Qiang Tang, Yuheng Tseng, Yong Hu, Allison H. Baker, Frank O. Bryan, John Dennis, Haohuan Fu, and Guangwen Yang
Geosci. Model Dev., 9, 4209–4225, https://doi.org/10.5194/gmd-9-4209-2016,https://doi.org/10.5194/gmd-9-4209-2016, 2016
Short summary
Evaluating statistical consistency in the ocean model component of the Community Earth System Model (pyCECT v2.0)
Allison H. Baker, Yong Hu, Dorit M. Hammerling, Yu-heng Tseng, Haiying Xu, Xiaomeng Huang, Frank O. Bryan, and Guangwen Yang
Geosci. Model Dev., 9, 2391–2406, https://doi.org/10.5194/gmd-9-2391-2016,https://doi.org/10.5194/gmd-9-2391-2016, 2016
Short summary
A new ensemble-based consistency test for the Community Earth System Model (pyCECT v1.0)
A. H. Baker, D. M. Hammerling, M. N. Levy, H. Xu, J. M. Dennis, B. E. Eaton, J. Edwards, C. Hannay, S. A. Mickelson, R. B. Neale, D. Nychka, J. Shollenberger, J. Tribbia, M. Vertenstein, and D. Williamson
Geosci. Model Dev., 8, 2829–2840, https://doi.org/10.5194/gmd-8-2829-2015,https://doi.org/10.5194/gmd-8-2829-2015, 2015
Short summary

Related subject area

Climate and Earth system modeling
Multivariate adjustment of drizzle bias using machine learning in European climate projections
Georgia Lazoglou, Theo Economou, Christina Anagnostopoulou, George Zittis, Anna Tzyrkalli, Pantelis Georgiades, and Jos Lelieveld
Geosci. Model Dev., 17, 4689–4703, https://doi.org/10.5194/gmd-17-4689-2024,https://doi.org/10.5194/gmd-17-4689-2024, 2024
Short summary
Development and evaluation of the interactive Model for Air Pollution and Land Ecosystems (iMAPLE) version 1.0
Xu Yue, Hao Zhou, Chenguang Tian, Yimian Ma, Yihan Hu, Cheng Gong, Hui Zheng, and Hong Liao
Geosci. Model Dev., 17, 4621–4642, https://doi.org/10.5194/gmd-17-4621-2024,https://doi.org/10.5194/gmd-17-4621-2024, 2024
Short summary
A perspective on the next generation of Earth system model scenarios: towards representative emission pathways (REPs)
Malte Meinshausen, Carl-Friedrich Schleussner, Kathleen Beyer, Greg Bodeker, Olivier Boucher, Josep G. Canadell, John S. Daniel, Aïda Diongue-Niang, Fatima Driouech, Erich Fischer, Piers Forster, Michael Grose, Gerrit Hansen, Zeke Hausfather, Tatiana Ilyina, Jarmo S. Kikstra, Joyce Kimutai, Andrew D. King, June-Yi Lee, Chris Lennard, Tabea Lissner, Alexander Nauels, Glen P. Peters, Anna Pirani, Gian-Kasper Plattner, Hans Pörtner, Joeri Rogelj, Maisa Rojas, Joyashree Roy, Bjørn H. Samset, Benjamin M. Sanderson, Roland Séférian, Sonia Seneviratne, Christopher J. Smith, Sophie Szopa, Adelle Thomas, Diana Urge-Vorsatz, Guus J. M. Velders, Tokuta Yokohata, Tilo Ziehn, and Zebedee Nicholls
Geosci. Model Dev., 17, 4533–4559, https://doi.org/10.5194/gmd-17-4533-2024,https://doi.org/10.5194/gmd-17-4533-2024, 2024
Short summary
Parallel SnowModel (v1.0): a parallel implementation of a distributed snow-evolution modeling system (SnowModel)
Ross Mower, Ethan D. Gutmann, Glen E. Liston, Jessica Lundquist, and Soren Rasmussen
Geosci. Model Dev., 17, 4135–4154, https://doi.org/10.5194/gmd-17-4135-2024,https://doi.org/10.5194/gmd-17-4135-2024, 2024
Short summary
LB-SCAM: a learning-based method for efficient large-scale sensitivity analysis and tuning of the Single Column Atmosphere Model (SCAM)
Jiaxu Guo, Juepeng Zheng, Yidan Xu, Haohuan Fu, Wei Xue, Lanning Wang, Lin Gan, Ping Gao, Wubing Wan, Xianwei Wu, Zhitao Zhang, Liang Hu, Gaochao Xu, and Xilong Che
Geosci. Model Dev., 17, 3975–3992, https://doi.org/10.5194/gmd-17-3975-2024,https://doi.org/10.5194/gmd-17-3975-2024, 2024
Short summary

Cited articles

Ana, F. and de Haan, L.: On the block maxima method in extreme value theory, Ann. Stat., 43, 276–298, 2015.
Baker, A., Xu, H., Dennis, J., Levy, M., Nychka, D., Mickelson, S., Edwards, J., Vertenstein, M., and Wegener, A.: A Methodology for Evaluating the Impact of Data Compression on Climate Simulation Data, in: Proceedings of the 23rd International Symposium on High-performance Parallel and Distributed Computing, HPDC '14, 23–27 June 2014, Vancouver, Canada, 203–214, 2014.
Baker, A. H., Hammerling, D. M., Levy, M. N., Xu, H., Dennis, J. M., Eaton, B. E., Edwards, J., Hannay, C., Mickelson, S. A., Neale, R. B., Nychka, D., Shollenberger, J., Tribbia, J., Vertenstein, M., and Williamson, D.: A new ensemble-based consistency test for the Community Earth System Model (pyCECT v1.0), Geosci. Model Dev., 8, 2829–2840, https://doi.org/10.5194/gmd-8-2829-2015, 2015.
Beirlant, J., Goegebeur, Y., Segers, J., and Teugels, J.: Statistics of Extremes: Theory and Applications, Wiley Series in Probability and Statistics, Hoboken, USA, 2004.
Bicer, T., Yin, J., Chiu, D., Agrawal, G., and Schuchardt, K.: Integrating online compression to accelerate large-scale data analytics applications. IEEE International Symposium on Parallel and Distributed Processing (IPDPS), 20–24 May 2013, Boston, Massachusetts, USA, 1205–1216, https://doi.org/10.1109/IPDPS.2013.81, 2013.
Download
Short summary
We apply lossy data compression to output from the Community Earth System Model Large Ensemble Community Project. We challenge climate scientists to examine features of the data relevant to their interests and identify which of the ensemble members have been compressed, and we perform direct comparisons on features critical to climate science. We find that applying lossy data compression to climate model data effectively reduces data volumes with minimal effect on scientific results.