Articles | Volume 9, issue 12
https://doi.org/10.5194/gmd-9-4381-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
https://doi.org/10.5194/gmd-9-4381-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
Development and technical paper
| 
07 Dec 2016
Development and technical paper |
| 07 Dec 2016
Evaluating lossy data compression on climate simulation data within a large ensemble
The National Center for Atmospheric Research, Boulder, CO, USA
Dorit M. Hammerling
The National Center for Atmospheric Research, Boulder, CO, USA
Sheri A. Mickelson
The National Center for Atmospheric Research, Boulder, CO, USA
Haiying Xu
The National Center for Atmospheric Research, Boulder, CO, USA
Martin B. Stolpe
Institute for Atmospheric and Climate Science, ETH Zurich, Zurich, Switzerland
Phillipe Naveau
Laboratoire des Sciences du Climat et l'Environnement, Gif-sur-Yvette, France
Ben Sanderson
The National Center for Atmospheric Research, Boulder, CO, USA
Imme Ebert-Uphoff
Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA
Savini Samarasinghe
Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA
Francesco De Simone
CNR-Institute of Atmospheric Pollution Research, Division of Rende, UNICAL-Polifunzionale, Rende, Italy
Francesco Carbone
CNR-Institute of Atmospheric Pollution Research, Division of Rende, UNICAL-Polifunzionale, Rende, Italy
Christian N. Gencarelli
CNR-Institute of Atmospheric Pollution Research, Division of Rende, UNICAL-Polifunzionale, Rende, Italy
John M. Dennis
The National Center for Atmospheric Research, Boulder, CO, USA
Jennifer E. Kay
Department of Oceanic and Atmospheric Sciences, University of Colorado, Boulder, CO, USA
Peter Lindstrom
Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, Livermore, CA, USA
Viewed
Total article views: 5,580 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 25 Jul 2016)
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 3,272 | 2,128 | 180 | 5,580 | 263 | 204 |
- HTML: 3,272
- PDF: 2,128
- XML: 180
- Total: 5,580
- BibTeX: 263
- EndNote: 204
Total article views: 4,998 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 07 Dec 2016)
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 2,934 | 1,888 | 176 | 4,998 | 248 | 196 |
- HTML: 2,934
- PDF: 1,888
- XML: 176
- Total: 4,998
- BibTeX: 248
- EndNote: 196
Total article views: 582 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 25 Jul 2016)
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 338 | 240 | 4 | 582 | 15 | 8 |
- HTML: 338
- PDF: 240
- XML: 4
- Total: 582
- BibTeX: 15
- EndNote: 8
Cited
32 citations as recorded by crossref.
- A statistical analysis of lossily compressed climate model data A. Poppick et al. 10.1016/j.cageo.2020.104599
- WaveRange: wavelet-based data compression for three-dimensional numerical simulations on regular grids D. Kolomenskiy et al. 10.1007/s12650-021-00813-8
- Compression Challenges in Large Scale Partial Differential Equation Solvers S. Götschel & M. Weiser 10.3390/a12090197
- Axially symmetric models for global data: A journey between geostatistics and stochastic generators E. Porcu et al. 10.1002/env.2555
- The effect of lossy compression of numerical weather prediction data on data analysis: a case study using enstools-compression 2023.11 O. Tintó Prims et al. 10.5194/gmd-17-8909-2024
- Visuelle Analyse großer Daten in der Klimaforschung N. Röber & M. Böttinger 10.1007/s00287-019-01222-w
- Reducing storage of global wind ensembles with stochastic generators J. Jeong et al. 10.1214/17-AOAS1105
- Lossy compression of Earth system model data based on a hierarchical tensor with Adaptive-HGFDR (v1.0) Z. Yu et al. 10.5194/gmd-14-875-2021
- On Preserving Scientific Integrity for Climate Model Data in the HPC Era A. Baker 10.1109/MCSE.2021.3119509
- Analyzing the Effect and Performance of Lossy Compression on Aeroacoustic Simulation of Gas Injector S. Najmabadi et al. 10.3390/computation5020024
- Connecting climate models to community needs A. Foley 10.1080/00167487.2021.1970930
- Rejoinder on ‘Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach’ H. Huang et al. 10.1007/s13253-023-00542-5
- Advancing data compression via noise detection D. Hammerling & A. Baker 10.1038/s43588-021-00167-z
- Z-checker: A framework for assessing lossy compression of scientific data D. Tao et al. 10.1177/1094342017737147
- Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing F. Cappello et al. 10.1016/j.future.2024.05.022
- BurstZ+: Eliminating The Communication Bottleneck of Scientific Computing Accelerators via Accelerated Compression G. Sun et al. 10.1145/3476831
- Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files X. Delaunay et al. 10.5194/gmd-12-4099-2019
- Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach H. Huang et al. 10.1007/s13253-022-00518-x
- Lossy Data Compression Effects on Wall-bounded Turbulence: Bounds on Data Reduction E. Otero et al. 10.1007/s10494-018-9923-5
- Impact of Lossy Compression Errors on Passive Seismic Data Analyses A. Issah & E. Martin 10.1785/0220230314
- Requirements for a global data infrastructure in support of CMIP6 V. Balaji et al. 10.5194/gmd-11-3659-2018
- Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data A. Baker et al. 10.1111/cgf.13707
- A Multivariate Global Spatiotemporal Stochastic Generator for Climate Ensembles M. Edwards et al. 10.1007/s13253-019-00352-8
- A data model of the Climate and Forecast metadata conventions (CF-1.6) with a software implementation (cf-python v2.1) D. Hassell et al. 10.5194/gmd-10-4619-2017
- Use cases of lossy compression for floating-point data in scientific data sets F. Cappello et al. 10.1177/1094342019853336
- Discussion on “Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach” A. Poppick 10.1007/s13253-023-00537-2
- Do Southern Ocean Cloud Feedbacks Matter for 21st Century Warming? W. Frey et al. 10.1002/2017GL076339
- Compression of climate simulations with a nonstationary global SpatioTemporal SPDE model G. Fuglstad & S. Castruccio 10.1214/20-AOAS1340
- Compressing atmospheric data into its real information content M. Klöwer et al. 10.1038/s43588-021-00156-2
- New Methods for Data Storage of Model Output from Ensemble Simulations P. Düben et al. 10.1175/MWR-D-18-0170.1
- Error Analysis of ZFP Compression for Floating-Point Data J. Diffenderfer et al. 10.1137/18M1168832
- A diagnostic interface for the ICOsahedral Non-hydrostatic (ICON) modelling framework based on the Modular Earth Submodel System (MESSy v2.50) B. Kern & P. Jöckel 10.5194/gmd-9-3639-2016
31 citations as recorded by crossref.
- A statistical analysis of lossily compressed climate model data A. Poppick et al. 10.1016/j.cageo.2020.104599
- WaveRange: wavelet-based data compression for three-dimensional numerical simulations on regular grids D. Kolomenskiy et al. 10.1007/s12650-021-00813-8
- Compression Challenges in Large Scale Partial Differential Equation Solvers S. Götschel & M. Weiser 10.3390/a12090197
- Axially symmetric models for global data: A journey between geostatistics and stochastic generators E. Porcu et al. 10.1002/env.2555
- The effect of lossy compression of numerical weather prediction data on data analysis: a case study using enstools-compression 2023.11 O. Tintó Prims et al. 10.5194/gmd-17-8909-2024
- Visuelle Analyse großer Daten in der Klimaforschung N. Röber & M. Böttinger 10.1007/s00287-019-01222-w
- Reducing storage of global wind ensembles with stochastic generators J. Jeong et al. 10.1214/17-AOAS1105
- Lossy compression of Earth system model data based on a hierarchical tensor with Adaptive-HGFDR (v1.0) Z. Yu et al. 10.5194/gmd-14-875-2021
- On Preserving Scientific Integrity for Climate Model Data in the HPC Era A. Baker 10.1109/MCSE.2021.3119509
- Analyzing the Effect and Performance of Lossy Compression on Aeroacoustic Simulation of Gas Injector S. Najmabadi et al. 10.3390/computation5020024
- Connecting climate models to community needs A. Foley 10.1080/00167487.2021.1970930
- Rejoinder on ‘Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach’ H. Huang et al. 10.1007/s13253-023-00542-5
- Advancing data compression via noise detection D. Hammerling & A. Baker 10.1038/s43588-021-00167-z
- Z-checker: A framework for assessing lossy compression of scientific data D. Tao et al. 10.1177/1094342017737147
- Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing F. Cappello et al. 10.1016/j.future.2024.05.022
- BurstZ+: Eliminating The Communication Bottleneck of Scientific Computing Accelerators via Accelerated Compression G. Sun et al. 10.1145/3476831
- Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files X. Delaunay et al. 10.5194/gmd-12-4099-2019
- Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach H. Huang et al. 10.1007/s13253-022-00518-x
- Lossy Data Compression Effects on Wall-bounded Turbulence: Bounds on Data Reduction E. Otero et al. 10.1007/s10494-018-9923-5
- Impact of Lossy Compression Errors on Passive Seismic Data Analyses A. Issah & E. Martin 10.1785/0220230314
- Requirements for a global data infrastructure in support of CMIP6 V. Balaji et al. 10.5194/gmd-11-3659-2018
- Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data A. Baker et al. 10.1111/cgf.13707
- A Multivariate Global Spatiotemporal Stochastic Generator for Climate Ensembles M. Edwards et al. 10.1007/s13253-019-00352-8
- A data model of the Climate and Forecast metadata conventions (CF-1.6) with a software implementation (cf-python v2.1) D. Hassell et al. 10.5194/gmd-10-4619-2017
- Use cases of lossy compression for floating-point data in scientific data sets F. Cappello et al. 10.1177/1094342019853336
- Discussion on “Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach” A. Poppick 10.1007/s13253-023-00537-2
- Do Southern Ocean Cloud Feedbacks Matter for 21st Century Warming? W. Frey et al. 10.1002/2017GL076339
- Compression of climate simulations with a nonstationary global SpatioTemporal SPDE model G. Fuglstad & S. Castruccio 10.1214/20-AOAS1340
- Compressing atmospheric data into its real information content M. Klöwer et al. 10.1038/s43588-021-00156-2
- New Methods for Data Storage of Model Output from Ensemble Simulations P. Düben et al. 10.1175/MWR-D-18-0170.1
- Error Analysis of ZFP Compression for Floating-Point Data J. Diffenderfer et al. 10.1137/18M1168832
Latest update: 25 Dec 2024
Short summary
We apply lossy data compression to output from the Community Earth System Model Large Ensemble Community Project. We challenge climate scientists to examine features of the data relevant to their interests and identify which of the ensemble members have been compressed, and we perform direct comparisons on features critical to climate science. We find that applying lossy data compression to climate model data effectively reduces data volumes with minimal effect on scientific results.
We apply lossy data compression to output from the Community Earth System Model Large Ensemble...
