Articles | Volume 9, issue 9
https://doi.org/10.5194/gmd-9-3199-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
https://doi.org/10.5194/gmd-9-3199-2016
© Author(s) 2016. This work is distributed under
the Creative Commons Attribution 3.0 License.
the Creative Commons Attribution 3.0 License.
Development and technical paper
| 
19 Sep 2016
Development and technical paper |
| 19 Sep 2016
Bit Grooming: statistically accurate precision-preserving quantization with compression, evaluated in the netCDF Operators (NCO, v4.4.8+)
Departments of Earth System Science and Computer Science, University of California, Irvine, Irvine, CA 92697-3100, USA
Viewed
Total article views: 5,849 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 21 Apr 2016)
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 3,792 | 1,737 | 320 | 5,849 | 864 | 330 | 377 |
- HTML: 3,792
- PDF: 1,737
- XML: 320
- Total: 5,849
- Supplement: 864
- BibTeX: 330
- EndNote: 377
Total article views: 5,192 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 19 Sep 2016)
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 3,507 | 1,376 | 309 | 5,192 | 631 | 318 | 368 |
- HTML: 3,507
- PDF: 1,376
- XML: 309
- Total: 5,192
- Supplement: 631
- BibTeX: 318
- EndNote: 368
Total article views: 657 (including HTML, PDF, and XML)
Cumulative views and downloads
(calculated since 21 Apr 2016)
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 285 | 361 | 11 | 657 | 233 | 12 | 9 |
- HTML: 285
- PDF: 361
- XML: 11
- Total: 657
- Supplement: 233
- BibTeX: 12
- EndNote: 9
Cited
24 citations as recorded by crossref.
- Advancing data compression via noise detection D. Hammerling & A. Baker https://doi.org/10.1038/s43588-021-00167-z
- Discussion on “Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach” J. Bessac et al. https://doi.org/10.1007/s13253-023-00540-7
- Black-box statistical prediction of lossy compression ratios for scientific data R. Underwood et al. https://doi.org/10.1177/10943420231179417
- The effect of lossy compression of numerical weather prediction data on data analysis: a case study using enstools-compression 2023.11 O. Tintó Prims et al. https://doi.org/10.5194/gmd-17-8909-2024
- A Survey on Error-Bounded Lossy Compression for Scientific Datasets S. Di et al. https://doi.org/10.1145/3733104
- The impact of altering emission data precision on compression efficiency and accuracy of simulations of the community multiscale air quality model M. Walters & D. Wong https://doi.org/10.5194/gmd-16-1179-2023
- Compressing atmospheric data into its real information content M. Klöwer et al. https://doi.org/10.1038/s43588-021-00156-2
- Array DBMS R. Zalipynis https://doi.org/10.14778/3476311.3476404
- Lossy Neural Compression for Geospatial Analytics: A review C. Gomes et al. https://doi.org/10.1109/MGRS.2025.3546527
- Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints Y. Liu et al. https://doi.org/10.1109/TPDS.2022.3194695
- The compression–error trade-off for large gridded data sets J. Silver & C. Zender https://doi.org/10.5194/gmd-10-413-2017
- Improving Performance of SLAV Model for Medium Range Weather Prediction R. Fadeev et al. https://doi.org/10.1134/S1995080224603874
- Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files X. Delaunay et al. https://doi.org/10.5194/gmd-12-4099-2019
- Parallel Implementation of a PETSc-Based Framework for the General Curvilinear Coastal Ocean Model M. Valera et al. https://doi.org/10.3390/jmse7060185
- New Methods for Data Storage of Model Output from Ensemble Simulations P. Düben et al. https://doi.org/10.1175/MWR-D-18-0170.1
- Striking the balance between speed and compression ratio: A fast bit-grouping algorithm and adaptive compressor selection for scientific data M. Middlezong https://doi.org/10.1016/j.future.2026.108370
- Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data A. Baker et al. https://doi.org/10.1111/cgf.13707
- Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data V. Sagan et al. https://doi.org/10.1109/TGRS.2021.3091409
- A statistical analysis of lossily compressed climate model data A. Poppick et al. https://doi.org/10.1016/j.cageo.2020.104599
- GenomicScores: seamless access to genomewide position-specific scores from R and Bioconductor P. Puigdevall et al. https://doi.org/10.1093/bioinformatics/bty311
- Technical note: A flexible framework for precision reduction of WRF inputs and outputs to balance storage efficiency and scientific fidelity S. Wu et al. https://doi.org/10.5194/acp-26-7261-2026
- Evaluating lossy data compression on climate simulation data within a large ensemble A. Baker et al. https://doi.org/10.5194/gmd-9-4381-2016
- Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing F. Cappello et al. https://doi.org/10.1016/j.future.2024.05.022
- A note on precision-preserving compression of scientific data R. Kouznetsov https://doi.org/10.5194/gmd-14-377-2021
24 citations as recorded by crossref.
- Advancing data compression via noise detection D. Hammerling & A. Baker https://doi.org/10.1038/s43588-021-00167-z
- Discussion on “Saving Storage in Climate Ensembles: A Model-Based Stochastic Approach” J. Bessac et al. https://doi.org/10.1007/s13253-023-00540-7
- Black-box statistical prediction of lossy compression ratios for scientific data R. Underwood et al. https://doi.org/10.1177/10943420231179417
- The effect of lossy compression of numerical weather prediction data on data analysis: a case study using enstools-compression 2023.11 O. Tintó Prims et al. https://doi.org/10.5194/gmd-17-8909-2024
- A Survey on Error-Bounded Lossy Compression for Scientific Datasets S. Di et al. https://doi.org/10.1145/3733104
- The impact of altering emission data precision on compression efficiency and accuracy of simulations of the community multiscale air quality model M. Walters & D. Wong https://doi.org/10.5194/gmd-16-1179-2023
- Compressing atmospheric data into its real information content M. Klöwer et al. https://doi.org/10.1038/s43588-021-00156-2
- Array DBMS R. Zalipynis https://doi.org/10.14778/3476311.3476404
- Lossy Neural Compression for Geospatial Analytics: A review C. Gomes et al. https://doi.org/10.1109/MGRS.2025.3546527
- Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints Y. Liu et al. https://doi.org/10.1109/TPDS.2022.3194695
- The compression–error trade-off for large gridded data sets J. Silver & C. Zender https://doi.org/10.5194/gmd-10-413-2017
- Improving Performance of SLAV Model for Medium Range Weather Prediction R. Fadeev et al. https://doi.org/10.1134/S1995080224603874
- Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files X. Delaunay et al. https://doi.org/10.5194/gmd-12-4099-2019
- Parallel Implementation of a PETSc-Based Framework for the General Curvilinear Coastal Ocean Model M. Valera et al. https://doi.org/10.3390/jmse7060185
- New Methods for Data Storage of Model Output from Ensemble Simulations P. Düben et al. https://doi.org/10.1175/MWR-D-18-0170.1
- Striking the balance between speed and compression ratio: A fast bit-grouping algorithm and adaptive compressor selection for scientific data M. Middlezong https://doi.org/10.1016/j.future.2026.108370
- Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data A. Baker et al. https://doi.org/10.1111/cgf.13707
- Data-Driven Artificial Intelligence for Calibration of Hyperspectral Big Data V. Sagan et al. https://doi.org/10.1109/TGRS.2021.3091409
- A statistical analysis of lossily compressed climate model data A. Poppick et al. https://doi.org/10.1016/j.cageo.2020.104599
- GenomicScores: seamless access to genomewide position-specific scores from R and Bioconductor P. Puigdevall et al. https://doi.org/10.1093/bioinformatics/bty311
- Technical note: A flexible framework for precision reduction of WRF inputs and outputs to balance storage efficiency and scientific fidelity S. Wu et al. https://doi.org/10.5194/acp-26-7261-2026
- Evaluating lossy data compression on climate simulation data within a large ensemble A. Baker et al. https://doi.org/10.5194/gmd-9-4381-2016
- Multifacets of lossy compression for scientific data in the Joint-Laboratory of Extreme Scale Computing F. Cappello et al. https://doi.org/10.1016/j.future.2024.05.022
- A note on precision-preserving compression of scientific data R. Kouznetsov https://doi.org/10.5194/gmd-14-377-2021
Saved (final revised paper)
Latest update: 12 Jun 2026
Short summary
We introduce Bit Grooming, a lossy compression algorithm that removes the bloat due to false precision, those bits and bytes beyond the meaningful precision of the data. Bit Grooming is statistically unbiased, applies to all floating-point numbers, and is easy to use. Bit Grooming reduces data storage requirements by 25–80 %. Unlike its best-known competitor Linear Packing, Bit Grooming imposes no software overhead on users, and guarantees its precision throughout the whole floating-point range.
We introduce Bit Grooming, a lossy compression algorithm that removes the bloat due to false...
