Articles | Volume 9, issue 9
Geosci. Model Dev., 9, 3199–3211, 2016
Geosci. Model Dev., 9, 3199–3211, 2016
Development and technical paper
19 Sep 2016
Development and technical paper | 19 Sep 2016

Bit Grooming: statistically accurate precision-preserving quantization with compression, evaluated in the netCDF Operators (NCO, v4.4.8+)

Charles S. Zender

Related authors

The E3SM Diagnostics Package (E3SM Diags v2.7): a Python-based diagnostics package for Earth system model evaluation
Chengzhu Zhang, Jean-Christophe Golaz, Ryan Forsyth, Tom Vo, Shaocheng Xie, Zeshawn Shaheen, Gerald L. Potter, Xylar S. Asay-Davis, Charles S. Zender, Wuyin Lin, Chih-Chieh Chen, Chris R. Terai, Salil Mahajan, Tian Zhou, Karthik Balaguru, Qi Tang, Cheng Tao, Yuying Zhang, Todd Emmenegger, Susannah Burrows, and Paul A. Ullrich
Geosci. Model Dev., 15, 9031–9056,,, 2022
Short summary
The Fully Coupled Regionally Refined Model of E3SM Version 2: Overview of the Atmosphere, Land, and River
Qi Tang, Jean-Christophe Golaz, Luke P. Van Roekel, Mark A. Taylor, Wuyin Lin, Benjamin R. Hillman, Paul A. Ullrich, Andrew M. Bradley, Oksana Guba, Jonathan D. Wolfe, Tian Zhou, Kai Zhang, Xue Zheng, Yunyan Zhang, Meng Zhang, Mingxuan Wu, Hailong Wang, Cheng Tao, Balwinder Singh, Alan M. Rhoades, Yi Qin, Hong-Yi Li, Yan Feng, Yuying Zhang, Chengzhu Zhang, Charles S. Zender, Shaocheng Xie, Erika L. Roesler, Andrew F. Roberts, Azamat Mametjanov, Mathew E. Maltrud, Noel D. Keen, Robert L. Jacob, Christiane Jablonowski, Owen K. Hughes, Ryan M. Forsyth, Alan V. Di Vittorio, Peter M. Caldwell, Gautam Bisht, Renata B. McCoy, L. Ruby Leung, and David C. Bader
Geosci. Model Dev. Discuss.,,, 2022
Preprint under review for GMD
Short summary
The role of föhn winds in eastern Antarctic Peninsula rapid ice shelf collapse
Matthew K. Laffin, Charles S. Zender, Melchior van Wessem, and Sebastián Marinsek
The Cryosphere, 16, 1369–1381,,, 2022
Short summary
SNICAR-ADv4: a physically based radiative transfer model to represent the spectral albedo of glacier ice
Chloe A. Whicker, Mark G. Flanner, Cheng Dang, Charles S. Zender, Joseph M. Cook, and Alex S. Gardner
The Cryosphere, 16, 1197–1220,,, 2022
Short summary
SNICAR-ADv3: a community tool for modeling spectral snow albedo
Mark G. Flanner, Julian B. Arnheim, Joseph M. Cook, Cheng Dang, Cenlin He, Xianglei Huang, Deepak Singh, S. McKenzie Skiles, Chloe A. Whicker, and Charles S. Zender
Geosci. Model Dev., 14, 7673–7704,,, 2021
Short summary

Related subject area

Numerical methods
Predicting peak daily maximum 8 h ozone and linkages to emissions and meteorology in Southern California using machine learning methods (SoCAB-8HR V1.0)
Ziqi Gao, Yifeng Wang, Petros Vasilakos, Cesunica E. Ivey, Khanh Do, and Armistead G. Russell
Geosci. Model Dev., 15, 9015–9029,,, 2022
Short summary
Transfer learning for landslide susceptibility modeling using domain adaptation and case-based reasoning
Zhihao Wang, Jason Goetz, and Alexander Brenning
Geosci. Model Dev., 15, 8765–8784,,, 2022
Short summary
ISMIP-HOM benchmark experiments using Underworld
Till Sachau, Haibin Yang, Justin Lang, Paul D. Bons, and Louis Moresi
Geosci. Model Dev., 15, 8749–8764,,, 2022
Short summary
spyro: a Firedrake-based wave propagation and full-waveform-inversion finite-element solver
Keith J. Roberts, Alexandre Olender, Lucas Franceschini, Robert C. Kirby, Rafael S. Gioria, and Bruno S. Carmo
Geosci. Model Dev., 15, 8639–8667,,, 2022
Short summary
Spatial filtering in a 6D hybrid-Vlasov scheme to alleviate adaptive mesh refinement artifacts: a case study with Vlasiator (versions 5.0, 5.1, and 5.2.1)
Konstantinos Papadakis, Yann Pfau-Kempf, Urs Ganse, Markus Battarbee, Markku Alho, Maxime Grandin, Maxime Dubart, Lucile Turc, Hongyang Zhou, Konstantinos Horaites, Ivan Zaitsev, Giulia Cozzani, Maarja Bussov, Evgeny Gordeev, Fasil Tesema, Harriet George, Jonas Suni, Vertti Tarvus, and Minna Palmroth
Geosci. Model Dev., 15, 7903–7912,,, 2022
Short summary

Cited articles

Burtscher, M. and Ratanaworabhan, P.: FPC: A high-speed compressor for double-precision floating-point data, IEEE T. Comput., 58, 18–31,, 2009.
Caron, J.: Compression by scaling and offset, available at: (last access: 13 September 2016), 2014a.
Caron, J.: Compression by bit shaving, available at: (last access: 13 September 2016), 2014b.
Collet, Y.: LZ4 lossless compression algorithm, available at: (last access: 13 September 2016), 2013.
Dennis, J. M., Edwards, J., Evans, K. J., Guba, O., Lauritzen, P. H., Mirin, A. A., St-Cyr, A., Taylor, M. A., and Worley, P. H.: CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model, Int. J. High Perform. C., 26, 74–89,, 2012.
Short summary
We introduce Bit Grooming, a lossy compression algorithm that removes the bloat due to false precision, those bits and bytes beyond the meaningful precision of the data. Bit Grooming is statistically unbiased, applies to all floating-point numbers, and is easy to use. Bit Grooming reduces data storage requirements by 25–80 %. Unlike its best-known competitor Linear Packing, Bit Grooming imposes no software overhead on users, and guarantees its precision throughout the whole floating-point range.