Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

IF value: 5.240
IF 5-year value: 5.768
IF 5-year
CiteScore value: 8.9
SNIP value: 1.713
IPP value: 5.53
SJR value: 3.18
Scimago H <br class='widget-line-break'>index value: 71
Scimago H
h5-index value: 51
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.
© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

  30 Jul 2020

30 Jul 2020

Review status
This preprint is currently under review for the journal GMD.

A note on precision-preserving compression of scientific data

Rostislav Kouznetsov1,2 Rostislav Kouznetsov
  • 1Finnish Meteorological Institute, Helsinki, Finland
  • 2Obukhov Institute for Atmospheric Physics, Moscow, Russia

Abstract. Lossy compression of scientific data arrays is a powerful tool to save network bandwidth and storage space. Properly applied lossy compression can reduce the size of a dataset by orders of magnitude keeping all essential information, whereas a wrong choice of lossy compression parameters leads to the loss of valuable data. The paper considers statistical properties of several lossy compression methods implemented in "NetCDF operators" (NCO), a popular tool for handling and transformation of numerical data in NetCDF format. We compare the effects of imprecisions and artifacts resulting from use of a lossy compression of floating-point data arrays. In particular, we show that a popular Bit Grooming algorithm (default in NCO) has sub-optimal accuracy and produces substantial artifacts in multipoint statistics. We suggest a simple implementation of two algorithms that are free from these artifacts and have twice higher precision. Besides that, we suggest a way to rectify the data already processed with Bit Grooming.

The algorithm has been contributed to NCO mainstream. The supplementary material contains the implementation of the algorithm in Python 3.

Rostislav Kouznetsov

Interactive discussion

Status: open (until 24 Sep 2020)
Status: open (until 24 Sep 2020)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Rostislav Kouznetsov

Rostislav Kouznetsov


Total article views: 112 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
87 23 2 112 8 0 0
  • HTML: 87
  • PDF: 23
  • XML: 2
  • Total: 112
  • Supplement: 8
  • BibTeX: 0
  • EndNote: 0
Views and downloads (calculated since 30 Jul 2020)
Cumulative views and downloads (calculated since 30 Jul 2020)

Viewed (geographical distribution)

Total article views: 144 (including HTML, PDF, and XML) Thereof 144 with geography defined and 0 with unknown origin.
Country # Views %
  • 1



No saved metrics found.


No discussed metrics found.
Latest update: 14 Aug 2020
Publications Copernicus
Short summary
Resetting of non-significant figures (precision trimming) enables efficient data compression and helps to avoid excessive usage of storage space and network bandwidth. The paper analyses accuracy of trimmed data and artifacts caused by trimming methods. The paper presents several methods with implementation, evaluation and illustrations. It is poses an extension and improvement over the methods introduced in a GMD paper By Zender (2016)
Resetting of non-significant figures (precision trimming) enables efficient data compression and...