A note on precision-preserving compression of scientific data

Kouznetsov, Rostislav

doi:https://doi.org/10.5194/gmd-14-377-2021

Articles | Volume 14, issue 1

Geosci. Model Dev., 14, 377–389, 2021

https://doi.org/10.5194/gmd-14-377-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Geosci. Model Dev., 14, 377–389, 2021

https://doi.org/10.5194/gmd-14-377-2021

© Author(s) 2021. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 14, issue 1

Development and technical paper 22 Jan 2021

Development and technical paper | 22 Jan 2021

A note on precision-preserving compression of scientific data

Rostislav Kouznetsov

Download

Final revised paper (published on 22 Jan 2021)
Supplement to the final revised paper
Preprint (discussion started on 30 Jul 2020)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

SC1: 'Comments: Tie rules and a bitwise algorithm.', Milan Klöwer, 30 Jul 2020
- AC1: 'Thank you!', Rostislav Kouznetsov, 15 Aug 2020
RC1: 'Paper review', Mario Acosta, 07 Sep 2020
- AC2: 'Thank you!', Rostislav Kouznetsov, 01 Oct 2020
RC2: 'Second paper review', Seth McGinnis, 11 Sep 2020
- AC3: 'Thank you!', Rostislav Kouznetsov, 01 Oct 2020
RC3: 'Utility of the method for data compression need elaboration', Ananda Kumar Das, 14 Sep 2020
- AC4: 'Thank you!', Rostislav Kouznetsov, 01 Oct 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Rostislav Kouznetsov on behalf of the Authors (17 Nov 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (29 Nov 2020) by Paul Ullrich

RR by Seth McGinnis (30 Nov 2020)

RR by Mario Acosta (04 Dec 2020)

ED: Publish subject to technical corrections (04 Dec 2020) by Paul Ullrich

Short summary

Resetting of non-significant figures (precision trimming) enables efficient data compression and helps to avoid excessive use of storage space and network bandwidth while having well-constrained distortion to the data. The paper analyses accuracy losses and artifacts caused by trimming methods and by the widely used linear packing method. The paper presents several methods with implementation, evaluation, and illustrations and includes subroutines directly usable in geoscientific models.