Articles | Volume 12, issue 9
Geosci. Model Dev., 12, 4099–4113, 2019
https://doi.org/10.5194/gmd-12-4099-2019
Geosci. Model Dev., 12, 4099–4113, 2019
https://doi.org/10.5194/gmd-12-4099-2019

Development and technical paper 23 Sep 2019

Development and technical paper | 23 Sep 2019

Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files

Xavier Delaunay et al.

Download

Interactive discussion

Status: closed
Status: closed
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
Printer-friendly Version - Printer-friendly version Supplement - Supplement

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision
AR by Anna Mirena Feist-Polner on behalf of the Authors (04 Apr 2019)  Author's response
ED: Referee Nomination & Report Request started (15 May 2019) by Steve Easterbrook
RR by Anonymous Referee #3 (20 May 2019)
RR by Anonymous Referee #1 (27 May 2019)
ED: Publish subject to minor revisions (review by editor) (28 May 2019) by Steve Easterbrook
AR by Xavier Delaunay on behalf of the Authors (07 Jun 2019)  Author's response    Manuscript
ED: Publish as is (03 Jul 2019) by Steve Easterbrook
Download
Short summary
This research aimed at finding a compression method suitable for the ground processing of CFOSAT and SWOT satellite datasets. Lossless algorithms did not allow enough compression. That is why we began studying lossy alternatives. This work introduces the digit rounding algorithm which reduces the volume of scientific datasets keeping only the significant digits in each sample value. The number of digits kept is relative to each sample so that both small and high values are similarly preserved.