The growing size of datasets used in climate science makes it difficult to store, analyze, and distribute dataset. Lossy compression algorithms can significantly reduce the disk space required to store datasets, but it can be difficult to understand and compare the behavior of different compression algorithms. ClimateBenchPress provides a benchmark to standardize comparisons between lossy compression algorithms and guide development of novel algorithms specifically targeted towards climate data.