Evaluating simulated climate patterns from the CMIP archives using satellite and reanalysis datasets using the Climate Model Assessment Tool (CMATv1)

Fasullo, John T.

doi:https://doi.org/10.5194/gmd-13-3627-2020

Articles | Volume 13, issue 8

https://doi.org/10.5194/gmd-13-3627-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/gmd-13-3627-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 13, issue 8

Methods for assessment of models

|

21 Aug 2020

Methods for assessment of models |

| 21 Aug 2020

Evaluating simulated climate patterns from the CMIP archives using satellite and reanalysis datasets using the Climate Model Assessment Tool (CMATv1)

John T. Fasullo

Download

Final revised paper (published on 21 Aug 2020)
Preprint (discussion started on 28 Feb 2020)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'nteractive comment on “Evaluating Simulated Climate Patterns from the CMIP Archives Using Satellite and Reanalysis Datasets” by John T. Fasullo', Anonymous Referee #1, 01 Apr 2020
- AC1: 'reply to feedback of RC1', John Fasullo, 22 Apr 2020
RC2: 'Review of the manuscript “Evaluating Simulated Climate Patterns from the CMIP Archives Using Satellite and Reanalysis Datasets“ by J.T. Fasullo', Anonymous Referee #2, 21 May 2020
- AC2: 'reply to feedback of RC2', John Fasullo, 26 May 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by John Fasullo on behalf of the Authors (04 Jun 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (17 Jun 2020) by Juan Antonio Añel

RR by Anonymous Referee #1 (19 Jun 2020)

Suggestions for revision or reasons for rejection

The manuscript has now undergone a significant improvement and most of my previous suggestions have been met. I particularly appreciated that the diagnostics have now been put in comparison with other existing sets, and that the Methods section now includes the way the performance metrics are generated from area-weighted correlation patterns.

I only have a few minor comments aimed at better clarifying a few points made in the previous round of revision. I recommend that the manuscript is accepted as is, provided that they are taken into account, if applicable.

------------

Specific comments
l. 82: what I mean when referring to Braverman et al. 2017, is that simple performance metrics do not usually encompass the full statistics of the model world, typically focusing on the first and second moments. Braverman et al. 2017, as well as others, adopt probability metrics, that allow to compare the full statistics of the model with the observations. Several distances are keys in defining these metrics, as surveyed by Gibbs and Su, 2002. Among these distances, the Wasserstein distance has been recently pointed out as particularly suitable for application in climate studies (Ghil 2015). I think this is an important aspect to mention while describing the approach here used.
ll. 116-119: I still do not find entirely convincing the motivation behind the selection of the variables, based on consensus among modelling communities. While the aim of a performance metrics is hindering at those aspects of model simulations that are particularly prone to errors and biases, they also have to provide a synthesis of the system that is reproduced. I do not see, though, how this can be further justified and I do not require additional clarification.

References
Ghil, M. (2015). A mathematical theory of climate sensitivity or, How to deal with both anthropogenic forcing and natural variability? In C.-P. Chang, M. Ghil, M. Latif, & J. Wallace (Eds.), Climate Change: Multidecadal and Beyond (Vol. 6, p. 31-52). Singapore: World Scientic Publishing Co.
Gibbs, A.L. and Su, F.E. (2002), On Choosing and Bounding Probability Metrics. International Statistical Review, 70: 419-435. doi:10.1111/j.1751-5823.2002.tb00178.x

Hide

RR by Juan Antonio Añel (27 Jun 2020)

ED: Publish subject to minor revisions (review by editor) (27 Jun 2020) by Juan Antonio Añel

AR by John Fasullo on behalf of the Authors (29 Jun 2020) Author's response Manuscript

ED: Publish as is (06 Jul 2020) by Juan Antonio Añel

Short summary

The fidelity of climate model simulations included in the WCRP Coupled Model Intercomparison Project Versions 3 through 6 is evaluated using best estimates of fields considered by the community to be critical for climate change projections. The analysis benchmarks patterns of the mean state and variability (seasonal/interannual) both within and across model generations, highlighting progress and quantifying persisting biases across models.