Evaluation of Model Intercomparison Projects
Evaluation of Model Intercomparison Projects
Editor(s): GMD topic editors | Coordinators: GMD executive editors
This special issue publishes model evaluation papers emerging from the Coupled Model Intercomparison Project Phase 6 (CMIP6) and other model intercomparison projects (MIPs). CMIP evaluation papers present particular challenges both for authors and for GMD’s policy on code and data availability.
This special issue interprets the code and data availability rules to make them applicable to CMIP evaluation papers and to provide clarity to authors as to what they need to provide in their manuscripts.

The guidance here must be read in conjunction with the GMD rules on manuscript types and code and data availability.
https://www.geoscientific-model-development.net/about/manuscript_types.html https://www.geoscientific-model-development.net/about/code_and_data_policy.html
The rules there must be followed, subject to the specific provisions outlined below.

Manuscript scope
The manuscript must meet the scope rules for a type of GMD manuscript. This will usually be “model evaluation” but may be another GMD-recognised type. In particular, this means that the subject of the manuscript must be the performance, behaviour, and/or design of the models. Manuscripts whose subject is the behaviour of some aspect of the Earth system or which draw conclusions about that system will be rejected as out of scope. Instead, they should be submitted to appropriate application area journals.

Publication of the MIP protocol
The experimental protocol for the MIP being evaluated must have been published in a peer-reviewed forum in advance of the MIP being conducted and must be cited in the manuscript. For CMIP6, this will have been in the GMD special issue “Coupled Model Intercomparison Project Phase 6 (CMIP6) Experimental Design and Organization”.
Where the evaluation uses model results which were produced in circumstances which deviate from the published protocol, this deviation must be documented either in the evaluation manuscript or in another paper cited in the manuscript.

Model code and data requirements
MIP evaluation papers will frequently span simulations conducted with many models, most or all of which are outside the control of the authors conducting the evaluation. The interpretation of the code and data availability rules presented here is designed to ensure the transparency and reproducibility of the evaluation itself.
1. All of the model outputs and observational data used in the evaluation must be publicly and persistently archived in accordance with GMD policy. For CMIP6, this will usually be on Earth System Grid Federation (ESGF). The manuscript must identify the data used with sufficient precision that a reader would be able to reproduce the analysis. GMD policy explicitly prohibits embargoes. If the data are embargoed, then manuscript submission must be delayed until the embargo expires.
2. The exact version of the analysis code used to conduct the evaluation must be publicly and persistently archived, subject only to the exceptions beyond the authors’ control which are detailed in the GMD code and data availability policy.
3. In the case of evaluations of multiple models participating in the same MIP, the usual requirement that there must be a published model description paper about the models being evaluated is waived. Where the evaluation is of a single model’s performance in a MIP, the usual requirement that there must be a previous model description paper applies.

Download citations of all papers

09 Mar 2021
Evaluation of regional climate models ALARO-0 and REMO2015 at 0.22° resolution over the CORDEX Central Asia domain
Sara Top, Lola Kotova, Lesley De Cruz, Svetlana Aniskevich, Leonid Bobylev, Rozemien De Troch, Natalia Gnatiuk, Anne Gobin, Rafiq Hamdi, Arne Kriegsmann, Armelle Reca Remedio, Abdulla Sakalli, Hans Van De Vyver, Bert Van Schaeybroeck, Viesturs Zandersons, Philippe De Maeyer, Piet Termonia, and Steven Caluwaerts
Geosci. Model Dev., 14, 1267–1293, https://doi.org/10.5194/gmd-14-1267-2021,https://doi.org/10.5194/gmd-14-1267-2021, 2021
Short summary
11 Nov 2020
European daily precipitation according to EURO-CORDEX regional climate models (RCMs) and high-resolution global climate models (GCMs) from the High-Resolution Model Intercomparison Project (HighResMIP)
Marie-Estelle Demory, Ségolène Berthou, Jesús Fernández, Silje L. Sørland, Roman Brogli, Malcolm J. Roberts, Urs Beyerle, Jon Seddon, Rein Haarsma, Christoph Schär, Erasmo Buonomo, Ole B. Christensen, James M. Ciarlo ̀, Rowan Fealy, Grigory Nikulin, Daniele Peano, Dian Putrasahan, Christopher D. Roberts, Retish Senan, Christian Steger, Claas Teichmann, and Robert Vautard
Geosci. Model Dev., 13, 5485–5506, https://doi.org/10.5194/gmd-13-5485-2020,https://doi.org/10.5194/gmd-13-5485-2020, 2020
Short summary
29 Sep 2020
Impact of horizontal resolution on global ocean–sea ice model simulations based on the experimental protocols of the Ocean Model Intercomparison Project phase 2 (OMIP-2)
Eric P. Chassignet, Stephen G. Yeager, Baylor Fox-Kemper, Alexandra Bozec, Frederic Castruccio, Gokhan Danabasoglu, Christopher Horvat, Who M. Kim, Nikolay Koldunov, Yiwen Li, Pengfei Lin, Hailong Liu, Dmitry V. Sein, Dmitry Sidorenko, Qiang Wang, and Xiaobiao Xu
Geosci. Model Dev., 13, 4595–4637, https://doi.org/10.5194/gmd-13-4595-2020,https://doi.org/10.5194/gmd-13-4595-2020, 2020
Short summary
21 Aug 2020
Evaluation of global ocean–sea-ice model simulations based on the experimental protocols of the Ocean Model Intercomparison Project phase 2 (OMIP-2)
Hiroyuki Tsujino, L. Shogo Urakawa, Stephen M. Griffies, Gokhan Danabasoglu, Alistair J. Adcroft, Arthur E. Amaral, Thomas Arsouze, Mats Bentsen, Raffaele Bernardello, Claus W. Böning, Alexandra Bozec, Eric P. Chassignet, Sergey Danilov, Raphael Dussin, Eleftheria Exarchou, Pier Giuseppe Fogli, Baylor Fox-Kemper, Chuncheng Guo, Mehmet Ilicak, Doroteaciro Iovino, Who M. Kim, Nikolay Koldunov, Vladimir Lapin, Yiwen Li, Pengfei Lin, Keith Lindsay, Hailong Liu, Matthew C. Long, Yoshiki Komuro, Simon J. Marsland, Simona Masina, Aleksi Nummelin, Jan Klaus Rieck, Yohan Ruprich-Robert, Markus Scheinert, Valentina Sicardi, Dmitry Sidorenko, Tatsuo Suzuki, Hiroaki Tatebe, Qiang Wang, Stephen G. Yeager, and Zipeng Yu
Geosci. Model Dev., 13, 3643–3708, https://doi.org/10.5194/gmd-13-3643-2020,https://doi.org/10.5194/gmd-13-3643-2020, 2020
Short summary
17 Jul 2020
Quantitative assessment of fire and vegetation properties in simulations with fire-enabled vegetation models from the Fire Model Intercomparison Project
Stijn Hantson, Douglas I. Kelley, Almut Arneth, Sandy P. Harrison, Sally Archibald, Dominique Bachelet, Matthew Forrest, Thomas Hickler, Gitta Lasslop, Fang Li, Stephane Mangeon, Joe R. Melton, Lars Nieradzik, Sam S. Rabin, I. Colin Prentice, Tim Sheehan, Stephen Sitch, Lina Teckentrup, Apostolos Voulgarakis, and Chao Yue
Geosci. Model Dev., 13, 3299–3318, https://doi.org/10.5194/gmd-13-3299-2020,https://doi.org/10.5194/gmd-13-3299-2020, 2020
Short summary
CC BY 4.0