Articles | Volume 17, issue 9
https://doi.org/10.5194/gmd-17-3919-2024
https://doi.org/10.5194/gmd-17-3919-2024
Model evaluation paper
 | 
15 May 2024
Model evaluation paper |  | 15 May 2024

Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3

Jiwoo Lee, Peter J. Gleckler, Min-Seop Ahn, Ana Ordonez, Paul A. Ullrich, Kenneth R. Sperber, Karl E. Taylor, Yann Y. Planton, Eric Guilyardi, Paul Durack, Celine Bonfils, Mark D. Zelinka, Li-Wei Chao, Bo Dong, Charles Doutriaux, Chengzhu Zhang, Tom Vo, Jason Boutte, Michael F. Wehner, Angeline G. Pendergrass, Daehyun Kim, Zeyu Xue, Andrew T. Wittenberg, and John Krasting

Related authors

The DOE E3SM version 2.1: overview and assessment of the impacts of parameterized ocean submesoscales
Katherine M. Smith, Alice M. Barthel, LeAnn M. Conlon, Luke P. Van Roekel, Anthony Bartoletti, Jean-Christophe Golaz, Chengzhu Zhang, Carolyn Branecky Begeman, James J. Benedict, Gautam Bisht, Yan Feng, Walter Hannah, Bryce E. Harrop, Nicole Jeffery, Wuyin Lin, Po-Lun Ma, Mathew E. Maltrud, Mark R. Petersen, Balwinder Singh, Qi Tang, Teklu Tesfa, Jonathan D. Wolfe, Shaocheng Xie, Xue Zheng, Karthik Balaguru, Oluwayemi Garuba, Peter Gleckler, Aixue Hu, Jiwoo Lee, Ben Moore-Maley, and Ana C. Ordoñez
Geosci. Model Dev., 18, 1613–1633, https://doi.org/10.5194/gmd-18-1613-2025,https://doi.org/10.5194/gmd-18-1613-2025, 2025
Short summary
A new metrics framework for quantifying and intercomparing atmospheric rivers in observations, reanalyses, and climate models
Bo Dong, Paul Ullrich, Jiwoo Lee, Peter Gleckler, Kristin Chang, and Travis A. O'Brien
Geosci. Model Dev., 18, 961–976, https://doi.org/10.5194/gmd-18-961-2025,https://doi.org/10.5194/gmd-18-961-2025, 2025
Short summary
The Coupled Model Intercomparison Project (CMIP): Reviewing project history, evolution, infrastructure and implementation
Paul J. Durack, Karl E. Taylor, Peter J. Gleckler, Gerald A. Meehl, Bryan N. Lawrence, Curt Covey, Ronald J. Stouffer, Guillaume Levavasseur, Atef Ben-Nasser, Sebastien Denvil, Martina Stockhause, Jonathan M. Gregory, Martin Juckes, Sasha K. Ames, Fabrizio Antonio, David C. Bader, John P. Dunne, Daniel Ellis, Veronika Eyring, Sandro L. Fiore, Sylvie Joussaume, Philip Kershaw, Jean-Francois Lamarque, Michael Lautenschlager, Jiwoo Lee, Chris F. Mauzey, Matthew Mizielinski, Paola Nassisi, Alessandra Nuzzo, Eleanor O’Rourke, Jeffrey Painter, Gerald L. Potter, Sven Rodriguez, and Dean N. Williams
EGUsphere, https://doi.org/10.5194/egusphere-2024-3729,https://doi.org/10.5194/egusphere-2024-3729, 2025
Short summary
Evaluating downscaled products with expected hydroclimatic co-variances
Seung H. Baek, Paul A. Ullrich, Bo Dong, and Jiwoo Lee
Geosci. Model Dev., 17, 8665–8681, https://doi.org/10.5194/gmd-17-8665-2024,https://doi.org/10.5194/gmd-17-8665-2024, 2024
Short summary
Evaluating precipitation distributions at regional scales: a benchmarking framework and application to CMIP5 and 6 models
Min-Seop Ahn, Paul A. Ullrich, Peter J. Gleckler, Jiwoo Lee, Ana C. Ordonez, and Angeline G. Pendergrass
Geosci. Model Dev., 16, 3927–3951, https://doi.org/10.5194/gmd-16-3927-2023,https://doi.org/10.5194/gmd-16-3927-2023, 2023
Short summary

Related subject area

Climate and Earth system modeling
Advanced climate model evaluation with ESMValTool v2.11.0 using parallel, out-of-core, and distributed computing
Manuel Schlund, Bouwe Andela, Jörg Benke, Ruth Comer, Birgit Hassler, Emma Hogan, Peter Kalverla, Axel Lauer, Bill Little, Saskia Loosveldt Tomas, Francesco Nattino, Patrick Peglar, Valeriu Predoi, Stef Smeets, Stephen Worsley, Martin Yeo, and Klaus Zimmermann
Geosci. Model Dev., 18, 4009–4021, https://doi.org/10.5194/gmd-18-4009-2025,https://doi.org/10.5194/gmd-18-4009-2025, 2025
Short summary
ICON-HAM-lite 1.0: simulating the Earth system with interactive aerosols at kilometer scales
Philipp Weiss, Ross Herbert, and Philip Stier
Geosci. Model Dev., 18, 3877–3894, https://doi.org/10.5194/gmd-18-3877-2025,https://doi.org/10.5194/gmd-18-3877-2025, 2025
Short summary
Process-based modeling framework for sustainable irrigation management at the regional scale: integrating rice production, water use, and greenhouse gas emissions
Yan Bo, Hao Liang, Tao Li, and Feng Zhou
Geosci. Model Dev., 18, 3799–3817, https://doi.org/10.5194/gmd-18-3799-2025,https://doi.org/10.5194/gmd-18-3799-2025, 2025
Short summary
Implementing deep soil and dynamic root uptake in Noah-MP (v4.5): impact on Amazon dry-season transpiration
Carolina A. Bieri, Francina Dominguez, Gonzalo Miguez-Macho, and Ying Fan
Geosci. Model Dev., 18, 3755–3779, https://doi.org/10.5194/gmd-18-3755-2025,https://doi.org/10.5194/gmd-18-3755-2025, 2025
Short summary
Reducing time and computing costs in EC-Earth: an automatic load-balancing approach for coupled Earth system models
Sergi Palomas, Mario C. Acosta, Gladys Utrera, and Etienne Tourigny
Geosci. Model Dev., 18, 3661–3679, https://doi.org/10.5194/gmd-18-3661-2025,https://doi.org/10.5194/gmd-18-3661-2025, 2025
Short summary

Cited articles

Adler, R. F., Sapiano, M. R., Huffman, G. J., Wang, J. J., Gu, G., Bolvin, D., Chiu, L., Schneider, U., Becker, A., Nelkin, E., Xie, P., Ferraro, R., and Shin, D.-B.: The Global Precipitation Climatology Project (GPCP) monthly analysis (new version 2.3) and a review of 2017 global precipitation. Atmosphere, 9, 138, https://doi.org/10.3390/atmos9040138, 2018. 
Ahn, M.-S., Kim, D. H., Sperber, K. R., Kang, I.-S., Maloney, E. D., Waliser, D. E., and Hendon, H. H.: MJO simulation in CMIP5 climate models: MJO skill metrics and process-oriented diagnosis, Clim. Dynam., 49, 4023–4045, https://doi.org/10.1007/s00382-017-3558-4, 2017. 
Ahn, M.-S., Gleckler, P. J., Lee, J., Pendergrass, A. G., and Jakob, C.: Benchmarking Simulated Precipitation Variability Amplitude across Time Scales, J. Climate, 35, 3173–3196, https://doi.org/10.1175/jcli-d-21-0542.1, 2022. 
Ahn, M.-S., Ullrich, P. A., Gleckler, P. J., Lee, J., Ordonez, A. C., and Pendergrass, A. G.: Evaluating precipitation distributions at regional scales: a benchmarking framework and application to CMIP5 and 6 models, Geosci. Model Dev., 16, 3927–3951, https://doi.org/10.5194/gmd-16-3927-2023, 2023. 
Anaconda pcmdi_metrics: https://anaconda.org/conda-forge/pcmdi_metrics, last access: 8 May 2024. 
Download
Short summary
We introduce an open-source software, the PCMDI Metrics Package (PMP), developed for a comprehensive comparison of Earth system models (ESMs) with real-world observations. Using diverse metrics evaluating climatology, variability, and extremes simulated in thousands of simulations from the Coupled Model Intercomparison Project (CMIP), PMP aids in benchmarking model improvements across generations. PMP also enables efficient tracking of performance evolutions during ESM developments.
Share