Articles | Volume 10, issue 4
Geosci. Model Dev., 10, 1403–1422, 2017

Special issue: Agricultural Model Intercomparison and Improvement Project...

Geosci. Model Dev., 10, 1403–1422, 2017

Model evaluation paper 04 Apr 2017

Model evaluation paper | 04 Apr 2017

Global gridded crop model evaluation: benchmarking, skills, deficiencies and implications

Christoph Müller1, Joshua Elliott2,3, James Chryssanthacopoulos2,3, Almut Arneth4, Juraj Balkovic5,6, Philippe Ciais7, Delphine Deryng2,3, Christian Folberth5,8, Michael Glotter9, Steven Hoek10, Toshichika Iizumi11, Roberto C. Izaurralde12,13, Curtis Jones12, Nikolay Khabarov5, Peter Lawrence14, Wenfeng Liu15, Stefan Olin16, Thomas A. M. Pugh4,17, Deepak K. Ray18, Ashwan Reddy12, Cynthia Rosenzweig19,3, Alex C. Ruane19,3, Gen Sakurai11, Erwin Schmid20, Rastislav Skalsky5, Carol X. Song21, Xuhui Wang7,22, Allard de Wit10, and Hong Yang15,23 Christoph Müller et al.
  • 1Potsdam Institute for Climate Impact Research, 14473 Potsdam, Germany
  • 2University of Chicago and ANL Computation Institute, Chicago, IL 60637, USA
  • 3Columbia University Center for Climate Systems Research, New York, NY 10025, USA
  • 4Karlsruhe Institute of Technology, IMK-IFU, 82467 Garmisch-Partenkirchen, Germany
  • 5International Institute for Applied Systems Analysis, Ecosystem Services and Management Program, 2361 Laxenburg, Austria
  • 6Department of Soil Science, Comenius University in Bratislava, 842 15 Bratislava, Slovak Republic
  • 7Laboratoire des Sciences du Climat et de l'Environnement. CEA CNRS UVSQ Orme des Merisiers, 91191 Gif-sur-Yvette, France
  • 8Department of Geography, Ludwig Maximilian University, 80333 Munich, Germany
  • 9Department of the Geophysical Sciences, University of Chicago, Chicago, IL 60637, USA
  • 10Earth Observation and Environmental Informatics, Alterra Wageningen University and Research Centre, 6708PB Wageningen, the Netherlands
  • 11Institute for Agro-Environmental Sciences, National Agriculture and Food Research Organization, Tsukuba, 305-8604, Japan
  • 12Department of Geographical Sciences, University of Maryland, College Park, MD 20742, USA
  • 13Texas AgriLife Research and Extension, Texas A&M University, Temple, TX 76502, USA
  • 14Earth System Laboratory, National Center for Atmospheric Research, Boulder, CO 80307, USA
  • 15Eawag, Swiss Federal Institute of Aquatic Science and Technology, 8600 Duebendorf, Switzerland
  • 16Department of Physical Geography and Ecosystem Science, Lund University, 223 62 Lund, Sweden
  • 17School of Geography, Earth & Environmental Sciences and Birmingham Institute of Forest Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK
  • 18Institute on the Environment, University of Minnesota, Saint Paul, MN, USA
  • 19National Aeronautics and Space Administration Goddard Institute for Space Studies, New York, NY 10025, USA
  • 20Institute for Sustainable Economic Development, University of Natural Resources and Life Sciences, 1180 Vienna, Austria
  • 21Rosen Center for Advanced Computing, Purdue University, West Lafayette, IN, USA
  • 22Sino-French Institute of Earth System Sciences, Peking University, 100871 Beijing, China
  • 23Department of Environmental Sciences, University of Basel, Petersplatz 1, 4003 Basel, Switzerland

Abstract. Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

Short summary
Crop models are increasingly used in climate change impact research and integrated assessments. For the Agricultural Model Intercomparison and Improvement Project (AgMIP), 14 global gridded crop models (GGCMs) have supplied crop yield simulations (1980–2010) for maize, wheat, rice and soybean. We evaluate the performance of these models against observational data at global, national and grid cell level. We propose an open-access benchmark system against which future model versions can be tested.