Submitted as: methods for assessment of models 27 Nov 2020

Submitted as: methods for assessment of models | 27 Nov 2020

Review status: this preprint is currently under review for the journal GMD.

An improved multivariable integrated evaluation method and NCL code for multimodel intercomparison (MVIETool version 1.0)

Meng-Zhuo Zhang1,2, Zhongfeng Xu2,3, Ying Han2, and Weidong Guo1,3 Meng-Zhuo Zhang et al.
  • 1Institute for Climate and Global Change Research, School of Atmospheric Sciences, Nanjing University, Nanjing, China
  • 2CAS Key Laboratory of Regional Climate and Environment for Temperate East Asia, Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing 100029, China
  • 3Jiangsu Collaborative Innovation Centerfor Climate Change, Nanjing University, Nanjing, China

Abstract. An evaluation of a model's overall performance in simulating multiple fields is fundamental to model intercomparison and development. A multivariable integrated evaluation (MVIE) method was proposed previously based on a vector field evaluation (VFE) diagram, which can provide quantitative and comprehensive evaluation on multiple fields. In this study, we make further improvements to this method from the following aspects. (1) We take area weighting into account in the definition of statistics in the VFE diagram and MVIE method, which is particularly important for a global evaluation. (2) We consider the combination of multiple scalar fields and vector fields against multiple scalar fields alone in the previous MVIE method. (3) A multivariable integrated skill score (MISS) is proposed as a flexible index to measure a model’s ability to simulate multiple fields. Compared with the MIEI proposed in the previous study, MISS is a normalized index that can adjust the relative importance of different aspects of model performance. (4) A simple-to-use and straightforward tool, the Multivariable Integrated Evaluation Tool (MVIETool), is developed to facilitate an intercomparison of the performance of various models. The tool is coded with the open-source NCAR Command Language (NCL), which provides a calculation of MVIE statistics and plotting. With the support of this tool, one can easily evaluate model performance in terms of each individual variable and/or multiple variables.

Meng-Zhuo Zhang et al.

Status: final response (author comments only)
Status: final response (author comments only)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Login for authors/topical editors] [Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Meng-Zhuo Zhang et al.

Meng-Zhuo Zhang et al.


Total article views: 276 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
220 52 4 276 2 3
  • HTML: 220
  • PDF: 52
  • XML: 4
  • Total: 276
  • BibTeX: 2
  • EndNote: 3
Views and downloads (calculated since 27 Nov 2020)
Cumulative views and downloads (calculated since 27 Nov 2020)

Viewed (geographical distribution)

Total article views: 244 (including HTML, PDF, and XML) Thereof 236 with geography defined and 8 with unknown origin.
Country # Views %
  • 1
Latest update: 23 Jan 2021
Short summary
Overall model performance evaluation in simulating multiple fields is crucial for geoscientific model development, inter-comparison, and application with increasing models available recently. We make key improvements for the Multivariable Integrated Evaluation (MVIE) method and develop a simple-to-use and straightforward tool, Multivariable Integrated Evaluation Tool (MVIETool) based on the improved MVIE, which will assist researchers to efficiently evaluate multivariable model performance.