Preprints
https://doi.org/10.5194/gmd-2023-134
https://doi.org/10.5194/gmd-2023-134
Submitted as: methods for assessment of models
 | 
22 Aug 2023
Submitted as: methods for assessment of models |  | 22 Aug 2023
Status: a revised version of this preprint is currently under review for the journal GMD.

Bergen Metrics: composite error metrics for assessing performance of climate models using EURO-CORDEX simulations

Alok Samantaray, Priscilla A. Mooney, and Carla A. Vivacqua

Abstract. Error metrics are useful for evaluating model performance and have been used extensively in climate change studies. Despite the abundance of error metrics in the literature, most studies use only one or two metrics. Since each metric evaluates a specific aspect of the relationship between the reference data and model data, restricting the comparison to just one or two metrics limits the range of insights derived from the analysis. This study proposes a new framework and composite error metrics called Bergen Metrics to summarise the overall performance of climate models and to ease interpretation of results from multiple error metrics. The framework of Bergen Metrics are based on the p-norm, and the first norm is selected to evaluate the climate models. The framework includes the application of a non-parametric clustering technique to multiple error metrics to reduce the number of error metrics with minimum information loss. An example of Bergen Metrics is provided through its application to the large ensemble of regional climate simulations available from the EURO-CORDEX initiative. This study calculates 38 different error metrics to assess the performance of 89 regional climate simulations of precipitation and temperature over Europe. The non-parametric clustering technique is applied to these 38 metrics to reduce the number of metrics to be used in Bergen Metrics for 8 different sub-regions in Europe. These provide useful information about the performance of the error metrics in different regions. Results show it is possible to observe contradictory behaviour among error metrics when examining a single model. Therefore, the study also underscores the significance of employing multiple error metrics depending on the specific use case to achieve a thorough understanding of the model behaviour.

Alok Samantaray, Priscilla A. Mooney, and Carla A. Vivacqua

Status: final response (author comments only)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CEC1: 'Comment on gmd-2023-134', Juan Antonio Añel, 06 Oct 2023
    • AC1: 'Reply on CEC1', Alok Samantaray, 09 Oct 2023
      • CEC2: 'Reply on AC1', Juan Antonio Añel, 09 Oct 2023
        • AC2: 'Reply on CEC2', Alok Samantaray, 10 Oct 2023
  • RC1: 'Comment on gmd-2023-134', Anonymous Referee #1, 10 Oct 2023
    • AC4: 'Reply on RC1', Alok Samantaray, 16 Jan 2024
  • RC2: 'Comment on gmd-2023-134', Anonymous Referee #2, 29 Dec 2023
    • AC3: 'Reply on RC2', Alok Samantaray, 16 Jan 2024
Alok Samantaray, Priscilla A. Mooney, and Carla A. Vivacqua
Alok Samantaray, Priscilla A. Mooney, and Carla A. Vivacqua

Viewed

Total article views: 420 (including HTML, PDF, and XML)
HTML PDF XML Total Supplement BibTeX EndNote
294 100 26 420 32 16 8
  • HTML: 294
  • PDF: 100
  • XML: 26
  • Total: 420
  • Supplement: 32
  • BibTeX: 16
  • EndNote: 8
Views and downloads (calculated since 22 Aug 2023)
Cumulative views and downloads (calculated since 22 Aug 2023)

Viewed (geographical distribution)

Total article views: 402 (including HTML, PDF, and XML) Thereof 402 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 02 Feb 2024
Download
Short summary
Any interpretation of climate modelled data requires a comprehensive evaluation of the model performance. Numerous error metrics exist for this purpose and each focus on a specific aspect of the relationship between reference and model data. Thus, a comprehensive evaluation demands the use of multiple error metrics. However, this can lead to confusion. We propose a clustering technique to reduce the number of error metrics needed and a composite error metric to simplify the interpretation.