Skill and independence weighting for multi-model assessments

Sanderson, Benjamin M.; Wehner, Michael; Knutti, Reto

doi:https://doi.org/10.5194/gmd-10-2379-2017

Articles | Volume 10, issue 6

https://doi.org/10.5194/gmd-10-2379-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

https://doi.org/10.5194/gmd-10-2379-2017

© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 3.0 License.

Articles | Volume 10, issue 6

Methods for assessment of models

|

28 Jun 2017

Methods for assessment of models |

| 28 Jun 2017

Skill and independence weighting for multi-model assessments

Benjamin M. Sanderson, Michael Wehner, and Reto Knutti

Download

Final revised paper (published on 28 Jun 2017)
Preprint (discussion started on 21 Dec 2016)
Supplement to the preprint

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review', Anonymous Referee #1, 27 Feb 2017
- AC1: 'Response to Reviewer 1', Ben Sanderson, 04 Apr 2017
RC2: 'Interactive comment on “Skill and independence weighting for multi-model assessments” by Benjamin Sanderson et al.', Craig H. Bishop, 06 Mar 2017
- AC2: 'Response to Reviewer 2 (Craig Bishop)', Ben Sanderson, 04 Apr 2017

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Ben Sanderson on behalf of the Authors (27 Apr 2017) Author's response Manuscript

ED: Publish subject to minor revisions (Editor review) (29 May 2017) by Steve Easterbrook

Thank you for your detailed response to the two reviews, and for the substantial revisions to the manuscript. I believe the paper is now suitable for acceptance, but on reading the revised manuscript, I noticed a few places where the text still is not as clear as it might be. Given that this paper will form the archived record for the weighting method for NCA4, some extra clarity is important, especially on how the weights are derived. Hence I would like to see the following (minor) revisions before publication:

49-51: "...ability of a model to project a certain change". It's not clear to me what this means. I assume you're talking about fitness-for-purpose here. E.g. a model's skill might vary for different regions of the world, or for different variables of interest. Please expand this sentence to make it clearer.

126: "...contains on the ..." A typo, I think?

139: should say "For each variable, v, ..."

144: "monthly climatology". But table 1 says "seasonal". Is there another aggregation step here? Or is one of these incorrect?

146-147: "area-weighted root mean square difference over the domain". This needs a little more explanation. Presumably, model output is gridded (but on different grids, depending on the model), and some (all?) of the observational data is gridded. I guess "area-weighted" suggests you've done something to handle the grid disparities. But I don't understand the steps involved in getting from the individual data points to an overall distance measure for each variable. Please elaborate (if only so that others could follow the same steps if they want to use the same analysis for other purposes).

147 "The matrix is then normalized". I think this should say "Each matrix...", or did I misunderstand? And is it normalized against the entire (inter-model) matrix mean, or just the mean distances of each model to the observations?

160: "The inter-model distance matrix is also computed from the inter-model distance matrix". This sentence doesn't make any sense.

202: The notation here is very awkward. Is it necessary to include the 20c superscript? You don't reference it anywhere in the text.

Figure 6: Please label the x axis.

386: "tailer" -> "tailor"

Hide

AR by Ben Sanderson on behalf of the Authors (08 Jun 2017) Author's response Manuscript

ED: Publish as is (16 Jun 2017) by Steve Easterbrook

AR by Ben Sanderson on behalf of the Authors (19 Jun 2017)

Short summary

How should climate model simulations be combined to produce an overall assessment that reflects both their performance and their interdependencies? This paper presents a strategy for weighting climate model output such that models that are replicated or models that perform poorly in a chosen set of metrics are appropriately weighted. We perform sensitivity tests to show how the method results depend on variables and parameter values.