Articles | Volume 15, issue 9
© Author(s) 2022. This work is distributed underthe Creative Commons Attribution 4.0 License.
Training a supermodel with noisy and sparse observations: a case study with CPT and the synch rule on SPEEDO – v.1
- Final revised paper (published on 12 May 2022)
- Preprint (discussion started on 17 Aug 2021)
Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor |
: Report abuse
- RC1: 'Comment on gmd-2021-221', Anonymous Referee #1, 17 Sep 2021
- RC2: 'Comment on gmd-2021-221', Anonymous Referee #2, 10 Nov 2021
- AC1: 'Comment on gmd-2021-221', Francine Schevenhoven, 21 Dec 2021
Peer review completion
AR: Author's response | RR: Referee report | ED: Editor decision
AR by Francine Schevenhoven on behalf of the Authors (02 Feb 2022)  Author's response Author's tracked changes Manuscript
ED: Publish subject to minor revisions (review by editor) (09 Feb 2022) by Julia Hargreaves
AR by Francine Schevenhoven on behalf of the Authors (01 Mar 2022)  Author's response Author's tracked changes Manuscript
ED: Publish as is (25 Mar 2022) by Julia Hargreaves
The paper applied two approaches, CPT and synchronization based, to “combine” multiple imperfect models to form a so called supermodel to noisy and sparse observations. The general concept is quite interesting and potentially could be very valuable to multi-model forecasting. The authors inherited previous works on CPT and synch rule, and relaxed noise-free assumption, proposed adaptations and demonstrated in SPEEDO experiments. Overall I think this paper is publishable with some revisions. Please find my concerns listed below.
1) It seems that the “imperfect” scenario described in the paper (for example section 2) are limited in perfect model class with imperfect model parameters. Is the proposed methodology applicable to a more general scenario where the model class is imperfect, i.e. there is no parameter value would lead to a perfect model.
2) I am strongly against the idea (Line 28,44) that the benefit of using MME is due to “errors tend to cancel each other”, this misinterpretation is a result of only considering the ensemble mean. Note that the purpose of using the ensemble forecasts (Leutbecher and Palmer (2008)) is to account for uncertainty.
3) The authors proposed the possibility of negative weights when form the supermodel. Personally I am a little concern about this approach, especially how would one interpret the negative weights? Can we first bias correct the models (for example Line 158, models 1 and before combine them?
4) It is a little unclear what exactly is a supermodel. is the sentence at (Line 35) a formal definition of the “supermodel”?
5) Line 98-99, it seems that achieving a synchronized state is a good thing, please define what is a synchronized supermodel state and clarify why it is desirable.
6) Line 87, “observations were available”, this statement needs to be clarified as the observations were not mention early or in the equations 1a-c.
7) Section 3.3, please give more details about how the CPT trajectories are generated.
8) Paragraph at Line 240, It seems that the authors suggest that when the models are linear, the CPT approach works well. Why it doesn’t if the model is highly nonlinear?
9) Table 4 and Figure 3 shows the weights for supermodel trained by CPT and synch rule, it would be interesting to see the forecast performance of CPT compared with synch rule.