The Sailor diagram – A new diagram for the verification of two-dimensional vector data   from multiple models

Sáenz, Jon; Carreno-Madinabeitia, Sheila; Esnaola, Ganix; González-Rojí, Santos J.; Ibarra-Berastegi, Gabriel; Ulazia, Alain

doi:https://doi.org/10.5194/gmd-13-3221-2020

Articles | Volume 13, issue 7

https://doi.org/10.5194/gmd-13-3221-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/gmd-13-3221-2020

© Author(s) 2020. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 13, issue 7

Methods for assessment of models

|

15 Jul 2020

Methods for assessment of models |

| 15 Jul 2020

The Sailor diagram – A new diagram for the verification of two-dimensional vector data from multiple models

Jon Sáenz, Sheila Carreno-Madinabeitia, Ganix Esnaola, Santos J. González-Rojí, Gabriel Ibarra-Berastegi, and Alain Ulazia

Download

Final revised paper (published on 15 Jul 2020)
Preprint (discussion started on 19 Nov 2019)

Interactive discussion

Status: closed

AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment

- Printer-friendly version

- Supplement

RC1: 'Review of "The Sailor diagram"', Anonymous Referee #1, 03 Jan 2020
- AC1: 'Reply to comments by Reviewer 1', Jon Saenz, 13 Jan 2020
  - AC2: 'New Figure 1', Jon Saenz, 13 Jan 2020
  - AC3: 'New Table 1', Jon Saenz, 13 Jan 2020
  - AC4: 'Potential change to last figure (right)', Jon Saenz, 13 Jan 2020
RC2: 'Comments on the Sailor diagram', Anonymous Referee #2, 13 Jan 2020
- AC5: 'Reply to comments by Reviewer 2', Jon Saenz, 20 Jan 2020

Peer-review completion

AR: Author's response | RR: Referee report | ED: Editor decision

AR by Jon Saenz on behalf of the Authors (01 Feb 2020) Author's response Manuscript

ED: Reconsider after major revisions (18 Feb 2020) by Olivier Marti

AR by Jon Saenz on behalf of the Authors (25 Feb 2020) Author's response Manuscript

ED: Referee Nomination & Report Request started (02 Mar 2020) by Olivier Marti

RR by Anonymous Referee #3 (14 May 2020)

Suggestions for revision or reasons for rejection

Review of “The Sailor diagram. A new diagram for the verification of two-dimensional vector data from multiple models”, by Sáenz et al.

This paper presents a graphical way of comparing two dimensional fields (like wind speed), like Taylor diagrams, designed to compare model simulations.
The methodology is well described and the applications are clear.

I have a few concerns on the methodology:
The way the results are shown implies using colors, which prevents its use by color blind researchers. For example, the lines in Figs. 6b, 7b, 8 are very hard to distinguish. Taylor diagrams can easily show results for tens of models. Sailor diagrams for more than two models might be awkward to decipher. Sailor diagrams do not directly provide the values of the diagnostics (unlike Taylor diagrams). This caveat could be discussed in the paper, even briefly.

My second issue is the apparent complication of using 2d EOFs. Why not compute the correlation between complex numbers, as 2d fields can be represented as complex numbers? Hence, one retrieves the phase and amplitude of the (complex) correlation. And then produce something like Taylor diagrams. I am not saying that Sailor diagrams are a bad idea or wrong… but a relatively simpler formulation with complex numbers would allow using the existing machinery of multivariate spectral analysis, and make a use of Taylor diagrams. I won’t ask the authors to do this (or even perform a comparison, although this is conceptually easy), but at least justify their option.

Minor comments
What is the use of Fig. 1a?

Should I deduce from Fig. 8 that the two models have the right wind direction?

The diagnostics on the MIROC and IPSL simulations reveal that one is positively biased, the other is negatively biased, but the difference between the two models is <0.03 m/s. Is this meaningful? I would have prefered that the authors chose examples where the differences are really important, which would give an incentive for bias correction.

Hide

ED: Publish subject to minor revisions (review by editor) (18 May 2020) by Olivier Marti

AR by Jon Saenz on behalf of the Authors (26 May 2020) Manuscript

ED: Publish as is (08 Jun 2020) by Olivier Marti

AR by Jon Saenz on behalf of the Authors (08 Jun 2020) Manuscript

Download

The requested paper has a corresponding corrigendum published. Please read the corrigendum first before downloading the article.

Article (2806 KB)
Full-text XML

Short summary

A new diagram for the verification of vector variables (wind, current, etc.) generated by multiple models against a set of observations is presented in this package. It has been designed as a generalization of the Taylor diagram for two-dimensional quantities. It is based on the analysis of the two-dimensional structure of the mean squared error matrix between model and observations, and it allows for an easy assessment of both bias and directional errors as well.