Assessment of stochastic weather forecast of precipitation near European cities, based on analogs of circulation

Krouma, Meriem; Yiou, Pascal; Déandreis, Céline; Thao, Soulivanh

doi:https://doi.org/10.5194/gmd-15-4941-2022

Articles | Volume 15, issue 12

https://doi.org/10.5194/gmd-15-4941-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/gmd-15-4941-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 15, issue 12

Model evaluation paper

|

28 Jun 2022

Model evaluation paper |

| 28 Jun 2022

Assessment of stochastic weather forecast of precipitation near European cities, based on analogs of circulation

Meriem Krouma, Pascal Yiou, Céline Déandreis, and Soulivanh Thao

Download

Final revised paper (published on 28 Jun 2022)
Preprint (discussion started on 24 Mar 2021)

Interactive discussion

Status: closed

RC1:
'Comment on gmd-2021-36', Anonymous Referee #1, 20 Apr 2021

The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-36/gmd-2021-36-RC1-supplement.pdf

Citation: https://doi.org/10.5194/gmd-2021-36-RC1
- AC1: 'Reply on RC1', Meriem Krouma, 31 Jul 2021
  
  We thank the reviewer for the positive and constructive comments. Please find our answers in the supplement.
  
  Citation: https://doi.org/10.5194/gmd-2021-36-AC1
RC2:
'Comment on gmd-2021-36', Anonymous Referee #2, 05 Jul 2021

Review of the manuscript "Assessment of stochastic weather forecast of precipitation near European cities, based on analogs of circulation" by M. Krouma et al.

This is a very interesting manuscript, owning a good potential to become a high impact paper with positive repercussions on different societal sectors. A stochastic rain generator is produced exploiting the relationship between Z500 and precipitation in different European cities. The work is worth publication, but it needs a substantial revision about three distinct points:

1 - an improved description of the methodology is needed, in order to better understand the workflow and some of the choices that have been employed.

2 - the use of the prolonged ERA5 dataset (since 1950) is urged, in order to understand whether the differences in skill with NCEP are actually due to the length of the analog database, or to the database itself.

3 - a thorough and comprehensive revision of the English language is needed. Many subject-predicate inconsistencies, missing s', wrong sentence structures make some parts of the manuscript very hard to read.

Specific comments are stored in the attached files. In bold font, those that pertain to the above-mentioned major observations.

Citation: https://doi.org/10.5194/gmd-2021-36-RC2
- AC2: 'Reply on RC2', Meriem Krouma, 31 Jul 2021
  
  We thank the reviewer for the positive and constructive comments Please find our answers in the supplement.
  
  Citation: https://doi.org/10.5194/gmd-2021-36-AC2

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Meriem Krouma on behalf of the Authors (15 Oct 2021) Author's response Author's tracked changes

EF by Vitaly Muravyev (20 Oct 2021) Manuscript

ED: Referee Nomination & Report Request started (21 Nov 2021) by Chiel van Heerwaarden

RR by Anonymous Referee #1 (03 Dec 2021)

Suggestions for revision or reasons for rejection

General comments :
The authors have made a good job in accounting for the reviewers’ comments. The article is clearer and easier to read. My only significant point is the comparison with ECMWF and in particular what regards Fig 8 that seems surprising to me given the previous results of Section 4.5 (see below).

Detailed comments :
- abstract : « Results show significant positive skill score » : do you refer here to the comparison with ECMWF or with the persistence/climatology ?
- l 100 : brackets
- l 136 : SGW
- l 151 : brackets
- eq (2) is still wrong. You have P and xa on the left hand side while you have P_{a,t} and xa on the right hand-side. Both sides should match.
- eq (3) : please specify that bar is the mean
- l 173 : brackets
- Table 2 : why do you consider two periods for ERA5 ? ERA5 extended might be enough.
In the caption : not only winter.
- l 245 brackets
- l 272 « albeit significant » : what did yoou test exactly ?
- l 285 : brackets
- l 296-297 : « We found that the values of CRPS of ECMWF forecast and SWG forecast are 80%, 39% 50% and 40 % equal or near to zero for respectively Orly, Berlin, Madrid and Toulouse » : there are two forecasts and 4 cities but you give only five % so I don’t understand what they correspond to.
Also do you mean CRPSS ?
- l 291 « They are about 0.16 in the summer (JJA) and 0.25 in the winter (DJF) for a lead time of T = 5 days » : are these numbers right ? This seems quite bad (=low) for CRPSS. I’d have expected better CRPSS given Fig 7.
- l 293 : « The CRPSS of SWG for a lead time of T = 5 days showed in Table 2, and this suggests.. » : syntax issue
- Fig 7 : The unit of CRPSS is not mm
- Fig 8 : I’m sorry but I still don’t understand these results (see my previous review). If I understood correctly, CRPSS here is 1-mean(CRPS ECMWF)/mean(CRPS SWG) so a CRPSS value of 0.5 means that mean(CRPS ECMWF)=1.5*mean(CRPS SWG). Given Fig 7, it’s hard to believe that for Orly mean(CRPS ECMWF)=1.5*mean(CRPS SWG). From my eyes, they are almost equal so I’d have expected a CRPSS value fot T=5 around 0. For Madrid, it seems to me that the average CRPS is much lower for ECMWF than for SWG so again I’d not have expected negative CRPSS.
Maybe my eyes are confused. If so it could be useful to add a line showing the means in Fig 7. Another possibility would be to replace Fig 7 by boxplots.
For the case of Orly, I’m also still surprised (see my previous review) about the strong change in CRPSS between 5-10-20 days. In your next response, I’d be happy to see the equivalent of Fig 7 for T=10 and 20.

Hide

RR by Anonymous Referee #2 (13 Dec 2021)

Suggestions for revision or reasons for rejection

Despite the revised version has improved some aspects of the manuscript, e.g. throughout the use of ERA5 from 1950 for comparison to NCEP, the methodological choices are still poorly explained. In my opinion, the work still needs to address a few important points before being accepted for publication in GMD.

In particular, I am still concerned about the choice of the analogs domain:
- First of all, it is not clear why this information is in the Results section, and not in the methods. I see there was a Paragraph 3.3 (erased), which is probably the right place where this details should be expanded.
- Figure 1 is not useful to answer this question, since it only shows four boxes where the sensitivity to the large scale was tested.
- Table 1 compares two of these boxes in terms of correlation, to indicate that the chosen box is better than the biggest of the remaining three boxes.
- What I'd like to see, is a regression map of Z500 for rainfall in each of the four cities. This map will finally show the relationship between gridpoint precipitation and Z500 in the Euro-Atlantic region. This map could be done for one season, while the other season may be shown in the supp. mat.
- Also, the fact that two seasons are analyzed in this work is not at all clear since the beginning. JJA and DJF come out of the blue in tables 1 and 2.
- I appreciate the effort of explaining the choice of the 20 analogs, since a larger amount does not improve the results, but I think this description is incomplete. What about less analogs (5, 10, 12...)? Also, once we have 20 dates, are precipitation averaged over these days in order to obtain rainfall prediction? This should be explained.
- Section 3.4. The detailed description of weather regimes is not needed here, nor it is figure 2. In my understanding, WR are only needed to explain the forecast skill dependence on them, therefore this section should ONLY describe how the authors assess the influence of WRs on the forecast quality.

Finally, as I reported in my first review, I would like to see some more discussion on figure 4. This plot is key and it needs better descriptions, some discussion (for example about why the forecast vs persistence improves/deteriorates with time, with no clear pattern) and more accurateness (e.g. line 294, "the CRPSS for persistence" should be "the CRPSS against the persistence reference").

Hide

ED: Reconsider after major revisions (10 Jan 2022) by Chiel van Heerwaarden

AR by Meriem Krouma on behalf of the Authors (20 Feb 2022) Author's response Author's tracked changes

EF by Sarah Buchmann (22 Feb 2022) Manuscript

ED: Referee Nomination & Report Request started (27 Feb 2022) by Chiel van Heerwaarden

RR by Anonymous Referee #1 (16 Mar 2022)

RR by Anonymous Referee #2 (23 Mar 2022)

ED: Publish subject to minor revisions (review by editor) (24 Apr 2022) by Chiel van Heerwaarden

AR by Meriem Krouma on behalf of the Authors (13 May 2022) Author's response Author's tracked changes Manuscript

ED: Publish as is (20 May 2022) by Chiel van Heerwaarden

AR by Meriem Krouma on behalf of the Authors (29 May 2022) Manuscript

Short summary

We evaluated the skill of a stochastic weather generator (SWG) to forecast precipitation at different time scales and in different areas of western Europe from analogs of Z500 hPa. The SWG has the skill to simulate precipitation for 5 and 10 d. We found that forecast weaknesses can be associated with specific weather patterns. The comparison with ECMWF forecasts confirms the skill of our model. This work is important because it provides information about weather forecasts over specific areas.