Assimilation of GPM-retrieved ocean surface meteorology data for two snowstorm events during ICE-POP 2018

Li, Xuanli; Roberts, Jason B.; Srikishen, Jayanthi; Case, Jonathan L.; Petersen, Walter A.; Lee, Gyuwon; Hain, Christopher R.

doi:https://doi.org/10.5194/gmd-15-5287-2022

Articles | Volume 15, issue 13

https://doi.org/10.5194/gmd-15-5287-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Special issue:

Winter weather research in complex terrain during ICE-POP...

https://doi.org/10.5194/gmd-15-5287-2022

© Author(s) 2022. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 15, issue 13

Development and technical paper

|

12 Jul 2022

Development and technical paper |

| 12 Jul 2022

Assimilation of GPM-retrieved ocean surface meteorology data for two snowstorm events during ICE-POP 2018

Xuanli Li, Jason B. Roberts, Jayanthi Srikishen, Jonathan L. Case, Walter A. Petersen, Gyuwon Lee, and Christopher R. Hain

Download

Final revised paper (published on 12 Jul 2022)
Preprint (discussion started on 20 Sep 2021)

Interactive discussion

Status: closed

RC1:
'Comment on gmd-2021-161', Anonymous Referee #1, 24 Oct 2021
Review of

Assimilation of GPM-retrieved Ocean Surface Meteorology Data for Two Snowstorm Events during ICE-POP 2018

by Li et al.

General comments

This manuscript intends to investigate the impact of the assimilation of the Global Precipitation Measurement (GPM) retrieved ocean surface observations on two winter storms during 2018 using GSI and WRF model. The discussions primarily focus on the analysis impacts in the observation space, and the precipitation predictions for each case. This study has potentials in improving our understanding of how to optimally utilize the GPM retrievals. But the unclear motivation and lack of in-depth discussions make the paper reading frustrating. I would not recommend publication of this work until substantial revisions have been made. The detailed comments are listed below.

Major comments/concerns:

One major concern I have about this manuscript is the unclear goal. If this study is focusing on the assimilation of GPM retrievals as suggested by the title, then the introduction should provide more information about the GPM retrievals, such as how it was (or was not) previously applied/researched in the field of snowstorm predictions (or other similar fields), and what’s the novelty of this current study (not just the technical specifics in section 2.1). Also, the results section should contain more discussions on how the corrections from this dataset are physically benefiting the storm predictions, not just showing there is a difference and the model is improved. This is a case study after all. Please consider re-organize the manuscript/title to better represent your point and outstand your novelty.

The results section spends large portions of contents on how the background is modified towards the observations for each snowstorm event. But the discussions on either A-B or O-B can just tell us that the DA is working properly since the verification is dependent. Any successful DA should make the analysis closer to observations than the background, this is from the mathematical nature of the minimization of the cost function. As indicated between L126-132, there should be plenty of other observations (e.g. D3R) available in addition to the retrievals. Why not use those independent observations to verify your analysis? It will objectively measure if the DA is really improving the analysis.

Minor but still important comments:

L220: The cost function is not just to measure the difference between the model and observations (otherwise it should be just yobs-Hxb). It is the sum of weighted (B and R) differences between the analysis estimation xa and model background xb (Jb), and between xa and observations y (Jo). The goal of DA is to find an optimal estimation xa that minimizes this J.

L225: xguess is xb, please use just one subscription for consistency. This y term is also commonly known as innovation in recent DA literatures.

L240: This comment may be trivial, but can you provide more details on how you perform the cycling? There are two start mode in WRF, restart and cold start. In the restart mode, WRF is able to continue the model integration without interruption by using both the analysis at the current time step and the tendencies from the previous time step. However, our current DA methods usually only updates the analysis, not the tendencies. I'm wondering if you saw any discontinuity issue if you are using this mode. On the other hand, if you are using cold start mode (restart=.false. by default, which only uses the analysis at the current step), it will interrupt previous integrations and perform differently with your control experiments even without DA. You have to let the control exp stop at the same time for consistency.

L261: Current Fig. 5 may not be the best way to show those info. I don't think it's possible to retrieve the percentile info directly from the figure.

L261-271: What do these statistics mean? Some of the o-b standard deviations are almost comparable or smaller than the corresponding observation errors (e.g. specific humidity). Are you suggesting your model background is very good already?

L291-293: Maybe just show the RMSD of O-A and O-B to better present the point if you are not going to physically explain how those improvements can benefit the storm analysis or predictions?

L299-301: I’m confused by the motivation of the discussion here. Differences should be there as long as you changed the initial analysis. Of cause the differences will spread out as the model tries to balance the changes. But without observations in those areas, how do we know if this “spread of information” is correct or not?

L308-315: What is your localization length scale? Are these reducing differences in the area of your observations and increasing differences in the remote area reasonable? What’s the physical explanations or guesses?

L324-329: How should we interpret this result from Fig. 8? Is it reasonable for a continuously cycled DA experiment to become more alike the NoDA experiment after several cycles? It appears to me that the impact of DA is fading after cycling. Is this true? Also, the increasing RMSD of wind speed at almost all levels from 21 UTC to 22 UTC (Fig. 8d) seems to be inconsistent with the reducing total RMSD in Fig. 8a. Why?

Fig. 9 -14: Can you use something like ETS score to quantitively verify the precipitation predictions? Also, in your discussions with the precipitation patterns, can you physically relate them to your previous analysis impacts? E.g. how does a warmer temperature in the analysis within certain area result in more precipitation predictions.

L371-380 and others: Why does the accuracy of this IMERG matter? How is this related to the focus of this paper? It reads to me that the discussions on the IMERG is out of no where. If you think this is a important part of your project and is somehow related to the focus of this paper, please add more information in the intro.

L407-410: Why does the specific humidity increase while the innovations are mostly negative?

L444-445: What makes the Feb case less significant than the Mar case? Any hypothesis? It appears to me that the O-B differences in Feb case is larger than the Mar case (Fig. 6 v.s. Fig. 12). Could it be related to my previous comment on the specific humidity?
Citation: https://doi.org/10.5194/gmd-2021-161-RC1
- AC2: 'Reply on RC1', Xuanli Li, 24 Nov 2021
  
  The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-161/gmd-2021-161-AC2-supplement.pdf
  
  Citation: https://doi.org/10.5194/gmd-2021-161-AC2
CEC1:
'Comment on gmd-2021-161', Juan Antonio Añel, 25 Oct 2021

Dear authors,
After checking your manuscript, it has come to our attention that it does not comply with our Code and Data Policy.

https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
The GPM-retrieved ocean surface meteorology data is stored in "https://www.nsstc.uah.edu/public/xuanli.li/GMD/". However, this is not a suitable repostory. Therefore, please, publish your data in one of the appropriate repositories (e.g., Zenodo).

In this way, before the Discussions stage is closed, you must reply to this comment with the link to the repository for the code and the corresponding DOI.
Also, remember including in a potential reviewed version of your manuscript the modified 'Code and Data Availability' section with the DOI of the new repository.
Regards,
Juan A. Añel
Geosc. Mod. Dev. Executive Editor

Citation: https://doi.org/10.5194/gmd-2021-161-CEC1
- AC1: 'Reply on CEC1', Xuanli Li, 25 Oct 2021
  
  The GPM-retrieved ocean surface meteorology data has now been published at Zenodo (https://doi.org/10.5281/zenodo.5597153). And the sentence will be updated in the revised article.
  
  Citation: https://doi.org/10.5194/gmd-2021-161-AC1
RC2:
'Comment on gmd-2021-161', Anonymous Referee #2, 29 Oct 2021

The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-161/gmd-2021-161-RC2-supplement.pdf

Citation: https://doi.org/10.5194/gmd-2021-161-RC2
- AC3: 'Reply on RC2', Xuanli Li, 24 Nov 2021
  
  The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-161/gmd-2021-161-AC3-supplement.pdf
  
  Citation: https://doi.org/10.5194/gmd-2021-161-AC3

Peer review completion

AR: Author's response | RR: Referee report | ED: Editor decision | EF: Editorial file upload

AR by Xuanli Li on behalf of the Authors (24 Nov 2021) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (07 Feb 2022) by Chiel van Heerwaarden

RR by Anonymous Referee #1 (16 Feb 2022)

Suggestions for revision or reasons for rejection

Re-Review of
Assimilation of GPM-retrieved Ocean Surface Meteorology Data for Two Snowstorm Events during ICE-POP 2018
by
Li et al.

General comments
This is the second-round review of the manuscript. The authors have made efforts to address my previous concerns. Nevertheless, I am not satisfied with the revisions made so far. The revised manuscript is still lacking a clear goal, and too many unnecessary details are in fact very distractive. I still do not think this is a paper ready for publication in its current form. The detailed comments are listed below.

Major comments:
1. In summary, the manuscript basically shows that there are significant changes made to the background and the forecasts in the temperature, precipitation and moisture flux divergence can be improved. Also, it shows that the changes made to the background can be different between Feb and Mar cases. Speculations are due to the different kinematic and thermodynamic processes involved.
As a case study, we cannot draw any statistically robust conclusions from the results. Yet the manuscript failed to provide any physical link on how the changes to the background can lead to the changes in the forecasts. The authors should try identifying at least one major contribution of the DA towards the better predictions. For example, which one of the T/Q/WSD fields is more important to the better predictions? Can the increased/reduced temperature analysis trigger more/less convection-induced precipitations in certain areas, which leads to better precipitation forecasts? What exact kinematic or thermodynamic process is the key for the Feb and Mar differences? etc… Sensitivity experiments are strongly recommended to explore the impact of DA for your single case study.

2. L313-341: I see the authors have reduced the corresponding discussions. Yet I still don't see the value of this O-B vs A-B comparison. Yes, DA can perform differently, and increment/influence can be large/small depends on the methodology/configurations. But in this current paper, the authors never compared any DA schemes other than a single scheme in two cases. I don't get what to expect from these comparisons. Again, it is a common knowledge that a successful DA is supposed to drag the analysis towards observations. Then, what else can we get from here? Are the increments balanced? Can the corrections be validated from other independent obs? I think my point is that you can use a few words/O-B/O-A statistics to support the idea that DA is working properly, but no need to use the figure and too much paragraph. Otherwise, try dig something new/unexpected.

3. L342-359: I don't think this exp design is a good approach to show how long or how much the DA impact lasts. First, CTRL_Mar is not initialized from the same background as DA_Mar, so the differences are not purely from DA. Second, since you are not verified against truth/nature run, the differences are not informative. In an extreme example, if the whole system is shifted from south to north (say, closer to the truth), yet you can still get the same average profile, and the difference won't grow if the parallel system moves similarly. Current configuration only shows that the DA forecast is being balanced after roughly 4~6 hours. Instead, you should use the statistics from the RMSE of Fig. 8 (or some other observation verifications) to do the impact study.
PS: I also still don’t understand why the authors want to use only the South Korean Surface Analysis dataset for verification. Isn’t it better to use the datasets with more confidence (e.g. examined before) for verification purposes?

4. L360-377, L380-390 & others: Throughout the manuscript, there are too many tedious fact descriptions and number counting without clear goals, which make the readers lose focus. Unless there is any physically important point, the detail listing is very distractive and unnecessary. For example, what's the major point here other than DA_Mar has less RMSE? Why are those spots in Taebaek Mountains or Sobaek Mountains so important? Can they physically influence the storm predictions somehow? You should really connect the details to specific improvements to make a meaningful point, otherwise, just summarize them with one or two sentences.

5. L435-449: I still do not think it is a good way to discuss the quality of GPM precipitation data here. It is off topic, and you should evaluate this irrelevant (to the goal of this manuscript) dataset in another paper with more thorough analyses.

Minor comments:
1. L115: 2m -> 2-m.

2. L119: Please try to reduce the use of however.

3. L122: part -> partly

4. L301-302: How to deduce this implication from the figures or percentiles?

5. L303-310: Did you assume the observations to be unbiased? And again, what do these numbers mean? Model is biased?

6. L338-341: Directly calculate O-A and compare it with O-B will be more straightforward than A-B.

Hide

ED: Reconsider after major revisions (27 Feb 2022) by Chiel van Heerwaarden

AR by Xuanli Li on behalf of the Authors (10 Apr 2022) Author's response Author's tracked changes Manuscript

ED: Referee Nomination & Report Request started (24 Apr 2022) by Chiel van Heerwaarden

RR by Anonymous Referee #1 (01 May 2022)

Suggestions for revision or reasons for rejection

Re-Review of
Assimilation of GPM-retrieved Ocean Surface Meteorology Data for Two Snowstorm Events during ICE-POP 2018
by
Li et al.

General comments
This is the third version of the manuscript. The authors have changed the focus of the manuscript and further removed some unnecessary discussions. The manuscript is now more focused and refined. I appreciate the efforts of the authors. Although I personally feel like the content is a little too thin (more in-depth diagnostics are recommended), I believe the manuscript is now closer to publication with a few more comments.

Comments:
1. L309-334: I agree with the authors that “Since this paper focuses on case studies for assimilating a new dataset with high resolution regional simulation, it is always a good idea to check the background, the observation innovation (O – B) and the analysis increment (A – B) and see whether or not the data assimilation is effective (similar plots can be found in numerous past studies such as Xiao et al. 2007, Bi et al. 2011, Chen et al. 2021, etc).”. However, as far as I can tell from the three citations and lots of other studies, a reasonable check for the DA effectiveness usually starts with O-B vs O-A (e.g. Fig. 6 in Xiao et al. 2007; Fig. 1& 4 in Bi et al. 2011; Fig. 4 in Chen et al. 2021), not O-B vs A-B. One simplified example is A=3, O=2, B=1, O-B=1 & A-B=2. You’ll see both red in Figure 6, yet you cannot say the overcorrection is an effective DA. It is not wrong to use A-B, just less straightforward. Also, why does A-B look much smoother than O-B? Are you plotting one in observation space, and the other one in model space? If so, please use a consistent approach.

2. Fig. 7 & L335-375: I’m still not fond of this comparison. The evolution of the difference between CTL & DA can tell us very limited information. It can only show us that the model is being stabilized after a while, but it cannot tell us if the changes are good or bad. If the authors cannot find appropriate observations, ECMWF analysis or FNL re-analysis can be other potential sources for verifications and are commonly used in early studies.

3. Although the authors have improved the manuscript, there are still some irrelevant number listings. For example,
(a) L297: What is the point of the median values of the observations? They are time-/case-sensitive values. What can the readers learn from them?
(b) L300-308: Although the skewness can be calculated in a variety of ways, I don’t believe only the mean and standard deviation can be enough (at least a median value of those departures is needed depending on the calculation, but then the earlier median values can be misleading). Again, what is the point of those values here? It might be better to use skewness values here instead, or you can just use the figure to illustrate the points.

Hide

ED: Publish subject to minor revisions (review by editor) (30 May 2022) by Chiel van Heerwaarden

AR by Xuanli Li on behalf of the Authors (08 Jun 2022) Author's response Author's tracked changes Manuscript

ED: Publish as is (19 Jun 2022) by Chiel van Heerwaarden

AR by Xuanli Li on behalf of the Authors (21 Jun 2022) Manuscript

Post-review adjustments

AA: Author's adjustment | EA: Editor approval

AA by Xuanli Li on behalf of the Authors (11 Jul 2022) Author's adjustment Manuscript

EA: Adjustments approved (11 Jul 2022) by Chiel van Heerwaarden

Short summary

This research assimilated the Global Precipitation Measurement (GPM) satellite-retrieved ocean surface meteorology data into the Weather Research and Forecasting (WRF) model with the Gridpoint Statistical Interpolation (GSI) system. This was for two snowstorms during the International Collaborative Experiments for PyeongChang 2018 Olympic and Paralympic Winter Games' (ICE-POP 2018) field experiments. The results indicated a positive impact of the data for short-term forecasts for heavy snowfall.