the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Experiments with the modified Rotating Shallow Water model (modRSW, v.1.0): assessing the relevance for convective-scale data assimilation research
Abstract. Following the development of the modified rotating shallow water model (modRSW), which includes simplified dynamics of convection and precipitation, we present the results of a series of idealised forecast-assimilation experiments to demonstrate its relevance for data assimilation research at convective scales, focusing on its ability to imitate an operational Numerical Weather Prediction system. We address in a rigorous manner how to ascertain whether an idealised model is relevant for data assimilation research by comparing our modRSW model configuration – based on a twin-setting approach in which synthetic observations are assimilated hourly into a Deterministic Ensemble Kalman filter – with the most common properties of an operational system. We demonstrate forecast-assimilation experiments that produce values of error growth rates (6–9 hours) and observational influence on the analyses (around 30 %) comparable to those found in operational systems. In addition, we provide a description of the approach we took to reach a well-tuned configuration by comparing the ensemble spread with the Root Mean Square Error of the ensemble mean, and examining the Continuous Ranked Probability Score. We also provide subjective assessments of forecasts at different lead times.
This preprint has been withdrawn.
-
Withdrawal notice
This preprint has been withdrawn.
-
Preprint
(5568 KB)
Interactive discussion
Status: closed
-
EC1: 'Comment on gmd-2022-269', Yuefei Zeng, 30 Nov 2022
- AC1: 'Reply on EC1', Luca Cantarello, 23 May 2023
-
RC1: 'Comment on gmd-2022-269', Anonymous Referee #1, 08 Dec 2022
This manuscript aims to show the ability of the DA experiments using modRSW to imitate some behaviors of an operational NWP so that one can utilize such inexpensive configurations to study DA in an operational-like environment. The topic is attractive, especially for researchers not in an operational center and who cannot run a real operational NWP due to computational resource limitations. This work is valuable, but the experiments are not well-designed concerning imitating the convective-scale cycling DA. My recommendation is a major revision before publication.
Major comments
1. The authors perform many experiments and give the experimental configurations relevant to the operational environment in terms of criteria such as grid spacing, ensemble size, localization, inflation, and RMSE. But what are the authors’ suggestions on the improvement of localization and covariance inflation? If a new localization scheme or inflation scheme produces good results but cannot satisfy the criteria listed in this manuscript, should they be excluded from the operational NWP development? Showing the ability to imitate an operational-like environment is a good start, but further suggestions are also necessary. At least, the authors should tell readers what criteria should not be violated and what criteria can be adjusted when readers try to improve the DA performance in the simulated operational environment. Is a larger OID (>60%) not acceptable for imitating the convective-scale DA?
2. As far as I know, most operation centers use the variational DA algorithm or the variational-based hybrid algorithm. Only a few centers use the pure EnKF algorithm. Giving a reason for choosing the pure EnKF algorithm to imitate the operational environment is necessary. I do not ask for conducting experiments with a variational DA algorithm, but a brief discussion on the selection should be helpful.
3. With respect to the convective-scale DA, the model and DA configurations are not so representative.
(1) The precipitation procedure in modRSW is more like a cumulus parameterization scheme that estimates the precipitation according to the large-scale thermodynamic environment (see the high correlation between r and h in Figure 6), while a feature of convective-scale NWP is using a microphysics scheme that explicitly simulates the physical procedures in the cloud. The difference in complexity is a gap between a convective-scale NWP and a synoptic-scale NWP. Heavy rain may occur with no large-scale forcing. So I think referring the DA experiments using modRSW as “convective-scale” DA is not proper.
(2) The observation density is too sparse for the convective-scale DA, especially in the case of assimilating radar and satellite data. The resolution of radar data is often 1 or 2 km. The lack of high-resolution observations is a flaw in the experiments aiming to imitate the convective-scale DA. Using multiscale observations is also a feature of convective-scale DA and is not considered or discussed in the manuscript.
(3) The precipitation r in the manuscript is more like a simultaneous quantity, e.g., the precipitation rate in a time step. The accumulated precipitation, 3-h or 6-h, is used in real data assimilation. In this respect, the DA configuration in the manuscript does not imitate the real scenarios. This situation should be stated.
(4) In convective-scale DA, we must face that many model variables are not directly observed. This issue implies that some model variables must be updated through cross-variable covariance. It is better to show results without r and h observations.
(5) The operators of remote observations are often nonlinear, such as those of radar reflectivity and satellite observations. The inaccurate operator is also an issue in the convective-scale DA, but this issue is not discussed.
In general, the DA configurations in the manuscript are more suitable for a synoptic-scale DA study; many issues that the convective-scale DA has to face are not discussed. Since it is an idealized study, doing DA experiments with multiscale observations and without r and h observations should not be difficult. Observing the precipitation area with high-resolution u observations should result in a much smaller RMSE, similar to radar data assimilation.
4. The text is not well organized. The description of the model, DA algorithm, and experimental design should be more concise. Too many details distract readers from focusing on the main idea of this paper. For example, I don’t think section 4.3.5 is necessary. There is no need to list the formulas of RMSE and ensemble spread. In addition, section 6 is more like a summary, not a conclusion.
Minor comments:
L320-323: With respect to imitating LFC with Hc, I have some reservations.
L407: Is K in Equation (19) identical to Ke in Eq. (6b)? If so, use Ke please. If not, what is the difference?
L421: If a new DA method or a new configuration has a much larger influence (OID) in convective-scale DA, what is the authors’ suggestion?
L606: “the limitations of the EnKF” What are the limitations?
L611: Should “Fig. 11” be “Figure 11” at the beginning of a sentence?
L616: It seems that “).” is missed after (Fig. 10.
L623: “This shows the impact of a well-calibrated Pfe matrix” How do authors define a well-calibrated P?
Citation: https://doi.org/10.5194/gmd-2022-269-RC1 - AC2: 'Reply on RC1', Luca Cantarello, 23 May 2023
-
RC2: 'Comment on gmd-2022-269', Anonymous Referee #2, 10 Jan 2023
The authors aim to show the relevance of using the modRSW model as a tool for mimicking key aspects of convective scale data assimilation in order to justify the transfer of knowledge from a simplified and cheaper setup to an operational configuration. In my option, this is a very important and complicated topic which is often overlooked, because it is not straightforward how to tackle it. I think the authors made a good attempt and I encourage the publication of this article, after the authors have considered the following points.
General comments:
1) The topic of this article is very tricky, since much of the relevance of the setup with the modRSW will depend on the purpose of the research. I think the authors should put more focus on the type of research that would be and would not be appropriate with the modRSW. For example, the authors note based on the snapshots that DA can recover the location of convection but struggles with the intensity. We know that operational convective scale DA does have problems with location errors. So for research that seeks to deal with location errors the modRSW may not be suitable. Another important topic among toy model users is non-Gaussianity and positivity constraints on hydrometeors. Is the non-Gaussianity and non-linearity in the modRSW comparable to an operational model? Does the rain get negative values and does it influence the DA results in a similar way as in an operational setup? I encourage the authors to discuss what type of research would be and would not be appropriate with the modRSW.
2) Comparing this modRSW setup to an operational setup skips the natural step of comparing to an idealised setup with operational model. I think it would be helpful to design a similar idealalised setup with an operational model to compare to the modRSW setup, to distinguish between model-caused differences and any other errors sources that come with the use of real observations. After all, in this work we are interested in the relevance of the modRSW, so we want to isolate its role in the DA experiments. Could the authors provide some thoughts on this matter?
Specific comments:
L365: Observations → members , right?
L370: By discarding negative observations, one creates a positive bias. Is this bias comparable to operational convective scale DA? As mentioned in general comment 1), non-Gaussianity and non-negativity is a popular topic among toy-model users, so I think this point should be explored more elaborately.
L412: the OID of 0.18, is for real observation experiments I assume. Would we expect a lower value for an idealised setup with operational model? As mentioned in general comment 2) shouldn’t that be the value to compare the modRSW setup to?
L421: I don’t fully understand how the thresholds of 20 and 40% are chosen, given the numbers mentioned in the previous paragraph.
Citation: https://doi.org/10.5194/gmd-2022-269-RC2 - AC3: 'Reply on RC2', Luca Cantarello, 23 May 2023
Interactive discussion
Status: closed
-
EC1: 'Comment on gmd-2022-269', Yuefei Zeng, 30 Nov 2022
- AC1: 'Reply on EC1', Luca Cantarello, 23 May 2023
-
RC1: 'Comment on gmd-2022-269', Anonymous Referee #1, 08 Dec 2022
This manuscript aims to show the ability of the DA experiments using modRSW to imitate some behaviors of an operational NWP so that one can utilize such inexpensive configurations to study DA in an operational-like environment. The topic is attractive, especially for researchers not in an operational center and who cannot run a real operational NWP due to computational resource limitations. This work is valuable, but the experiments are not well-designed concerning imitating the convective-scale cycling DA. My recommendation is a major revision before publication.
Major comments
1. The authors perform many experiments and give the experimental configurations relevant to the operational environment in terms of criteria such as grid spacing, ensemble size, localization, inflation, and RMSE. But what are the authors’ suggestions on the improvement of localization and covariance inflation? If a new localization scheme or inflation scheme produces good results but cannot satisfy the criteria listed in this manuscript, should they be excluded from the operational NWP development? Showing the ability to imitate an operational-like environment is a good start, but further suggestions are also necessary. At least, the authors should tell readers what criteria should not be violated and what criteria can be adjusted when readers try to improve the DA performance in the simulated operational environment. Is a larger OID (>60%) not acceptable for imitating the convective-scale DA?
2. As far as I know, most operation centers use the variational DA algorithm or the variational-based hybrid algorithm. Only a few centers use the pure EnKF algorithm. Giving a reason for choosing the pure EnKF algorithm to imitate the operational environment is necessary. I do not ask for conducting experiments with a variational DA algorithm, but a brief discussion on the selection should be helpful.
3. With respect to the convective-scale DA, the model and DA configurations are not so representative.
(1) The precipitation procedure in modRSW is more like a cumulus parameterization scheme that estimates the precipitation according to the large-scale thermodynamic environment (see the high correlation between r and h in Figure 6), while a feature of convective-scale NWP is using a microphysics scheme that explicitly simulates the physical procedures in the cloud. The difference in complexity is a gap between a convective-scale NWP and a synoptic-scale NWP. Heavy rain may occur with no large-scale forcing. So I think referring the DA experiments using modRSW as “convective-scale” DA is not proper.
(2) The observation density is too sparse for the convective-scale DA, especially in the case of assimilating radar and satellite data. The resolution of radar data is often 1 or 2 km. The lack of high-resolution observations is a flaw in the experiments aiming to imitate the convective-scale DA. Using multiscale observations is also a feature of convective-scale DA and is not considered or discussed in the manuscript.
(3) The precipitation r in the manuscript is more like a simultaneous quantity, e.g., the precipitation rate in a time step. The accumulated precipitation, 3-h or 6-h, is used in real data assimilation. In this respect, the DA configuration in the manuscript does not imitate the real scenarios. This situation should be stated.
(4) In convective-scale DA, we must face that many model variables are not directly observed. This issue implies that some model variables must be updated through cross-variable covariance. It is better to show results without r and h observations.
(5) The operators of remote observations are often nonlinear, such as those of radar reflectivity and satellite observations. The inaccurate operator is also an issue in the convective-scale DA, but this issue is not discussed.
In general, the DA configurations in the manuscript are more suitable for a synoptic-scale DA study; many issues that the convective-scale DA has to face are not discussed. Since it is an idealized study, doing DA experiments with multiscale observations and without r and h observations should not be difficult. Observing the precipitation area with high-resolution u observations should result in a much smaller RMSE, similar to radar data assimilation.
4. The text is not well organized. The description of the model, DA algorithm, and experimental design should be more concise. Too many details distract readers from focusing on the main idea of this paper. For example, I don’t think section 4.3.5 is necessary. There is no need to list the formulas of RMSE and ensemble spread. In addition, section 6 is more like a summary, not a conclusion.
Minor comments:
L320-323: With respect to imitating LFC with Hc, I have some reservations.
L407: Is K in Equation (19) identical to Ke in Eq. (6b)? If so, use Ke please. If not, what is the difference?
L421: If a new DA method or a new configuration has a much larger influence (OID) in convective-scale DA, what is the authors’ suggestion?
L606: “the limitations of the EnKF” What are the limitations?
L611: Should “Fig. 11” be “Figure 11” at the beginning of a sentence?
L616: It seems that “).” is missed after (Fig. 10.
L623: “This shows the impact of a well-calibrated Pfe matrix” How do authors define a well-calibrated P?
Citation: https://doi.org/10.5194/gmd-2022-269-RC1 - AC2: 'Reply on RC1', Luca Cantarello, 23 May 2023
-
RC2: 'Comment on gmd-2022-269', Anonymous Referee #2, 10 Jan 2023
The authors aim to show the relevance of using the modRSW model as a tool for mimicking key aspects of convective scale data assimilation in order to justify the transfer of knowledge from a simplified and cheaper setup to an operational configuration. In my option, this is a very important and complicated topic which is often overlooked, because it is not straightforward how to tackle it. I think the authors made a good attempt and I encourage the publication of this article, after the authors have considered the following points.
General comments:
1) The topic of this article is very tricky, since much of the relevance of the setup with the modRSW will depend on the purpose of the research. I think the authors should put more focus on the type of research that would be and would not be appropriate with the modRSW. For example, the authors note based on the snapshots that DA can recover the location of convection but struggles with the intensity. We know that operational convective scale DA does have problems with location errors. So for research that seeks to deal with location errors the modRSW may not be suitable. Another important topic among toy model users is non-Gaussianity and positivity constraints on hydrometeors. Is the non-Gaussianity and non-linearity in the modRSW comparable to an operational model? Does the rain get negative values and does it influence the DA results in a similar way as in an operational setup? I encourage the authors to discuss what type of research would be and would not be appropriate with the modRSW.
2) Comparing this modRSW setup to an operational setup skips the natural step of comparing to an idealised setup with operational model. I think it would be helpful to design a similar idealalised setup with an operational model to compare to the modRSW setup, to distinguish between model-caused differences and any other errors sources that come with the use of real observations. After all, in this work we are interested in the relevance of the modRSW, so we want to isolate its role in the DA experiments. Could the authors provide some thoughts on this matter?
Specific comments:
L365: Observations → members , right?
L370: By discarding negative observations, one creates a positive bias. Is this bias comparable to operational convective scale DA? As mentioned in general comment 1), non-Gaussianity and non-negativity is a popular topic among toy-model users, so I think this point should be explored more elaborately.
L412: the OID of 0.18, is for real observation experiments I assume. Would we expect a lower value for an idealised setup with operational model? As mentioned in general comment 2) shouldn’t that be the value to compare the modRSW setup to?
L421: I don’t fully understand how the thresholds of 20 and 40% are chosen, given the numbers mentioned in the previous paragraph.
Citation: https://doi.org/10.5194/gmd-2022-269-RC2 - AC3: 'Reply on RC2', Luca Cantarello, 23 May 2023
Data sets
GMD paper data T. Kent, L. Cantarello, G. Inverarity https://doi.org/10.5281/zenodo.7244173
Model code and software
modRSW_DEnKF T. Kent, L. Cantarello, G. Inverarity https://doi.org/10.5281/zenodo.7241058
Viewed
HTML | XML | Total | BibTeX | EndNote | |
---|---|---|---|---|---|
781 | 215 | 49 | 1,045 | 37 | 39 |
- HTML: 781
- PDF: 215
- XML: 49
- Total: 1,045
- BibTeX: 37
- EndNote: 39
Viewed (geographical distribution)
Country | # | Views | % |
---|
Total: | 0 |
HTML: | 0 |
PDF: | 0 |
XML: | 0 |
- 1