Experiments with the modified Rotating Shallow Water model (modRSW, v.1.0): assessing the relevance for convective-scale data assimilation research

Kent, Thomas; Cantarello, Luca; Inverarity, Gordon; Tobias, Steven; Bokhove, Onno

doi:https://doi.org/10.5194/gmd-2022-269

Preprints

https://doi.org/10.5194/gmd-2022-269

Preprints

Submitted as: model experiment description paper

29 Nov 2022

Submitted as: model experiment description paper |

| 29 Nov 2022

Status: this preprint was under review for the journal GMD. A final paper is not foreseen.

Experiments with the modified Rotating Shallow Water model (modRSW, v.1.0): assessing the relevance for convective-scale data assimilation research

Thomas Kent, Luca Cantarello, Gordon Inverarity, Steven Tobias, and Onno Bokhove

Abstract. Following the development of the modified rotating shallow water model (modRSW), which includes simplified dynamics of convection and precipitation, we present the results of a series of idealised forecast-assimilation experiments to demonstrate its relevance for data assimilation research at convective scales, focusing on its ability to imitate an operational Numerical Weather Prediction system. We address in a rigorous manner how to ascertain whether an idealised model is relevant for data assimilation research by comparing our modRSW model configuration – based on a twin-setting approach in which synthetic observations are assimilated hourly into a Deterministic Ensemble Kalman filter – with the most common properties of an operational system. We demonstrate forecast-assimilation experiments that produce values of error growth rates (6–9 hours) and observational influence on the analyses (around 30 %) comparable to those found in operational systems. In addition, we provide a description of the approach we took to reach a well-tuned configuration by comparing the ensemble spread with the Root Mean Square Error of the ensemble mean, and examining the Continuous Ranked Probability Score. We also provide subjective assessments of forecasts at different lead times.

This preprint has been withdrawn.

Received: 08 Nov 2022 – Discussion started: 29 Nov 2022

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.

Download & links

Preprint (PDF, 5568 KB)

Withdrawal notice
This preprint has been withdrawn.
Preprint (5568 KB)

Download & links

This preprint has been withdrawn.

Thomas Kent, Luca Cantarello, Gordon Inverarity, Steven Tobias, and Onno Bokhove

Interactive discussion

Status: closed

EC1:
'Comment on gmd-2022-269', Yuefei Zeng, 30 Nov 2022

I have included my comments in the attached file

Citation: https://doi.org/10.5194/gmd-2022-269-EC1
- AC1: 'Reply on EC1', Luca Cantarello, 23 May 2023
  
  Please see attachment.
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC1
RC1:
'Comment on gmd-2022-269', Anonymous Referee #1, 08 Dec 2022

This manuscript aims to show the ability of the DA experiments using modRSW to imitate some behaviors of an operational NWP so that one can utilize such inexpensive configurations to study DA in an operational-like environment. The topic is attractive, especially for researchers not in an operational center and who cannot run a real operational NWP due to computational resource limitations. This work is valuable, but the experiments are not well-designed concerning imitating the convective-scale cycling DA. My recommendation is a major revision before publication.

Major comments

1. The authors perform many experiments and give the experimental configurations relevant to the operational environment in terms of criteria such as grid spacing, ensemble size, localization, inflation, and RMSE. But what are the authors’ suggestions on the improvement of localization and covariance inflation? If a new localization scheme or inflation scheme produces good results but cannot satisfy the criteria listed in this manuscript, should they be excluded from the operational NWP development? Showing the ability to imitate an operational-like environment is a good start, but further suggestions are also necessary. At least, the authors should tell readers what criteria should not be violated and what criteria can be adjusted when readers try to improve the DA performance in the simulated operational environment. Is a larger OID (>60%) not acceptable for imitating the convective-scale DA?

2. As far as I know, most operation centers use the variational DA algorithm or the variational-based hybrid algorithm. Only a few centers use the pure EnKF algorithm. Giving a reason for choosing the pure EnKF algorithm to imitate the operational environment is necessary. I do not ask for conducting experiments with a variational DA algorithm, but a brief discussion on the selection should be helpful.

3. With respect to the convective-scale DA, the model and DA configurations are not so representative.

(1) The precipitation procedure in modRSW is more like a cumulus parameterization scheme that estimates the precipitation according to the large-scale thermodynamic environment (see the high correlation between r and h in Figure 6), while a feature of convective-scale NWP is using a microphysics scheme that explicitly simulates the physical procedures in the cloud. The difference in complexity is a gap between a convective-scale NWP and a synoptic-scale NWP. Heavy rain may occur with no large-scale forcing. So I think referring the DA experiments using modRSW as “convective-scale” DA is not proper.

(2) The observation density is too sparse for the convective-scale DA, especially in the case of assimilating radar and satellite data. The resolution of radar data is often 1 or 2 km. The lack of high-resolution observations is a flaw in the experiments aiming to imitate the convective-scale DA. Using multiscale observations is also a feature of convective-scale DA and is not considered or discussed in the manuscript.

(3) The precipitation r in the manuscript is more like a simultaneous quantity, e.g., the precipitation rate in a time step. The accumulated precipitation, 3-h or 6-h, is used in real data assimilation. In this respect, the DA configuration in the manuscript does not imitate the real scenarios. This situation should be stated.

(4) In convective-scale DA, we must face that many model variables are not directly observed. This issue implies that some model variables must be updated through cross-variable covariance. It is better to show results without r and h observations.

(5) The operators of remote observations are often nonlinear, such as those of radar reflectivity and satellite observations. The inaccurate operator is also an issue in the convective-scale DA, but this issue is not discussed.

In general, the DA configurations in the manuscript are more suitable for a synoptic-scale DA study; many issues that the convective-scale DA has to face are not discussed. Since it is an idealized study, doing DA experiments with multiscale observations and without r and h observations should not be difficult. Observing the precipitation area with high-resolution u observations should result in a much smaller RMSE, similar to radar data assimilation.

4. The text is not well organized. The description of the model, DA algorithm, and experimental design should be more concise. Too many details distract readers from focusing on the main idea of this paper. For example, I don’t think section 4.3.5 is necessary. There is no need to list the formulas of RMSE and ensemble spread. In addition, section 6 is more like a summary, not a conclusion.

Minor comments:

L320-323: With respect to imitating LFC with H_c, I have some reservations.

L407: Is K in Equation (19) identical to K_e in Eq. (6b)? If so, use K_e please. If not, what is the difference?

L421: If a new DA method or a new configuration has a much larger influence (OID) in convective-scale DA, what is the authors’ suggestion?

L606: “the limitations of the EnKF” What are the limitations?

L611: Should “Fig. 11” be “Figure 11” at the beginning of a sentence?

L616: It seems that “).” is missed after (Fig. 10.

L623: “This shows the impact of a well-calibrated P^f_e matrix” How do authors define a well-calibrated P?

Citation: https://doi.org/10.5194/gmd-2022-269-RC1
- AC2: 'Reply on RC1', Luca Cantarello, 23 May 2023
  
  Please see attachment
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC2
RC2:
'Comment on gmd-2022-269', Anonymous Referee #2, 10 Jan 2023

The authors aim to show the relevance of using the modRSW model as a tool for mimicking key aspects of convective scale data assimilation in order to justify the transfer of knowledge from a simplified and cheaper setup to an operational configuration. In my option, this is a very important and complicated topic which is often overlooked, because it is not straightforward how to tackle it. I think the authors made a good attempt and I encourage the publication of this article, after the authors have considered the following points.

General comments:

1) The topic of this article is very tricky, since much of the relevance of the setup with the modRSW will depend on the purpose of the research. I think the authors should put more focus on the type of research that would be and would not be appropriate with the modRSW. For example, the authors note based on the snapshots that DA can recover the location of convection but struggles with the intensity. We know that operational convective scale DA does have problems with location errors. So for research that seeks to deal with location errors the modRSW may not be suitable. Another important topic among toy model users is non-Gaussianity and positivity constraints on hydrometeors. Is the non-Gaussianity and non-linearity in the modRSW comparable to an operational model? Does the rain get negative values and does it influence the DA results in a similar way as in an operational setup? I encourage the authors to discuss what type of research would be and would not be appropriate with the modRSW.

2) Comparing this modRSW setup to an operational setup skips the natural step of comparing to an idealised setup with operational model. I think it would be helpful to design a similar idealalised setup with an operational model to compare to the modRSW setup, to distinguish between model-caused differences and any other errors sources that come with the use of real observations. After all, in this work we are interested in the relevance of the modRSW, so we want to isolate its role in the DA experiments. Could the authors provide some thoughts on this matter?

Specific comments:

L365: Observations → members , right?

L370: By discarding negative observations, one creates a positive bias. Is this bias comparable to operational convective scale DA? As mentioned in general comment 1), non-Gaussianity and non-negativity is a popular topic among toy-model users, so I think this point should be explored more elaborately.

L412: the OID of 0.18, is for real observation experiments I assume. Would we expect a lower value for an idealised setup with operational model? As mentioned in general comment 2) shouldn’t that be the value to compare the modRSW setup to?

L421: I don’t fully understand how the thresholds of 20 and 40% are chosen, given the numbers mentioned in the previous paragraph.

Citation: https://doi.org/10.5194/gmd-2022-269-RC2
- AC3: 'Reply on RC2', Luca Cantarello, 23 May 2023
  
  Please see attachment.
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC3

Interactive discussion

Status: closed

EC1:
'Comment on gmd-2022-269', Yuefei Zeng, 30 Nov 2022

I have included my comments in the attached file

Citation: https://doi.org/10.5194/gmd-2022-269-EC1
- AC1: 'Reply on EC1', Luca Cantarello, 23 May 2023
  
  Please see attachment.
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC1
RC1:
'Comment on gmd-2022-269', Anonymous Referee #1, 08 Dec 2022

This manuscript aims to show the ability of the DA experiments using modRSW to imitate some behaviors of an operational NWP so that one can utilize such inexpensive configurations to study DA in an operational-like environment. The topic is attractive, especially for researchers not in an operational center and who cannot run a real operational NWP due to computational resource limitations. This work is valuable, but the experiments are not well-designed concerning imitating the convective-scale cycling DA. My recommendation is a major revision before publication.

Major comments

1. The authors perform many experiments and give the experimental configurations relevant to the operational environment in terms of criteria such as grid spacing, ensemble size, localization, inflation, and RMSE. But what are the authors’ suggestions on the improvement of localization and covariance inflation? If a new localization scheme or inflation scheme produces good results but cannot satisfy the criteria listed in this manuscript, should they be excluded from the operational NWP development? Showing the ability to imitate an operational-like environment is a good start, but further suggestions are also necessary. At least, the authors should tell readers what criteria should not be violated and what criteria can be adjusted when readers try to improve the DA performance in the simulated operational environment. Is a larger OID (>60%) not acceptable for imitating the convective-scale DA?

2. As far as I know, most operation centers use the variational DA algorithm or the variational-based hybrid algorithm. Only a few centers use the pure EnKF algorithm. Giving a reason for choosing the pure EnKF algorithm to imitate the operational environment is necessary. I do not ask for conducting experiments with a variational DA algorithm, but a brief discussion on the selection should be helpful.

3. With respect to the convective-scale DA, the model and DA configurations are not so representative.

(1) The precipitation procedure in modRSW is more like a cumulus parameterization scheme that estimates the precipitation according to the large-scale thermodynamic environment (see the high correlation between r and h in Figure 6), while a feature of convective-scale NWP is using a microphysics scheme that explicitly simulates the physical procedures in the cloud. The difference in complexity is a gap between a convective-scale NWP and a synoptic-scale NWP. Heavy rain may occur with no large-scale forcing. So I think referring the DA experiments using modRSW as “convective-scale” DA is not proper.

(2) The observation density is too sparse for the convective-scale DA, especially in the case of assimilating radar and satellite data. The resolution of radar data is often 1 or 2 km. The lack of high-resolution observations is a flaw in the experiments aiming to imitate the convective-scale DA. Using multiscale observations is also a feature of convective-scale DA and is not considered or discussed in the manuscript.

(3) The precipitation r in the manuscript is more like a simultaneous quantity, e.g., the precipitation rate in a time step. The accumulated precipitation, 3-h or 6-h, is used in real data assimilation. In this respect, the DA configuration in the manuscript does not imitate the real scenarios. This situation should be stated.

(4) In convective-scale DA, we must face that many model variables are not directly observed. This issue implies that some model variables must be updated through cross-variable covariance. It is better to show results without r and h observations.

(5) The operators of remote observations are often nonlinear, such as those of radar reflectivity and satellite observations. The inaccurate operator is also an issue in the convective-scale DA, but this issue is not discussed.

In general, the DA configurations in the manuscript are more suitable for a synoptic-scale DA study; many issues that the convective-scale DA has to face are not discussed. Since it is an idealized study, doing DA experiments with multiscale observations and without r and h observations should not be difficult. Observing the precipitation area with high-resolution u observations should result in a much smaller RMSE, similar to radar data assimilation.

4. The text is not well organized. The description of the model, DA algorithm, and experimental design should be more concise. Too many details distract readers from focusing on the main idea of this paper. For example, I don’t think section 4.3.5 is necessary. There is no need to list the formulas of RMSE and ensemble spread. In addition, section 6 is more like a summary, not a conclusion.

Minor comments:

L320-323: With respect to imitating LFC with H_c, I have some reservations.

L407: Is K in Equation (19) identical to K_e in Eq. (6b)? If so, use K_e please. If not, what is the difference?

L421: If a new DA method or a new configuration has a much larger influence (OID) in convective-scale DA, what is the authors’ suggestion?

L606: “the limitations of the EnKF” What are the limitations?

L611: Should “Fig. 11” be “Figure 11” at the beginning of a sentence?

L616: It seems that “).” is missed after (Fig. 10.

L623: “This shows the impact of a well-calibrated P^f_e matrix” How do authors define a well-calibrated P?

Citation: https://doi.org/10.5194/gmd-2022-269-RC1
- AC2: 'Reply on RC1', Luca Cantarello, 23 May 2023
  
  Please see attachment
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC2
RC2:
'Comment on gmd-2022-269', Anonymous Referee #2, 10 Jan 2023

The authors aim to show the relevance of using the modRSW model as a tool for mimicking key aspects of convective scale data assimilation in order to justify the transfer of knowledge from a simplified and cheaper setup to an operational configuration. In my option, this is a very important and complicated topic which is often overlooked, because it is not straightforward how to tackle it. I think the authors made a good attempt and I encourage the publication of this article, after the authors have considered the following points.

General comments:

1) The topic of this article is very tricky, since much of the relevance of the setup with the modRSW will depend on the purpose of the research. I think the authors should put more focus on the type of research that would be and would not be appropriate with the modRSW. For example, the authors note based on the snapshots that DA can recover the location of convection but struggles with the intensity. We know that operational convective scale DA does have problems with location errors. So for research that seeks to deal with location errors the modRSW may not be suitable. Another important topic among toy model users is non-Gaussianity and positivity constraints on hydrometeors. Is the non-Gaussianity and non-linearity in the modRSW comparable to an operational model? Does the rain get negative values and does it influence the DA results in a similar way as in an operational setup? I encourage the authors to discuss what type of research would be and would not be appropriate with the modRSW.

2) Comparing this modRSW setup to an operational setup skips the natural step of comparing to an idealised setup with operational model. I think it would be helpful to design a similar idealalised setup with an operational model to compare to the modRSW setup, to distinguish between model-caused differences and any other errors sources that come with the use of real observations. After all, in this work we are interested in the relevance of the modRSW, so we want to isolate its role in the DA experiments. Could the authors provide some thoughts on this matter?

Specific comments:

L365: Observations → members , right?

L370: By discarding negative observations, one creates a positive bias. Is this bias comparable to operational convective scale DA? As mentioned in general comment 1), non-Gaussianity and non-negativity is a popular topic among toy-model users, so I think this point should be explored more elaborately.

L412: the OID of 0.18, is for real observation experiments I assume. Would we expect a lower value for an idealised setup with operational model? As mentioned in general comment 2) shouldn’t that be the value to compare the modRSW setup to?

L421: I don’t fully understand how the thresholds of 20 and 40% are chosen, given the numbers mentioned in the previous paragraph.

Citation: https://doi.org/10.5194/gmd-2022-269-RC2
- AC3: 'Reply on RC2', Luca Cantarello, 23 May 2023
  
  Please see attachment.
  
  Citation: https://doi.org/10.5194/gmd-2022-269-AC3

Thomas Kent, Luca Cantarello, Gordon Inverarity, Steven Tobias, and Onno Bokhove

Data sets

GMD paper data T. Kent, L. Cantarello, G. Inverarity https://doi.org/10.5281/zenodo.7244173

Model code and software

modRSW_DEnKF T. Kent, L. Cantarello, G. Inverarity https://doi.org/10.5281/zenodo.7241058

Thomas Kent, Luca Cantarello, Gordon Inverarity, Steven Tobias, and Onno Bokhove

Viewed

Total article views: 1,016 (including HTML, PDF, and XML)

HTML	PDF	XML	Total	BibTeX	EndNote
753	214	49	1,016	36	38

HTML: 753
PDF: 214
XML: 49
Total: 1,016
BibTeX: 36
EndNote: 38

Views and downloads (calculated since 29 Nov 2022)

Month	HTML	PDF	XML	Total
Nov 2022	91	13	3	107
Dec 2022	193	35	8	236
Jan 2023	56	16	2	74
Feb 2023	60	16	0	76
Mar 2023	49	4	2	55
Apr 2023	44	21	0	65
May 2023	45	15	7	67
Jun 2023	13	1	0	14
Jul 2023	7	11	0	18
Aug 2023	19	7	1	27
Sep 2023	15	6	1	22
Oct 2023	18	7	0	25
Nov 2023	4	3	0	7
Dec 2023	6	12	1	19
Jan 2024	3	6	1	10
Feb 2024	15	10	3	28
Mar 2024	7	11	2	20
Apr 2024	18	2	8	28
May 2024	16	6	2	24
Jun 2024	6	3	2	11
Jul 2024	8	3	2	13
Aug 2024	17	3	2	22
Sep 2024	7	1	0	8
Oct 2024	12	0	12
Nov 2024	24	2	2	28

Cumulative views and downloads (calculated since 29 Nov 2022)

Month	HTML	PDF	XML	Total
Nov 2022	91	13	3	107
Dec 2022	193	35	8	236
Jan 2023	56	16	2	74
Feb 2023	60	16	0	76
Mar 2023	49	4	2	55
Apr 2023	44	21	0	65
May 2023	45	15	7	67
Jun 2023	13	1	0	14
Jul 2023	7	11	0	18
Aug 2023	19	7	1	27
Sep 2023	15	6	1	22
Oct 2023	18	7	0	25
Nov 2023	4	3	0	7
Dec 2023	6	12	1	19
Jan 2024	3	6	1	10
Feb 2024	15	10	3	28
Mar 2024	7	11	2	20
Apr 2024	18	2	8	28
May 2024	16	6	2	24
Jun 2024	6	3	2	11
Jul 2024	8	3	2	13
Aug 2024	17	3	2	22
Sep 2024	7	1	0	8
Oct 2024	12	0	12
Nov 2024	24	2	2	28

Viewed (geographical distribution)

Total article views: 970 (including HTML, PDF, and XML) Thereof 970 with geography defined and 0 with unknown origin.

Country	#	Views	%

Latest update: 20 Nov 2024

Short summary

Data assimilation combines recent model forecasts and observations to estimate current atmospheric conditions for use as initial conditions for numerical weather prediction. We analyse the results of a series of data assimilation experiments using a simplified and inexpensive mathematical model of the atmosphere. We closely compare key properties of the models used by weather centres with our idealised setup, proving that it can help support operational data assimilation research.


Total:	0
HTML:	0
PDF:	0
XML:	0