the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Data assimilation for the Model for Prediction Across Scales – Atmosphere with the Joint Effort for Data assimilation Integration (JEDI-MPAS 2.0.0-beta): ensemble of 3D ensemble-variational (En-3DEnVar) assimilations
Jonathan J. Guerrette
Zhiquan Liu
Chris Snyder
Byoung-Joo Jung
Craig S. Schwartz
Junmei Ban
Steven Vahl
Yali Wu
Ivette Hernandez Banos
Yonggang G. Yu
Soyoung Ha
Yannick Tremolet
Thomas Auligne
Clementine Gas
Benjamin Menetrier
Anna Shlyaeva
Mark Miesch
Stephen Herbener
Emily Liu
Daniel Holdaway
Benjamin T. Johnson
Abstract. An ensemble of three-dimensional ensemble-variational (En-3DEnVar) data assimilations is demonstrated with the Joint Effort for Data assimilation Integration (JEDI) with the Model for Prediction Across Scales – Atmosphere (MPAS-A) (i.e., JEDI-MPAS). Basic software building blocks are reused from previously presented deterministic 3DEnVar functionality, and combined with a formal experimental workflow manager in MPAS-Workflow. En-3DEnVar is used to produce an 80-member ensemble of analyses, which are cycled with ensemble forecasts in a 1-month experiment. The ensemble forecasts approximate a purely flow-dependent background error covariance (BEC) at each analysis time. The En-3DEnVar BECs and prior ensemble mean forecast errors are compared to those produced by a similar experiment that uses the Data Assimilation Research Testbed (DART) Ensemble Adjustment Kalman Filter (EAKF). The experiment using En-3DEnVar produces similar ensemble spread to and slightly smaller errors than the EAKF. The ensemble forecasts initialized from En-3DEnVar and EAKF analyses are used as BECs in deterministic cycling 3DEnVar experiments, which are compared to a control experiment that uses 20-member MPAS-A forecasts initialized from Global Ensemble Forecast System (GEFS) initial conditions. The experimental ensembles achieve mostly equivalent or better performance than the off-the-shelf ensemble system in this deterministic cycling setting; although, there are many obvious differences in configuration between GEFS and the two MPAS ensemble systems. An additional experiment that uses hybrid 3DEnVar, which combines the En-3DEnVar ensemble BEC with a climatological BEC, increases tropospheric forecast quality compared to the corresponding pure 3DEnVar experiment. The JEDI-MPAS En-3DEnVar is technically working and useful for future research studies. Tuning of observation errors and spread is needed to improve performance and several algorithmic advancements are needed to improve computational efficiency for larger-scale applications.
Jonathan J. Guerrette et al.
Status: open (extended)
-
CC1: 'Comment on gmd-2023-54', Lili Lei, 17 Apr 2023
reply
Title: Data assimilation for the Model for Prediction Across Scales – Atmosphere with the Joint Effort for Data assimilation Integration (JEDI-MPAS 2.0.0-beta): ensemble of 3D ensemble-variational (En-3DEnVar) assimilations
Authors: Guerrette et al.
Recommendation: Minor revision
Summary
This manuscript demonstrated an ensemble of three-dimensional ensemble-variational (En3DEnVar) for MPAS with JEDI. En3DEnVAR is an extension of 3DEnVAR, but with ensemble assimilations and perturbed observations. En3DEnVAR is compared with EAKF in DART. Similar posteriors from En3DEnVAR and EAKF are obtained. Deterministic 3DEnVAR experiments with different BECs are conducted. Results show that the experimental ensembles achieve mostly equivalent or better performance than the off-the-shelf ensemble system. The manuscript is well written and could be a valuable contribution to the community. I have several comments as below.
- l46, model space localization can also be implemented in ensemble Kalman filters (Bishop et al. 2018; Lei et al. 2018; Lei et al. 2021).
- l102, since the reference Jung et al. (2023) is not available, it would be helpful to more details for the Bc.
- l127, why “the ensemble forecast in JEDI-MPAS do not at present account for model error”? Since an imperfect model is used, model error can be naturally embedded in the forecast.
- l187, if the pressure constraint has significant impact on the analyses, it is straightforward to have a mass adjustment to mimic the pressure constraint in EAKF.
- l272, since model space localization tends to have broader localization length scale than observation space localization, it might be more appropriate to have a larger localization length scale in EAKF than in 3DEnVAR.
- l273, why not use the same inflation method, either RTPS or RTPP, for both EAKF and En3DEnVAR?
- l305, please provide some explanations for the larger U and V errors of EAKF compared to En3DEnVAR. Are these larger errors resulted from the hybrid background error covariances? If yes, is it possible to have an En3DEnVAR with pure ensemble B, since the goal of the section is to demonstrate that En3DEnVAR has similar posteriors to EAKF?
- l381, this result is inconsistent with Fig 1c. Please provide some explanations.
- l380-410, the main difference between eda100 and dart100 is from the verification against AMV, but not from the other verification metrics. Why?
- l445-455, there multiple ?s that are not correctly displayed.
Citation: https://doi.org/10.5194/gmd-2023-54-CC1 -
RC1: 'Comment on gmd-2023-54', Anonymous Referee #1, 16 May 2023
reply
This paper is very well-written and presents its material clearly. It is the development of an ensemble of data assimilations, and this topic will be of interest to people using the JEDI system. There are hints of scientific results in section 4, but the differences between the systems being compared are too large to make concrete conclusions, as the authors acknowledge. My largest reservations about the paper are in the presentation of the results, which would benefit from some work. However, these issues are relatively minor and therefore I recommend accepting the paper after these changes are made.
The comments below are presented in the order of the line number on which they appear.
L80: Setting the sample mean of the perturbations to be exactly zero alters the distribution which the numbers are drawn from. This statistical artefact should be mentioned.
L152: Here and below the word "variational" appears in italics. I don't see a good cause for this highlight, and think it should be removed.
L249: Running an NWP experiment for a short period (1 month) can lead to issues with the results. It should be acknowledged that a single month is less than ideal.
L265: Not only is a higher-resolution model more accurate than a lower-resolution model, it is also typically more active. Therefore, one might expect to see faster growth of errors, leading to less requirement for inflation.
L272: A comparison of model- and observation-space localization was conducted by Greybush et al, MWR (2011) (https://doi.org/10.1175/2010MWR3328.1). This should be referenced. It should also be noted that they concluded that length-scales for observation-space localization should be shorter than those for model-space localization. This would suggest that it's questionable whether using 1200 km for the EAKF is a good choice.
L273: The value of RTPS being used (1) is shocking. This should be noted, and discussed since it implies that the ensemble spread does not grow during the sequence of DA cycles.
L309: The authors conclude that the regional differences in RMSE are directly related to the differences in spread between the ensembles in those regions. It is not clear from this short discussion whether that is a valid connection to make. Thus the authors should either provide evidence of the connection, or remove this sentence.
L335: Section 5 needs to begin with a clear description of the deterministic experiments which are being proposed, but this appears to be missing. Perhaps the authors could note that they are running deterministic 3DEnVar experiments following in the footsteps of Liu et al (2022). It should be stated whether the ensembles in the previous section are run prior to the experiments in section 5, or whether they are "coupled" to the DA experiments in any way.
L362: Error bars are estimated by a bootstrap resampling. The authors use 10,000 samples of 27 values. Using a large number of samples cannot mask the limitations of using a short period of the experiment. Either here or elsewhere the authors should acknowledge the short length of the trial, and note that the confidence intervals will not be fully representative of the uncertainty in the result (for instance no account is made of seasonal variability of the results).
L378: Figure 4 shows the RMSE difference averaged over all horizontal locations and vertical levels. Averaging in the vertical is a very strange thing to do, as the variability of certain variables will change substantially with height. Comparing with the results in Figure 7, which show improvements in T and U in the upper model levels I would assume that this weighting is towards those heights. Given the unusual nature of the metric the authors should at least acknowledge this issue. Perhaps better would be to promote Figure 7 to the front of the discussion, since this graphic seems to be the most informative.
L403: Refractivity is a function of pressure, temperature and humidity. Why is pressure not mentioned here in relation to GNSS-RO observations?
L444: Please add "The" in front of EDA.
L446: Here and below (L447, L453 and L445) question marks appear in the text, which looks like characters which have not been represented correctly. Please correct these errors.
L477: Please add "the" in front of EnKF.
Figure 2: Please could the x-axis start at zero, as I initially thought that the EDA spread in the tropics was much smaller than the DART spread (i.e. very close to zero)?
Table 1: Please change "Obsevation error distribution" to "Observation error standard deviation".
Citation: https://doi.org/10.5194/gmd-2023-54-RC1
Jonathan J. Guerrette et al.
Model code and software
JEDI-MPAS Data Assimilation System v2.0.0-beta Joint Center for Satellite Data Assimilation, and National Center for Atmospheric Research https://doi.org/10.5281/zenodo.7630054
Jonathan J. Guerrette et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 254 | 89 | 11 | 354 | 7 | 3 |
- HTML: 254
- PDF: 89
- XML: 11
- Total: 354
- BibTeX: 7
- EndNote: 3
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1