the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Implementation and application of ensemble optimal interpolation on an operational chemistry weather model for improving PM2.5 and visibility predictions
Siting Li
Zhaodong Liu
Wenjie Zhang
Hongli Liu
Yaqiang Wang
Huizheng Che
Xiaoye Zhang
Download
- Final revised paper (published on 25 Jul 2023)
- Preprint (discussion started on 07 Nov 2022)
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2022-207', P. Armand, 16 Nov 2022
The article by Siting et al. is about the implementation of an Ensemble Optimal Interpolation (EnOI) method in the numerical chemistry & weather prediction system GRAPES_Meso5.1 / CUACE and the application of the method to try to improve PM2.5 and visibility forecasts of pollution episodes in Eastern China. The authors strive to calibrate the parameters of the EnOI, namely the "localization length scale", which is the spatial range of the assimilation, the ensemble size, and the time at which assimilation should be carried out. They also investigate the impact of assimilating PM2.5 observations on the simulated PM2.5 concentration field. The mean error (ME) and the root mean square error (RMSE) on the initial PM2.5 concentration field is reduced when assimilating the data. According to the authors, the forecasts of the PM2.5 concentration and visibility fields are also improved throughout the lead time window of 24 hours, especially when the assimilation time is 1200 UTC because the discrepancy between simulation and observation is larger compared to 0000 UTC. Again according to the authors, the visibility forecasts by assimilating PM2.5 are further improved for light pollution episodes in comparison with heavy pollution episodes, which are more affected by humidity. Thus, for extreme low visibility during severe haze pollution, the authors recommend to assimilate both PM2.5 and humidity observations.
The work carried out by Siting et al. essentially consists of optimal interpolation where the model error covariance matrix is evaluated by an ensemble approach with the members of the ensemble being previous timeframes of the PM2.5 concentration field. Beyond the fact that the PM2.5 field which assimilates data at a given time and at the close following times actually presents better statistics compared to the observations, the results obtained do not seem very convincing to me. However, this work constitutes an approach which deserves to be studied and is therefore worthy of publication. The article could be improved by taking into account the following remarks.
L55 to 61 – Instead of listing a very large number of references, authors should briefly indicate what they contain.
L95 – Develop the acronym "EnOI" at least in the title of the section.
L102 – I don't understand the point of the double notation "x" and "psi". If there is a good reason, the authors should give it, otherwise the notations should be simplified.
L110 - In fact, the ensemble is used to estimate the error covariance matrix of the model. The authors could say it simply and directly.
L112 – The relationship between "psi_a", "psi_f" and "psi_i" should be given.
L113 - What the acronym "AF" corresponds to is not indicated (even if we understand well). The many acronyms used throughout the text should be made explicit and, in my opinion, fewer abbreviations should be used.
L114 – As the scalar "alpha" is used to weight the model and the observations, one would expect to see "1 – alpha" in formula (6). Are the authors sure about this formula?
L118 – Although the use of a length scale or spatial range in data assimilation is understood, this concept is poorly introduced. The expression "to avoid all observations…" is not clear at all and needs to be rephrased.
L118 – What does the "localization scheme" look like? The authors should give the formula of it.
L125 – We understand later in the reading of the article how the ensemble of PM2.5 concentration fields is constructed. It would be better if the authors explained it in this section of the article.
L129 – The sentence "Compared with the traditional EnOI..." can only be understood after reading the rest of the article. This sentence should be explained at this point in the article.
L134 – The horizontal and vertical dimensions and resolutions of the simulation domains used by the GRAPES_Meso5.1 and CUACE models should be indicated.
L166 – What do the authors call a "warm" restart? Is that really the right term?
L183 – Finally it is said that "the N hourly model forecasts before the assimilation moment were used as the ensemble samples to approximate" the model error covariance matrix. It could have been mentioned before.
L207 – If we look at figure 4, it seems very difficult to me to see what is the optimal "localization length scale" especially since the metrics on the correlation (CORR) and the errors (RMSE, MB and ME) are close.
L248 – This is the same problem as for my previous remark. The number of members in the ensemble seems to me chosen in a practical, if not "ad hoc", way. It should be studied whether other pollution episodes lead to the same choice of parameters "ensemble size" and "localization length scale".
L265 – Thanks to the authors for expanding the acronyms to facilitate reading.
L271 – What is a "sheet-like" concentration distribution? Is it really the correct term?
L287 – In Figure 8, there are very significant differences between the forecasts both with and without data assimilation and the observations, in particular on December 19 and December 23. Not only the amplitude, but mainly the dynamics of the concentrations are very different. Do the authors have an explanation on this point?
L391 – The final part of the sentence "… and relatively consistent in (20)" seems to be lacking. Please, correct.
L305 – Figure 9 does not seem to me to show convincingly that the assimilation at 1200 UTC (DA12) is better than the assimilation at 0000 UTC (DA00). Have the authors looked at, from a more fundamental point of view, why this should be the case?
L311 – Is it a general property that assimilation at 1200 UTC would be better than assimilation at 0000 UTC? The authors should be more careful about their assertion.
L323 – There is no linear relationship between visibility and PM2.5 concentration. Thus, it is not surprising that the result of assimilation improves the result on visibility for light pollution episodes, whereas this improvement does not exist or is insignificant for heavy pollution episodes.
L334 – Also in Figure 11 (as reported for Figure 8), there are large discrepancies between the PM2.5 concentration predictions with or without data assimilation and the observations. Would the authors have an explanation on not only the amplitudes, but above all the dynamics which are very different?
Citation: https://doi.org/10.5194/gmd-2022-207-RC1 - AC1: 'Reply on RC1', Ping Wang, 08 Apr 2023
-
RC2: 'Comment on gmd-2022-207', Anonymous Referee #2, 09 Dec 2022
The authors present the implementation of an optimal interpolation framework for PM2.5 based on the regional model GRAPES_Meso5.1/CUACE, which computes on-line meteorology as well as chemistry, and PM2.5 atmospheric observations from surface sites in China. The aim of the data assimilation scheme implemented here is to provide initial fields of PM2.5 concentrations in order to improve the forecasted PM2.5 fields and visibility. After sensitivity tests to adjust some parameters of the ensemble optimal interpolation scheme, the impact of the analysed initial PM2.5 field on the forecasted PM2.5 and visibility for the day are assessed.
General comments
The data assimilation scheme described in this paper is well-known and the interesting points are its adaptation to the application to PM2.5 and visibility in a large domain, China, with very constrasted concentrations both in time and in space. Therefore, such a topic is relevant for publication in GMD. Nevertheless, I am not convinced that the work described here actually brings anything new or sate-of-the-art regarding data assimilation for air quality forecast.The introduction to the paper does not give all the required information e.g. definitions of "haze period" (l.37), "the forecast accuracy of air quality forecasting" (l.46), "advantageous" (l.68) are not defined, nor are the terms directly refereing to the method such as "spatial length scale of the covariance localization" (l.70); it is not always clear if the references are relevant: they must not be cited as lists of works but each of them must be linked to an aspect of the topic of the paper; conversely, references seem to be missing e.g. "4D-Var requires coding the adjoint model, which is difficult to perform for complex systems": true, but it has been done nevertheless, even for PMs.
In Section 2, the presentation of the EnOI is not very clear regarding which parts are the well-known mathematical method and which are the contributions of this work. For example, l.118 "To avoid all observations affecting the same model grid,[...]": why should it be an issue? It generally is not, it may be quite the reverse. Other examples include values which are assigned to parameters without any explanation such as "alpha is taken as 0.9 in this study" (l.115) or "the time step is 100s" (l.164).
In Section 3, the sensitivity experiments are not designed in a very convincing way. The case with only two sites is more a pedagogical illustration than a result or proof. Some possibilities are not explored or even discarded with an explanation: for example, why not adapt the localization length-scale to the representativeness of each station? Why don't we obtain the expected conclusion on the size of the ensemble i.e the larger the ensemble, the better the results of the DA in Section 3.2? The aim of Section 3.3 is not very clear: the first paragraph seems to be dedicated to evaluating the impact of DA against non-assimilated sites but the following ones seem to use all sites for DA (l.261). Therefore, the results showing the impact of DA are trivial: the assimilation is designed to obtain better statistics at the assimilated sites; they just show that there is not an obvious bug in the implementation. The comments on the 40-km length scale and the "discrete" aspect of the posterior field are a bit strange. Obtaining such a result suggests that the spatial resolution of the model (what is it?) is not relevant compared to the representativity of the measurements. This can be linked to the issue of the "overlapping" of the influence areas of sites. If the corrections for each site are independent from the others, then the reasoning in term of "field" (i.e. continuous mapping) is not really valid.
The impact on forecasted concentration fields and visibility are not presented in a clear enough way to be convincing. This is linked to the above remarks, also to the ones in the Specific comments, but mainly to the lack of information of what is aimed at for the forecast: is the aim to get better scores about peak detection / visibility loss forecasting? To avoid false positive? False negative? Moreover, the possibilities of a simpler correction of the initial field or improvements to the model are not discussed.Several important remarks on the way the text is written may explain why it is not convincing to the reader:
- please avoid using "etc": either provide the full information or indicate in the sentence that it's only the most recent/most relevant examples which are given.
- please don't use any phrase such as "it is obvious that" (l.258), "is evident" (l.283). Nothing is obvious or evident that is not trivial in a scientific paper.
- qualitative statements do not bring actual information: what is an "unreasonable result" (e.g. l. 234)? What does "significantly" means here (e.g. l.240, l.258)? What is "light" or "heavy" pollution (l.265-266)?
- please justify why so many significant numbers are provided, in particular for statistical indicators such as MB, ME, RMSE - or stick to a meaningful rounding.The legends of the Figures are not detailed enough: several do not contain all the information required to understand what is shown without looking up in the text which color is which variable or what an acronym means.
Specific comments
Introduction
- l.37: what is "the haze period"?
- l.39: please detail the "etc": health?
- l.44-45: "a deviation of air quality forecast results from observed comparisons, which can reach 30-50%": which results compared to which measurements can lead to a difference of 30 to 50% (and not -30 to -50%?)? Do you compare PM2.5 concentrations? Visibility? Other variables?
- l.46: " the forecast accuracy": what is intended here: to forecast concentrations of PM2.5 and/or visibility or to forecast air quality in terms of peaks and other indicators? The "accuracy" is not the same depending on waht is intended.
- l.54: what is in "etc"?
- l.50 to 61: aAvoid lists of methods and references if there is no message relevant to the topic of the paper to be derived from them.
- l.62-63: "which is difficult to perform for complex systems": it has nevertheless be done once or twice, please look up the references.
- l.68: "which is an advantageous data assimilation method": advantageous with regards to what? Practical implementation? Computing time and/or power? Why is it not always used (eg what about non-linear problems, such as is the case here)?
- l.69-75: too many terms which are not defined before, not adapted to an introduction.
- l.79: "the CTMs are strongly non-linear systems and the assumption of Gaussain variables and non-biased do not apply": 1) the sentence is not clear, at least one word is missing after "non-biased" and "variables" is not clear (shouldn't it be errors??) 2) it looks like the linearity and the Gaussian assumption are linked, which is not the case.
- l.85-86: "which makes the computation greatly reduced": at least one work is missing: the computation time?
- l.90: "still relatively rare": not very precise, it's been done in the past; also, explain why this is the case: are there any drawbacks? Which are they? Or maybe it's not efficient?
- l.92: same remark as before about the "forecast accuracy" but is it all the more crucial here that the assessment of the results depend on this definition.
Methods and Data
Generally, in this section, the difference between the well-known mathematics of the method and the choices made for this particular study must be made clearer.- Eq.1: the notation used for H indicate that it is considered linear. But l.79, it is stated that the case studied here is strongly non-linear - which is what is expected with PMs. An explanation of why assuming H to be linear is justified is required.
- please ensure that all notations and acronyms are defined at the fiest use. Example: AF l.113.
- l.115: "(0,1]": why (? Shouldn't it be ]?
- l.115: "alpha is taken as 0.9 in this study": why?
- l.118: the localization scheme is not part of the general method. A explanation of why it is needed, and stating its formulation and link with the previous equations is required, probably in a new subsection.
- generally, in this
- l.119-120: "limit the influence of a single observation by the Kalman update equation[...]": this sentence is not clear for me.
- l.124: "the observations can be reused": for what?
- l.135-137: long list of references but waht is the message?
- l.143: "Etc": which processes are included in this etc?
- l.155: which sectors are included in the "etc"?
- l.164: "the time step is 100s": why this choice?
- l.179: "the missing localization": not clear, do you mean "not using a localization scheme"?
- l.180: "localization was performed in selecting the optimal ensemble size": not clear what the link between localization and the ensemble size may be. Please explain.Results and discussion
- l.191-205: this two-site case is interesting as a pedagogical illustration but does not seem very relevant for the case study.
- l.206: "the initial fields from 15 to 23 December 2016 were performed": not clear, DA is performed on these fields?
- l.208: "means that more the simulation is closer to the observation": broken English.
- l.210: "[...] of the DA experiment are smaller that those of the CR": it is not clear if the statistical indicators are computed against to the validation sites or against assimilated sites. If it is the later, then the results are trivial.
- l.219: what are "spurious increments"? How are they detected?
- l.221: what is "an unreasonable initial field"?
- l.222: concluding on the localization length-scale as the one allowing "the best assimilation" is too strong based on the previous results. What is the "best assimilation"? The one for which initial fields match the validation data? The one which provides forecast concentrations closer to validation data?
- l.227: what is "the r field"?
- l.229-230: "the range of positive CORR at sites A and B gradually increases with the range of CORR greater than 0.7" I don't understand this sentence, please rephrase.
- l.230 : can be considered small": relative to what?
- l.232: "which exaggerates the correlation of each area": waht does "exaggerates" means here? Overestimates? The correlation of each area with what?
- l.234: what are "unreasonable results"?
- l.240: "changed significantly": 1) define "significant" for each statistical indicator here; 2) if the statistics are computed against sites used in the DA, the result is trivial.
- l.240-245: put numbers in a Table.
- l.247-248: the expected result would be that the larger the ensemble, the better the results. Why this is not the case must be explained.
- l.258: "it is obvious that":please avoid this kind of phrase, everything must be demonstrated.
- l.258: what is "significant" for a correction of the initial field?
- l.260: "affects other areas": that is the role of the localization length-scale, isn't it?
- l.265: "BF, AF< AFI": please ensure all abbreviations are defined at the first occurrence.
- l.265-266: please define "light" and "heavy" pollution. Are there official thresholds involved?
- l.270: "after assimilating the BFs": strangely put. Usually, it's more relevant to explain which observations are assimilated.
- l.270-271: "the AFs PM2.5 concentration distribution changes from sheet-like to discrete, which is due to the update of the model data in a length-scale of 40 km range": as stated in the General Comments, this looks loke a discrepancy between the use of a large modelled domain (with waht horizontal resolution?) and a comparatively short localization length. Please justify fully.
- l.274-276: this sentence is not complete, please re-write.
- l.277-282: please summarize the results in a Table so that the text is easier to read. Beware also of the number of significant numbers!
- l.282-283: "The results show that the correction effect of DA on the initial fields is evident": 1) nothing is evident or 2) if it is evaluated against assimilated sites, it is trivial.
- l.286-289: the sentence is too long.
- l.289-291: please put the information about which color is what in the legend of the Figures. The same applies below in the section.
- l.291: "relatively consistent in (20)": I don't understand, please rephrase.
- l.293: "gradually overlaps": not very precise or clear. Do you mean the effect of DA is at a very short term (a few hours)?
- l.296: "assimilating the initial field improves the PM2.5 forecast field throughout the assimilation tme window": this is not very clear. The assimilation time window is 0 since DA is only performed for the initial field. This field is then used for simulating forecast for a given simulation length but no DA is performed during this period.
- l.297: "strongest": please quantify.
- l.297-303: what is the message?
- l.311-313: "assimilating the initial field [...] a significant impact": all this is not clear for me. Please rephrase.
- l.313-316: this concluding remark is very general and is not specific to the case study. It would be more logical to have it in the method section.
- l.322-323: same remark about the "discrete" DA fields as for the PM2.5 concentrations.
- l.324-329: description of the Figure, not required in the text.
- l.329-330: "It proves that [...]": what is the link with the DA discussed here?
- l.334seq: put elements about Figure 11 in the legend of said Figure.
- l.342: "It is obvious that": no, everything that is worth mentionning must be explained/demonstrated.
- l.346: "the inaccuracy of the humidity simulation here and inaccurate visibility parameterization scheme for the model": wouldn't it be more relevant to improve the model than to perform DA? With a poorly adapted model, the impact of improving initial fields cannot be very large.
- l.348-349: "other objects of assimilation": the priority really seems to be the improvement of the model.Conclusion
- l.364: "the DA can significantly improve the model initial field": see above 1) define what is signifciant 2) trivial if not evaluated against validation sites.
- l.369-370: "was most pronounced in the first 12 hours and gradually decreased": very vagie statement for the actual result expected from the implementation of DA.
- l.372-373: "efficiency is highest with the largest distance between the model simulation and observation": here again, if the comparison is made against assimilated sites, it is trivial. The poorer the prior compared to the assimilated observations, the more spectacular the shift towards the observations after assimilation - provided there are no bugs in the implementation.
- l.379-380: "but this positive correlation is not particularly obvious": a correlation is so much, it is not obvious or not.
- l.383-384: considering the improvements which could be done in the model and the difficulties of assimilating such data streams as satellite and surface together, this perspective seems very ambitious.Tables and Figures
- Tables 2 and 3: please check the number of significant nunmbers.
- Figure 1: "ensemble generate": explain how; "calculate B": this is not shown in the equations; "C-B": what is C? It is not defined in the text when the reader is referred to the Fig; "verify assimilation result": how?
- Figure 4: "The num":? ; also make sure that all abbreviations are defined in the legend (same reamrk for all Figures)
- Figure 7: what ia a "most serious" PM2.5 pollution?Technical corrections
- l.158: "Fig. 1" -> should be Fig.2Citation: https://doi.org/10.5194/gmd-2022-207-RC2 - AC2: 'Reply on RC2', Ping Wang, 08 Apr 2023