the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 20 Nov 2024
FastCTM (v1.0): Atmospheric chemical transport modelling with a principle-informed neural network for air quality simulations
Abstract. Chemical transport models (CTM) have wide and profound applications in air quality simulations and managements. However, its applications are often constrained by high computational burdens. In this study, we developed a neural network based CTM model (FastCTM) to efficiently simulate ten air pollutant composition variables, including major PM2.5 species of SO42-, NO3-, NH4+, organic matters and other inorganic components, coarse part of PM10, SO2, NO2, CO and O3. The FastCTM has a principle-informed structure by explicitly encoding atmospheric physical and chemical processes in a basic simulator. Specifically, in the simulator, five neural network modules are proposed to respectively represent five major atmospheric processes of primary emissions, transport, diffusion, chemical reactions and depositions. Given 1-hour initial condition data, the FastCTM is able to simulate future 24-hour concentrations of the ten air pollutants with corresponding meteorology fields and emissions as input. The FastCTM is trained with operational forecast data from a numerical CTM model named Community Multiscale Air Quality (CMAQ) in 2018–2022. The well-trained FastCTM is evaluated comparing to the long-term CMAQ forecast in an independent year 2023, and achieves high agreements with mean RMSE values of 9.1, 11.9, 4.4, 4.0, 48.9 and 10.9 μg/m3 and R2 values of 0.8, 0.81, 0.8, 0.83, 0.9 and 0.7 for PM2.5, PM10, SO2, NO2, CO, and O3. Besides, assessed against hourly site observations of six criteria pollutants, the RMSE values of FastCTM have small relative differences of 4.3 %, 4.2 %, -2.8 %, -1.7 %, -0.3 % and -3.2 % compared to that of CMAQ. The FastCTM model also exhibited reasonable responses of air quality to meteorological variables of air temperature, wind speed and planetary boundary layer height, as well as to input pollutant emissions. Furthermore, due to the principles-oriented structure, internal process analysis could be performed by FastCTM to quantify the specific contribution from each of the five processes for hourly air pollutant concentration changes. In a nutshell, FastCTM has multi-functional advantages in air pollutant concentration simulations, sensitivity analysis and internal process analysis with high computation efficiencies on GPU and accuracy.
- Preprint
(2266 KB) - Metadata XML
-
Supplement
(896 KB) - BibTeX
- EndNote
Status: open (extended)
-
RC1: 'Comment on gmd-2024-198', Anonymous Referee #1, 10 Feb 2025
reply
The authors describe a new, neural network-based reduced-order model of atmospheric chemistry and transport (FastCTM) which has been trained using an extensive dataset of output from CMAQ. FastCTM uses a novel and interesting approach, building physics-informed networks for five separate operators. The authors show that FastCTM is able to reproduce the general patterns of concentrations calculated by CMAQ for an out-of-training-data year (2023), and that the sensitivities of FastCTM to key meteorological variables or nation-wide changes in emissions mostly follow expected patterns. If FastCTM can be shown to be reliable in policy-relevant contexts then it could be a very useful tool.
This approach to modelling is interesting, and this methodological advance has the potential to significantly accelerate air quality scenario analysis. A CTM which can respect key physical constraints (e.g. mass conservation) while also accurately reproducing the effect of different perturbations to emissions and meteorological fields would have great value. However, the manuscript as written does not quite live up to this promise. Along with some minor concerns, the key challenge is that the authors do not show evidence that this new model can fulfil the roles of a CTM and produce accurate results for one of the most common use cases (i.e. understanding the effects of different perturbations). I explain this concern in more detail below, and until this concern is addressed I do not believe the manuscript should be accepted for publication by GMD.
Major commentsThe most significant concern relates to the validation/evaluation of the model. The authors appear to have trained the five physical operators based on several years of output from the CMAQ chemistry transport model. While I have some questions regarding the training process, I will take it as read for the moment that the training was done in such a way as to avoid overfitting. However, the verification of the model rests on its ability to predict, from the 2018-2022 data, the performance in 2023. This approach is inadequate for two reasons. First, the authors do not compare the performance of the model to simpler approaches with the same data such as generalized additive models, gradient boosting, or linear regression with land use (see e.g. Wong et al., 2021 and Cheng et al., 2021). Without such a comparison to evaluate how such models would have performed in predicting 2023, it is difficult to say what the magnitude of FastCTM’s advance is. This is exacerbated by the relatively shallow quantitative assessment in section 3.1.1. RMSE and R2 values are provided, but it is not clear how these were calculated; given that these are calculated as a function of time, are these calculated based on the difference in each of the 158,742 grid cells between CMAQ and FastCTM? A deeper analysis which investigates how model performance varies between (e.g.) rural and urban areas, coastal and inland areas, winter and summer, and so on would provide a much more robust test of the model’s ability to predict the effect of changing meteorology. This could be informed by (e.g.) taking the difference of FastCTM for 2023 against CMAQ for 2023, and comparing that to the difference between CMAQ for 2023 and the average of CMAQ from 2018 to 2022. This would at least demonstrate whether FastCTM provides more explanatory power for the mean atmospheric state than taking the average concentration from the previous five years.
Second, and perhaps more importantly, the function of FastCTM is to reproduce the results of high-fidelity CTMs at a fraction of the computational cost – specifically to support air pollution simulations, sensitivity analysis, and internal process analysis (abstract lines 30-32). The comparison to 2023 only tells us that FastCTM can reproduce the general pattern of air pollution in 2023, but does not tell us whether FastCTM will accurately predict the effect of interventions. The sensitivity tests in section 3.2 have no basis for comparison, and are in any case so broad (representing nationwide changes in temperature, PBL height, or emissions) that they are a limited test of the CTM’s capabilities. At the very minimum, an evaluation is needed which shows that FastCTM’s trends actually match the underlying trends in WRF-CMAQ; this should be straightforward for the emissions cases. Since the goal of FastCTM is to reduce computational costs, it is critical that FastCTM be shown to be faithful to its parent model for realistic applications such as projecting the impact of a change in emissions. Going further and comparing sensitivities for local or single-sector emissions changes would provide even more powerful proof, and I strongly recommend that the authors consider such a comparison.
Without these kinds of quantitative comparisons I can only judge the model’s success based on data such as Figure 3, where I am concerned because the patterns do not – speaking qualitatively – appear to match that well between CMAQ and FastCTM. I am particularly concerned that the model may be mostly reproducing emissions maps and historical scalings, rather than accurately representing chemistry and transport (especially given that transport is 2-D only). A more critical, quantitative analysis of the models strengths and weaknesses would be necessary before I would recommend its use in a scientific or regulatory context.
Minor commentsThe description of the five operators does not quite seem to verify that physical constraints are being satisfied but this may simply be a misinterpretation on my part. For example, can you confirm that the method you used to generate the convolution kernels (Eq 5-7) for transport ensures mass conservation? This seems to depend on Ci being in units of molec/cm3 rather than ppbv, but the units of Ci are not clearly specified.
A related concern is that surface layer winds are used and treated conservatively, which neglects the fact of rapid vertical mixing. Can the authors provide evidence that the surface winds (which would be expected to be slower than the mean wind speed in the boundary layer) are accurately predicting pollutant motion? It seems that any model which is designed to predict transport using only the horizontal near-surface winds will underestimate overall transport. Should the model not be using the PBL-averaged horizontal winds in Eq. 4 instead of the surface winds at 10-meter height (lines 83-84)?
It would be helpful to get more detail on how components such as the diffusion encoder were trained. Currently the manuscript states that 5 years of data (2018 – 2022 inclusive) were used in training, but not how the five different models were trained using that data. A naïve assessment would assume that all five sub-models were trained based simply on hour-to-hour pollutant concentrations, but that would suggest that the models were each trying to represent all atmospheric processes simultaneously.
Line 172 says that the reaction encoder in Equation 12 “has the same structure as that of reaction and deposition encoder models (Eq. 10)”. This is recursive, but also Eq. 10 refers to the diffusion module?
On line 194, “We did not use the fixed area as that in the previous studies (Xing et al., 2022)” – can you elaborate? It was not clear to me what this meant.
The y-axis labels on Figure 5 say “Percentage”, but from context it appears these must really be the factor difference from the baseline (as all cross at 1.0).
Finally, there are numerous minor grammatical errors (e.g. L14: simulations and managements; L67: interpretations of the FastCTM are also widely vowed; L70: including and major; and so on). This is not important for judgment of the paper’s appropriateness for publication, but I recommend the authors take another look at the paper to correct such minor issues.
CitationsWong et al., “Using a land use regression model with machine learning to estimate ground level PM2.5.” Environ. Pollut., 2021.
Cheng et al. "Influence of weather and air pollution on concentration change of PM2. 5 using a generalized additive model and gradient boosting machine." Atmospheric environment, 2021.
Citation: https://doi.org/10.5194/gmd-2024-198-RC1
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 185 | 40 | 13 | 238 | 20 | 15 | 14 |
- HTML: 185
- PDF: 40
- XML: 13
- Total: 238
- Supplement: 20
- BibTeX: 15
- EndNote: 14
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1