the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 06 Jan 2025
Optimized step size control within the Rosenbrock solvers for stiff chemical ODE systems in KPP version 2.2.3_rs4
Abstract. Numerical integration of multiphase chemical kinetics in atmospheric models is challenging. The underlying system of ordinary differential equations (ODEs) is stiff and thus difficult to solve. Rosenbrock solvers are a popular choice for such tasks. These solvers provide the desired stability and accuracy of results at an affordable yet large computational cost. The latter is crucially dependent on the efficiency of the step size control. Our analysis indicates that the local error, which is the key factor for the step size selection, is often overestimated, leading to very small sub-steps. In this study, we optimized the first-order step size controller most commonly employed in Rosenbrock solvers. Furthermore, we compared its efficiency to a second-order step size controller. We assessed the performance of the controllers in both a box and a global model for very stiff ODEs. Significant reductions of the computation time were accomplished with only marginal deviations in the results compared to the standard first-order controller. This was achieved not only for gas-phase chemistry but also for the more complex aqueous-phase chemistry in cloud droplets and deliquescent aerosols. Depending on the selected chemical mechanism, significant improvements were already achieved by simply adjusting heuristic parameters of the default controller. However, especially for the global model, the best results were achieved with the second-order controller, which reduced the number of function evaluations by 43 %, 27 % and 16 % for gas-phase, cloud and aerosol chemistry, respectively. The overall computational time was reduced by over 11 % while requiring only minimal adjustments to the original code. Analysis of a 1-year integration period showed that with the second-order controller, the deviations from the reference simulation stays below 1 % for the main tropospheric oxidants. The results presented here show the possibility of more efficient atmospheric chemistry simulations without compromising accuracy.
- Preprint
(5842 KB) - Metadata XML
-
Supplement
(26288 KB) - BibTeX
- EndNote
Status: open (until 03 Mar 2025)
-
RC1: 'Comment on gmd-2024-166', Anonymous Referee #1, 26 Jan 2025
reply
Dreger et al. present an optimized first-order step size controller for the Rosenbrock solver within the Kinetic Pre-Processor (KPP) package, which is often used for solving stiff systems of ODEs used in atmospheric chemistry. The updated step size controller achieves double-digit improvements in computational performance in gas-phase and complex aqueous-phase chemistry with minimal (<1%) errors for main tropospheric oxidants in a 1-year simulation period. This work is important to the development of chemical solvers in atmospheric chemistry; it tackles a new aspect of improvement in contrast to previous efforts in reducing the ODE system size, improved solvers, and use of GPUs, by improving the step size controller, and implements the work in a widely used KPP software package making it adoptable by other atmospheric chemistry models immediately. The work is well written and in scope for GMD, and I recommend its publication. I have a few questions for clarification on some aspects of the work presented by the authors.
Comments:
1. L33: "atmospheric chemical kinetic models satisfy this criterion when the mechanism contains very fast as well as very slow reactions..." I would rephrase to emphasize that atmospheric chemistry inherently has this property because of the large range of time-scales involved in a typical atmosphere - "when the mechanism contains" implies to me that one can select a mechanism that doesn't have this property, but this wouldn't be representative of the typical (Earth) atmosphere.2. Figure 2 notes that the first model step requires significantly more substeps than the others. I understand this may be because the initial condition provided to the solver is a condition that was perturbed by other non-chemistry operators (e.g., advection, emissions) and the solver needs to first adapt to this new condition. I see that in L280-289 the authors have experimented with changing this initial step size. I am curious about this further optimization - could the authors elaborate on "for the following model steps the step size always grew at the beginning" - is this only observed in the box model environment? Are there other operations on the chemical concentrations between integrator time steps in the box model? I wonder if such an optimization could be more advantageous in a global model where operator splitting would have a greater effect on the the solver's initial time step.
3. Eq. 7 - is "equal with exclamation mark" here denoting an estimation or a note to the reader?
4. Table 3 - could the authors clarify the difference between "the reduction within the corresponding ODE system" vs. "the whole accumulated number of function evaluations"? If it's not referring to the function evaluations, is this reduction a reduction in wall-clock time?
5. In Table 5 I would note the % of function calls reduction as well which I believe is a better representation of the improvement in the solver itself (for the same arguments made in the manuscript that the model has other processes like I/O which also take significant amount of computational time), even if run time reduction is most relevant to model users.
6. Section 5.2 Error analysis Figure 7 - the authors show main tropospheric oxidants (OH, O3, and NO3) percentage difference compared to a simulation without the improved step size controller. I was wondering if a plot including all the species in a lighter color in the background (similar to Figure 7 in Shen et al., 2022 - https://doi.org/10.5194/gmd-15-1677-2022) would help visualize the effect on other species and identify potential outliers in relative differences.
7. Figure 8 d) and f) are quite saturated in color scale; as the authors have noted this may be just numerical noise because of concentrations that are too small. Maybe the same figure could be made to filter out (maybe mark as grey) small concentrations (by some percentile or by the single-precision limit?) so the % difference on grid boxes with "meaningful" (non-numerical noise) values can be seen better?
8. The authors included a discussion of load imbalance in Section 6, which is a known problem in atmospheric chemistry models where the terminator zone has a much greater number of internal step sizes, making parallel simulations spend wall-clock time on "waiting" (e.g. MPI barriers). Figure 9 is a good visualization, the # of step sizes has been used in many papers to show this imbalance, but in terms of visualizing improvement in this imbalance between the original and the optimized solver, I wonder if the authors would be able to plot a histogram of the distribution of the # of function evaluations in # of grid boxes, and show if the distribution of function evaluations has changed between the original and optimized solver.
9. Another note on the load imbalance issue, the authors note in caption of Figure 9 that "Load imbalance could increase in the worst case if many red boxes are calculated by the same CPU-core" - I suppose this depends on the MPI/OpenMP decomposition used by the MECCA model. How does MECCA decompose the domain? Different models have different approaches, in the GEOS-Chem High Performance model (10.1029/2020MS002064 Figure D1) the domain is decomposed in a trivial way on each face of the cubed-sphere, but other models like CESM have more elaborate decompositions which aim to distribute sunlight evenly across cores (for the benefit of load balancing radiation code) which would benefit atmospheric chemistry.
10. Do the authors plan to contribute this improved step-size controller to the mainline KPP code for use by other models by default in the Rosenbrock solver?
Citation: https://doi.org/10.5194/gmd-2024-166-RC1 -
RC2: 'Comment on gmd-2024-166', Anonymous Referee #2, 05 Feb 2025
reply
Review for Optimized step size control within the Rosenbrock solvers for stiff chemical ODE systems in KPP version 2.2.3_rs4 by Dreger et al.
This is a nice paper examining how step size control/optimization impacts computational efficiency and accuracy when simulating atmospheric (multiphase) chemistry in box and 3D global chemical transport modeling. It is relatively clear and well-written and presents novel results that will be of significant value to atmospheric chemistry modeling community, particularly for operational 3D modeling where efficient yet accurate approaches are required. It is well-suited for GMD. I recommend publication with the consideration of minor edits/questions.
Line 67. From “represents the currently used controller used in our applications that we investigate” to “represents the controller we investigate…”
Line 91. From “robuster” to “more robust”
Table 1 caption. Change “over” to “of”
Lines 127-132. Are these three scenarios specified only by changing input parameters or are there changes to the mechanism treated? (You might want to mention this here and define the changes that are made to represent the different scenarios somewhere.)
Line 145. Remove the word “binary”?
Line 153. Should “GMXE” be changed to “GMXe”?
Lines 186-187. This is unclear to me. Output is written every 23rd hour? Or every day at hour 23? How does this lead to output for every hour of the day? Do you just mean that there are outputs for each hour (0-23) as you cycle through the days of the year (writing every 23rd hour) but not outputs for every hour of each day?
Line 233. Typo – change “suite” to “suit”
Line 262. Remove “for qmax” (redundant)
Line 272. From “decreasing” to “decrease”.
Line 279. From “good” to “well”.
Line 281. From “tough” to “though”.
Line 301. From “scenario, respectively.” to “scenarios.”
Section 4.1 and elsewhere (e.g., line 315): it might be better to say “fewer” function evaluations than “less”.
Also, any thoughts on why the qmax changes had more influence in the global simulations wrt MECCA compared to SCAV and the box modeling where it was not very impactful?
Line 326. Change “over” to “of”.
Table 3 caption: Change from “qmax and [safety factor] for MECCA and SCAV” to “qmax and [safety factor] for MECCA, [safety factor] for SCAV,”. Change “brackets” to “parentheses”.
Line 361. Remove comma
Line 365. From “an precision” to “a precision”
Line 377. Change “areas” to “area”
Line 380. Remove “proceeded rather consistent and”?
Line 390. What is this reference?
Would it be worthwhile to look at the relative differences for an aerosol species that has important chemistry in cloud and aerosol water (e.g., sulfate)?
Line 423: Change “worth” to “worthwhile”
Line 448. Change “…to the Eq. 6, we only have two” to “…to Eq. 6 with only two”
Citation: https://doi.org/10.5194/gmd-2024-166-RC2
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 181 | 20 | 8 | 209 | 115 | 3 | 5 |
- HTML: 181
- PDF: 20
- XML: 8
- Total: 209
- Supplement: 115
- BibTeX: 3
- EndNote: 5
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1