the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 27 Mar 2023
Positive semi-definite variants of CBM4 and CBM05 chemistry schemes for atmospheric composition models
Abstract. Carbon bond chemistry models are widely used to describe tropospheric chemistry reactions in atmospheric chemistry transport models. The standard implementation of these carbon bond models includes a species for paraffin carbon bond which is positioned in chemistry reactions such that its concentration can be easily driven to negative unless additional mechanisms have been implemented around the scheme. Here we describe an alternative formulation which is positively semi-definite, while preserving the main structure of the carbon bond mechanism. We apply the formulation to CBM4 and CBM05 schemes. The new formulation substantially improves the numerical robustness of the schemes and relaxes the need for external mechanisms to enforce positive concentrations.
This preprint has been withdrawn.
-
Withdrawal notice
This preprint has been withdrawn.
-
Preprint
(681 KB)
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2023-3', William Carter, 31 Mar 2023
This paper proposes revised methods to implement two Carbon Bond mechanisms, CBM4 and CBM05, into airshed model software that should remove a potential for numerical instability when used in atmospheric models. Unfortunately, this work has several serious problems that prevent it from being suitable for publication. The serious problems are (1) they did not describe how they came up with the changes for the two mechanisms and gave no guidance on how to do this for future mechanisms of this type, and (2) the one scenario that was simulated to test the transformations was not comprehensive or representative of actual applications where CBM may be used. In addition, the authors have not made a convincing case that it is worth the effort for current CBM4 and CBM05 users to change their mechanism implementation software and their emissions processing assignments.
Although I am not an airshed modeler myself, as a developer of chemical mechanisms who has worked with airshed modelers, I do not remember any of them complaining about numerical instability of the many models that run the widely-used CBM mechanisms. In fact, many working modelers prefer CBM because of how fast simulations using CBM are compared to other mechanisms, despite the greater chemical approximations compared to the larger mechanisms. The paper does not actually make the claim that this is a problem with actual airshed applications using these mechanisms, and cites no references supporting this. They just state that that it might be a problem under certain circumstances, but no specific examples were cited. The apparent problem they noted around line 120 where [PAR]0 ~ [OLE]0 does not look like a realistic atmospheric situation.
The paper is correct in observing that these mechanisms can predict negative concentrations under certain situations, leading to numerical problems. However, these CBM's were designed, adjusted, and optimized for use with ambient mixtures dominated primarily by anthropogenic emissions, in part because their development was funded primarily for emissions impact assessment applications. These anthropogenic mixtures always have much more PAR when represented by CBM than OLE or other VOC model species, both because of the large amounts of alkanes emitted, and because of the use of PAR to represent carbons in other types of larger molecules. Although perhaps not initially designed for biogenic simulations, these mechanisms have a more explicit representation of isoprene, and terpene emissions introduce large amounts of PAR into the system, so numerical problems when simulating biogenic-dominated mixtures is also not expected.
The one test simulation that is presented is a static simulation using approximately equal amounts of PAR and OLE, which is the same thing as simulating a mixture of only propene and NOx (plus a small amount of some alkane). None of the CBMs are designed for simulations involving single organic compounds or even simple mixtures. If a single compound is deemed to be important, as has happened in the past with compounds such as isoprene, ethanol, and MTBE, then an explicit representation is just added to the mechanism.
It is very unlikely that present users of CBM4 and CBM05 would do all the work needed to revise the implementation of these mechanisms in their modeling software and emissions assignments just so the models will be more stable when the mechanism is applied to scenarios for which the use of CBM is not appropriate. Neither of these are the current version of CBM (CBM06 is in CMAQ), so if CBM users wanted to make all these changes, they might as well change to the latest version, such as CBM06. The main reason out-of-date mechanisms such as CBM4 and CBM05 continue to be used in models is the effort and cost involved in changing the mechanisms in the models.
However, this work COULD be a useful contribution to the literature if provided an algorithm of software that could be used to make corrections to any mechanisms that are not positive semi-definite, such as future versions of CBM or even new mechanisms that might use novel reduction approaches. However, the paper just shows the reaction changes (Tables 2 and 3), but does not show how they were derived. Without this information, this paper is relevant only to users of CB4 and CB05, and, as discussed above, it is unlikely they will find it useful. Note that just showing a series of equations is not sufficient -- they need to specify an objective algorithm or software that any user can apply to any applicable mechanisms.
Information on how the transformations were derived is also necessary to assess whether the transformations are in fact mathematically appropriate or at least a good approximation, and to determine the conditions where they are valid. They may still be acceptable for use by CBM4 or CB05 users if the paper shows it gives the same results as the original version for the full range of conditions where the models may be used, but this was not done. Instead they did a single simulation of what amounts to a propene - NOx mixture with a small alkane impurity and showed that the results were the same for all model species (shown) except for PAR. Does this work for simulations of PAR - NOx - air systems as well, such a system dominated by alkane emissions? It seems to me that the proposed changes would change the rate of consumption of PAR if it is in excess, affecting simulations of products formed when PAR reacts. But they also need to include simulations of realistic anthropogenic and biogenic mixtures. Those more realistic simulations using the original mechanisms with higher PAR levels may not have the numerical problems of their propene simulation, but it is necessary to show that the modification does not affect their results.
In Section 3 they state that one of the changes proposed for CB4 included "defining" "ROR4" as ROR + 2.1 PAR. This does not make sense because ROR is a model species representing alkoxy radicals that rapidly formed and reacts, and has a very low concentration compared to model species representing non-radical compounds such as PAR. Apparently what they actually mean is that the ROR is replaced by ROR4 -2.1 PAR in reactions forming it. They need to make this clearer.
The proposed changes for CBM4 would require re-speciating emissions for the many types of emitted VOCs that are represented by OLE. However, what about compounds such as 1,3-butadiene, that are represented by OLE alone, which would be represented by negative amounts of PAR? From Table 3 it is clear that the changes to CBM05 would require changing assignments for compounds represented by NTR, CRES, TOL, IOLE, ETHA, ISOP, and TERP, as well as OLE, but the paper does not indicate exactly how such species should be mapped. Therefore, the paper does not provide information on how emissions should be changed when modifying CBM05 as suggested.
I do not think a complete listing of the CB05 mechanism is necessary to include in this paper. Files implementing the standard and modified CB05 are in the supplement, and a formatted table could also be included in the supplement. The journal space would be better used describing how the CB05 corrections were derived, how emissions assignments need to be modified using it, and showing results of a more comprehensive set of test calculations.
Citation: https://doi.org/10.5194/gmd-2023-3-RC1 -
AC1: 'Reply on RC1', Risto Hänninen, 16 May 2023
Thank you very much for pointing out the ways to improve the presentation of our method! We shall certainly follow the recommendation in the revised paper. In particular, we shall describe the way emission data should be pre-processed to be consistent with the scheme modification.
We will also describe in more details about the iterative process for constructing the variable transformation. The article space is better occupied by moving the complete list of modified CBM05 equations into the appendix.
Citation: https://doi.org/10.5194/gmd-2023-3-AC1 -
AC3: 'Final reply on RC1', Risto Hänninen, 31 May 2023
First of all, we would like to thank the reviewers for their efforts and comments, which helped us to improve the manuscript and better position it in the scientific context. Below, we put point-by-point responses to the comments.
*****
RC1: "This paper proposes revised methods to implement two Carbon Bond mechanisms, CBM4 and CBM05, into airshed model software that should remove a potential for numerical instability when used in atmospheric models. Unfortunately, this work has several serious problems that prevent it from being suitable for publication. The serious problems are (1) they did not describe how they came up with the changes for the two mechanisms and gave no guidance on how to do this for future mechanisms of this type, ..."
-------
Answer: Thank you for the note! This omission has been rectified. The revised paper includes the procedure explained as a step-by-step algorithm, which is deduced from the Definitions 1-4 and the proposition equations (2) and (3). It shows that the requirement of the scheme being positive semi-definite (i.e. always giving physically meaningful solutions with all concentrations non-negative) leads to the modifications presented in the following sections.
*****
RC1: "... and (2) the one scenario that was simulated to test the transformations was not comprehensive or representative of actual applications where CBM may be used. In addition, the authors have not made a convincing case that it is worth the effort for current CBM4 and CBM05 users to change their mechanism implementation software and their emissions processing assignments."
-------
Answer: The box-model simulations shown in Figure 1 was made as simple as possible in order to demonstrate the way the modified scheme works, in comparison with the initial formulations. Also, it is not stated in the manuscript that original formulation is wrong, but one just needs to be aware the numerical problems it may cause.
*****
RC1: "Although I am not an airshed modeler myself, as a developer of chemical mechanisms who has worked with airshed modelers, I do not remember any of them complaining about numerical instability of the many models that run the widely-used CBM mechanisms. In fact, many working modelers prefer CBM because of how fast simulations using CBM are compared to other mechanisms, despite the greater chemical approximations compared to the larger mechanisms. The paper does not actually make the claim that this is a problem with actual airshed applications using these mechanisms, and cites no references supporting this. They just state that that it might be a problem under certain circumstances, but no specific examples were cited. The apparent problem they noted around line 120 where [PAR]0 ~ [OLE]0 does not look like a realistic atmospheric situation."
-------
Answer: We respectfully disagree. We faced such relations between the species in real-life global simulations of the SILAM model. The study was a very pragmatic exercise from the very beginning: we were debugging a randomly crashing / stalled runs, and it took us a great deal of efforts to find where the problem comes from. And this paper is the very publication reporting the problem, with a hope that other modellers can benefit from this effort. Therefore, the presented modifications were definitely worth the effort: they (i) adjust the scheme making it positive semi-definite, which is a valuable mathematical property of any scheme, and (ii) preclude random crashes of the models in large-scale simulations if non-negative concentrations are assumed by the algorithms.
*****
RC1: "The paper is correct in observing that these mechanisms can predict negative concentrations under certain situations, leading to numerical problems. However, these CBM's were designed, adjusted, and optimized for use with ambient mixtures dominated primarily by anthropogenic emissions, in part because their development was funded primarily for emissions impact assessment applications. These anthropogenic mixtures always have much more PAR when represented by CBM than OLE or other VOC model species, both because of the large amounts of alkanes emitted, and because of the use of PAR to represent carbons in other types of larger molecules. Although perhaps not initially designed for biogenic simulations, these mechanisms have a more explicit representation of isoprene, and terpene emissions introduce large amounts of PAR into the system, so numerical problems when simulating biogenic-dominated mixtures is also not expected."
-------
Answer: We agree that in real-life troposphere, standard concentrations [PAR] > [OLE], but it may be not the case in numerical simulations, especially in large-scale global problems: numerical uncertainties, peculiarities of emission speciation, randomization during the data assimilation process (e.g., by Ensemble Calman Filter), etc, can lead to any kind of relations between the species in remote regions or high above the surface. Practically the only restriction that can be guaranteed is that concentrations are always non-negative (at least in SILAM it always holds).
*****
RC1: "The one test simulation that is presented is a static simulation using approximately equal amounts of PAR and OLE, which is the same thing as simulating a mixture of only propene and NOx (plus a small amount of some alkane). None of the CBMs are designed for simulations involving single organic compounds or even simple mixtures. If a single compound is deemed to be important, as has happened in the past with compounds such as isoprene, ethanol, and MTBE, then an explicit representation is just added to the mechanism."
-------
Answer: As mentioned above, the presented case was deliberately made very simple, to demonstrate the essence of the problem. Presenting the full-scale simulations would only complicate the analysis. This is now clarified in the revised paper.
*****
RC1: "It is very unlikely that present users of CBM4 and CBM05 would do all the work needed to revise the implementation of these mechanisms in their modeling software and emissions assignments just so the models will be more stable when the mechanism is applied to scenarios for which the use of CBM is not appropriate. Neither of these are the current version of CBM (CBM06 is in CMAQ), so if CBM users wanted to make all these changes, they might as well change to the latest version, such as CBM06. The main reason out-of-date mechanisms such as CBM4 and CBM05 continue to be used in models is the effort and cost involved in changing the mechanisms in the models."
-------
Answer: We included CMB4 and CBM05 in the paper because these were the schemes implemented in SILAM. However, exactly the same issue exists in CMB06, which we now include in the revised paper, together with the appropriate modifications. Therefore, our findings are applicable also to newer versions of the CB mechanism.
*****
RC1: "However, this work COULD be a useful contribution to the literature if provided an algorithm of software that could be used to make corrections to any mechanisms that are not positive semi-definite, such as future versions of CBM or even new mechanisms that might use novel reduction approaches. However, the paper just shows the reaction changes (Tables 2 and 3), but does not show how they were derived. Without this information, this paper is relevant only to users of CB4 and CB05, and, as discussed above, it is unlikely they will find it useful. Note that just showing a series of equations is not sufficient -- they need to specify an objective algorithm or software that any user can apply to any applicable mechanisms."
-------
Answer: Thank you for the advice! The revised paper includes: (i) detailed step-by-step procedure for identifying and applying the needed changes, (ii) demonstration of the modifications for CMB06.
*****
RC1: "Information on how the transformations were derived is also necessary to assess whether the transformations are in fact mathematically appropriate or at least a good approximation, and to determine the conditions where they are valid. They may still be acceptable for use by CBM4 or CB05 users if the paper shows it gives the same results as the original version for the full range of conditions where the models may be used, but this was not done. Instead they did a single simulation of what amounts to a propene - NOx mixture with a small alkane impurity and showed that the results were the same for all model species (shown) except for PAR. Does this work for simulations of PAR - NOx - air systems as well, such a system dominated by alkane emissions? It seems to me that the proposed changes would change the rate of consumption of PAR if it is in excess, affecting simulations of products formed when PAR reacts. But they also need to include simulations of realistic anthropogenic and biogenic mixtures. Those more realistic simulations using the original mechanisms with higher PAR levels may not have the numerical problems of their propene simulation, but it is necessary to show that the modification does not affect their results."
-------
Answer: The proposed changes are mathematically equivalent substitution of variables, for a condition of [PAR] >> [OLE] (environmentally-relevant as the reviewer called it), In such conditions there cannot be any change of results of any species. However in the situation [PAR] < [OLE] which is not impossible the original version of mechanisms leads to negative concentrations. The adjusted mechanism does not lead to negative concentrations as long as both PAR4 and OLE4 are non-negative. The latter condition is way simpler to maintain in a model. The updated manuscript now illustrates how variable changes impact results in a more realistic scenario.
*****
RC1: "In Section 3 they state that one of the changes proposed for CB4 included "defining" "ROR4" as ROR + 2.1 PAR. This does not make sense because ROR is a model species representing alkoxy radicals that rapidly formed and reacts, and has a very low concentration compared to model species representing non-radical compounds such as PAR. Apparently what they actually mean is that the ROR is replaced by ROR4 -2.1 PAR in reactions forming it. They need to make this clearer."
-------
Answer: The revised paper contains the necessary clarification.
*****
RC1: "The proposed changes for CBM4 would require re-speciating emissions for the many types of emitted VOCs that are represented by OLE. However, what about compounds such as 1,3-butadiene, that are represented by OLE alone, which would be represented by negative amounts of PAR? From Table 3 it is clear that the changes to CBM05 would require changing assignments for compounds represented by NTR, CRES, TOL, IOLE, ETHA, ISOP, and TERP, as well as OLE, but the paper does not indicate exactly how such species should be mapped. Therefore, the paper does not provide information on how emissions should be changed when modifying CBM05 as suggested."
-------
Answer: Indeed, emission mapping has to be modified accordingly. We apologize for the omission. Corrected in the revised paper.
*****
RC1: "I do not think a complete listing of the CB05 mechanism is necessary to include in this paper. Files implementing the standard and modified CB05 are in the supplement, and a formatted table could also be included in the supplement. The journal space would be better used describing how the CB05 corrections were derived, how emissions assignments need to be modified using it, and showing results of a more comprehensive set of test calculations."
-------
Answer: The paper is modified as suggested.
Citation: https://doi.org/10.5194/gmd-2023-3-AC3
-
AC1: 'Reply on RC1', Risto Hänninen, 16 May 2023
-
RC2: 'Comment on gmd-2023-3', Anonymous Referee #2, 01 Apr 2023
The manuscript describes modifications to two gas-phase chemical mechanisms, CBM-IV (Gery et al., 1989) and CB05 (Yarwood et al., 2005), that are intended to improve numerical stability when either mechanism is applied in the SILAM chemical transport model. The authors suggest that their modifications may be useful to others. The specific issue considered is the existence of negative stoichiometric coefficients in these condensed mechanisms, for example -1 PAR in reaction 57 of CMB-IV:
(R57) OH + OLE = FORM + ALD2 + XO2 + HO2 - PAR
This reaction creates the potential for OLE to drive [PAR] negative if [OLE] > [PAR] (square brackets denote concentration). The authors propose that several reactions can be re-written to avoid this possibility but their strategy necessitates changing the number of Carbon atoms (C-number) contained in several model species, e.g., they change the definition of OLE from 2 to 3 Carbons. There are disadvantages to this strategy: (a) emissions must be revised to match the adjusted C-numbers , and (b) conservation of Carbon will be destroyed if emissions are not revised as needed. It is worth considering that conserving Carbon through the process of mapping VOC emissions to model species (OLE, PAR, etc.) is a design principle of CBM-IV and CB05 which distinguishes these mechanism from other condensed mechanisms that do not conserve moles of Carbon as VOC emissions are mapped to model species.
Gery et al. (1989) introduce a simple strategy to achieving the aims of this work that doesn’t have the disadvantages identified above. Appendix A of Gery et al. (1989) describes the CBM-EX mechanism (from which CBM-IV was condensed) and includes the following reactions:
(A97) OH + OLE = MEO2 + ALD2 + X
(A86) X + PAR =
Here, using the operator X to remove PAR in A86 avoids having a negative stoichiometric coefficient in A97 (i.e., - PAR is replaced by + X) which meets the authors first objective of eliminating negative stoichiometry. PAR could still be driven negative by this scheme if [OLE] > [PAR] but that can be prevented very simply by adding a reaction:
(Anew) X =
with a rate constant that makes Anew much slower than A86 except when [PAR] approaches zero. Gery et al. assigned a rate constant of 6.8E-12 cm-3 molecule-1 s-1 to reaction A86 in which case a rate constant of 0.1 s-1 for reaction Anew would prevent X from driving [PAR] below ~0.1 ppb. This simple strategy would achieve the authors second objective of avoiding negative [PAR].
The authors mention that the condition [OLE] > [PAR] arises in SILAM which is surprising considering that PAR represents paraffinic Carbon, predominantly alkanes, whereas OLE represents a terminal olefin groups (C=C) in alkenes. Generally, the mass of alkane emissions greatly exceeds the mass of alkene emissions and tends to produce [PAR] >> [OLE]. Furthermore, PAR is destroyed slowly by OH-reaction which tends to produce a ubiquitous background of [PAR] in models that works against the condition [OLE] > [PAR] arising. Perhaps the SILAM instability that the authors are encountering is a consequence of how total VOC emissions are being assigned to individual VOC compounds in the emission preparation step for SILAM.
Gery, M.W., Whitten, G.Z., Killus, J.P. and Dodge, M.C., 1989. A photochemical kinetics mechanism for urban and regional scale computer modeling. Journal of Geophysical Research: Atmospheres, 94(D10), pp.12925-12956.
Citation: https://doi.org/10.5194/gmd-2023-3-RC2 -
AC2: 'Reply on RC2', Risto Hänninen, 16 May 2023
We are certainly familiar with the Gery et al baseline paper, just used the updated version of the coefficients in our text. This is corrected now. However, we do not think that the solution with a fake substance is good. Apart from evident computational problems with the fast reaction, it does not remove the negative-mass cases, just allows the solver to pass them through. But the essence of the problem remains. A more extensive description will be added to the final paper version.
Citation: https://doi.org/10.5194/gmd-2023-3-AC2 -
AC4: 'Final reply on RC2', Risto Hänninen, 31 May 2023
First of all, we would like to thank the reviewers for their efforts and comments, which helped us to improve the manuscript and better position it in the scientific context. Below, we put point-by-point responses to the comments.
*****
RC2: "The manuscript describes modifications to two gas-phase chemical mechanisms, CBM-IV (Gery et al., 1989) and CB05 (Yarwood et al., 2005), that are intended to improve numerical stability when either mechanism is applied in the SILAM chemical transport model. The authors suggest that their modifications may be useful to others. The specific issue considered is the existence of negative stoichiometric coefficients in these condensed mechanisms, for example -1 PAR in reaction 57 of CMB-IV:
(R57) OH + OLE = FORM + ALD2 + XO2 + HO2 - PAR
This reaction creates the potential for OLE to drive [PAR] negative if [OLE] > [PAR] (square brackets denote concentration). The authors propose that several reactions can be re-written to avoid this possibility but their strategy necessitates changing the number of Carbon atoms (C-number) contained in several model species, e.g., they change the definition of OLE from 2 to 3 Carbons. There are disadvantages to this strategy: (a) emissions must be revised to match the adjusted C-numbers , and (b) conservation of Carbon will be destroyed if emissions are not revised as needed. It is worth considering that conserving Carbon through the process of mapping VOC emissions to model species (OLE, PAR, etc.) is a design principle of CBM-IV and CB05 which distinguishes these mechanism from other condensed mechanisms that do not conserve moles of Carbon as VOC emissions are mapped to model species."
-------
Answer: Yes, indeed, emission must be modified accordingly, thanks for pointing out the omission! As stated above (answers to the Reviewer 1), the revised paper includes the procedure of the emission modification.
*****
RC2: "Gery et al. (1989) introduce a simple strategy to achieving the aims of this work that doesn’t have the disadvantages identified above. Appendix A of Gery et al. (1989) describes the CBM-EX mechanism (from which CBM-IV was condensed) and includes the following reactions:
(A97) OH + OLE = MEO2 + ALD2 + X
(A86) X + PAR =
Here, using the operator X to remove PAR in A86 avoids having a negative stoichiometric coefficient in A97 (i.e., - PAR is replaced by + X) which meets the authors first objective of eliminating negative stoichiometry. PAR could still be driven negative by this scheme if [OLE] > [PAR] but that can be prevented very simply by adding a reaction:
(Anew) X =
with a rate constant that makes Anew much slower than A86 except when [PAR] approaches zero. Gery et al. assigned a rate constant of 6.8E-12 cm-3 molecule-1 s-1 to reaction A86 in which case a rate constant of 0.1 s-1 for reaction Anew would prevent X from driving [PAR] below ~0.1 ppb. This simple strategy would achieve the authors second objective of avoiding negative [PAR]."
-------
Answer: Indeed, Gery et al have noticed the problem too and suggested this approach as a way out (we are certainly aware about that ground-setting work and now cite it in the revised paper). Being comparatively simple, it has some disadvantages in practical applications. Most-importantly, it does not eliminate the problem of negative concentrations. If [X] > 0 at the end of a time step, i.e., there happened to be no PAR to consume, one still gets negative mass and all related problems. Forcing it to zero breaks down the carbon budget, leaving intact with a hope that it will eventually be consumed just postpones the problem. Apart from that, there are practical issues: (i) extra species, which either has to be transported similar to others consuming computational resources or somehow handled as an exception complicating the code and creating points of failure, (ii) an additional fast reaction, which rate has no physical meaning and thus has to be set "fast enough", potentially increasing stiffness of the scheme. Our solution, albeit more complicated in implementation, is free from these disadvantages. The corresponding discussion is inserted in the Discussion section of the revised paper.
*****
RC2: "The authors mention that the condition [OLE] > [PAR] arises in SILAM which is surprising considering that PAR represents paraffinic Carbon, predominantly alkanes, whereas OLE represents a terminal olefin groups (C=C) in alkenes. Generally, the mass of alkane emissions greatly exceeds the mass of alkene emissions and tends to produce [PAR] >> [OLE]. Furthermore, PAR is destroyed slowly by OH-reaction which tends to produce a ubiquitous background of [PAR] in models that works against the condition [OLE] > [PAR] arising. Perhaps the SILAM instability that the authors are encountering is a consequence of how total VOC emissions are being assigned to individual VOC compounds in the emission preparation step for SILAM."
-------
Answer: As mentioned in the answer to the Reviewer 1, the situation shows up in large-scale global applications going way above the boundary layer, etc. A specific feature of global runs is that all inaccuracies in all schemes tend to accumulate: they cannot be blown away through the boundaries, and dry deposition at the surface is slow. As we mentioned in the paper and stressed in its revised version, the issue is not frequent and practically never happens in limited-area simulations where boundary conditions restrict the model behavior. But in multi-decadal climate-scale chemical composition computations of a closed troposphere-stratosphere system even small inaccuracies matter. In a typical every-day operational SILAM simulation the scheme is evaluated about 1e12 times. The number is even bigger in ensemble-data-assimilation runs. Therefore making a scheme theoretically stable is necessary. This is now stressed in the revised paper.
Citation: https://doi.org/10.5194/gmd-2023-3-AC4
-
AC2: 'Reply on RC2', Risto Hänninen, 16 May 2023
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2023-3', William Carter, 31 Mar 2023
This paper proposes revised methods to implement two Carbon Bond mechanisms, CBM4 and CBM05, into airshed model software that should remove a potential for numerical instability when used in atmospheric models. Unfortunately, this work has several serious problems that prevent it from being suitable for publication. The serious problems are (1) they did not describe how they came up with the changes for the two mechanisms and gave no guidance on how to do this for future mechanisms of this type, and (2) the one scenario that was simulated to test the transformations was not comprehensive or representative of actual applications where CBM may be used. In addition, the authors have not made a convincing case that it is worth the effort for current CBM4 and CBM05 users to change their mechanism implementation software and their emissions processing assignments.
Although I am not an airshed modeler myself, as a developer of chemical mechanisms who has worked with airshed modelers, I do not remember any of them complaining about numerical instability of the many models that run the widely-used CBM mechanisms. In fact, many working modelers prefer CBM because of how fast simulations using CBM are compared to other mechanisms, despite the greater chemical approximations compared to the larger mechanisms. The paper does not actually make the claim that this is a problem with actual airshed applications using these mechanisms, and cites no references supporting this. They just state that that it might be a problem under certain circumstances, but no specific examples were cited. The apparent problem they noted around line 120 where [PAR]0 ~ [OLE]0 does not look like a realistic atmospheric situation.
The paper is correct in observing that these mechanisms can predict negative concentrations under certain situations, leading to numerical problems. However, these CBM's were designed, adjusted, and optimized for use with ambient mixtures dominated primarily by anthropogenic emissions, in part because their development was funded primarily for emissions impact assessment applications. These anthropogenic mixtures always have much more PAR when represented by CBM than OLE or other VOC model species, both because of the large amounts of alkanes emitted, and because of the use of PAR to represent carbons in other types of larger molecules. Although perhaps not initially designed for biogenic simulations, these mechanisms have a more explicit representation of isoprene, and terpene emissions introduce large amounts of PAR into the system, so numerical problems when simulating biogenic-dominated mixtures is also not expected.
The one test simulation that is presented is a static simulation using approximately equal amounts of PAR and OLE, which is the same thing as simulating a mixture of only propene and NOx (plus a small amount of some alkane). None of the CBMs are designed for simulations involving single organic compounds or even simple mixtures. If a single compound is deemed to be important, as has happened in the past with compounds such as isoprene, ethanol, and MTBE, then an explicit representation is just added to the mechanism.
It is very unlikely that present users of CBM4 and CBM05 would do all the work needed to revise the implementation of these mechanisms in their modeling software and emissions assignments just so the models will be more stable when the mechanism is applied to scenarios for which the use of CBM is not appropriate. Neither of these are the current version of CBM (CBM06 is in CMAQ), so if CBM users wanted to make all these changes, they might as well change to the latest version, such as CBM06. The main reason out-of-date mechanisms such as CBM4 and CBM05 continue to be used in models is the effort and cost involved in changing the mechanisms in the models.
However, this work COULD be a useful contribution to the literature if provided an algorithm of software that could be used to make corrections to any mechanisms that are not positive semi-definite, such as future versions of CBM or even new mechanisms that might use novel reduction approaches. However, the paper just shows the reaction changes (Tables 2 and 3), but does not show how they were derived. Without this information, this paper is relevant only to users of CB4 and CB05, and, as discussed above, it is unlikely they will find it useful. Note that just showing a series of equations is not sufficient -- they need to specify an objective algorithm or software that any user can apply to any applicable mechanisms.
Information on how the transformations were derived is also necessary to assess whether the transformations are in fact mathematically appropriate or at least a good approximation, and to determine the conditions where they are valid. They may still be acceptable for use by CBM4 or CB05 users if the paper shows it gives the same results as the original version for the full range of conditions where the models may be used, but this was not done. Instead they did a single simulation of what amounts to a propene - NOx mixture with a small alkane impurity and showed that the results were the same for all model species (shown) except for PAR. Does this work for simulations of PAR - NOx - air systems as well, such a system dominated by alkane emissions? It seems to me that the proposed changes would change the rate of consumption of PAR if it is in excess, affecting simulations of products formed when PAR reacts. But they also need to include simulations of realistic anthropogenic and biogenic mixtures. Those more realistic simulations using the original mechanisms with higher PAR levels may not have the numerical problems of their propene simulation, but it is necessary to show that the modification does not affect their results.
In Section 3 they state that one of the changes proposed for CB4 included "defining" "ROR4" as ROR + 2.1 PAR. This does not make sense because ROR is a model species representing alkoxy radicals that rapidly formed and reacts, and has a very low concentration compared to model species representing non-radical compounds such as PAR. Apparently what they actually mean is that the ROR is replaced by ROR4 -2.1 PAR in reactions forming it. They need to make this clearer.
The proposed changes for CBM4 would require re-speciating emissions for the many types of emitted VOCs that are represented by OLE. However, what about compounds such as 1,3-butadiene, that are represented by OLE alone, which would be represented by negative amounts of PAR? From Table 3 it is clear that the changes to CBM05 would require changing assignments for compounds represented by NTR, CRES, TOL, IOLE, ETHA, ISOP, and TERP, as well as OLE, but the paper does not indicate exactly how such species should be mapped. Therefore, the paper does not provide information on how emissions should be changed when modifying CBM05 as suggested.
I do not think a complete listing of the CB05 mechanism is necessary to include in this paper. Files implementing the standard and modified CB05 are in the supplement, and a formatted table could also be included in the supplement. The journal space would be better used describing how the CB05 corrections were derived, how emissions assignments need to be modified using it, and showing results of a more comprehensive set of test calculations.
Citation: https://doi.org/10.5194/gmd-2023-3-RC1 -
AC1: 'Reply on RC1', Risto Hänninen, 16 May 2023
Thank you very much for pointing out the ways to improve the presentation of our method! We shall certainly follow the recommendation in the revised paper. In particular, we shall describe the way emission data should be pre-processed to be consistent with the scheme modification.
We will also describe in more details about the iterative process for constructing the variable transformation. The article space is better occupied by moving the complete list of modified CBM05 equations into the appendix.
Citation: https://doi.org/10.5194/gmd-2023-3-AC1 -
AC3: 'Final reply on RC1', Risto Hänninen, 31 May 2023
First of all, we would like to thank the reviewers for their efforts and comments, which helped us to improve the manuscript and better position it in the scientific context. Below, we put point-by-point responses to the comments.
*****
RC1: "This paper proposes revised methods to implement two Carbon Bond mechanisms, CBM4 and CBM05, into airshed model software that should remove a potential for numerical instability when used in atmospheric models. Unfortunately, this work has several serious problems that prevent it from being suitable for publication. The serious problems are (1) they did not describe how they came up with the changes for the two mechanisms and gave no guidance on how to do this for future mechanisms of this type, ..."
-------
Answer: Thank you for the note! This omission has been rectified. The revised paper includes the procedure explained as a step-by-step algorithm, which is deduced from the Definitions 1-4 and the proposition equations (2) and (3). It shows that the requirement of the scheme being positive semi-definite (i.e. always giving physically meaningful solutions with all concentrations non-negative) leads to the modifications presented in the following sections.
*****
RC1: "... and (2) the one scenario that was simulated to test the transformations was not comprehensive or representative of actual applications where CBM may be used. In addition, the authors have not made a convincing case that it is worth the effort for current CBM4 and CBM05 users to change their mechanism implementation software and their emissions processing assignments."
-------
Answer: The box-model simulations shown in Figure 1 was made as simple as possible in order to demonstrate the way the modified scheme works, in comparison with the initial formulations. Also, it is not stated in the manuscript that original formulation is wrong, but one just needs to be aware the numerical problems it may cause.
*****
RC1: "Although I am not an airshed modeler myself, as a developer of chemical mechanisms who has worked with airshed modelers, I do not remember any of them complaining about numerical instability of the many models that run the widely-used CBM mechanisms. In fact, many working modelers prefer CBM because of how fast simulations using CBM are compared to other mechanisms, despite the greater chemical approximations compared to the larger mechanisms. The paper does not actually make the claim that this is a problem with actual airshed applications using these mechanisms, and cites no references supporting this. They just state that that it might be a problem under certain circumstances, but no specific examples were cited. The apparent problem they noted around line 120 where [PAR]0 ~ [OLE]0 does not look like a realistic atmospheric situation."
-------
Answer: We respectfully disagree. We faced such relations between the species in real-life global simulations of the SILAM model. The study was a very pragmatic exercise from the very beginning: we were debugging a randomly crashing / stalled runs, and it took us a great deal of efforts to find where the problem comes from. And this paper is the very publication reporting the problem, with a hope that other modellers can benefit from this effort. Therefore, the presented modifications were definitely worth the effort: they (i) adjust the scheme making it positive semi-definite, which is a valuable mathematical property of any scheme, and (ii) preclude random crashes of the models in large-scale simulations if non-negative concentrations are assumed by the algorithms.
*****
RC1: "The paper is correct in observing that these mechanisms can predict negative concentrations under certain situations, leading to numerical problems. However, these CBM's were designed, adjusted, and optimized for use with ambient mixtures dominated primarily by anthropogenic emissions, in part because their development was funded primarily for emissions impact assessment applications. These anthropogenic mixtures always have much more PAR when represented by CBM than OLE or other VOC model species, both because of the large amounts of alkanes emitted, and because of the use of PAR to represent carbons in other types of larger molecules. Although perhaps not initially designed for biogenic simulations, these mechanisms have a more explicit representation of isoprene, and terpene emissions introduce large amounts of PAR into the system, so numerical problems when simulating biogenic-dominated mixtures is also not expected."
-------
Answer: We agree that in real-life troposphere, standard concentrations [PAR] > [OLE], but it may be not the case in numerical simulations, especially in large-scale global problems: numerical uncertainties, peculiarities of emission speciation, randomization during the data assimilation process (e.g., by Ensemble Calman Filter), etc, can lead to any kind of relations between the species in remote regions or high above the surface. Practically the only restriction that can be guaranteed is that concentrations are always non-negative (at least in SILAM it always holds).
*****
RC1: "The one test simulation that is presented is a static simulation using approximately equal amounts of PAR and OLE, which is the same thing as simulating a mixture of only propene and NOx (plus a small amount of some alkane). None of the CBMs are designed for simulations involving single organic compounds or even simple mixtures. If a single compound is deemed to be important, as has happened in the past with compounds such as isoprene, ethanol, and MTBE, then an explicit representation is just added to the mechanism."
-------
Answer: As mentioned above, the presented case was deliberately made very simple, to demonstrate the essence of the problem. Presenting the full-scale simulations would only complicate the analysis. This is now clarified in the revised paper.
*****
RC1: "It is very unlikely that present users of CBM4 and CBM05 would do all the work needed to revise the implementation of these mechanisms in their modeling software and emissions assignments just so the models will be more stable when the mechanism is applied to scenarios for which the use of CBM is not appropriate. Neither of these are the current version of CBM (CBM06 is in CMAQ), so if CBM users wanted to make all these changes, they might as well change to the latest version, such as CBM06. The main reason out-of-date mechanisms such as CBM4 and CBM05 continue to be used in models is the effort and cost involved in changing the mechanisms in the models."
-------
Answer: We included CMB4 and CBM05 in the paper because these were the schemes implemented in SILAM. However, exactly the same issue exists in CMB06, which we now include in the revised paper, together with the appropriate modifications. Therefore, our findings are applicable also to newer versions of the CB mechanism.
*****
RC1: "However, this work COULD be a useful contribution to the literature if provided an algorithm of software that could be used to make corrections to any mechanisms that are not positive semi-definite, such as future versions of CBM or even new mechanisms that might use novel reduction approaches. However, the paper just shows the reaction changes (Tables 2 and 3), but does not show how they were derived. Without this information, this paper is relevant only to users of CB4 and CB05, and, as discussed above, it is unlikely they will find it useful. Note that just showing a series of equations is not sufficient -- they need to specify an objective algorithm or software that any user can apply to any applicable mechanisms."
-------
Answer: Thank you for the advice! The revised paper includes: (i) detailed step-by-step procedure for identifying and applying the needed changes, (ii) demonstration of the modifications for CMB06.
*****
RC1: "Information on how the transformations were derived is also necessary to assess whether the transformations are in fact mathematically appropriate or at least a good approximation, and to determine the conditions where they are valid. They may still be acceptable for use by CBM4 or CB05 users if the paper shows it gives the same results as the original version for the full range of conditions where the models may be used, but this was not done. Instead they did a single simulation of what amounts to a propene - NOx mixture with a small alkane impurity and showed that the results were the same for all model species (shown) except for PAR. Does this work for simulations of PAR - NOx - air systems as well, such a system dominated by alkane emissions? It seems to me that the proposed changes would change the rate of consumption of PAR if it is in excess, affecting simulations of products formed when PAR reacts. But they also need to include simulations of realistic anthropogenic and biogenic mixtures. Those more realistic simulations using the original mechanisms with higher PAR levels may not have the numerical problems of their propene simulation, but it is necessary to show that the modification does not affect their results."
-------
Answer: The proposed changes are mathematically equivalent substitution of variables, for a condition of [PAR] >> [OLE] (environmentally-relevant as the reviewer called it), In such conditions there cannot be any change of results of any species. However in the situation [PAR] < [OLE] which is not impossible the original version of mechanisms leads to negative concentrations. The adjusted mechanism does not lead to negative concentrations as long as both PAR4 and OLE4 are non-negative. The latter condition is way simpler to maintain in a model. The updated manuscript now illustrates how variable changes impact results in a more realistic scenario.
*****
RC1: "In Section 3 they state that one of the changes proposed for CB4 included "defining" "ROR4" as ROR + 2.1 PAR. This does not make sense because ROR is a model species representing alkoxy radicals that rapidly formed and reacts, and has a very low concentration compared to model species representing non-radical compounds such as PAR. Apparently what they actually mean is that the ROR is replaced by ROR4 -2.1 PAR in reactions forming it. They need to make this clearer."
-------
Answer: The revised paper contains the necessary clarification.
*****
RC1: "The proposed changes for CBM4 would require re-speciating emissions for the many types of emitted VOCs that are represented by OLE. However, what about compounds such as 1,3-butadiene, that are represented by OLE alone, which would be represented by negative amounts of PAR? From Table 3 it is clear that the changes to CBM05 would require changing assignments for compounds represented by NTR, CRES, TOL, IOLE, ETHA, ISOP, and TERP, as well as OLE, but the paper does not indicate exactly how such species should be mapped. Therefore, the paper does not provide information on how emissions should be changed when modifying CBM05 as suggested."
-------
Answer: Indeed, emission mapping has to be modified accordingly. We apologize for the omission. Corrected in the revised paper.
*****
RC1: "I do not think a complete listing of the CB05 mechanism is necessary to include in this paper. Files implementing the standard and modified CB05 are in the supplement, and a formatted table could also be included in the supplement. The journal space would be better used describing how the CB05 corrections were derived, how emissions assignments need to be modified using it, and showing results of a more comprehensive set of test calculations."
-------
Answer: The paper is modified as suggested.
Citation: https://doi.org/10.5194/gmd-2023-3-AC3
-
AC1: 'Reply on RC1', Risto Hänninen, 16 May 2023
-
RC2: 'Comment on gmd-2023-3', Anonymous Referee #2, 01 Apr 2023
The manuscript describes modifications to two gas-phase chemical mechanisms, CBM-IV (Gery et al., 1989) and CB05 (Yarwood et al., 2005), that are intended to improve numerical stability when either mechanism is applied in the SILAM chemical transport model. The authors suggest that their modifications may be useful to others. The specific issue considered is the existence of negative stoichiometric coefficients in these condensed mechanisms, for example -1 PAR in reaction 57 of CMB-IV:
(R57) OH + OLE = FORM + ALD2 + XO2 + HO2 - PAR
This reaction creates the potential for OLE to drive [PAR] negative if [OLE] > [PAR] (square brackets denote concentration). The authors propose that several reactions can be re-written to avoid this possibility but their strategy necessitates changing the number of Carbon atoms (C-number) contained in several model species, e.g., they change the definition of OLE from 2 to 3 Carbons. There are disadvantages to this strategy: (a) emissions must be revised to match the adjusted C-numbers , and (b) conservation of Carbon will be destroyed if emissions are not revised as needed. It is worth considering that conserving Carbon through the process of mapping VOC emissions to model species (OLE, PAR, etc.) is a design principle of CBM-IV and CB05 which distinguishes these mechanism from other condensed mechanisms that do not conserve moles of Carbon as VOC emissions are mapped to model species.
Gery et al. (1989) introduce a simple strategy to achieving the aims of this work that doesn’t have the disadvantages identified above. Appendix A of Gery et al. (1989) describes the CBM-EX mechanism (from which CBM-IV was condensed) and includes the following reactions:
(A97) OH + OLE = MEO2 + ALD2 + X
(A86) X + PAR =
Here, using the operator X to remove PAR in A86 avoids having a negative stoichiometric coefficient in A97 (i.e., - PAR is replaced by + X) which meets the authors first objective of eliminating negative stoichiometry. PAR could still be driven negative by this scheme if [OLE] > [PAR] but that can be prevented very simply by adding a reaction:
(Anew) X =
with a rate constant that makes Anew much slower than A86 except when [PAR] approaches zero. Gery et al. assigned a rate constant of 6.8E-12 cm-3 molecule-1 s-1 to reaction A86 in which case a rate constant of 0.1 s-1 for reaction Anew would prevent X from driving [PAR] below ~0.1 ppb. This simple strategy would achieve the authors second objective of avoiding negative [PAR].
The authors mention that the condition [OLE] > [PAR] arises in SILAM which is surprising considering that PAR represents paraffinic Carbon, predominantly alkanes, whereas OLE represents a terminal olefin groups (C=C) in alkenes. Generally, the mass of alkane emissions greatly exceeds the mass of alkene emissions and tends to produce [PAR] >> [OLE]. Furthermore, PAR is destroyed slowly by OH-reaction which tends to produce a ubiquitous background of [PAR] in models that works against the condition [OLE] > [PAR] arising. Perhaps the SILAM instability that the authors are encountering is a consequence of how total VOC emissions are being assigned to individual VOC compounds in the emission preparation step for SILAM.
Gery, M.W., Whitten, G.Z., Killus, J.P. and Dodge, M.C., 1989. A photochemical kinetics mechanism for urban and regional scale computer modeling. Journal of Geophysical Research: Atmospheres, 94(D10), pp.12925-12956.
Citation: https://doi.org/10.5194/gmd-2023-3-RC2 -
AC2: 'Reply on RC2', Risto Hänninen, 16 May 2023
We are certainly familiar with the Gery et al baseline paper, just used the updated version of the coefficients in our text. This is corrected now. However, we do not think that the solution with a fake substance is good. Apart from evident computational problems with the fast reaction, it does not remove the negative-mass cases, just allows the solver to pass them through. But the essence of the problem remains. A more extensive description will be added to the final paper version.
Citation: https://doi.org/10.5194/gmd-2023-3-AC2 -
AC4: 'Final reply on RC2', Risto Hänninen, 31 May 2023
First of all, we would like to thank the reviewers for their efforts and comments, which helped us to improve the manuscript and better position it in the scientific context. Below, we put point-by-point responses to the comments.
*****
RC2: "The manuscript describes modifications to two gas-phase chemical mechanisms, CBM-IV (Gery et al., 1989) and CB05 (Yarwood et al., 2005), that are intended to improve numerical stability when either mechanism is applied in the SILAM chemical transport model. The authors suggest that their modifications may be useful to others. The specific issue considered is the existence of negative stoichiometric coefficients in these condensed mechanisms, for example -1 PAR in reaction 57 of CMB-IV:
(R57) OH + OLE = FORM + ALD2 + XO2 + HO2 - PAR
This reaction creates the potential for OLE to drive [PAR] negative if [OLE] > [PAR] (square brackets denote concentration). The authors propose that several reactions can be re-written to avoid this possibility but their strategy necessitates changing the number of Carbon atoms (C-number) contained in several model species, e.g., they change the definition of OLE from 2 to 3 Carbons. There are disadvantages to this strategy: (a) emissions must be revised to match the adjusted C-numbers , and (b) conservation of Carbon will be destroyed if emissions are not revised as needed. It is worth considering that conserving Carbon through the process of mapping VOC emissions to model species (OLE, PAR, etc.) is a design principle of CBM-IV and CB05 which distinguishes these mechanism from other condensed mechanisms that do not conserve moles of Carbon as VOC emissions are mapped to model species."
-------
Answer: Yes, indeed, emission must be modified accordingly, thanks for pointing out the omission! As stated above (answers to the Reviewer 1), the revised paper includes the procedure of the emission modification.
*****
RC2: "Gery et al. (1989) introduce a simple strategy to achieving the aims of this work that doesn’t have the disadvantages identified above. Appendix A of Gery et al. (1989) describes the CBM-EX mechanism (from which CBM-IV was condensed) and includes the following reactions:
(A97) OH + OLE = MEO2 + ALD2 + X
(A86) X + PAR =
Here, using the operator X to remove PAR in A86 avoids having a negative stoichiometric coefficient in A97 (i.e., - PAR is replaced by + X) which meets the authors first objective of eliminating negative stoichiometry. PAR could still be driven negative by this scheme if [OLE] > [PAR] but that can be prevented very simply by adding a reaction:
(Anew) X =
with a rate constant that makes Anew much slower than A86 except when [PAR] approaches zero. Gery et al. assigned a rate constant of 6.8E-12 cm-3 molecule-1 s-1 to reaction A86 in which case a rate constant of 0.1 s-1 for reaction Anew would prevent X from driving [PAR] below ~0.1 ppb. This simple strategy would achieve the authors second objective of avoiding negative [PAR]."
-------
Answer: Indeed, Gery et al have noticed the problem too and suggested this approach as a way out (we are certainly aware about that ground-setting work and now cite it in the revised paper). Being comparatively simple, it has some disadvantages in practical applications. Most-importantly, it does not eliminate the problem of negative concentrations. If [X] > 0 at the end of a time step, i.e., there happened to be no PAR to consume, one still gets negative mass and all related problems. Forcing it to zero breaks down the carbon budget, leaving intact with a hope that it will eventually be consumed just postpones the problem. Apart from that, there are practical issues: (i) extra species, which either has to be transported similar to others consuming computational resources or somehow handled as an exception complicating the code and creating points of failure, (ii) an additional fast reaction, which rate has no physical meaning and thus has to be set "fast enough", potentially increasing stiffness of the scheme. Our solution, albeit more complicated in implementation, is free from these disadvantages. The corresponding discussion is inserted in the Discussion section of the revised paper.
*****
RC2: "The authors mention that the condition [OLE] > [PAR] arises in SILAM which is surprising considering that PAR represents paraffinic Carbon, predominantly alkanes, whereas OLE represents a terminal olefin groups (C=C) in alkenes. Generally, the mass of alkane emissions greatly exceeds the mass of alkene emissions and tends to produce [PAR] >> [OLE]. Furthermore, PAR is destroyed slowly by OH-reaction which tends to produce a ubiquitous background of [PAR] in models that works against the condition [OLE] > [PAR] arising. Perhaps the SILAM instability that the authors are encountering is a consequence of how total VOC emissions are being assigned to individual VOC compounds in the emission preparation step for SILAM."
-------
Answer: As mentioned in the answer to the Reviewer 1, the situation shows up in large-scale global applications going way above the boundary layer, etc. A specific feature of global runs is that all inaccuracies in all schemes tend to accumulate: they cannot be blown away through the boundaries, and dry deposition at the surface is slow. As we mentioned in the paper and stressed in its revised version, the issue is not frequent and practically never happens in limited-area simulations where boundary conditions restrict the model behavior. But in multi-decadal climate-scale chemical composition computations of a closed troposphere-stratosphere system even small inaccuracies matter. In a typical every-day operational SILAM simulation the scheme is evaluated about 1e12 times. The number is even bigger in ensemble-data-assimilation runs. Therefore making a scheme theoretically stable is necessary. This is now stressed in the revised paper.
Citation: https://doi.org/10.5194/gmd-2023-3-AC4
-
AC2: 'Reply on RC2', Risto Hänninen, 16 May 2023
Data sets
Numerical example on how to convert CBM4 chemistry scheme into positively semi-definite Risto Hänninen, Rostislav Kouznetsov, and Mikhail Sofiev https://doi.org/10.5281/zenodo.7763872
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 555 | 165 | 45 | 765 | 30 | 24 |
- HTML: 555
- PDF: 165
- XML: 45
- Total: 765
- BibTeX: 30
- EndNote: 24
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1