the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Order of magnitude wall time improvement of variational methane inversions by physical parallelization: a demonstration using TM5-4DVAR
- 1SRON Netherlands Institute for Space Research, Utrecht, the Netherlands
- 2Department of Earth Sciences, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- 3TNO Department of Climate, Air and Sustainability, Utrecht, The Netherlands
- 1SRON Netherlands Institute for Space Research, Utrecht, the Netherlands
- 2Department of Earth Sciences, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
- 3TNO Department of Climate, Air and Sustainability, Utrecht, The Netherlands
Abstract. Atmospheric inversions are used to constrain the emissions of trace gases from atmospheric mole fraction measurements. The variational (4DVAR) inversion approach allows optimization of the emissions at a much higher temporal and spatial resolution than the ensemble or analytical approaches but provides limited opportunities for scalable parallelization as the optimization is performed iteratively. Multidecadal variational inversions are used to optimally extract information from the long measurement records of long-lived atmospheric trace gases like carbon dioxide and methane. However, the wall clock time needed—up to months— complicates these multidecadal inversions. The physical parallelization method introduced by Chevallier (2013) addresses this problem for CO2 inversions by splitting the time period of the chemical transport model into blocks that are run in parallel. Here we present a new implementation of the physical parallelization for variational inversion (PPVI) approach that is suitable for methane inversions as it accounts for methane’s atmospheric lifetime. The performance of PPVI is tested in an 11-year inversion using a TM5-4DVAR inversion setup that assimilates surface observations to optimize methane emissions at grid-scale. We find that the PPVI inversion approach improves the wall clock time performance by a factor of 5 and shows excellent agreement with the posterior emissions of a full serial inversion with identical configuration (global mean emissions difference = 0.06 % with an interannual variation correlation R = 99 %; regional mean emission difference < 5 % and interannual variation R > 0.95). The wall clock time improvement using the PPVI method increases with the size of the inversion period. The PPVI approach is planned to be used in future releases of the CAMS (Copernicus Atmosphere Monitoring Service) multidecadal methane reanalysis.
Sudhanshu Pandey et al.
Status: closed
-
CEC1: 'Comment on gmd-2021-339', Juan Antonio Añel, 19 Nov 2021
Dear authors,
After checking your manuscript, it has come to our attention that it does not comply with our Code and Data Policy.
https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
You have archived the TM5 code in SourceForge. However, SourceForge is not a suitable repository for long-term archival and publishing. Therefore, please, publish your code in one of the appropriate repositories.
In this way, before the Discussions stage is closed, you must reply to this comment with the link to the repository for the code and the corresponding DOI.
Also, you must include in a potential reviewed version of your manuscript the modified 'Code and Data Availability' section and the DOI of the code. Also, in the SourceForge repository, I have not seen a license listed for TM5. If you do not include a license, the code continues to be your property and can not be used by others. Therefore, when uploading the model's code to the repository, you could want to choose a free software/open-source (FLOSS) license. We recommend the GPLv3. You only need to include the file 'https://www.gnu.org/licenses/gpl-3.0.txt' as LICENSE.txt with your code. Also, you can choose other options that Zenodo provides: GPLv2, Apache License, MIT License, etc.Finally, in the same way, it would be good if you archive and make available a snapshot of the exact file with the methane data used in your work, with its corresponding DOI. We can not consider the NOAA site as a permanent, long-term link.
Please, reply as soon as possible to this comment with the link for it so that it is available for the peer-review process, as it should be.Regards,
Juan A. Añel
Geosc. Mod. Dev. Executive Editor
-
AC1: 'Reply on CEC1', Sudhanshu Pandey, 06 Mar 2022
Dear Editor,
We have put the TM5-4DVAR-PP code developed in this study as well as the input data used in the simulations (surface observations, initial mole fraction fields) on Zenodo:
“Data Availability. NOAA ESRL methane observations used in this study are available on Zenodo in the input folder of the TM5-4DVAR-PP code (https://doi.org/10.5281/zenodo.6326373, Pandey et al., 2022).
Code availability. The TM5-4DVAR-PP version 1.0-beta-1 code used in this study for the simulations can be downloaded from Zenodo (https://doi.org/10.5281/zenodo.6326373, Pandey et al., 2022). The TM5 model is described in detail on http://tm5.sourceforge.net/. ”
-
AC1: 'Reply on CEC1', Sudhanshu Pandey, 06 Mar 2022
-
RC1: 'Comment on gmd-2021-339', Anonymous Referee #1, 02 Dec 2021
The authors present an implementation of a so-called physical parallelization for variational flux inversions (PPVI): from a previously described PPVI aimed at carbon dioxide (CO2), they add developments to take into account the chemical reactivity of methane (CH4).
General comments
The developments described in this paper are particularly relevant since long-term methane inversions are now run by several teams and the issue of the trends in methane emissions by various types of sources is still under study. Nevertheless, I think the presentation is too sloppy as it is: the work must be better introduced and described. technically, several notations are unclear in the mathematical description. Moreover, although I am not an native English speaker, I think the writing has to be improved.
The introduction to the paper is off the mark. It remains very general and not precise enough on variational inversion. Some examples: the state vector, in most inversions, not only consists in emissions but also includes initial conditions or boundary conditions for area-limited domains; the analytical approach is alluded to compared to the variational one but it is never explicitly stated that the analytical approach cannot be used for non-linear problems (which may be the case with reactive species); conversely, it is not stated that the variational approach does not provide full posterior uncertainties as a by-product of an inversion (either none are obtained, or truncated ones). I think the introduction does not target the right readers: people who may be interested in PPVI already know the whys and hows of analytical and variational inversion. It would be more useful to clearly state in which cases and why this implementation of PPVI is interesting e.g. for variational inversions of reactive species at scales at which chemistry is to be taken into account but the precision is not so important i.e. not for non-linear chemistry.
In Section 2 Physical parallelization for variational inversions, it must be made very clear which parts are the general or Chevallier (2013) developments and which are specific to this work and therefore, to methane. It should make it possible to understand whether the developments are also applicable to other species (e.g. CO). A discussion on the assumptions required to apply this PPVI and its limitations is necessary, either in this part or in the introduction or in the discussion.
In Section 3PPVI Performance test, not all the information required to understand (and reproduce) the simulations are available. The main information missing is how the posterior uncertainties are obtained: which approach is used? What are the assumptions? Even the simple approach of using Congrad as a minimizer and using the uncertainties obtained with a truncation requires to specify at least this truncation threshold and how it is expected to affect the resulting uncertainties estimates.
Specific comments
- Section 2 Physical parallelization for variational inversions:
- p.3 l.93 in Eq.2: it should be H* and not HT - or the assumptions which make HT equal to H* should be stated. It would also be safer to add a bracket: H* [ R-1(H(xi)-y) ]
- p.3 l.96: same remark as above: H* is the adjoint, if HT is used, it means that the problem is linear, which must be stated explicitly from the beginning.
- p.4 l.116: why this conversion factor?
- p.4 l.119 in Eq. 5: the notation for H changes suddenly from italics i.e. an operator with no particular characteristics to bold i.e. a matrix (probably): see above for the issue about H and its various spin-offs being linear or not and adjust notations accordingly.
- p.5 l.134 in Eq 6 and seq.: the notation * for the adjoint appears here: please make this consistent with the beginning of the Section. Moreover, H is bold so probably a matrix i.e. for a linear problem so that * and transpose are the same: this is not clear at all for the reader.
- Section 3 PPVI performance test
- p.7 l.210-211: the difference between both posterior emissions must be compared to the difference with the prior to be said to be small - or not. Better still, the uncertainties on the three estimates must be taken into account for such a comparison.
- p.7 l.213-214: same remark as above for the regions: how much is the deviation from the prior compared to the 5% between the two inversions? What about the uncertainties on the emission estimates?
- p.7 l.214: the posterior uncertainty is alluded to here but nowhere is it stated how it is computed. Since the full posterior uncertainties are not a by-product of the variational inversion, the way they are computed must be described (truncated from Congrad? ensemble? Monte-Carlo? other method?).
- p.7 l.215 seq.: I guess the correlation coefficient used here is simply the correlation of the time series. There are other characteristics of the inter-annual variability which could be interesting to look at e.g. are the uncertainties the same?
- p.8 l.233-234: what about the uncertainties? Without an explanation on how they are computed, the times given here are read as times for one inversion and may therefore be a lot smaller than what is actually required to get the full range of results (i.e. emission estimates + uncertainties).
- p.9 l.263 seq.: the specification of the OH fields is one of the main issues for methane inversions today, particularly as the vertical distribution of OH is crucial when using satellite data. A sink defined as simply as suggested here (even with an annual change) does not really solve the scientific issue. The optimization of the sink, as described in Section 4.3 is one of the possible ways forward.
- Figure 1
It would be useful to distinguish between the general (CO2) PPVI and the elements which are particular to this work i.e. the CH4. for example, the sink does not appear in this figure. Please also check the consistency of the notations (matrices, operators, vectors,...).- Figures 5, 6 and 7
How are the uncertainties obtained? Does a 2-sigma interval make sense?Technical corrections
Throughout the text, "a priori" and "a posteriori" are used: shouldn't it be "prior" and "posterior" instead?There are many writing mistakes, such as sentences where words are missing (e.g. p.7 l.194: "the PPVI results are good agreement with the results from serial") or superfluous words remain: the text must be re-read carefully by the authors before being checked by a native speaker.
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The authors present an implementation of a so-called physical parallelization for variational flux inversions (PPVI): from a previously described PPVI aimed at carbon dioxide (CO2), they add developments to take into account the chemical reactivity of methane (CH4).
General comments
The developments described in this paper are particularly relevant since long-term methane inversions are now run by several teams and the issue of the trends in methane emissions by various types of sources is still under study. Nevertheless, I think the presentation is too sloppy as it is: the work must be better introduced and described. technically, several notations are unclear in the mathematical description. Moreover, although I am not an native English speaker, I think the writing has to be improved.
The introduction to the paper is off the mark. It remains very general and not precise enough on variational inversion. Some examples: the state vector, in most inversions, not only consists in emissions but also includes initial conditions or boundary conditions for area-limited domains; the analytical approach is alluded to compared to the variational one but it is never explicitly stated that the analytical approach cannot be used for non-linear problems (which may be the case with reactive species); conversely, it is not stated that the variational approach does not provide full posterior uncertainties as a by-product of an inversion (either none are obtained, or truncated ones). I think the introduction does not target the right readers: people who may be interested in PPVI already know the whys and hows of analytical and variational inversion. It would be more useful to clearly state in which cases and why this implementation of PPVI is interesting e.g. for variational inversions of reactive species at scales at which chemistry is to be taken into account but the precision is not so important i.e. not for non-linear chemistry.
In Section 2 Physical parallelization for variational inversions, it must be made very clear which parts are the general or Chevallier (2013) developments and which are specific to this work and therefore, to methane. It should make it possible to understand whether the developments are also applicable to other species (e.g. CO). A discussion on the assumptions required to apply this PPVI and its limitations is necessary, either in this part or in the introduction or in the discussion.
In Section 3PPVI Performance test, not all the information required to understand (and reproduce) the simulations are available. The main information missing is how the posterior uncertainties are obtained: which approach is used? What are the assumptions? Even the simple approach of using Congrad as a minimizer and using the uncertainties obtained with a truncation requires to specify at least this truncation threshold and how it is expected to affect the resulting uncertainties estimates.
Specific comments
- Section 2 Physical parallelization for variational inversions:
- p.3 l.93 in Eq.2: it should be H* and not HT - or the assumptions which make HT equal to H* should be stated. It would also be safer to add a bracket: H* [ R-1(H(xi)-y) ]
- p.3 l.96: same remark as above: H* is the adjoint, if HT is used, it means that the problem is linear, which must be stated explicitly from the beginning.
- p.4 l.116: why this conversion factor?
- p.4 l.119 in Eq. 5: the notation for H changes suddenly from italics i.e. an operator with no particular characteristics to bold i.e. a matrix (probably): see above for the issue about H and its various spin-offs being linear or not and adjust notations accordingly.
- p.5 l.134 in Eq 6 and seq.: the notation * for the adjoint appears here: please make this consistent with the beginning of the Section. Moreover, H is bold so probably a matrix i.e. for a linear problem so that * and transpose are the same: this is not clear at all for the reader.
- Section 3 PPVI performance test
- p.7 l.210-211: the difference between both posterior emissions must be compared to the difference with the prior to be said to be small - or not. Better still, the uncertainties on the three estimates must be taken into account for such a comparison.
- p.7 l.213-214: same remark as above for the regions: how much is the deviation from the prior compared to the 5% between the two inversions? What about the uncertainties on the emission estimates?
- p.7 l.214: the posterior uncertainty is alluded to here but nowhere is it stated how it is computed. Since the full posterior uncertainties are not a by-product of the variational inversion, the way they are computed must be described (truncated from Congrad? ensemble? Monte-Carlo? other method?).
- p.7 l.215 seq.: I guess the correlation coefficient used here is simply the correlation of the time series. There are other characteristics of the inter-annual variability which could be interesting to look at e.g. are the uncertainties the same?
- p.8 l.233-234: what about the uncertainties? Without an explanation on how they are computed, the times given here are read as times for one inversion and may therefore be a lot smaller than what is actually required to get the full range of results (i.e. emission estimates + uncertainties).
- p.9 l.263 seq.: the specification of the OH fields is one of the main issues for methane inversions today, particularly as the vertical distribution of OH is crucial when using satellite data. A sink defined as simply as suggested here (even with an annual change) does not really solve the scientific issue. The optimization of the sink, as described in Section 4.3 is one of the possible ways forward.
- Figure 1
It would be useful to distinguish between the general (CO2) PPVI and the elements which are particular to this work i.e. the CH4. for example, the sink does not appear in this figure. Please also check the consistency of the notations (matrices, operators, vectors,...).- Figures 5, 6 and 7
How are the uncertainties obtained? Does a 2-sigma interval make sense?Technical corrections
Throughout the text, "a priori" and "a posteriori" are used: shouldn't it be "prior" and "posterior" instead?There are many writing mistakes, such as sentences where words are missing (e.g. p.7 l.194: "the PPVI results are good agreement with the results from serial") or superfluous words remain: the text must be re-read carefully by the authors before being checked by a native speaker.
-
RC2: 'Comment on gmd-2021-339', Anonymous Referee #2, 07 Dec 2021
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-339/gmd-2021-339-RC2-supplement.pdf
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-339/gmd-2021-339-RC2-supplement.pdf
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
-
RC3: 'Comment on gmd-2021-339', Anonymous Referee #3, 12 Dec 2021
The authors have developed a kind of “window-splitting” scheme for a variational inverse analysis of atmospheric CH4, which can be performed by parallel computing. For a multi-decadal analysis of long-lived species such as CO2 and CH4, a variational inverse analysis would be time consuming even when a massive amount of computational resources are available. This is because a variational analysis is basically a serial computation algorithm, which requires iterative calculations. In this regard, the developed method is worthy of publication from GMD, though its basic idea is already published by Chevallier (2013). Before publication, however, the reviewer would like the authors to revise the manuscript considering comments described below.
It is difficult to follow the description of the scheme, whose major reason is that many matrices and vectors are not written in bold fonts. This is very confusing. Furthermore, the reviewer strongly recommend that the author should clearly describe what is new and different from the original scheme of Chevallier (2013).
Although the reviewer is not a native English speaker, the reviewer thinks that the English writing of the manuscript has much room to improve. Therefore, a native check is also recommended.
The authors claim that the developed scheme is effective for a long-term inverse analysis in terms of wall clack time. The reviewer has no doubt about it, but would like the authors to discuss its relative effectiveness comparing with other approaches. For instance, a MPI parallelization (much more scalable parallelization than OpenMP) on the transport model could also shorten the wall clack time.
Specific comments:
L11: “variational (4DVAR)” => “four-dimensional variational (4DVAR)”
L21: “by a factor of 5” its computational effectiveness should be also described. How much computational resources are increased?
L25: “CAMS (Copernicus Atmosphere Monitoring Service)” => “Copernicus Atmosphere Monitoring Service (CAMS)”
L39: “CTM (chemical transport model)” => “a chemical transport model (CTM)”
L42: Which is “this study”, the study by the authors or the one by Saunois et al.?
Maybe it is the latter, but it should be clarified for more general readers.
L56: “representing the sensitivities by a statistical ensemble” is not clear.
L61: “is obtained” would be better than “is computed”
L67: “computational efficiency” might be inappropriate, because the computational resources used in the inversion were increased.
L76: Chevallier (2013) named the scheme as “physical parallelization (PP)”, but the authors here named their scheme as “physical parallelization for variational inversion (PPVI)”. Are they the same? If that is the case, it would be better to use PP rather than PPVI to respect the original idea of Chevallier (2013).
Somewhere in Introduction: More introduction about CH4 inverse analyses other than Saunois et al. (2020) would be beneficial.
L87: transpose “T” is missing. “(x-xa)B-1(x-xa)” => “(x-xa)TB-1(x-xa)”, “(H(x) - y)R-1(H(x) - y)” => “(H(x) - y)TR-1(H(x) - y)”
L89 and elsewhere: “In here” => “Here”
L89: “the a” => “the”
L118: Why can the CTM that calculates the initial mole fraction fields be performed at the coarser resolution?
L119: What is the “methane perturbation”?
L116: Please describe how the mole fraction conversion factor (=0.361) is derived.
Eqs. (3)-(5): Are c0, xi, ni scalars or vectors? If they are scalars, are they the global totals?
L124: Please elaborate the sufficiency of the e-folding decay function, because this might be the new and different from the original scheme of Chevallier (2013).
L138: What is “the adjoint test”? Please elaborate it.
L152: “uniform” is better than “unity”, isn’t it?
L192-193: Are “78 ppb” and “28 ppb” the results of serial or PPVI?
L203-204: “For both inversions, the good fit …. a gradient reduction of 1000 is sufficient” The fit to the observations cannot be used to determine the sufficiency of the convergence.
L207-208: “The parallelized … in the serial inversion” is not clear.
Section 3.1: One may want to see differences of more small scales (e.g., flux patterns, seasonal cycles).
Section 4.1: This section would be better to be moved to Introduction.
L257: “if future” => “in future”?
L278: Please spell out “SWIR” and TIR, because they appear first here.
L280-281: “These studies … small in an inversion.” Is not clear.
L282: Does “the methane lifetimes in the S operator would be scaled in each iteration” mean that S is included in the control variables?
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The authors have developed a kind of “window-splitting” scheme for a variational inverse analysis of atmospheric CH4, which can be performed by parallel computing. For a multi-decadal analysis of long-lived species such as CO2 and CH4, a variational inverse analysis would be time consuming even when a massive amount of computational resources are available. This is because a variational analysis is basically a serial computation algorithm, which requires iterative calculations. In this regard, the developed method is worthy of publication from GMD, though its basic idea is already published by Chevallier (2013). Before publication, however, the reviewer would like the authors to revise the manuscript considering comments described below.
It is difficult to follow the description of the scheme, whose major reason is that many matrices and vectors are not written in bold fonts. This is very confusing. Furthermore, the reviewer strongly recommend that the author should clearly describe what is new and different from the original scheme of Chevallier (2013).
Although the reviewer is not a native English speaker, the reviewer thinks that the English writing of the manuscript has much room to improve. Therefore, a native check is also recommended.
The authors claim that the developed scheme is effective for a long-term inverse analysis in terms of wall clack time. The reviewer has no doubt about it, but would like the authors to discuss its relative effectiveness comparing with other approaches. For instance, a MPI parallelization (much more scalable parallelization than OpenMP) on the transport model could also shorten the wall clack time.
Specific comments:
L11: “variational (4DVAR)” => “four-dimensional variational (4DVAR)”
L21: “by a factor of 5” its computational effectiveness should be also described. How much computational resources are increased?
L25: “CAMS (Copernicus Atmosphere Monitoring Service)” => “Copernicus Atmosphere Monitoring Service (CAMS)”
L39: “CTM (chemical transport model)” => “a chemical transport model (CTM)”
L42: Which is “this study”, the study by the authors or the one by Saunois et al.?
Maybe it is the latter, but it should be clarified for more general readers.
L56: “representing the sensitivities by a statistical ensemble” is not clear.
L61: “is obtained” would be better than “is computed”
L67: “computational efficiency” might be inappropriate, because the computational resources used in the inversion were increased.
L76: Chevallier (2013) named the scheme as “physical parallelization (PP)”, but the authors here named their scheme as “physical parallelization for variational inversion (PPVI)”. Are they the same? If that is the case, it would be better to use PP rather than PPVI to respect the original idea of Chevallier (2013).
Somewhere in Introduction: More introduction about CH4 inverse analyses other than Saunois et al. (2020) would be beneficial.
L87: transpose “T” is missing. “(x-xa)B-1(x-xa)” => “(x-xa)TB-1(x-xa)”, “(H(x) - y)R-1(H(x) - y)” => “(H(x) - y)TR-1(H(x) - y)”
L89 and elsewhere: “In here” => “Here”
L89: “the a” => “the”
L118: Why can the CTM that calculates the initial mole fraction fields be performed at the coarser resolution?
L119: What is the “methane perturbation”?
L116: Please describe how the mole fraction conversion factor (=0.361) is derived.
Eqs. (3)-(5): Are c0, xi, ni scalars or vectors? If they are scalars, are they the global totals?
L124: Please elaborate the sufficiency of the e-folding decay function, because this might be the new and different from the original scheme of Chevallier (2013).
L138: What is “the adjoint test”? Please elaborate it.
L152: “uniform” is better than “unity”, isn’t it?
L192-193: Are “78 ppb” and “28 ppb” the results of serial or PPVI?
L203-204: “For both inversions, the good fit …. a gradient reduction of 1000 is sufficient” The fit to the observations cannot be used to determine the sufficiency of the convergence.
L207-208: “The parallelized … in the serial inversion” is not clear.
Section 3.1: One may want to see differences of more small scales (e.g., flux patterns, seasonal cycles).
Section 4.1: This section would be better to be moved to Introduction.
L257: “if future” => “in future”?
L278: Please spell out “SWIR” and TIR, because they appear first here.
L280-281: “These studies … small in an inversion.” Is not clear.
L282: Does “the methane lifetimes in the S operator would be scaled in each iteration” mean that S is included in the control variables?
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
We thank the three anonymous reviewers for their detailed comments. These comments have led to a significant improvement in the quality and presentation of our manuscript. Our responses to the comments of all the reviewers are given in the attached document.
Status: closed
-
CEC1: 'Comment on gmd-2021-339', Juan Antonio Añel, 19 Nov 2021
Dear authors,
After checking your manuscript, it has come to our attention that it does not comply with our Code and Data Policy.
https://www.geoscientific-model-development.net/policies/code_and_data_policy.html
You have archived the TM5 code in SourceForge. However, SourceForge is not a suitable repository for long-term archival and publishing. Therefore, please, publish your code in one of the appropriate repositories.
In this way, before the Discussions stage is closed, you must reply to this comment with the link to the repository for the code and the corresponding DOI.
Also, you must include in a potential reviewed version of your manuscript the modified 'Code and Data Availability' section and the DOI of the code. Also, in the SourceForge repository, I have not seen a license listed for TM5. If you do not include a license, the code continues to be your property and can not be used by others. Therefore, when uploading the model's code to the repository, you could want to choose a free software/open-source (FLOSS) license. We recommend the GPLv3. You only need to include the file 'https://www.gnu.org/licenses/gpl-3.0.txt' as LICENSE.txt with your code. Also, you can choose other options that Zenodo provides: GPLv2, Apache License, MIT License, etc.Finally, in the same way, it would be good if you archive and make available a snapshot of the exact file with the methane data used in your work, with its corresponding DOI. We can not consider the NOAA site as a permanent, long-term link.
Please, reply as soon as possible to this comment with the link for it so that it is available for the peer-review process, as it should be.Regards,
Juan A. Añel
Geosc. Mod. Dev. Executive Editor
-
AC1: 'Reply on CEC1', Sudhanshu Pandey, 06 Mar 2022
Dear Editor,
We have put the TM5-4DVAR-PP code developed in this study as well as the input data used in the simulations (surface observations, initial mole fraction fields) on Zenodo:
“Data Availability. NOAA ESRL methane observations used in this study are available on Zenodo in the input folder of the TM5-4DVAR-PP code (https://doi.org/10.5281/zenodo.6326373, Pandey et al., 2022).
Code availability. The TM5-4DVAR-PP version 1.0-beta-1 code used in this study for the simulations can be downloaded from Zenodo (https://doi.org/10.5281/zenodo.6326373, Pandey et al., 2022). The TM5 model is described in detail on http://tm5.sourceforge.net/. ”
-
AC1: 'Reply on CEC1', Sudhanshu Pandey, 06 Mar 2022
-
RC1: 'Comment on gmd-2021-339', Anonymous Referee #1, 02 Dec 2021
The authors present an implementation of a so-called physical parallelization for variational flux inversions (PPVI): from a previously described PPVI aimed at carbon dioxide (CO2), they add developments to take into account the chemical reactivity of methane (CH4).
General comments
The developments described in this paper are particularly relevant since long-term methane inversions are now run by several teams and the issue of the trends in methane emissions by various types of sources is still under study. Nevertheless, I think the presentation is too sloppy as it is: the work must be better introduced and described. technically, several notations are unclear in the mathematical description. Moreover, although I am not an native English speaker, I think the writing has to be improved.
The introduction to the paper is off the mark. It remains very general and not precise enough on variational inversion. Some examples: the state vector, in most inversions, not only consists in emissions but also includes initial conditions or boundary conditions for area-limited domains; the analytical approach is alluded to compared to the variational one but it is never explicitly stated that the analytical approach cannot be used for non-linear problems (which may be the case with reactive species); conversely, it is not stated that the variational approach does not provide full posterior uncertainties as a by-product of an inversion (either none are obtained, or truncated ones). I think the introduction does not target the right readers: people who may be interested in PPVI already know the whys and hows of analytical and variational inversion. It would be more useful to clearly state in which cases and why this implementation of PPVI is interesting e.g. for variational inversions of reactive species at scales at which chemistry is to be taken into account but the precision is not so important i.e. not for non-linear chemistry.
In Section 2 Physical parallelization for variational inversions, it must be made very clear which parts are the general or Chevallier (2013) developments and which are specific to this work and therefore, to methane. It should make it possible to understand whether the developments are also applicable to other species (e.g. CO). A discussion on the assumptions required to apply this PPVI and its limitations is necessary, either in this part or in the introduction or in the discussion.
In Section 3PPVI Performance test, not all the information required to understand (and reproduce) the simulations are available. The main information missing is how the posterior uncertainties are obtained: which approach is used? What are the assumptions? Even the simple approach of using Congrad as a minimizer and using the uncertainties obtained with a truncation requires to specify at least this truncation threshold and how it is expected to affect the resulting uncertainties estimates.
Specific comments
- Section 2 Physical parallelization for variational inversions:
- p.3 l.93 in Eq.2: it should be H* and not HT - or the assumptions which make HT equal to H* should be stated. It would also be safer to add a bracket: H* [ R-1(H(xi)-y) ]
- p.3 l.96: same remark as above: H* is the adjoint, if HT is used, it means that the problem is linear, which must be stated explicitly from the beginning.
- p.4 l.116: why this conversion factor?
- p.4 l.119 in Eq. 5: the notation for H changes suddenly from italics i.e. an operator with no particular characteristics to bold i.e. a matrix (probably): see above for the issue about H and its various spin-offs being linear or not and adjust notations accordingly.
- p.5 l.134 in Eq 6 and seq.: the notation * for the adjoint appears here: please make this consistent with the beginning of the Section. Moreover, H is bold so probably a matrix i.e. for a linear problem so that * and transpose are the same: this is not clear at all for the reader.
- Section 3 PPVI performance test
- p.7 l.210-211: the difference between both posterior emissions must be compared to the difference with the prior to be said to be small - or not. Better still, the uncertainties on the three estimates must be taken into account for such a comparison.
- p.7 l.213-214: same remark as above for the regions: how much is the deviation from the prior compared to the 5% between the two inversions? What about the uncertainties on the emission estimates?
- p.7 l.214: the posterior uncertainty is alluded to here but nowhere is it stated how it is computed. Since the full posterior uncertainties are not a by-product of the variational inversion, the way they are computed must be described (truncated from Congrad? ensemble? Monte-Carlo? other method?).
- p.7 l.215 seq.: I guess the correlation coefficient used here is simply the correlation of the time series. There are other characteristics of the inter-annual variability which could be interesting to look at e.g. are the uncertainties the same?
- p.8 l.233-234: what about the uncertainties? Without an explanation on how they are computed, the times given here are read as times for one inversion and may therefore be a lot smaller than what is actually required to get the full range of results (i.e. emission estimates + uncertainties).
- p.9 l.263 seq.: the specification of the OH fields is one of the main issues for methane inversions today, particularly as the vertical distribution of OH is crucial when using satellite data. A sink defined as simply as suggested here (even with an annual change) does not really solve the scientific issue. The optimization of the sink, as described in Section 4.3 is one of the possible ways forward.
- Figure 1
It would be useful to distinguish between the general (CO2) PPVI and the elements which are particular to this work i.e. the CH4. for example, the sink does not appear in this figure. Please also check the consistency of the notations (matrices, operators, vectors,...).- Figures 5, 6 and 7
How are the uncertainties obtained? Does a 2-sigma interval make sense?Technical corrections
Throughout the text, "a priori" and "a posteriori" are used: shouldn't it be "prior" and "posterior" instead?There are many writing mistakes, such as sentences where words are missing (e.g. p.7 l.194: "the PPVI results are good agreement with the results from serial") or superfluous words remain: the text must be re-read carefully by the authors before being checked by a native speaker.
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The authors present an implementation of a so-called physical parallelization for variational flux inversions (PPVI): from a previously described PPVI aimed at carbon dioxide (CO2), they add developments to take into account the chemical reactivity of methane (CH4).
General comments
The developments described in this paper are particularly relevant since long-term methane inversions are now run by several teams and the issue of the trends in methane emissions by various types of sources is still under study. Nevertheless, I think the presentation is too sloppy as it is: the work must be better introduced and described. technically, several notations are unclear in the mathematical description. Moreover, although I am not an native English speaker, I think the writing has to be improved.
The introduction to the paper is off the mark. It remains very general and not precise enough on variational inversion. Some examples: the state vector, in most inversions, not only consists in emissions but also includes initial conditions or boundary conditions for area-limited domains; the analytical approach is alluded to compared to the variational one but it is never explicitly stated that the analytical approach cannot be used for non-linear problems (which may be the case with reactive species); conversely, it is not stated that the variational approach does not provide full posterior uncertainties as a by-product of an inversion (either none are obtained, or truncated ones). I think the introduction does not target the right readers: people who may be interested in PPVI already know the whys and hows of analytical and variational inversion. It would be more useful to clearly state in which cases and why this implementation of PPVI is interesting e.g. for variational inversions of reactive species at scales at which chemistry is to be taken into account but the precision is not so important i.e. not for non-linear chemistry.
In Section 2 Physical parallelization for variational inversions, it must be made very clear which parts are the general or Chevallier (2013) developments and which are specific to this work and therefore, to methane. It should make it possible to understand whether the developments are also applicable to other species (e.g. CO). A discussion on the assumptions required to apply this PPVI and its limitations is necessary, either in this part or in the introduction or in the discussion.
In Section 3PPVI Performance test, not all the information required to understand (and reproduce) the simulations are available. The main information missing is how the posterior uncertainties are obtained: which approach is used? What are the assumptions? Even the simple approach of using Congrad as a minimizer and using the uncertainties obtained with a truncation requires to specify at least this truncation threshold and how it is expected to affect the resulting uncertainties estimates.
Specific comments
- Section 2 Physical parallelization for variational inversions:
- p.3 l.93 in Eq.2: it should be H* and not HT - or the assumptions which make HT equal to H* should be stated. It would also be safer to add a bracket: H* [ R-1(H(xi)-y) ]
- p.3 l.96: same remark as above: H* is the adjoint, if HT is used, it means that the problem is linear, which must be stated explicitly from the beginning.
- p.4 l.116: why this conversion factor?
- p.4 l.119 in Eq. 5: the notation for H changes suddenly from italics i.e. an operator with no particular characteristics to bold i.e. a matrix (probably): see above for the issue about H and its various spin-offs being linear or not and adjust notations accordingly.
- p.5 l.134 in Eq 6 and seq.: the notation * for the adjoint appears here: please make this consistent with the beginning of the Section. Moreover, H is bold so probably a matrix i.e. for a linear problem so that * and transpose are the same: this is not clear at all for the reader.
- Section 3 PPVI performance test
- p.7 l.210-211: the difference between both posterior emissions must be compared to the difference with the prior to be said to be small - or not. Better still, the uncertainties on the three estimates must be taken into account for such a comparison.
- p.7 l.213-214: same remark as above for the regions: how much is the deviation from the prior compared to the 5% between the two inversions? What about the uncertainties on the emission estimates?
- p.7 l.214: the posterior uncertainty is alluded to here but nowhere is it stated how it is computed. Since the full posterior uncertainties are not a by-product of the variational inversion, the way they are computed must be described (truncated from Congrad? ensemble? Monte-Carlo? other method?).
- p.7 l.215 seq.: I guess the correlation coefficient used here is simply the correlation of the time series. There are other characteristics of the inter-annual variability which could be interesting to look at e.g. are the uncertainties the same?
- p.8 l.233-234: what about the uncertainties? Without an explanation on how they are computed, the times given here are read as times for one inversion and may therefore be a lot smaller than what is actually required to get the full range of results (i.e. emission estimates + uncertainties).
- p.9 l.263 seq.: the specification of the OH fields is one of the main issues for methane inversions today, particularly as the vertical distribution of OH is crucial when using satellite data. A sink defined as simply as suggested here (even with an annual change) does not really solve the scientific issue. The optimization of the sink, as described in Section 4.3 is one of the possible ways forward.
- Figure 1
It would be useful to distinguish between the general (CO2) PPVI and the elements which are particular to this work i.e. the CH4. for example, the sink does not appear in this figure. Please also check the consistency of the notations (matrices, operators, vectors,...).- Figures 5, 6 and 7
How are the uncertainties obtained? Does a 2-sigma interval make sense?Technical corrections
Throughout the text, "a priori" and "a posteriori" are used: shouldn't it be "prior" and "posterior" instead?There are many writing mistakes, such as sentences where words are missing (e.g. p.7 l.194: "the PPVI results are good agreement with the results from serial") or superfluous words remain: the text must be re-read carefully by the authors before being checked by a native speaker.
-
RC2: 'Comment on gmd-2021-339', Anonymous Referee #2, 07 Dec 2021
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-339/gmd-2021-339-RC2-supplement.pdf
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2021-339/gmd-2021-339-RC2-supplement.pdf
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
-
RC3: 'Comment on gmd-2021-339', Anonymous Referee #3, 12 Dec 2021
The authors have developed a kind of “window-splitting” scheme for a variational inverse analysis of atmospheric CH4, which can be performed by parallel computing. For a multi-decadal analysis of long-lived species such as CO2 and CH4, a variational inverse analysis would be time consuming even when a massive amount of computational resources are available. This is because a variational analysis is basically a serial computation algorithm, which requires iterative calculations. In this regard, the developed method is worthy of publication from GMD, though its basic idea is already published by Chevallier (2013). Before publication, however, the reviewer would like the authors to revise the manuscript considering comments described below.
It is difficult to follow the description of the scheme, whose major reason is that many matrices and vectors are not written in bold fonts. This is very confusing. Furthermore, the reviewer strongly recommend that the author should clearly describe what is new and different from the original scheme of Chevallier (2013).
Although the reviewer is not a native English speaker, the reviewer thinks that the English writing of the manuscript has much room to improve. Therefore, a native check is also recommended.
The authors claim that the developed scheme is effective for a long-term inverse analysis in terms of wall clack time. The reviewer has no doubt about it, but would like the authors to discuss its relative effectiveness comparing with other approaches. For instance, a MPI parallelization (much more scalable parallelization than OpenMP) on the transport model could also shorten the wall clack time.
Specific comments:
L11: “variational (4DVAR)” => “four-dimensional variational (4DVAR)”
L21: “by a factor of 5” its computational effectiveness should be also described. How much computational resources are increased?
L25: “CAMS (Copernicus Atmosphere Monitoring Service)” => “Copernicus Atmosphere Monitoring Service (CAMS)”
L39: “CTM (chemical transport model)” => “a chemical transport model (CTM)”
L42: Which is “this study”, the study by the authors or the one by Saunois et al.?
Maybe it is the latter, but it should be clarified for more general readers.
L56: “representing the sensitivities by a statistical ensemble” is not clear.
L61: “is obtained” would be better than “is computed”
L67: “computational efficiency” might be inappropriate, because the computational resources used in the inversion were increased.
L76: Chevallier (2013) named the scheme as “physical parallelization (PP)”, but the authors here named their scheme as “physical parallelization for variational inversion (PPVI)”. Are they the same? If that is the case, it would be better to use PP rather than PPVI to respect the original idea of Chevallier (2013).
Somewhere in Introduction: More introduction about CH4 inverse analyses other than Saunois et al. (2020) would be beneficial.
L87: transpose “T” is missing. “(x-xa)B-1(x-xa)” => “(x-xa)TB-1(x-xa)”, “(H(x) - y)R-1(H(x) - y)” => “(H(x) - y)TR-1(H(x) - y)”
L89 and elsewhere: “In here” => “Here”
L89: “the a” => “the”
L118: Why can the CTM that calculates the initial mole fraction fields be performed at the coarser resolution?
L119: What is the “methane perturbation”?
L116: Please describe how the mole fraction conversion factor (=0.361) is derived.
Eqs. (3)-(5): Are c0, xi, ni scalars or vectors? If they are scalars, are they the global totals?
L124: Please elaborate the sufficiency of the e-folding decay function, because this might be the new and different from the original scheme of Chevallier (2013).
L138: What is “the adjoint test”? Please elaborate it.
L152: “uniform” is better than “unity”, isn’t it?
L192-193: Are “78 ppb” and “28 ppb” the results of serial or PPVI?
L203-204: “For both inversions, the good fit …. a gradient reduction of 1000 is sufficient” The fit to the observations cannot be used to determine the sufficiency of the convergence.
L207-208: “The parallelized … in the serial inversion” is not clear.
Section 3.1: One may want to see differences of more small scales (e.g., flux patterns, seasonal cycles).
Section 4.1: This section would be better to be moved to Introduction.
L257: “if future” => “in future”?
L278: Please spell out “SWIR” and TIR, because they appear first here.
L280-281: “These studies … small in an inversion.” Is not clear.
L282: Does “the methane lifetimes in the S operator would be scaled in each iteration” mean that S is included in the control variables?
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
The authors have developed a kind of “window-splitting” scheme for a variational inverse analysis of atmospheric CH4, which can be performed by parallel computing. For a multi-decadal analysis of long-lived species such as CO2 and CH4, a variational inverse analysis would be time consuming even when a massive amount of computational resources are available. This is because a variational analysis is basically a serial computation algorithm, which requires iterative calculations. In this regard, the developed method is worthy of publication from GMD, though its basic idea is already published by Chevallier (2013). Before publication, however, the reviewer would like the authors to revise the manuscript considering comments described below.
It is difficult to follow the description of the scheme, whose major reason is that many matrices and vectors are not written in bold fonts. This is very confusing. Furthermore, the reviewer strongly recommend that the author should clearly describe what is new and different from the original scheme of Chevallier (2013).
Although the reviewer is not a native English speaker, the reviewer thinks that the English writing of the manuscript has much room to improve. Therefore, a native check is also recommended.
The authors claim that the developed scheme is effective for a long-term inverse analysis in terms of wall clack time. The reviewer has no doubt about it, but would like the authors to discuss its relative effectiveness comparing with other approaches. For instance, a MPI parallelization (much more scalable parallelization than OpenMP) on the transport model could also shorten the wall clack time.
Specific comments:
L11: “variational (4DVAR)” => “four-dimensional variational (4DVAR)”
L21: “by a factor of 5” its computational effectiveness should be also described. How much computational resources are increased?
L25: “CAMS (Copernicus Atmosphere Monitoring Service)” => “Copernicus Atmosphere Monitoring Service (CAMS)”
L39: “CTM (chemical transport model)” => “a chemical transport model (CTM)”
L42: Which is “this study”, the study by the authors or the one by Saunois et al.?
Maybe it is the latter, but it should be clarified for more general readers.
L56: “representing the sensitivities by a statistical ensemble” is not clear.
L61: “is obtained” would be better than “is computed”
L67: “computational efficiency” might be inappropriate, because the computational resources used in the inversion were increased.
L76: Chevallier (2013) named the scheme as “physical parallelization (PP)”, but the authors here named their scheme as “physical parallelization for variational inversion (PPVI)”. Are they the same? If that is the case, it would be better to use PP rather than PPVI to respect the original idea of Chevallier (2013).
Somewhere in Introduction: More introduction about CH4 inverse analyses other than Saunois et al. (2020) would be beneficial.
L87: transpose “T” is missing. “(x-xa)B-1(x-xa)” => “(x-xa)TB-1(x-xa)”, “(H(x) - y)R-1(H(x) - y)” => “(H(x) - y)TR-1(H(x) - y)”
L89 and elsewhere: “In here” => “Here”
L89: “the a” => “the”
L118: Why can the CTM that calculates the initial mole fraction fields be performed at the coarser resolution?
L119: What is the “methane perturbation”?
L116: Please describe how the mole fraction conversion factor (=0.361) is derived.
Eqs. (3)-(5): Are c0, xi, ni scalars or vectors? If they are scalars, are they the global totals?
L124: Please elaborate the sufficiency of the e-folding decay function, because this might be the new and different from the original scheme of Chevallier (2013).
L138: What is “the adjoint test”? Please elaborate it.
L152: “uniform” is better than “unity”, isn’t it?
L192-193: Are “78 ppb” and “28 ppb” the results of serial or PPVI?
L203-204: “For both inversions, the good fit …. a gradient reduction of 1000 is sufficient” The fit to the observations cannot be used to determine the sufficiency of the convergence.
L207-208: “The parallelized … in the serial inversion” is not clear.
Section 3.1: One may want to see differences of more small scales (e.g., flux patterns, seasonal cycles).
Section 4.1: This section would be better to be moved to Introduction.
L257: “if future” => “in future”?
L278: Please spell out “SWIR” and TIR, because they appear first here.
L280-281: “These studies … small in an inversion.” Is not clear.
L282: Does “the methane lifetimes in the S operator would be scaled in each iteration” mean that S is included in the control variables?
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
-
AC2: 'Author response to the reviewer comments', Sudhanshu Pandey, 06 Mar 2022
We thank the three anonymous reviewers for their detailed comments. These comments have led to a significant improvement in the quality and presentation of our manuscript. Our responses to the comments of all the reviewers are given in the attached document.
Sudhanshu Pandey et al.
Sudhanshu Pandey et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 367 | 112 | 22 | 501 | 6 | 8 |
- HTML: 367
- PDF: 112
- XML: 22
- Total: 501
- BibTeX: 6
- EndNote: 8
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1