the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 21 Nov 2023
Implementation and evaluation of diabatic advection in the Lagrangian transport model MPTRAC 2.6
Abstract. Diabatic transport schemes with hybrid zeta coordinates, which follow isentropes in the stratosphere, are known to greatly improve Lagrangian transport calculations compared to the kinematic approach. However, some Lagrangian transport calculations with a diabatic approach, such as the Chemical Lagrangian Transport Model of the Atmosphere (CLaMS), show low computational performance on modern high-performance computing (HPC) architectures. Here, we implemented and evaluated a new diabatic transport scheme in the Massive-Parallel Trajectory Calculations (MPTRAC) model. While MPTRAC effectively exploits modern HPC architectures, it was previously limited to kinematic trajectories on pressure coordinates. The extended modelling approach now enables the use of either kinematic or diabatic vertical velocities and the coupling of different MPTRAC modules based on pressure or hybrid zeta coordinates.
The evaluation of the new transport scheme in MPTRAC shows that after 90-day forward calculations distributions of air parcels in the upper troposphere and lower stratosphere (UTLS) are almost identical for MPTRAC and CLaMS. No significant bias between the two Lagrangian models was found. Furthermore, after one day, internal uncertainties (e.g., due to interpolation or the numerical integration method) in the Lagrangian transport calculations are at least one order of magnitude smaller than external uncertainties (e.g., from reanalysis selection or downsampling of ERA5). Differences between trajectories using either CLaMS or MPTRAC are on the order of the combined internal uncertainties within MPTRAC. Since the largest systematic differences are caused by the reanalysis and the vertical velocity (diabatic vs. kinematic) the results support the development efforts for trajectory codes that can access the full resolution of ERA5 in combination with diabatic vertical velocities. This work is part of a larger effort to adapt Lagrangian transport in state-of-the-art models such as CLaMS and MPTRAC to current and future HPC architectures and exascale applications.
- Preprint
(9847 KB) - Metadata XML
- BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on gmd-2023-214', Anonymous Referee #1, 03 Jan 2024
This article describes the adaptation of a Lagrangian transport model, MPTRAC, to use diabatic velocity coordinates, the use of which are beneficial to Lagrangian transport calculations. The main component of this is an interpolation scheme to compute these diabatic velocities at particle locations, which requires a vertical search.
The adapted model is benchmarked against another model, CLaMS, using a sensible set of case studies, where the errors are compared to those within MPTRAC upon variation of parameters. This is a good practical approach.
My main point about the paper is that it is motivated by performance on next generation HPC, but nothing is said about performance in this paper. I appreciate that performance optimisation is a second step after checking that the numerical approach is sufficient (that's the main content of this paper). However the general reader would appreciate a bit more discussion. So,
1. What is it about MPTRAC that makes it expected to perform better than CLaMS on GPUs etc? Are these properties preserved by the new adaptation? Can this be argued from a performance model?
2. Can you provide a performance profile of the new adaptation versus the standard MPTRAC to see where potential bottlenecks have arisen, and discuss (if necessary) how they would be addressed in performance optimisation?
I think that addressing these questions would provide a better documentation of this stage of the development of MPTRAC.
Citation: https://doi.org/10.5194/gmd-2023-214-RC1 -
RC2: 'Comment on gmd-2023-214', Anonymous Referee #2, 10 Jan 2024
The paper by Clemens et al. describes a model development, more precise the implementation of a vertical diabatic velocity, including a new vertical hybrid zeta coordinate into the massive parallel trajectory model MPTRAC. The paper fits into the scope of GMD. The model results are evaluated in the paper against the CLaMS model. Further, a sensitivity study with respect to the driving data (ERA-interim, ERA, downscaled data), physical processes (vertical velocity, diffusion) and other processes is performed to estimate the uncertainty. The paper presents the implementation of a relevant modelling approach (diabatic velocity) as used in CLaMS to another trajectory model suitable for massive parallel architectures. Although the applied concept (diabatic velocity) itself is not novel, its implementation is a step to further make model concepts sustainable and applicable on modern computer architectures.
General comments:
The abstract is well written and provides a concise and short summary of the work. The paper is structured well. The main text (result and conclusion section), however, is in parts hard to read. Further, the publication is rather long and thus should be shortened. The publication should be excepted for publication after revision.
Specific comments:
Page 2, line 41: What implies an improved “interoperability” between CLaMS and MPTRAC? Does this mean, that the advection of trajectories is calculated by MPTRAC and the mixing process by CLaMS? As far as I know, in CLaMS, the calculation of mixing is started by gathering the trajectories onto one CPU. If that is correct, doesn’t this step slow down the performance of MPTRAC? What is the “coupled mode” in MPTRAC?
Page 4, line 94: .. model differences are evaluated. Better say: ..model results are evaluated.
Page 5, line 126: Formulae 1 and the corresponding text do not match. The zeta coordinate at the sigma_r-level is the last value of a smooth transformation to potential temperature and not the beginning of the transformation. Please correct the sentence.
Page 5, Eq. 2: Is the last term of Eq. 2 correct? I mean that the inner derivation of the sinus function (of Eq. 1) should be:
-sigma_dot(p)/(1-sigma_r)
Page 5, line 132: To my knowledge, diabatic heating rates are also calculated from convective and diffusion tendencies. Only in the stratosphere, radiative heating rates dominate.
Page 5, line 138: You state that there is a need to smooth zeta profiles, that are not monotonic with height. My question is: Did you smooth the zeta profiles after you calculated them with Eq.1 ? If yes, how can you guarantee an exact transformation from zeta to pressure and vice versa? Don’t you violate equation 1 then?
Page 5, line 139: You mention that “many processes in the troposphere are not diabatic (e.g. convection)”. How much is “many” and why is this statement necessary? To my knowledge, convection is a diabatic process in the troposphere.
Chapter 2.1.3: The chapter on “interpolation” confuses me. The more I read the less I understand. Please, see my specific remarks below:
Page 6, line 166: “Interpolation with positions given only in eta coordinates therefore requires additional consideration”. You should shortly describe, what the additional considerations are.
Page 6, line 168: You perform a time interpolation locally for each air parcel in MPTRAC, and in CLaMS you interpolate the wind field globally for the time step procedure. Maybe there is some information missing in this sentence? I do not understand what you want to say.
Page 6: line 175: Please reformulate the sentence beginning with “At the beginning of ...” to for instance: “In CLaMS at first, the interpolation in time is performed”.
Page 7, Figure 1: A figure should make a description in the text more clearly, however, the representation of 6 in my view identical cubes do not help understand, how the interpolation is performed. Moreover, the description of the left upper cube is “zeta”, with a head line “3-d data is in eta coordinates” (which seems to be the ECMWF data). This description confuses me. I would suggest to describe the procedure of interpolations in a bullet list and delete of the upper part of Fig. 1. Note: It would be easier to read figure 1 if it would be made larger. The same applies for Fig.2.
Page 7, line 190: “If a different box is used for re-interpolation..”. I would suggest to delete this part of the sentence, just say “Then, significant errors may...”
Page 8, line 199: each of these columns....
Page 9, line 228: The statement of this sentence is not clear: You “compare” the time interpolation for the air parcel in MPTRAC with the meteorological field as in CLaMS? A similar sentence can be found on page 6. Please reformulate the sentence.
Chapter 2.3.1: I hope that I have not overlooked it in the text, but what I found missing here, is that you explicitly state and emphasize that you evaluate a MPTRAC simulation with a CLaMS simulation. I note this, because it is mentioned as the first result in the abstract! Further, a reference uncertainty source (I guess it is the “Transport” uncertainty source) should clearly be mentioned, otherwise the uncertainties have no (physical) reference value.
Page 15, figure 3c middle: Can you explain the pronounced daily cycle in the lower stratosphere only for CLaMS-default, MPTRAC-bestfit, MPTRAC-default?
Page 15, figure 3 caption, “theupper”
Page 16, figure 4: Colors are difficult to differentiate on paper!
Page 16, caption fig. 4: Please explain, why it is acceptable to exclude these outliers. I wonder, if the estimation of a maximum value is meaningful at all, if you ignore outliers? Moreover, could you please describe, why the selected value of 1.5 of the inter-quartiles is an appropriate value? Is it possible to show a figure, where the outliers are included?
Page 19, line 362: Here you state for the first time in the description of Fig. 4-6 the “overall transport median”. In the figures 4-6 it is shown as the transport AHTDs or AVTDs. I hope I have not overlooked it elsewhere, but what I miss in this chapter is that you clearly mention the reference value to which all of the calculated uncertainties are related to. As the “transport” is defined as the difference between start and end point of the trajectory, I would not see it as an uncertainty quantity, rather a (physical) difference reference value, where uncertainties should be compared with.
Page 19, line 370: This statement is not represented in Fig.4-6. Please, include the respective values of uncertainty or a reference to another figure, where these results are presented. I think this is an important result, where you show, that your model development in MPTRAC is in accordance with CLaMS, which is a widely published and evaluated model!
Page 19, line 372: Is it really worth to mention differences in results due to compilation flags? The question, however, is: Are results still reliable or equal in a statistical sense, when model components or even a different computer architecture, compiler etc. is used?
Page 20, line 412: The maximum of the normalized mean AVTD for “diffusion” can be found in the tropopause region. What can be concluded from the differences in vertical structure of normalized AVTD of “diffusion” in comparison with mean AVTD? Why are the profiles for diffusion of mean AVTD and normalized AVTD not equal in shape, if the normalization is the mean vertical path length of CLaMS, e.g. one value?
Page 20, line 413: “largest source of uncertainty”, isn’t it the largest relative source?
Page 20, line 413: Afterwards is not the correct adverb in this sentence.
Page 21 fig 7: Please add “average AVTDs for MPTRAC scenarios (see also Table 2)” in the figure caption.
Page 22, line 432: Say: vertical transport deviations remain smaller...
Page 25, lines 459: You write that the convergence of air parcels (Fig. 10) near the tropopause and near the surface is a consequence of up- and downdrafts in the troposphere including the tropopause and surface as a transport barrier. Though, I would suggest that the convergence of parcels near the surface/tropopause seems to be a result of missing mixing and convective transports, as the troposphere is a well-mixed region. Please comment.
Page 25, line 468: You describe the green lines in Fig. 11a as contour lines of air parcel frequencies. Are they absolute number of parcels of the CLaMS default simulation? If yes, please specify this also in the Fig 11 caption.
Page 25, line 473: This sentence is unclear. What is meant by “patterns”, or “smoothed peaks”? green contours shrink?
Page 26, lines 490: What did you want to say with “climatological findings”? That more air parcels are found at higher altitude around latitude of 45° with ERA-interim? I would suggest to replace “climatological findings” with “the results by Ploeger et al.” Please comment.
Page 28, line 521: Say:.. differences between scenarios are small...
Page 29, line 525: “to assess the implementation of the zeta coordinate...”, shouldn’t it be “to evaluate the implemented zeta coordinate in MPTRAC, we conducted simulations using...”?
Chapter 4: The conclusion section is rather long and repeats the description of results in detail. I would suggest to shorten the conclusion section (more a summary and conclusion section) and focus on the main results of the evaluation of the new model and the main results of the analysis (scenarios, uncertainties, effect on parcel distribution ..)
Citation: https://doi.org/10.5194/gmd-2023-214-RC2 - AC1: 'Comment on gmd-2023-214', Jan Heinrich Clemens, 06 Mar 2024
Status: closed
-
RC1: 'Comment on gmd-2023-214', Anonymous Referee #1, 03 Jan 2024
This article describes the adaptation of a Lagrangian transport model, MPTRAC, to use diabatic velocity coordinates, the use of which are beneficial to Lagrangian transport calculations. The main component of this is an interpolation scheme to compute these diabatic velocities at particle locations, which requires a vertical search.
The adapted model is benchmarked against another model, CLaMS, using a sensible set of case studies, where the errors are compared to those within MPTRAC upon variation of parameters. This is a good practical approach.
My main point about the paper is that it is motivated by performance on next generation HPC, but nothing is said about performance in this paper. I appreciate that performance optimisation is a second step after checking that the numerical approach is sufficient (that's the main content of this paper). However the general reader would appreciate a bit more discussion. So,
1. What is it about MPTRAC that makes it expected to perform better than CLaMS on GPUs etc? Are these properties preserved by the new adaptation? Can this be argued from a performance model?
2. Can you provide a performance profile of the new adaptation versus the standard MPTRAC to see where potential bottlenecks have arisen, and discuss (if necessary) how they would be addressed in performance optimisation?
I think that addressing these questions would provide a better documentation of this stage of the development of MPTRAC.
Citation: https://doi.org/10.5194/gmd-2023-214-RC1 -
RC2: 'Comment on gmd-2023-214', Anonymous Referee #2, 10 Jan 2024
The paper by Clemens et al. describes a model development, more precise the implementation of a vertical diabatic velocity, including a new vertical hybrid zeta coordinate into the massive parallel trajectory model MPTRAC. The paper fits into the scope of GMD. The model results are evaluated in the paper against the CLaMS model. Further, a sensitivity study with respect to the driving data (ERA-interim, ERA, downscaled data), physical processes (vertical velocity, diffusion) and other processes is performed to estimate the uncertainty. The paper presents the implementation of a relevant modelling approach (diabatic velocity) as used in CLaMS to another trajectory model suitable for massive parallel architectures. Although the applied concept (diabatic velocity) itself is not novel, its implementation is a step to further make model concepts sustainable and applicable on modern computer architectures.
General comments:
The abstract is well written and provides a concise and short summary of the work. The paper is structured well. The main text (result and conclusion section), however, is in parts hard to read. Further, the publication is rather long and thus should be shortened. The publication should be excepted for publication after revision.
Specific comments:
Page 2, line 41: What implies an improved “interoperability” between CLaMS and MPTRAC? Does this mean, that the advection of trajectories is calculated by MPTRAC and the mixing process by CLaMS? As far as I know, in CLaMS, the calculation of mixing is started by gathering the trajectories onto one CPU. If that is correct, doesn’t this step slow down the performance of MPTRAC? What is the “coupled mode” in MPTRAC?
Page 4, line 94: .. model differences are evaluated. Better say: ..model results are evaluated.
Page 5, line 126: Formulae 1 and the corresponding text do not match. The zeta coordinate at the sigma_r-level is the last value of a smooth transformation to potential temperature and not the beginning of the transformation. Please correct the sentence.
Page 5, Eq. 2: Is the last term of Eq. 2 correct? I mean that the inner derivation of the sinus function (of Eq. 1) should be:
-sigma_dot(p)/(1-sigma_r)
Page 5, line 132: To my knowledge, diabatic heating rates are also calculated from convective and diffusion tendencies. Only in the stratosphere, radiative heating rates dominate.
Page 5, line 138: You state that there is a need to smooth zeta profiles, that are not monotonic with height. My question is: Did you smooth the zeta profiles after you calculated them with Eq.1 ? If yes, how can you guarantee an exact transformation from zeta to pressure and vice versa? Don’t you violate equation 1 then?
Page 5, line 139: You mention that “many processes in the troposphere are not diabatic (e.g. convection)”. How much is “many” and why is this statement necessary? To my knowledge, convection is a diabatic process in the troposphere.
Chapter 2.1.3: The chapter on “interpolation” confuses me. The more I read the less I understand. Please, see my specific remarks below:
Page 6, line 166: “Interpolation with positions given only in eta coordinates therefore requires additional consideration”. You should shortly describe, what the additional considerations are.
Page 6, line 168: You perform a time interpolation locally for each air parcel in MPTRAC, and in CLaMS you interpolate the wind field globally for the time step procedure. Maybe there is some information missing in this sentence? I do not understand what you want to say.
Page 6: line 175: Please reformulate the sentence beginning with “At the beginning of ...” to for instance: “In CLaMS at first, the interpolation in time is performed”.
Page 7, Figure 1: A figure should make a description in the text more clearly, however, the representation of 6 in my view identical cubes do not help understand, how the interpolation is performed. Moreover, the description of the left upper cube is “zeta”, with a head line “3-d data is in eta coordinates” (which seems to be the ECMWF data). This description confuses me. I would suggest to describe the procedure of interpolations in a bullet list and delete of the upper part of Fig. 1. Note: It would be easier to read figure 1 if it would be made larger. The same applies for Fig.2.
Page 7, line 190: “If a different box is used for re-interpolation..”. I would suggest to delete this part of the sentence, just say “Then, significant errors may...”
Page 8, line 199: each of these columns....
Page 9, line 228: The statement of this sentence is not clear: You “compare” the time interpolation for the air parcel in MPTRAC with the meteorological field as in CLaMS? A similar sentence can be found on page 6. Please reformulate the sentence.
Chapter 2.3.1: I hope that I have not overlooked it in the text, but what I found missing here, is that you explicitly state and emphasize that you evaluate a MPTRAC simulation with a CLaMS simulation. I note this, because it is mentioned as the first result in the abstract! Further, a reference uncertainty source (I guess it is the “Transport” uncertainty source) should clearly be mentioned, otherwise the uncertainties have no (physical) reference value.
Page 15, figure 3c middle: Can you explain the pronounced daily cycle in the lower stratosphere only for CLaMS-default, MPTRAC-bestfit, MPTRAC-default?
Page 15, figure 3 caption, “theupper”
Page 16, figure 4: Colors are difficult to differentiate on paper!
Page 16, caption fig. 4: Please explain, why it is acceptable to exclude these outliers. I wonder, if the estimation of a maximum value is meaningful at all, if you ignore outliers? Moreover, could you please describe, why the selected value of 1.5 of the inter-quartiles is an appropriate value? Is it possible to show a figure, where the outliers are included?
Page 19, line 362: Here you state for the first time in the description of Fig. 4-6 the “overall transport median”. In the figures 4-6 it is shown as the transport AHTDs or AVTDs. I hope I have not overlooked it elsewhere, but what I miss in this chapter is that you clearly mention the reference value to which all of the calculated uncertainties are related to. As the “transport” is defined as the difference between start and end point of the trajectory, I would not see it as an uncertainty quantity, rather a (physical) difference reference value, where uncertainties should be compared with.
Page 19, line 370: This statement is not represented in Fig.4-6. Please, include the respective values of uncertainty or a reference to another figure, where these results are presented. I think this is an important result, where you show, that your model development in MPTRAC is in accordance with CLaMS, which is a widely published and evaluated model!
Page 19, line 372: Is it really worth to mention differences in results due to compilation flags? The question, however, is: Are results still reliable or equal in a statistical sense, when model components or even a different computer architecture, compiler etc. is used?
Page 20, line 412: The maximum of the normalized mean AVTD for “diffusion” can be found in the tropopause region. What can be concluded from the differences in vertical structure of normalized AVTD of “diffusion” in comparison with mean AVTD? Why are the profiles for diffusion of mean AVTD and normalized AVTD not equal in shape, if the normalization is the mean vertical path length of CLaMS, e.g. one value?
Page 20, line 413: “largest source of uncertainty”, isn’t it the largest relative source?
Page 20, line 413: Afterwards is not the correct adverb in this sentence.
Page 21 fig 7: Please add “average AVTDs for MPTRAC scenarios (see also Table 2)” in the figure caption.
Page 22, line 432: Say: vertical transport deviations remain smaller...
Page 25, lines 459: You write that the convergence of air parcels (Fig. 10) near the tropopause and near the surface is a consequence of up- and downdrafts in the troposphere including the tropopause and surface as a transport barrier. Though, I would suggest that the convergence of parcels near the surface/tropopause seems to be a result of missing mixing and convective transports, as the troposphere is a well-mixed region. Please comment.
Page 25, line 468: You describe the green lines in Fig. 11a as contour lines of air parcel frequencies. Are they absolute number of parcels of the CLaMS default simulation? If yes, please specify this also in the Fig 11 caption.
Page 25, line 473: This sentence is unclear. What is meant by “patterns”, or “smoothed peaks”? green contours shrink?
Page 26, lines 490: What did you want to say with “climatological findings”? That more air parcels are found at higher altitude around latitude of 45° with ERA-interim? I would suggest to replace “climatological findings” with “the results by Ploeger et al.” Please comment.
Page 28, line 521: Say:.. differences between scenarios are small...
Page 29, line 525: “to assess the implementation of the zeta coordinate...”, shouldn’t it be “to evaluate the implemented zeta coordinate in MPTRAC, we conducted simulations using...”?
Chapter 4: The conclusion section is rather long and repeats the description of results in detail. I would suggest to shorten the conclusion section (more a summary and conclusion section) and focus on the main results of the evaluation of the new model and the main results of the analysis (scenarios, uncertainties, effect on parcel distribution ..)
Citation: https://doi.org/10.5194/gmd-2023-214-RC2 - AC1: 'Comment on gmd-2023-214', Jan Heinrich Clemens, 06 Mar 2024
Data sets
Implementation and evaluation of diabatic advection in the Lagrangian transport model MPTRAC 2.6 - code and data for review Creators Jan Clemens https://doi.org/10.5281/zenodo.10050089
Model code and software
Massive-Parallel Trajectory Calculations (MPTRAC) L. Hoffmann, J. Clemens, S. Griessbach, K. Haghighi Mood, F. Khosrawi, M. Liu, Y.-S. Lu, J. Sonnabend, and L. Zou https://doi.org/10.5281/zenodo.10067751
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 183 | 50 | 22 | 255 | 12 | 16 |
- HTML: 183
- PDF: 50
- XML: 22
- Total: 255
- BibTeX: 12
- EndNote: 16
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1