the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
A new precipitation emulator (PREMU v1.0) for lower-complexity models
Gang Liu
Chris Huntingford
Download
- Final revised paper (published on 22 Feb 2023)
- Supplement to the final revised paper
- Preprint (discussion started on 07 Jul 2022)
- Supplement to the preprint
Interactive discussion
Status: closed
-
RC1: 'Comment on gmd-2022-144', Anonymous Referee #1, 05 Aug 2022
General Comments:
PREMU presents a novel method for emulating precipitation by considering modes of temperature variability, thus providing a more precise approach towards representing such a dynamic climate variable. I believe this paper would be useful for the GMD reader base and provide new avenues for tackling emulation of other climate variables that do not scale so directly with Global Mean Temperature. In general, the text could undergo some restructuring for understandability and clarity purposes with some suggestions to do so provided in the specific comments below. Some more general comments on the methodological approach that could also be improved upon are as follows:
- The methods overall make sense but could be more explicit in terms of the PCA analysis done: how are the spatial patterns derived, how sensitive are these patterns to sample size, do these patterns make sense/have physical explanations behind them that are identified in previous literature. Additionally, there are parts of the discussion where it is mentioned that the sensitivity of the PCA based coefficient matrix is tested (L271-L281), this could be mentioned before as a subsection under Methods and Results.
- The 3-month mean of temperature does not necessarily mean that lag effects are captured, and the authors should elaborate on this choice as compared to e.g. a multi-linear regression. Again the discussion brings about some tests of lag effects using premu-1mon and premu-6mon but this still considers monthly averages, furthermore this part would also fit better as a subsection under methods and results. In terms of assessing memory effects, the evaluation could be enriched by, for instance, looking at lag-1,2 and 3 correlation coefficients between subsequent month values unless the authors are not interested in preserving month-to-month correlations within PREMU predictions which should then be justified.
- The results only focus on MAP, inter-annual variations and trends and annual spatial patterns, given that PREMU is monthly, readers would also benefit from seeing these analyses done on a seasonal/monthly level which would also be more impact relevant.
Specific Comments:
L8: Elaborate on the term rainfall features, otherwise it is a bit vague
L20: MESMER-M is a monthly extension of MESMER, given that PREMU takes monthly input this may be relevant to mention (Nath et al. https://doi.org/10.5194/esd-13-851-2022)
L23-26: consider moving the policy relevance of LCMs higher up in the abstract, the final sentences should focus on PREMU itself.
Introduction: Generally a good overview into the problem this study is trying to tackle, however it tends to downplay the real value of this work and could benefit from some restructuring to make the points of this study clearer:
- L28-L32: Consider merging this with the next paragraph, otherwise it comes across as a bit redundant.
- L34-L36: Paints a good picture of LCMs, in relevance to this study and generally for LCMs it should be emphasised that they focus on reproducing only a few climate variables that are most impact relevant. This allows them to have such reduced computational expenses as well as differentiates them from ESMs. If this is what is meant by “highly parameterized macro-properties” it should still be elaborated upon, in general this term can imply a lot of things so it should be made more specific.
- L42: The following reference could be insightful for the author/readers when mentioning pattern scaling:
- Tebaldi, C., & Arblaster, J. M. (2014). Pattern scaling: Its strengths and limitations, and an update on the latest model simulations. Climatic Change, 122(3), 459–471. https://doi.org/10.1007/s10584-013-1032-9
- Tebaldi, C., & Knutti, R. (2018). Evaluating the accuracy of climate change pattern emulation for low warming targets. Environmental Research Letters, 13(5). https://doi.org/10.1088/1748-9326/aabef2
- L43: It could be good to add that there is already some work that successfully considers modes of variability within emulation, see:
- Mckinnon and Deser 2018: https://doi.org/10.1175/JCLI-D-17-0901.1
- Mckinnon and Deser 2021: DOI: 10.1175/JCLI-D-21-0251.1
- L45: I feel like the fact that precipitation is a crucial variable but has statistical features that make its representation within traditional LCM approaches difficult is a strong message to highlight. Consider restructuring the first three paragraphs to emphasise this i.e.
- First paragraph:
- There is increasing demand for climate information however lack of sufficient computational power for running ESMs for all potential future emission scenarios etc. .
- LCMs are a solution to this representing key climate variables, however in terms of gridded data most use pattern scaling based off GMT which is suited for local temperatures.
- Second paragraph:
- Precipitation has high spatio-temporal variability and is affected by atmospheric dynamics, inter-annual modes of variability etc. making representation within LCMs using only GMT as an explanatory variable difficult e.g. OSCAR, IMOGEN
- Nevertheless precipitation is a crucial component of the water cycle (Eltahir and Bras, 1996; Trenberth et al., 2003; Sun et al., 2018), has key societal implications and is closely associated with the functioning of terrestrial ecosystems
- First paragraph:
- L65-L67: Here it may also be a good idea to elaborate on the rainfall features this study finds especially relevant to preserve and provide relevant literature references (is it just inter-annual variability, spatial and seasonal variance, or are there more to think of under a changing climate? What are the drivers of inter-annual variability in precipitation e.g. El-Nino, Indian Ocean Dipole?) and why basing off gridded temperature provides a more promising avenue as compared to GMT to achieve this.
- L66: The end-use of the resulting emulator should already be introduced, it seems like it can have multiple uses in terms of looking at future emission pathways e.g.
- Stand in for OSCAR and IMOGEN’s precip. pattern scaling (here it should be noted that these LCMs have interactive atmospheric chemistry and endogenous calculations of e.g. biomass burning which may lead to differences from ESMs in the PCA analysis and therefore the resulting emulator is not an ESM one but simply a module within the aforementioned LCMs)
- An extension of the MAGICC-MESMER emulation chain
L70: Consider making Data and Methods two independent sections, it seems like they have that role in the text anyways.
L71: The text structure could benefit from introducing the approach before going into the subsections i.e. 1) you divide the emulator calibration into that done on the historical period 1901-2016 and that done on the future period 2016-2100 2) for the historical calibration, you use observational data while for future calibration you use ESM data.
L72-73: the use of “first tested” is a bit misleading since you both calibrated and tested PREMU on historical data, maybe replace with “we first demonstrate the applicability of the emulator on observational data provided by the Global Soil Wetness Project Phase 3.”
L77: The statement: ‘yet provides additional high frequency signals which are lacking in previous products' leaves the reader wanting more, if possible elaborate a bit on what this means.
L78: Elaborate on what Tair is, is it 2m air temperature? For translation to ESMs this is quite important.
L80: Does the author perform regridding to 2.5°x2.5° so that these variables are on the same grid with each other as well as ESMs used? This should be elaborated on and if not done, justified.
L92: Out of curiosity, why first-order? Other archives (e.g. Brunner L., M. Hauser, R. Lorenz, and U. Beyerle (2020).The ETH Zurich CMIP6 next generation archive: technical documentation. DOI: 10.5281/zenodo.3734128) use second order.
L94: When calibrating, do you use all initial condition ensemble member available per ESM for SSP 5-8.5? Same goes for testing on the other SSPs?
L94: Instead of going straight into calibration the author should also first provide the PREMU framework, consider therefore breaking up the text to introduce general approach (gridded temperature, PCR etc. ) , emulator framework (here introduce equations and key variables alongside Table 2 and Figure 1), calibration etc.
L113: It is not convincing that taking a 3-month average across the month of interest and the preceding 2 months will sufficiently capture the influence from the previous 2 months. Moreover this could average out the general seasonal transitions in temperature, does the author have a justification for this instead of using e.g. a multiple linear regression
L113-L115: “ESM may under-represent the effects of topography and aerosols on the precipitation from observations“ do you have a reference to back this up with? I see this later on in the discussion, consider bringing them up here too.
L120: How the spatial patterns of the PCA components is obtained needs to be elaborated upon
Fig 2: It is strange that there is such consistently high coefficient values occurring in the Arctic, is there a reason to this? If possible, the explanatory power of such spatial patterns towards precipitation should somehow be evaluated in terms of how representative they are of global circulation patterns and inter-annual modes of variability. In such, one suggestion could be to check how the coefficients change when fitted using a leave-one-year out approach.
L141: I find equation 3 redundant as it is the same as equation 2 only with grid-point specific coefficients, for simplicity’s sakes perhaps these two equations could be merged
L142-L143: Redundant as the same as before but only for grid-point level
L153, L156, L157: These equations are repetitive, again consider simply referencing to the ones already introduced before.
Validation: Generally this section cuts off to abruptly and should be restructured as well as more detailed. There seems to be too much emphasis on defining the predicted variables and how they are obtained from the test set such that the actual method for validation is lost. The author should try be more explicit on what they are validating for i.e.
- Validation is performed for each predicted variable, T^{PCA}, P^{global}, P^{grid}
- What is validated for? It should be specified that the MAP, the inter annual variance and the trend are checked etc. Evaluation metrics should be provided (e.g. percentage error, correlation) and how they are calculated should also be explained.
- Given the high spatio-temporal variability of precipitation, it would be interesting to see how well PREMU does in preserving spatio/temporal autocorrelations, e.g. lag-1 correlations between months and spatial cross-correlations
- Readers are left wondering if any evaluation on the PCA is done e.g. comparison between ESMs and observations, linkage to existing literature. There should be some physical relevance to the PCA components obtained as well as stability checks e.g. is 70% of variance explained by 10 PCA components within ESMs too, does this apply for all months, what happens to the spatial patterns when some training years are left out.
L162, Eq 7: This is a methodological addition and seems out of place in the validation section, consider moving it to methods (e.g. “from further tests etc. we decided to apply a correction factor etc.”)
L170: when taking the areally-average is it a weighted average according to latitude, if not why? Areally-averaged could also be replaced by simply average or average over grid points as areally-averaged reads ambiguously.
L172: How do you check the similarities in inter annual variations i.e. using pearson correlation coefficient between the inter-annual variance or absolute values or applying a low-pass filter to extract the variance before checking the correlations?
Results: Quite nice findings! It should be made explicit the annual precipitation is mainly verified for, given that PREMU is a monthly emulator. It raises the question of the seasonal performance of PREMU, is there a reason the authors did not show this?
L271-L305: These seem to be more points within the results, in general it would be good to first introduce that these things (e.g. memory effect) were tested for an how in the methods and then provide the results in the Results. Otherwise the Discussion is lacking structure and quite long.
L311: Different version of calibrated PREMUs should probably be introduced in their own subsection
Conclusion: Generally the discussion is quite long and covers myriad of possibilities, consider merging some parts into the results (as mentioned above) some parts in conclusion and dividing into further subsections e.g. Further Developments/Versions (PREMU-1MON, PREMU-LAND etc.)
Table 3: It is hard to contrast so many numbers, perhaps providing percentage difference relative to actual trend value or something along those lines would be better?
Figure 1: Having read the text I understand this figure, however by itself it is quite complicated and could benefit from some more graphics and simplification of the text.
Figure 4: Readers would be interested in seeing differences in trends between GWSP and PREMU characterised, it seems like there are areas where the direction in trend is different (e.g. Australia, West Africa) which also matters vs simply looking at over/underestimation of changes. The discussion point that more differences can be seen in this figure as compared to Figure 8 due to topography and aerosols should also be mentioned when describing this figure as it enriches the analysis.
Figure 8: again it may be interesting to characterise the differences: where direction of changes are properly captured and where they are opposite
Editorial Comments:
L28: I would start the sentence off with the subject rather than verb, especially if it is at the beginning of the paragraph, i.e.
Earth system models (ESMs) are the primary tools to study the impact of greenhouse gas emissions on our climate, representing all the important Earth system processes (IPCC, 2013).
L30: run the
Figure captions: emulations are referred to as our emulations, I am not sure how formal this is.
Section 2.3.2: the Table should be labelled and given a number
Eq 4: T has the superscript Timeseries,val which is not in line with that of eq 1, is this intentional?
Citation: https://doi.org/10.5194/gmd-2022-144-RC1 -
RC2: 'Comment on gmd-2022-144', Anonymous Referee #2, 14 Aug 2022
General Comments:
The authors present a novel method for precipitation emulation (PREMU) that uses gridded temperature patterns over time to (re)construct global and local precipitation time series by means of principle component analysis (PCA). The method provides high accuracy against existing precipitation estimates, suggesting its potential power for situations where a temperature projection can be easily constructed but a precipitation projection cannot, such as with lower complexity models (LCMs) or novel projections of temperature outside of the most common future scenarios in more complex earth system models (ESMs). The organization of the text does not make this message as clearly as it needs to, though, and there are a few outstanding questions about the methodology as well that need elaboration.
The effort put into defending the ability of PREMU to accurately reconstruct ESM precipitation overwhelms any description of the applicability of the method to LCM temperature data, making that reproducibility appear to be the main message instead. While the accuracy of PREMU in this respect is impressive, it minimizes the importance of this novel method because it gives the impression that no new precipitation has been added. More emphasis on the potential application of PREMU is needed, and better framing of the ESM matching would greatly improve the article as well.
In particular, with PREMU being much less computationally intense than an ESM simulation, this paper should include at least one application of PREMU to an LCM projection of future temperature as an example. The example should also include a precipitation simulation based on that LCM using the traditional linear method so that the audience can see what differences (in theory, improvements) there are between the linear method and PREMU.
Some specific suggestions for providing better framework and organization for the core message are presented below, along with questions and comments about the methodology. Overall, I do believe that this is a valuable new tool for the community that should be shared; it just needs slightly more explanation and a better presentation.
Specific Comments:
Abstract: The mention of LCMs should be mentioned much earlier, along with the issue with linear scaling for precipitation. Then present PREMU as a solution to that issue, followed by the justification statistics from the historical and ESM comparisons. L22 onward can remain as the conclusion sentence. L12 “better estimate and represent precipitation simulated by Earth system models (ESMs)” wording should be changed as there is no justification in the paper suggesting the PREMU simulations are better than the ESM data they are recreating.
Introduction: Generally good, though precipitation is not mentioned until L47. This section should be rearranged to better emphasize the problems associated with precipitation estimation outside of the common ESM future scenarios; it does not have to lead the section but should be mentioned higher up and expounded upon. The final paragraph also needs to be fleshed out more, particularly in the sentence describing section 4 where you need to draw the connection between the presented ESM validation and the potential use with LCMs. The potential addition of a direct LCM-PREMU precipitation example would also be mentioned in these last two sentences.
Data & Methods: With the existing structure in this section, it appears that the Methods might benefit from being its own top-level section, as Methods is the only section of the paper with third-level subsections. There are also many choices (listed in detail below) made in the methodology that are ultimately justified in the Discussion section but present lingering questions when presented alone in this section. Full justification of these choices cannot be done before the Results section, but you should telegraph some of the comments that will be made in the Discussion section so that your audience recognizes you are already aware of some of the potential issues with your assumptions and choices.
L89-90: Calibrating to your endpoint (here, the hottest future temperature scenario) is good in the sense that your validation will only involve interpolation instead of extrapolation, but since we know that the atmosphere can respond nonlinearly with warming, this introduces the concern that this extreme warming scenario may not produce precipitation patterns that are representative of cooler scenarios. This is addressed later (e.g. L203-205, L271-292) but should be acknowledged here.
L91 “we constructed the emulator for each ESM respectively” On first reading this, it sounds like you performed the entire PCA individually for ESM. Upon reading further, it instead looks like only the coefficients differ for each ESM, as Fig. S2+S3 state that the same ten temperature modes are used for all ESMs. This should be stated clearer.
L95 Calibration: This is another section that should be split in half, as most of the first half of the section (L96-177?) focuses on the temperature PCA itself. This description should also use a little bit more elaboration for readers who may not be as familiar with the mathematics of the process.
L107 “used in climate research” requires at least two example references.
L111-112: You further investigate the effect of different lags later in the text, but this statement alone requires some elaboration and at least one reference to justify how earlier months must be considered.
L116-117: something like this should have been mentioned around L89-90, and similar statements (i.e. “discussed in Sect. 4”) are needed for other issues.
L118: It’s great to have the table for reference, but more of this information should also be in the text itself so readers don’t have to go back and forth between the table and the text repeatedly in order to understand the equations.
L127-128: More justification is needed as to why both the historical and ESM versions of PREMU were constructed with 10 modes of temperature. From the four figures cited here, in addition to S13+S14, it’s clear that the amount of warming in the chosen temperature data greatly affects how many modes are needed to hit any particular threshold for variance explained. This also goes back to the earlier concern of how the 8.5 scenario, which is so strongly dictated by worldwide warming, might not give reasonable precipitation results for scenarios with more varied temperature modes. This is later somewhat addressed in L275-278, but the stark differences between the first four figures mentioned in L128 will raise questions that should be partially addressed here.
L136,L141: The similarities between these two equations suggest that it should be possible to construct the global coefficients from the gridded ones, i.e. instead of doing the global analysis alone. I know some averages are computed and compared later, but unsure if that is the same as what I’m thinking here.
L159-164: if you say you found a slight difference, you should be more explicit as to what the difference, and potentially offer an explanation for why the difference is present. Without a physical reason given, this subsequent ratio correction feels like a poorly justified mathematical band-aid.
Results: This section seems thorough, but it often states a large number of different quantities in quick succession in many paragraphs, making it somewhat difficult to follow and occasionally feel like not every comparison is being directly stated. I would like to see a more organized pattern of describing results in each paragraph and/or another table stating all values of interest (average precip/yr, precip trend, year to year variance, error, correlation, proportion of grid cells with given error, etc.) for each set of simulations, both trainings and experiments, obs vs linear vs PREMU for historic and each ESM vs PREMU for future.
Fig. 3: correlation and error look like they might be strongly influenced by the extreme changes in the last year or two of the experiment time period. Out of curiosity, since PREMU is not too computationally intensive, could you do some sensitivity analysis where you vary the start and end years for both your trainings and experiments? I’m guessing it might not change much for your results and thus might not need to be put into a later version of this article, but it would be interesting to see.
L190-191: It feels very weird to end a section saying a particular part of the simulation is inaccurate - why not try to compare it to the linear scaling? Is it worse than the linear method, and that’s why the linear scaling isn’t mentioned here?
L194: This is the first time mentioning the possibility of PREMU working with novel trajectories for GHGs; something that should be mentioned earlier in the paper alongside the extra emphasis on using PREMU with LCM temperatures.
L201-202: While you later suggest that some of better variation simulation with ESMs might be due to topographic complication or aerosols, (L243-L252,) one thing that comes to mind with R values is the standard deviations in the data sets you’re comparing. I would like to see some mention of the underlying statistics here.
L204-205: This speaks to the previously mentioned issue of training to the 8.5 scenario, and you do address it later in the Discussion section. Nothing necessarily needs to be added here if you properly acknowledge the issue earlier in the text.
L207-209: The issue of over half of your ESMs doing visibly worse than the others deserves more than the passing comment here, especially as you’ve devoted a figure (6) to show it. There is a much larger discussion in L254-279, but if you don’t say more here, you should at least mention that the discussion is coming later. (Again, telegraph your discussion points so that if a reader starts to question a methodological point or an odd result, they know you’re aware of it and are planning to address that concern.)
L212-223: Another paragraph that feels disorderly with the amount of widely varying results strung together.
Discussion: This is a great consideration of all potential issues with the methodology and interesting components of the results previously shown. As stated before, several of these need to be telegraphed earlier in the paper so your audience isn’t reading through your results with too many questions. It has now been over six pages since “LCMs” was previously mentioned, which is why the article so far feels like the ESM comparison was the main goal, as opposed to the actual potential for use with temperatures from LCMs and other novel scenarios. Also, your audience should not get to this point without knowing exactly how one of these other experiments would be run, e.g. what would be the “training” precipitation data for an LCM experiment?
L254-279: These are all good suggestions for why some ESMs are better simulated with the PREMU method than others. However, it does raise the question: if we know different ESMs have different schemes relating atmospheric circulation to their precipitation, why was a set of 10 temperature modes chosen to use for all ESM experiments with the only differences being the subsequent coefficients, especially when we know the ESM temperature patterns themselves are not consistent between different ESMs to begin with? You could potentially better capture model-specific ENSOs and other such features this way.
L275-276: This paragraph is a great way to address some of the previously mentioned concerns about training to a scenario with such extreme warming. This particular sentence, though, still doesn’t seem entirely reasonable considering how different the 8.5 modes and 2.6 modes are. You do later state that the mode order is slightly shuffled, implying that you have clearly identified similar pairs of modes between the two scenarios; it would be nice to show a side-by-side comparison justifying to your audience that these two sets of modes are indeed similar enough to explain the >95% similarity in coefficients.
L283-292: More discussion of the issue of how much warming in your training affects the fit during the experimental phase, which is good to see. You point out that using a historical/cool training for a warmer scenario would be unwise; it might be better to state the opposite in acknowledgement of how you already pointed out that the 8.5 training generally produces worse results as you go from the 7.0 experiment to the 2.6 experiment. The last sentence here also is potentially very instructive for if/when you include an LCM-PREMU example, i.e. you would train PREMU based on the SSP whose future temperature most closely resembles the future temperatures from the LCM simulation in question.
L304-305: While this is one possible conclusion to draw from the ESM results being more robust versus lag than the historical results were, you did previously posit “an alternative argument” suggesting that apparently better results in ESMs might be due to underrepresentation of topography and aerosols. As these are potential sources of variation, the “robustness” versus lag with ESMs could also be due to the ESMs not showing enough variation for the lag differences to matter.
L309-310: This could also be a potential concern since SSTs have a much larger influence on total atmospheric water vapor than land temperatures do, both by being much more surface area and by being the main source of evaporation. With that consideration, it should follow that PREMU should be at least slightly worse in its precipitation simulation – potentially still very good, but still not as good as when ocean area is also considered. I would like to see the differences between the normal emulator and emulator-land explicitly shown in some manner. You do state that including the ocean is still the preferred method for now; it would be good to expound on that.
Table 3: You have space at the upper left for a proper title and/or stating the units; the latter in particular should be easily read somewhere outside of just the caption.
Figure 4: I would love to see difference plots e-a, e-c, f-b, and f-d.
Figure 5 onward: From here on out, your maps are smaller than earlier; between the small size and the fact that the first red+blue from the middle are similar to the grey, all of the maps here and onward are hard to read. I would strive to rearrange all of these figures so that you are no more than two maps wide per page – and potentially darken those first lighter colors a bit as well.
Figure 7: In trying to make this figure no more than two maps wide, try visualizing it as a 2x2 grid where each quadrant has three maps stacked vertically.
Figure 8: I understand that 8.5 is the training scenario, then followed by 2.6 to 7.0 in that order as the experiments, but it still looks odd that the overall order isn't uniformly increasing or decreasing.
Technical Corrections:
There seems to be a minor inconsistency in the paper between whether 2015 or 2016 is the starting point for the ESM data, L125 vs L132. Please make sure these are better aligned
L117: remove “the” form “discussed in the Sect. 4”
Citation: https://doi.org/10.5194/gmd-2022-144-RC2 -
AC1: 'Response to reviewers', S. Peng, 30 Nov 2022
The comment was uploaded in the form of a supplement: https://gmd.copernicus.org/preprints/gmd-2022-144/gmd-2022-144-AC1-supplement.pdf