the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 06 May 2024
A computationally light-weight model for ensemble forecasting of environmental hazard: General TAMSAT-ALERT v1.2.1
Abstract. Efficient methods for predicting weather-related hazards are crucial for stakeholders managing environmental risks. Many environmental hazards depend on the evolution of meteorological conditions over protracted periods, requiring assessments that account for evolving conditions. The TAMSAT-ALERT approach addresses this challenge by combining observational monitoring with a weighted climatological ensemble. As such, it enhances the utility of existing systems by enabling users to combine multiple streams of monitoring and forecasting data into holistic hazard assessments. TAMSAT-ALERT forecasts are now used in a number of regions in the Global South for soil moisture forecasting, drought early warning and agricultural decision support. The model presented here, General TAMSAT-ALERT, represents a significant scientific and functional advance on previous implementations. Notably, General TAMSAT-ALERT is applicable to any variable for which time series data are available. In addition, functionality has been introduced to account for climatological non-stationarity (for example due to climate change); large-scale modes of variability (for example El Nino), and persistence (for example of land-surface condition). In this paper, we present a full description of the model, along with case studies of its application to prediction of Central England Temperature, Pakistan vegetation condition and African precipitation.
- Preprint
(4652 KB) - Metadata XML
-
Supplement
(179 KB) - BibTeX
- EndNote
Status: closed
-
RC1: 'Comment on gmd-2024-75', Anonymous Referee #1, 07 Jun 2024
This paper documents a general method of producing forecasts by taking observations up to a point and combining with the historical distribution as a plausible ensemble of outcomes from that point. It is clearly a valuable tool (having already been used in several real-world contexts) and is worthy of publication, with the documentation allowing other researchers / users to deploy the method for themselves.
I have no hesitation recommending this for publication, barring a few minor comments listed below - mostly to improve clarity, although there is one suggestion that the years for the case studies should be changed such that the forecast method is "fair" and doesn't assume knowledge of future conditions from each forecast.
Minor commnets
L30 "hazard(s)"
L39 "drift in predictions" is this really a fundamental problem? Surely it is simply fixed with bias correction. OK bias correction is often not simple, but as written suggests that the problem is insurmountable. Perhaps a brief note here for completeness would be useful.
L46 "seamless integration of past and future conditions facilitates in-season updates for slowly developing hazards". Is this really unique to the method? NWP/seasonal forecast systems do exactly this - accounting for past conditions (represented by the initial state, eg. initialised soil moisture capturing antecedant rainfall) and evolving into the future. This is easy to be updated in-season - just run with a more recent initial state. Of course this doesn't stop it being a 'feature' of the GT-A method, but it would be clearer to really focus on it's uniquness (parsimonious and quick method of making a forecast for any variable by integrating current state with plausible future states, allowing testing of some simple assumptions around climate indices and non-stationarity)
L77 "continually" typo
L82 "tThe" typo
Figure 1 Nice figure. Would suggest simplifying the text in boxes as much as possible and making them active/imperative sentences rather than passive (i.e. put the verb at the front) e.g. "Define initialisation date and period of interest" (don't need to specifiy "user" since the box color tells us that); "Calculate gridded difference time series". Also, is it possible to add the numbered steps from the previous stage on to this figure? Not quite sure it is, but it may be unecessarily confusing to have two parallel descriptions of the method which don't map on to each other.
L134 "incrementing forecasts". Not sure this is explained fully. I think you mean, that the historical data is transformed into anomalies relative to the initial state for that year, and then added on to the current state? e.g. if June 1981 is 23C and July 1981 is 24C, and June 2024 is 20C, then that 'member' produces a forecast of 21C for July 2024. Unpacking the method here or elsewhere would help clarify.
L203 Setup 1 - essentially means that July FC is just the ensemble of values for historical Julys. This becomes obvious later but could be spelt out here.
L204 How are they weighted? I note the appendix giving details of the exponential weighitng, but this includes a coefficient. Which is used here?
L206 "Day of initialisation" is confusing since you're using monthly data in this example. Also missing a note to say you're making forecasts at a 1-month lead (June)
Figure 2 missing subheadings ( (a) etc, but could also include the name of the set-up rather htan needing to refer to caption). Can also get rid of legend in 3/4 plots and use the space to make the plots bigger. Also, y axis units missing.
Figure 4 Caption missing D
Figure 5 units missing on top plot. Also please mention what the subnational borders represent (e.g. states ... although you could easily show this plot with just country borders -since the distinction of the admin1 regions is not relevant or discussed, and is not included on subsequent plots)
Sec 3.1.1 Missing info on if forecasts were produced with weighitng, incremental option or both. Also now this case study uses gridded data, maybe worht pointing out that the method runs independently on gridpoints.
Figure 6a Is it possible to achieve negative NDVI? If so please comment on what this represents.
L273 The scores here are calculated now across all gridpoints for the single forecast?
L282 Did you test all the set-ups with Case study 2? If so do you find that the weighted years + perseistance brings minimal improvement beyond just using persistance?
Figure 7 please add an extra colour so the color changes match up with the labelled intervals
L306 "(a)ccumulations"
L314 Deriving probabilities from ensemble mean and spread requires Gaussian assumption. Why not just count percentage of members below a threshold since it doesn't assume anything? Does it make a difference? If this is avoided deliberately please document the reason.
L325 "negative anomalies are also (incorrectly) forecast in southern Africa" - although, negative anomalies are correct for South Africa
L425 I feel like it would help to discuss a little this weighting method in the main text. A few different weighting strengths are shown - are these discrete options for the user or can they set this to any value? It would probably also be helpful to give some guidance on which weighitng to choose (maybe the point is that it's up to the users to play around and decide what makes most sense to them - in which case that is also useful thing to say).
Figure A1: just looking at this it becomes clear that the forecasts in the case studies are made with knowledge of future conditions. This is not a fair forecasting test - really it should be an out-of-sample event, not including any 'future' information. I realise the paper is not primarily 'about' forecasting, but I would prefer to see the analysis reproduced taking this into account. For Case study 1 it could just be switched to July 2023 with presumably the same result. Also Case study 3 could also be switched to 2023 - which was also a very wet El Nino year in the region so should look similar. Case study 2 maybe more difficult, but is there a more recent drought that could be used (and, basing the forecast production on a period truncated to the year before that event?Citation: https://doi.org/10.5194/gmd-2024-75-RC1 -
AC1: 'Reply on RC1', Emily Black, 07 Aug 2024
- AC3: 'Reply on AC1', Emily Black, 07 Aug 2024
-
AC1: 'Reply on RC1', Emily Black, 07 Aug 2024
-
RC2: 'Comment on gmd-2024-75', Anonymous Referee #2, 05 Jul 2024
The manuscript presents General TAMSAT-ALERT which is an updated and extended version of an existing tool (TAMSAT-ALERT) to combine historical times series and climatological forecasts to obtain probabilistic forecasts that can consider climate variability and climate change by weighting members of the ensemble. It seems that General TAMSAT-ALERT is a very useful tool; it seems to be simple and easily applicable, and I think the presentation of General TAMSAT-ALERT deserves publication.
However, I feel that the manuscript is, in its current version, not very accessible to readers that are not deeply familiar with (meteorological) forecasting. Sentences like “… A key innovation is the option to increment variables from the initialisation date, enabling forecasts to account for persistence in time…” (Line 70) are not easy to understand outside the forecasting community. I had quite some problems in understanding the manuscript, and I could have provided a more detailed review if I had better understood the details. So, my main recommendation is to provide more information and to improve the understandability for readers outside the forecasting community.
Specific comments:
The manuscript needs careful correction, as it contains a few typos. Some examples: Line 82: “… the …” instead of “… tThe …”; Line 87 and Figure 1 (bottom right box): bracket missing; Line 169: “… perfect forecast and the observed …” instead of “… perfect forecast, in the observed …”
Title, abstract and short summary: The authors should make it very clear that they mean climatological forecasting when they speak about forecasting. This is mentioned in the abstract but not in the title and the short summary. I was confused about that. For instance, the sentence in Line 120 (enabling users to derive forecasts directly from observations and reanalysis, without the need for the use of land-surface/crop models or NWP forecasts) should come much earlier.
Line 40: “ … A more fundamental problem is the drift in predictions … If model predictions were to be spliced directly onto historical observations, the drift would cause systematic bias in yield assessments, the magnitude of which would depend on the stage of the growing season at which the meteorological forecasts were initiated. The TAMSAT-ALERT approach addresses these issues by splicing together historical time series for the past with a climatology for the future …”: Here, I am not sure whether I understand this argument correctly. Do you compare the TAMSAT-ALERT approach where you combine historical time series with forecasts (of today) to a situation where you combine historical time series with a model prediction that started in the past? In the latter approach: why would you not use historical data until today and then start the simulation?
Line 55: “… A strength of the methodology is that NWP output can be incorporated, even when forecasts are not available for the variable being assessed. In Kenya, for example, incorporation of skilful precipitation tercile forecast probabilities output by the ECMWF dynamical forecasting system improves the skill of NDVI and yield forecasts during the secondary rainy season …”: Here, I would like to have more information on how the use one variable as proxy for another variables works.
Section 2.1 and Fig. 1: I propose to align the 9 steps in the text with Fig. 1. Currently, it is not obvious which step belongs to which box in Fig. 1 and why there are much more boxes than steps. It should also be shown in this figure (if possible) what the change/additions in comparison to TAMSAR-ALERT are.
Line 120: Do I understand correctly that the predecessor system TAMSAT-ALERT needed simulations models (land-surface/crop models or NWP forecasts) but General TAMSAT-ALERT does not need them? I am confused. Please provide a more comprehensive description of TAMSAT-ALERT.
Line 166: in the introduction of the 2 skill scores, the predictand should be general, as only in the second case study NDVI is predicted.
Figure 3: Please add (A), (B), (C), (D) to the subplots.
Figure 5: This figure needs much more information to be understood. Please add the scale; what do the polygons mean? Where are the boundaries of your case study are? Maybe also show a few main cities, so that the reader can easily understand the figure. Why does the color bar end at 500 mm while in the text you write that there is rain up to 1000 mm?
Line 420: Please add a short Conclusions section.
Citation: https://doi.org/10.5194/gmd-2024-75-RC2 -
AC2: 'Reply on RC2', Emily Black, 07 Aug 2024
- AC4: 'Reply on AC2', Emily Black, 07 Aug 2024
-
AC2: 'Reply on RC2', Emily Black, 07 Aug 2024
Status: closed
-
RC1: 'Comment on gmd-2024-75', Anonymous Referee #1, 07 Jun 2024
This paper documents a general method of producing forecasts by taking observations up to a point and combining with the historical distribution as a plausible ensemble of outcomes from that point. It is clearly a valuable tool (having already been used in several real-world contexts) and is worthy of publication, with the documentation allowing other researchers / users to deploy the method for themselves.
I have no hesitation recommending this for publication, barring a few minor comments listed below - mostly to improve clarity, although there is one suggestion that the years for the case studies should be changed such that the forecast method is "fair" and doesn't assume knowledge of future conditions from each forecast.
Minor commnets
L30 "hazard(s)"
L39 "drift in predictions" is this really a fundamental problem? Surely it is simply fixed with bias correction. OK bias correction is often not simple, but as written suggests that the problem is insurmountable. Perhaps a brief note here for completeness would be useful.
L46 "seamless integration of past and future conditions facilitates in-season updates for slowly developing hazards". Is this really unique to the method? NWP/seasonal forecast systems do exactly this - accounting for past conditions (represented by the initial state, eg. initialised soil moisture capturing antecedant rainfall) and evolving into the future. This is easy to be updated in-season - just run with a more recent initial state. Of course this doesn't stop it being a 'feature' of the GT-A method, but it would be clearer to really focus on it's uniquness (parsimonious and quick method of making a forecast for any variable by integrating current state with plausible future states, allowing testing of some simple assumptions around climate indices and non-stationarity)
L77 "continually" typo
L82 "tThe" typo
Figure 1 Nice figure. Would suggest simplifying the text in boxes as much as possible and making them active/imperative sentences rather than passive (i.e. put the verb at the front) e.g. "Define initialisation date and period of interest" (don't need to specifiy "user" since the box color tells us that); "Calculate gridded difference time series". Also, is it possible to add the numbered steps from the previous stage on to this figure? Not quite sure it is, but it may be unecessarily confusing to have two parallel descriptions of the method which don't map on to each other.
L134 "incrementing forecasts". Not sure this is explained fully. I think you mean, that the historical data is transformed into anomalies relative to the initial state for that year, and then added on to the current state? e.g. if June 1981 is 23C and July 1981 is 24C, and June 2024 is 20C, then that 'member' produces a forecast of 21C for July 2024. Unpacking the method here or elsewhere would help clarify.
L203 Setup 1 - essentially means that July FC is just the ensemble of values for historical Julys. This becomes obvious later but could be spelt out here.
L204 How are they weighted? I note the appendix giving details of the exponential weighitng, but this includes a coefficient. Which is used here?
L206 "Day of initialisation" is confusing since you're using monthly data in this example. Also missing a note to say you're making forecasts at a 1-month lead (June)
Figure 2 missing subheadings ( (a) etc, but could also include the name of the set-up rather htan needing to refer to caption). Can also get rid of legend in 3/4 plots and use the space to make the plots bigger. Also, y axis units missing.
Figure 4 Caption missing D
Figure 5 units missing on top plot. Also please mention what the subnational borders represent (e.g. states ... although you could easily show this plot with just country borders -since the distinction of the admin1 regions is not relevant or discussed, and is not included on subsequent plots)
Sec 3.1.1 Missing info on if forecasts were produced with weighitng, incremental option or both. Also now this case study uses gridded data, maybe worht pointing out that the method runs independently on gridpoints.
Figure 6a Is it possible to achieve negative NDVI? If so please comment on what this represents.
L273 The scores here are calculated now across all gridpoints for the single forecast?
L282 Did you test all the set-ups with Case study 2? If so do you find that the weighted years + perseistance brings minimal improvement beyond just using persistance?
Figure 7 please add an extra colour so the color changes match up with the labelled intervals
L306 "(a)ccumulations"
L314 Deriving probabilities from ensemble mean and spread requires Gaussian assumption. Why not just count percentage of members below a threshold since it doesn't assume anything? Does it make a difference? If this is avoided deliberately please document the reason.
L325 "negative anomalies are also (incorrectly) forecast in southern Africa" - although, negative anomalies are correct for South Africa
L425 I feel like it would help to discuss a little this weighting method in the main text. A few different weighting strengths are shown - are these discrete options for the user or can they set this to any value? It would probably also be helpful to give some guidance on which weighitng to choose (maybe the point is that it's up to the users to play around and decide what makes most sense to them - in which case that is also useful thing to say).
Figure A1: just looking at this it becomes clear that the forecasts in the case studies are made with knowledge of future conditions. This is not a fair forecasting test - really it should be an out-of-sample event, not including any 'future' information. I realise the paper is not primarily 'about' forecasting, but I would prefer to see the analysis reproduced taking this into account. For Case study 1 it could just be switched to July 2023 with presumably the same result. Also Case study 3 could also be switched to 2023 - which was also a very wet El Nino year in the region so should look similar. Case study 2 maybe more difficult, but is there a more recent drought that could be used (and, basing the forecast production on a period truncated to the year before that event?Citation: https://doi.org/10.5194/gmd-2024-75-RC1 -
AC1: 'Reply on RC1', Emily Black, 07 Aug 2024
- AC3: 'Reply on AC1', Emily Black, 07 Aug 2024
-
AC1: 'Reply on RC1', Emily Black, 07 Aug 2024
-
RC2: 'Comment on gmd-2024-75', Anonymous Referee #2, 05 Jul 2024
The manuscript presents General TAMSAT-ALERT which is an updated and extended version of an existing tool (TAMSAT-ALERT) to combine historical times series and climatological forecasts to obtain probabilistic forecasts that can consider climate variability and climate change by weighting members of the ensemble. It seems that General TAMSAT-ALERT is a very useful tool; it seems to be simple and easily applicable, and I think the presentation of General TAMSAT-ALERT deserves publication.
However, I feel that the manuscript is, in its current version, not very accessible to readers that are not deeply familiar with (meteorological) forecasting. Sentences like “… A key innovation is the option to increment variables from the initialisation date, enabling forecasts to account for persistence in time…” (Line 70) are not easy to understand outside the forecasting community. I had quite some problems in understanding the manuscript, and I could have provided a more detailed review if I had better understood the details. So, my main recommendation is to provide more information and to improve the understandability for readers outside the forecasting community.
Specific comments:
The manuscript needs careful correction, as it contains a few typos. Some examples: Line 82: “… the …” instead of “… tThe …”; Line 87 and Figure 1 (bottom right box): bracket missing; Line 169: “… perfect forecast and the observed …” instead of “… perfect forecast, in the observed …”
Title, abstract and short summary: The authors should make it very clear that they mean climatological forecasting when they speak about forecasting. This is mentioned in the abstract but not in the title and the short summary. I was confused about that. For instance, the sentence in Line 120 (enabling users to derive forecasts directly from observations and reanalysis, without the need for the use of land-surface/crop models or NWP forecasts) should come much earlier.
Line 40: “ … A more fundamental problem is the drift in predictions … If model predictions were to be spliced directly onto historical observations, the drift would cause systematic bias in yield assessments, the magnitude of which would depend on the stage of the growing season at which the meteorological forecasts were initiated. The TAMSAT-ALERT approach addresses these issues by splicing together historical time series for the past with a climatology for the future …”: Here, I am not sure whether I understand this argument correctly. Do you compare the TAMSAT-ALERT approach where you combine historical time series with forecasts (of today) to a situation where you combine historical time series with a model prediction that started in the past? In the latter approach: why would you not use historical data until today and then start the simulation?
Line 55: “… A strength of the methodology is that NWP output can be incorporated, even when forecasts are not available for the variable being assessed. In Kenya, for example, incorporation of skilful precipitation tercile forecast probabilities output by the ECMWF dynamical forecasting system improves the skill of NDVI and yield forecasts during the secondary rainy season …”: Here, I would like to have more information on how the use one variable as proxy for another variables works.
Section 2.1 and Fig. 1: I propose to align the 9 steps in the text with Fig. 1. Currently, it is not obvious which step belongs to which box in Fig. 1 and why there are much more boxes than steps. It should also be shown in this figure (if possible) what the change/additions in comparison to TAMSAR-ALERT are.
Line 120: Do I understand correctly that the predecessor system TAMSAT-ALERT needed simulations models (land-surface/crop models or NWP forecasts) but General TAMSAT-ALERT does not need them? I am confused. Please provide a more comprehensive description of TAMSAT-ALERT.
Line 166: in the introduction of the 2 skill scores, the predictand should be general, as only in the second case study NDVI is predicted.
Figure 3: Please add (A), (B), (C), (D) to the subplots.
Figure 5: This figure needs much more information to be understood. Please add the scale; what do the polygons mean? Where are the boundaries of your case study are? Maybe also show a few main cities, so that the reader can easily understand the figure. Why does the color bar end at 500 mm while in the text you write that there is rain up to 1000 mm?
Line 420: Please add a short Conclusions section.
Citation: https://doi.org/10.5194/gmd-2024-75-RC2 -
AC2: 'Reply on RC2', Emily Black, 07 Aug 2024
- AC4: 'Reply on AC2', Emily Black, 07 Aug 2024
-
AC2: 'Reply on RC2', Emily Black, 07 Aug 2024
Model code and software
General TAMSAT-ALERT v1.2.1 John Ellis and Emily Black https://doi.org/10.5281/zenodo.10955490
Interactive computing environment
Demo of General TAMSAT-ALERT v1.2.1 John Ellis and Emily Black https://gws-access.jasmin.ac.uk/public/tamsat/tamsat_alert/gmd_paper/demo.zip
Viewed
| HTML | XML | Total | Supplement | BibTeX | EndNote | |
|---|---|---|---|---|---|---|
| 309 | 71 | 26 | 406 | 44 | 11 | 12 |
- HTML: 309
- PDF: 71
- XML: 26
- Total: 406
- Supplement: 44
- BibTeX: 11
- EndNote: 12
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1