The driving weather datasets for the evaluation of the model were daily time series extracted from the Watch Forcing Data ERA-Interim (WFDEI; Weedon et al., 2014) for shortwave radiation, maximum temperature, minimum temperature and rainfall. For the demonstration of the system at a point, the driving data were based on daily, quality-controlled station data provided by the Ghana Meteorological Agency. The station used is Tamale, which is located in northern Ghana (9.41∘ N, 0.85∘ W; Fig. 2). Precipitation and maximum–minimum temperature were measured directly, and shortwave radiation was derived from sunshine hours. We chose to use Tamale because it is in the northern part of Ghana (Fig. 2) where most of the maize is grown. The station in Tamale also has a long-term record of the driving data for the crop model. It should be noted that TAMSAT-ALERT can in principle be run using any gridded meteorological data, like satellite rainfall estimates (e.g., TAMSAT (https://www.tamsat.org.uk/data/rfe/index.cgi, last access: June 2018), with a resolution of 4 km (Maidment et al., 2017).

Tercile forecast data were downloaded from the publicly available IRI regional forecasts (http://iri.columbia.edu/our-expertise/climate/forecasts/seasonal-climate-forecasts/, last access: June 2018). The IRI forecasts are based on a hybrid dynamical–statistical method developed by the U.S. National Oceanographic and Atmospheric Administration North American Multi-Model Ensemble Project (NOAA-NMME; Kirtman et al., 2014). The seasonal forecasts are issued at the beginning of each month for precipitation and temperature at a global scale with a spatial resolution of 2.5∘ for precipitation and 2∘ for temperature (Barnston and Tippett, 2014). The IRI forecasts were chosen for this analysis because of their wide use by African meteorological services and regional climate outlook forums. In this study the seasonal forecast data were used in the form they are supplied to farmers, i.e., tercile probabilities of 3-month cumulative rainfall and 3-month mean temperature at a regional level.

In addition to meteorological time series, GLAM requires data on soil type and the agronomic properties of maize (Sect. 3.2.2). For this study, the soil texture was set to be sandy loam and the planting date was set to start from the 124th day of the year to the 154th day of the year, which allows for a 30-day planting window. The maize agronomic properties were taken from the published literature and are presented in the Supplement (Table S1).

GLAM was evaluated against national-level maize yield data released by the FAOSTAT (http://www.fao.org/faostat/, last access: June 2018; see Fig. 4). Although the FAO issues guidance on the compilation of these datasets, in practice there is little quality control and the data should be treated with caution.

As described in Sect. 2.1, the TAMSAT-ALERT system can be used to assess any metric of risk that can be output by a model driven with meteorological data. In this study, the General Large Area Model (GLAM) for annual crops is used to simulate maize yield and subsequently to monitor the probabilistic risk of poor harvest as the growing season progresses.

GLAM is a process-based crop simulation model, which incorporates sufficient processes to capture the impact of climate variability on crop yield (Challinor et al., 2004; Ramirez-Villegas et al., 2015b). GLAM uses a limited number of driving datasets and an intermediate complexity of crop development process representation. Nevertheless, previous studies have demonstrated that GLAM has skill in capturing the impact of weather on crops (Challinor et al., 2005, 2006). Such information enables users to translate time series of weather into a time series of yield estimates (Challinor and Wheeler, 2008). GLAM has also been used to model weather and climate change impact on crop yield and adaptation strategies (Parkes et al., 2015; Ramirez-Villegas et al., 2015a; Ramirez-Villegas and Challinor, 2016).

GLAM requires daily values of precipitation, shortwave radiation, maximum temperature and minimum temperature as driving weather data with additional inputs of soil properties and planting window (Watson and Challinor, 2013). GLAM accumulates the aboveground biomass, which is a product of daily transpiration and a predetermined transpiration efficiency value, within the growing season to determine total biomass production, which is converted into yield using a harvest index (Osborne et al., 2007). The planting date is either prescribed by the user or determined using GLAM's intelligent planting date system (the approach taken in this study). It is important to note that GLAM does not account, in a process-based fashion, for non-meteorological influences on crop growth, such as pests, diseases and fertilizer use. Rather, these factors are encapsulated in the yield gap parameter (YGP), which is determined by calibrating the model yield with observed yield (Challinor et al., 2004). The YGP is assigned a value between 0 and 1, where 1 represents the potential yield given the weather conditions, soil texture and crop development parameters (Challinor et al., 2005).

GLAM was used to simulate the yield from 1994 to 2014 using the WFDEI as a driving dataset. The WFDEI has a 0.5∘ by 0.5∘ resolution and so GLAM was output at this resolution. The simulated yield at each grid point was then weighted by the year 2000 season fraction of production area over each grid point to make a country average yield (Weedon et al., 2014; Monfreda et al., 2008). This country average yield was then compared with the FAO maize yield dataset for the same period. It is shown in Fig. 4 that maize production can be split into two distinct periods: 1994–2006 and 2007–2014. Because of the reported changes in agronomic practice and drought-tolerant maize variety introduction through the drought-tolerant maize for Africa (DTMA) project (Obeng-Antwi et al., 2013; Ragasa et al., 2013) the transpiration efficiency (TE) value was increased from 7.0 for the period 1997–2006 to 8.0 for the period 2007–2014. The YGP was maintained at 0.4 for the whole simulation period.

The results of the simulated crop yield are presented in Fig. 5 and the statistical values of the comparison are presented with the scatter plot in Fig. 6. GLAM was able to maintain the overall mean yield and, and as a result the normalized root mean square error (NRMSE) is very low (0.07). The overall correlation value is found to be 0.67 (Pearson) and the Spearman correlation, which is less affected by outliers, is 0.8. The difference in the Spearman and Pearson correlation coefficients is mainly due to the severe overestimation of 2001 season yield, probably resulting from a long dry spell, the impact of which on farming practices was not fully accounted for by GLAM (FAO/WFP-Global Information and Early Warning System on food and agriculture, 2002). Some of the correlation strength is due to capturing the change in mean yield from the 1994–2006 to 2007–2014 period, and this is done by changing the transpiration efficiency (TE) value for the two periods. The strength of the correlation of yield suggests that the link between Ghana-wide weather and yield is moderate – an important consideration for policy makers when they make use of information from TAMSAT-ALERT. This is primarily due to the myriad of factors that can affect yield, including agronomic practice, pests and disease, and socioeconomic problems. Nevertheless, in vulnerable regions, the meteorological risk to yield is, in itself, an important consideration for agricultural agencies because action can be taken to mitigate the hazard. This might include subsidizing drought-resistant varieties or encouraging early planting and replanting.

Figure 7 shows how GLAM has been incorporated into the TAMSAT-ALERT system. As described in Sect. 2, time series of driving data based on historical observations are used both to derive climatological yield and to generate an ensemble of predicted yield. Individual planting dates are determined for each ensemble member using GLAM's intelligent planting date system, and the crop is harvested when the growing degree-day requirement is fulfilled (Challinor et al., 2004; Challinor and Wheeler, 2008). Because of the way TAMSAT-ALERT is set up to incorporate observational data continually as the season progresses, once the optimum planting time has passed for the year being hindcast, the planting date for each ensemble member converges. Analogously, once the harvest date for the hindcast year has passed in the observations, the harvest date, and indeed the predicted yield, for each ensemble member is identical. In this implementation of GLAM, a climatological period of 30 years (1980–2009) was used for the yield forecast.

Figures 8, 9 and 10 illustrate the implementation of TAMSAT-ALERT for the 2011 growing season, which nationally was a low yield year compared to other post-2007 years (noting that we do not have yield data for Tamale). The hindcasts were initiated every 5 days. GLAM infers that planting occurred on 4 June and harvesting on 15 September to 20 September. Figure 8 depicts all ensemble members in the context of the climatological spread in yield. Figure 9 shows histograms of ensemble members at monthly intervals, starting 10 days after planting. Figure 10 shows a time series of ensemble spread (standard deviation of ensemble yield predictions).

At the outset of the season, the yield estimates are derived only from the meteorological climatology; no in-season observational data are incorporated. The spread is thus large (equivalent to the climatology). During the season, as in-season data are incorporated by TAMSAT-ALERT, the meteorological time series driving GLAM become progressively more similar. As a result, the ensemble rapidly converges. In this example, for instance, 2 months after planting, the ensemble standard deviation is 34 % of the climatology.

The yield forecasts can be communicated with end users in a probabilistic form, with the ensemble expressed as quintiles representing the following categories: above the 80th, 60th–80th, 40th–60th, 20th–40th and below 20th percentile. These categories can be equated to very high, high, average, low and very low yield, respectively. An example of such quintile forecasts at monthly intervals during the 2011 growing season is shown in Fig. 11. Consistent with Figs. 8 and 9, at the outset of the season, the categories are equally likely except the extreme categories, the difference in probability coming from the change in planting date for some years in the climatological period considered (1980–2009). As the season progresses, the average and low categories become more likely and the extreme categories (very high and very low) less likely.

It is evident from Fig. 11 that the ensemble mean tends towards average or low values, even 2 months ahead of the harvest date during 2011, suggesting a degree of precision, even towards the beginning of the growing season. Section 3.3.3 presents a formal evaluation of skill for the 2002–2011 period.

As described in Sect. 2.1, the TAMSAT-ALERT framework can use probabilistic information from meteorological forecasts to weight the yield forecast ensemble, providing a means of incorporating forecast information into the decision support system. In this study, we consider tercile forecasts of cumulative 90-day rainfall and mean 90-day temperature to reflect the information currently available to the Ghana Meteorological Agency. The forecasts are commonly issued at the start of every month. Hence, we have applied the forecasts only to the meteorological season being forecasted with the remaining season not included in the weighting estimation. For example, for running TAMSAT-ALERT on 4 June, the seasonal forecast of June–July–August is applied. To illustrate the process of including forecasts, we continue with the 2011 case study. We have used idealized tercile seasonal forecasts for total June–July–August (JJA) precipitation to weight the forecast on 4 June 2011, July–August–September (JAS) precipitation to weight the forecast on 4 July 2011, August–September–October (ASO) precipitation to weight the forecast on 4 August 2011 and September–October–November (SON) precipitation to weight the forecast on 4 September 2011.

To assess the potential value of tercile rainfall and temperature seasonal forecast information, we have weighted the ensemble as if the next 90 days of temperature and cumulative rain are known (i.e., perfect forecast experiment). So, we consider three probabilistic forecasts: tercile weightings of [0,0,1] for the lower, middle and upper tercile, respectively (perfect wet forecast), [0,1,0] for the lower, middle and upper tercile, respectively (perfect normal forecast), and [1,0,0] for the lower, middle and upper tercile, respectively (perfect dry forecast). The ensemble was weighted by these perfect tercile forecasts according to the actual total rainfall (perfect rainfall forecast) or the actual mean temperature (perfect temperature forecast) that ensued in the next 90 days following each TAMSAT-ALERT hindcast.

Figure 12 shows the yield forecast probabilities when the perfect rainfall forecast is used. When a perfect rainfall forecast is used to weight the ensemble, the probabilities of the quintile forecast show more rapid convergence, especially 2 months into the season. The improvement is less noticeable in June and July, perhaps reflecting the fact that, at least in the GLAM crop model, cumulative rainfall in this part of the season is comparatively less strongly correlated with yield.

An alternative approach is to use temperature forecasts to weight the ensemble. To investigate the effect of temperature forecasts, the ensemble was weighted using idealized June–July–August (JJA) tercile temperature forecasts to weight the forecast on 4 June 2011, July–August–September (JAS) tercile temperature forecasts to weight the forecast on 4 July 2011, August–September–October (ASO) tercile temperature forecasts to weight the forecast on 4 August 2011 and September–October–November (SON) tercile temperature forecasts to weight the forecast on 4 September 2011. As with rainfall, the upper, middle and lower terciles are weighted [1,0,0] for a “perfect cold forecast”, [0,1,0] for a “perfect normal forecast” and [0,0,1] for a “perfect warm forecast”. Figure 13 shows the forecast for the 2011 cropping season with a perfect average temperature forecast. Due to a negative correlation of the average temperature with maize yield, a warmer temperature forecast is associated with predictions of lower yield. Comparison between Figs. 12 and 13 suggests that temperature forecasts have a greater effect on the risk assessments than rainfall forecasts.

So far, only idealized forecasts have been considered. In the next section, we demonstrate the effect of using actual tercile forecast information issued by the International Research Institute (IRI) for rainfall and temperature. The seasonal forecasts from IRI for 2011 in northern Ghana are shown in Table 2.

Figure 14 shows the yield forecast probabilities based on weighting the yield ensembles by seasonal rainfall forecasts from IRI. Comparison with Fig. 11 suggests that the weighting has little effect. Figure 15 shows the quintile yield predictions when temperature forecast weightings from IRI are applied. As with rainfall, comparison with Fig. 15 shows that the weighting has little effect.

The results are summarized in Fig. 16, which represents the probability of each yield pentile at different lead times in the 2011 season yield forecast with no seasonal forecast, precipitation forecast or temperature forecast applied. For all lead time periods indicated, weighting by IRI seasonal forecast for the 2011 season showed no improvement in predicting the final yield compared to the non-weighted values. This is not surprising because the relationship between the seasonal cumulative rainfall and seasonal mean temperature with maize yield is very low (see Figs. S1 and S2 in the Supplement). The tercile weightings for the IRI forecast (Table 2) are close to climatology, and the previous discussion showed that even a perfect and precise seasonal forecast has relatively little impact.

In summary, Figs. 12 and 13 indicate that if meteorological forecasts have sufficient accuracy and precision, they can add information to the decision-making process, especially in the middle to later part of the growing season. However, Figs. 14–16 show that the tercile forecasts currently issued in northern Ghana do not have sufficient precision information to yield risk assessments. A further application of TAMSAT-ALERT could be to investigate the level of skill that is required for meteorological forecasts to contribute useful information to such decision-making processes.

The objective of TAMSAT-ALERT is to provide early warning of the meteorological risk to yield, which is not an observable quantity. For this reason, evaluations of TAMSAT-ALERT skill are carried out in a “perfect model” framework, in which we attempt to forecast the yield simulated by GLAM forced with observed weather data. It is important not to confuse these skill assessments with evaluation of GLAM (Sect. 3.2.3), although the usefulness of the framework depends to a large extent on the quality of the model and data incorporated within it.

Figure 17 shows GLAM hindcasts at four approximate lead times (i.e., ∼ 3, ∼ 2, ∼ 1, ∼ 0.5 months ahead of harvest) for 5 years. Towards the outset of the season, the hindcasts for each year are similar and close to the climatology, with the minor differences explained by variation in planting date. For all the lead times considered the spread of the ensembles is reduced as the season progresses. Only the years 2007–2012 are presented in Fig. 17 because the maize variety changed in 2007, making the hindcasts of these years more relevant to the present day than the 1994–2006 period (see Fig. S3).

As described in Sect. 3.3.1, the probabilistic ensemble forecasts will be presented as the likelihood of quintile categories. The skill of the probabilistic forecast was assessed using the ranked probability skill score (RPSS). The RPSS is a skill score formulated from the ranked probability score (RPS) that compares the cumulative squared probability error for climatological forecasts in each category identified. The RPSS is negatively biased with smaller ensemble sizes (< 40) and due to this a correction was done on the reference RPS used before calculating the final RPSS. The bias-corrected RPSS is called the discrete ranked probability skill score (RPSSD). Details on the calculation and bias-correction are given in Muller et al. (2005) and Weigel et al. (2007). Positive values indicate better skill than the climatology; a unit value represents a perfect score and zero or below-zero values indicate no skill in the forecast.

The RPSSD for Tamale was derived for the period 2002–2011. This period is used because IRI seasonal forecasts for precipitation and temperature issued on a monthly basis are only available from 2002. Figure 18 indicates the skill scores for the four lead times for the forecasts made using the TAMSAT-ALERT system. The skill scores are generally above 0.4 for ∼ 2-month lead time and over 0.6 for ∼ 1-month lead time over the 10-year period considered. There are some years in which the skill score was lower than the stated values and this is mainly because of shifts in forecast categories towards the end of the season, which tends to happen if the yield is near a category boundary. For example, the 2011 final yield was in the low category, but 1 month earlier than harvest the ensembles indicate 56 % in the average category and 44 % in the low category (see Fig. 11), which results in a low skill score for that year. The overall skill of the system is presented in Fig. 19, which shows a good skill even 2 months ahead of harvest. The average RPSSD shows an increase in skill as the lead time decreases, which is expected. Comparison of similar period skill scores for yield forecasted weighted by the IRI seasonal weather forecast of rainfall and temperature showed a similar result to that of the non-weighted forecast. This indicates that the seasonal forecasts have little impact in predicting the maize yield in the region, which is associated with both the low correlation of seasonal weather values and maize yield and with the vague nature of the forecasts.