The coupled model SPEEDO consists of an atmospheric component (SPEEDY), that exchanges information with a land (LBM) and an ocean-sea-ice component (CLIO). Detailed descriptions of SPEEDO can be found in and . SPEEDY describes the evolution of the horizontal wind components U (east-west) and V (north-south), temperature T and specific humidity q at eight vertical levels plus the surface pressure ps. The horizontal grid resolution has a spacing of 3.75∘ (48×96 grid cells). SPEEDY exchanges moisture and heat with the land model, LBM, which uses three soil layers and up to two snow layers to close the hydrological cycle over land. The horizontal discretization of the LBM is the same as for SPEEDY. Moreover, SPEEDY exchanges heat, water, and momentum with the ocean model, CLIO. CLIO describes the evolution of ocean currents, temperature and salinity on a computational grid with 3∘ horizontal resolution and 20 unevenly spaced layers in the vertical resolution. A three-layer thermodynamic-dynamic sea-ice model describes the evolution of sea ice.

The SPEEDO equations can be formally and compactly written as 3aa˙=faa;pa+gaeh,ew,em,3bo˙=foo;po+goPoeh,Poew,Poem,Por,3cl˙=fll;pl+glPleh,Plew,r, where a stands for atmosphere, o for ocean and sea-ice and l for land; eh represents the heat exchange between the atmosphere and surface, ew the water exchange, em the momentum exchange and r the river outflow describing the streaming of water from land to ocean. The exchange vectors depend on the state of the atmosphere and the surface, but this dependency is not made explicit in Eq. (–c) to simplify the notation. The projection operators P represent the conservative regridding operations between the computational grids of the different model components. The nonlinear functions f represent the cumulative contribution of the modeled physical processes to the change in the state vectors, and depend on the values of the parameter vectors p. The nonlinear functions g describe how the exchange of heat, water and momentum between atmosphere, ocean and land affects the change of the state vectors.

The synch rule was originally conceived for parameter optimization in . We follow here a similar setting. Let us assume that the parameters q∈Rm appear linearly in the system for state variables y∈Rn, such that y˙=f(y;q). The synch rule ensures convergence towards parameters p∈Rm of the system for state variables x∈Rn, x˙=f(x;p), provided that synchronization between the systems occurs if the parameters of both systems are equal: y(t)→x(t) if p=q. The update of parameter qj for the jth component of q reads: 5ax˙=f(x;p),5by˙=f(y;q)-K(y-x), 5cq˙j=-δj∑iei∂fi(y,q)∂qj, where f denotes the evolution function and K(y-x) a connecting term between the two systems that nudges y towards x. K∈Rn×n is a diagonal matrix of nudging coefficients, K=diag(k). Furthermore, ei=yi-xi denotes the ith component of the synchronization error, and δj an adjustable rate of the learning scaling factor.

We have extended the use of the synch rule to the training of supermodels . In this context q refers to the supermodel weights, y to the supermodel state and x to the observations. The synch rule is initialized with certain values for q and during training the weights are updated according to the rule, such that the supermodel synchronizes with the observations. In order to keep the supermodel in the vicinity of the observations, the supermodel is nudged towards the observations by the term K(y-x).

The CPT learning approach is based on an idea proposed by . It dynamically combines trajectories of different models, such that the solution space is virtually extended. The aim is to generate trajectories that more closely follow the truth. In , this idea has been developed into a supermodel training scheme.

The training phase of CPT starts from an observation. From the same initial state, the imperfect models run for a predefined cross pollination time, τ, until an observation is available. The individual model predictions are then compared to the observation, and the model state that is closest to the observation will serve as the initial condition for the next integration. In our experiments, the “closeness” to data is measured using the global root mean squared error (RMSE). In the case of a multidimensional (multivariate) model, such as SPEEDO, it is possible that at certain time steps different models are the closest to the truth for different state variables. In this case, the initial condition for the next run is constructed by combining the portion of the state vector of each closest model state. This choice, while providing the closest to data initial condition, is prone to create imbalances in the model integration. Nevertheless, we experienced that as long as the update is global, such that each grid point receives the state of the same model for a certain variable, these imbalances are not a big issue. Otherwise, a possible solution is to use techniques from data assimilation to make the initial condition suitable for the individual models. An example is given in by using pseudo orbit data assimilation (PDA).

After the training, a CPT trajectory is obtained as a combination of different imperfect models, and we count how often each model has produced the best prediction of a particular component of the state vector during the training. These frequencies are then used to compute weights W for the corresponding states of the models. The superposition of the weighted imperfect model states forms the supermodel state. Since the frequency is used to compute the supermodel weights, the weights automatically sum to 1, which is also functional to maintain physical balances. See for a more extensive and figurative explanation of the CPT training scheme.

Before we start training the supermodel, we need to decide when, where and how to let the models exchange their information in order to create a weighted supermodel.

Following , we use global weights for both CPT and the synch rule. This means that we use the same weight for all grid points. By doing so we mitigate, and in the best case prevent, numerical instabilities; however, note that different weights are allowed for each variable.

As long as there are enough observations to capture the global behavior of the different models, spatially sparse observations are not expected to be an issue when constructing a weighted supermodel. Given that we focus here on the data sparsity in time, in the experiments we assume that all grid points are observed.

The prognostic variables exchanged between models are temperature, vorticity and flow divergence. The weights for the fluxes from atmosphere to ocean and to land are given by the average of the weights for the three prognostic variables. The SPEEDO time step during training is set to δt=15 min.

Following , the training period for both CPT and the synch rule is 1 year. For CPT, the supermodel weights are calculated every week or every second week in the case of ΔT=24 h, and the model states are set back to the observations. For the synch rule the training period continues for an entire year. This amount of time is needed to obtain stable converged weights.

The codes for both training methods of CPT and the synch rule in the experiments in this paper are integrated into the SPEEDO code. After the individual models have made their individual time steps, their states are exchanged between the models with coupling routines. Once all models have shared their knowledge, they can calculate the new supermodel state and the update of the weight according to the training method. The SPEEDO CPT and synch rule supermodel training code is available in .

In the synch rule, rewritten from Eq. (–c), looked as follows: 9W˙i,j=-δjejfi,j, where Wi,j denotes the weight of model i for state variable j, fi,j the imperfect model tendency of model i and state variable j, ej the synchronization error between the supermodel state and the observations, and δj an adjustable rate of the learning scaling factor. This equation is derived without any prior assumption on the weights. In the context of noise-free and continuously available observations, the weights turned out to sum approximately to 1, which seems necessary in order to maintain physical balances. Nevertheless, when Eq. () is used in the case of noisy and sparse in time observations, the weights do not sum to 1 anymore. If the deviation from 1 is too large, the supermodel state will be either too small or too large compared to the imperfect model states, possibly resulting in loss of synchronization with the observations and an even worse estimation of the next weight update.

We adapt the synch rule such that the weights are imposed to sum to 1. This is achieved by using the tendency of the individual imperfect model fi, but also by subtracting the equally weighted supermodel tendency fE=1N∑i=1Nfi. The new synch rule is defined as (see Appendix A for a derivation): 10W˙i=-δefi-fE, where index j is omitted to simplify the notation. From Eq. () it can be seen that the total update of the weights for the N imperfect models equals 0: ∑i=1NW˙i=-δe∑i=1N(fi-fE)=-δeN(fE-fE)=0. Thus, if the initial weights sum to 1, they will sum to 1 continuously throughout the training.

In this section we assess to what extent observations can be noisy and sparse in time before the CPT or the synch rule training methods are no longer able to produce weights close to the optimum. To systematically evaluate this, we choose 4 different observation frequencies ΔT: 15 min, 1 h, 6 h and 24 h. Since for the standard CPT training time of 1 week the weights for ΔT=24 h would only be based on 7 steps, the training time for this observation frequency is doubled to 2 weeks. The error in the observations is unbiased and Gaussian distributed ∼N(0,σ), where standard deviation σ is chosen to be equal to either 0.5 %, 2.5 % or 5 % of the spatial standard deviation σX of the observations per prognostic variable X. Hence σX=1N∑iXi-X‾2, with i ranging over all N=96×48×8 grid points and X‾ denoting the spatial mean value. For temperature this corresponds to a standard deviation of ∼0.15, 0.75 and 1.5 ∘C. Table  denotes the chosen nudging coefficient K and the resulting weights together with their variance. For the experiments with Δt=15 min the same nudging strength K is chosen as in : K=1/24 h-1, which corresponds to K=1 d-1. All CPT experiments are performed with the iterative method.

Figure  shows the weights from Table  in one plot such that the differences between the methods become clear. The horizontal lines (continuous for model 1 and dashed for model 2) indicate the weights obtained by CPT and the synch rule in , in which case the observations were perfect and available at every time step. Despite the optimal weights for ΔT=15 min they are not necessarily expected to be optimal for, e.g., ΔT=24 h, in this particular experiment the 2 cases show similar weights. From Fig. a and b it can be seen that if observations are available at each time step (ΔT=15 min), the synch rule gives slightly better results for noisier observations than CPT. For the synch rule, the weights turn out to be almost exactly the same for all three levels of noise. For ΔT=1 h, the results for CPT and the synch rule seem similar for the two lowest noise levels. For the highest noise level, both methods seem to struggle a bit more to obtain good weights, since the models are more equally weighted. Decreasing the observation frequency further to ΔT=6 h results in the same pattern with CPT performing slightly better. Once good CPT weights have been found, they remain very consistent throughout the year, indicated by the standard deviation of 0. For the largest observation window ΔT=24 h, again the synch rule seems to encounter somewhat more difficulties in following the temperature observations for the highest noise level. Overall however, both methods perform well in the context of sparse and noisy observations.

From we know the performance of supermodels with CPT and synch rule weights trained with perfect, noise free observations. The weights for the different experiments in varied within a range of ±0.05 per variable. We did not find any significant difference in the forecast performance of the supermodels for these weights. Since the weights in the experiments in this paper are approximately within this range, we can foresee the outcome of model performance experiments. To see this point, we compared the short-term forecast performance of the supermodels trained by perfect observations (s-CPT/synch perf obs), and the supermodels trained by observations available every 24 h(s-CPT/synch noisy obs), with the highest noise level we used in this paper (see Fig. ). Not surprisingly, the supermodels trained with perfect observations are slightly better than the two supermodels trained with sparse and noisy observations. The supermodels trained with the observations available every 24 h, also combine in the forecast phase the states every 24 h, which introduces a small shock. The supermodel differences in the 2-week forecast, however, are very small. The synch rule supermodel trained with sparse and noisy observations performs least well, but one would expect this result as the weights for temperature are clearly a bit different from the other supermodels. Still, the model skill is not far from the other supermodel skills.

Imperfect models 1 and 2 complement each other in important physical variables such as temperature and wind. Model 1 tends to overestimate their global average values, while model 2 underestimates them. Together they form a convex hull , which results in positive supermodel weights. On the other hand, using model 1 and model 3 to construct a supermodel implies the need for negative weights. The synch rule naturally allows negative weights, since we did not impose any restrictions on the weights. In synch rule training has been performed with model 1 and model 3, resulting in a supermodel with partly negative weights that outperformed both imperfect models in short-term and long-term forecast quality.

CPT training does not automatically produce negative weights, since the weights are based on the frequency by which the imperfect models are chosen. Nevertheless, CPT training can give negative weights too, although with boundary restrictions. In the standard CPT training, one chooses whether one of the imperfect models is the closest to the observations, or in addition, whether the supermodel is closest in the iterative method. To obtain negative weights one can also choose a predefined combination of the imperfect models, for example: xneg1=αx1+(1-α)x3, with α∈R. If one defines an additional predefined combination xneg2=(1-α)x1+αx3, the range for weights w1 and w3 for imperfect models 1 and 3 is between α and 1-α if α∉[0,1].

In this experiment we choose α=-1, such that w1, w3∈[-1,2]. The experiment is the same as in : an observation is available for every time step , for every time step either xneg1 or xneg2 per variable is chosen as the closest model state and the training period is 1 week. Table  shows the weights and associated variance of the weights. The weights are remarkably similar to the weights of the synch rule experiment with negative weights in . The weights for vorticity and divergence differ by 0.16, the weights for temperature by only 0.03.

The statistics of a 40-year supermodel run with the weights from Table  are therefore quite similar to the climatology of the supermodel in . Table  shows that the supermodel outperforms both imperfect models in temperature, precipitation, wind, cloud cover and surface solar radiation compared to the values in Table .

Despite the application of the iterative method and nudging, the CPT and synch rule may still struggle to stay very close to the observations. To increase the chance to obtain a proper trajectory, one could work with an augmented ensemble of trajectories. This ensemble could consist of trajectories starting from slightly different initial conditions, or trajectories that emerge from a model nearby the closest model to an observation. One could make a comparison with the particle filter method (see e.g., ), where trajectories that do not fall within the likelihood of the observations are pruned and one continues with the trajectories within the likelihood of the observations from slightly perturbed initial conditions. In our case, the best trajectory after training can be obtained by comparing the RMSE between the trajectories and the observations.

Both training methods seem in principle more suitable for short rather than longer timescales, since for both training rules it is important that the imperfect models stay close enough to the observations. For longer timescales this can be difficult. Despite the action of the nudging, the models can be out of the data phase as long as time evolves. If the observations are lost, the “closest” model in the CPT training is not necessarily the one that contributes most to improving the supermodel dynamics. If during synch rule training the supermodel loses the observations, a new, non-optimal equilibrium for the weights can be found, as we have seen in Sect. . Having said this, both methods could still be useful if one prefers to combine models only on a seasonal or even longer timescale (under the assumption that with this limited amount of exchange the models are still synchronized to some extent). For both CPT and the synch rule, one can average over the observations to potentially obtain a correct sign whether the supermodel is either overestimating or underestimating the observations.

Until now the distance between models and model to data has been the RMSE. If one is training a supermodel with improved skill on longer timescales, it is possible that the appearance of specific climatological features of the models is of more importance than a small RMSE. In that case the distance between observations and models can be defined in a different way. For example, if the imperfect models suffer from an erroneous double intertropical convergence zone (ITCZ), one can increase the weight of the model which is on average closer to a single ITCZ. Additionally, one can define different weights for different periods of time, for example seasonally dependent weights. Despite these possibilities in adapting the training methods, there are some conditions that need to be fulfilled when CPT or the synch rule are used on longer timescales. The methods only work if the models can compensate for each other. For example, when both models have been spun up for a sufficient amount of time and are stable in state space, both CPT and the synch rule cannot give useful weights. In the case of CPT, the model that is on average closest to the observations will be repeatedly chosen. For the synch rule, the average model tendency will be zero over a sufficient amount of time, hence there will be no update of the weights on average over time. Therefore, for both training methods the imperfect models cannot already reside on their own attractor and the tendency towards their attractor needs to be visible.

To make the training methods suitable for state of the art models it needs to be taken into account that state of the art models can differ in grid point resolution and time steps. In this paper, for both CPT and the synch rule during training the imperfect model states are replaced, in the case of the synch rule the imperfect model states are replaced by the new supermodel state, and in the case of CPT the imperfect model states are replaced by the state of the closest model. To apply the training methods in state of the art models, techniques from data assimilation can be used to combine the states in a dynamically consistent manner . With the use of these techniques both CPT and the synch rule in principle seem to be suitable for training a state of the art supermodel.