Numerical climate models are no exception to the often quoted statement “all models are wrong, but some are useful” from . Indeed, their climate projections (simulated responses to a scenario of greenhouse gas and aerosol emissions) are useful to assess future climate change, but they vary widely from one climate model to another e.g. Fig. 4.2 in IPCC in:. There are now several dozen climate models around the world.

To assess the future value of a climate variable, such as global temperature in 2100, a traditional approach is to examine the distribution of projections of the variable simulated by an ensemble of climate models. The climate variable projected by climate models is hereafter referred to as the projected variable. The mean and standard deviation, which characterise the distribution of the projected variable, are usually used to define the so-called best guess and uncertainty of the projected variable, respectively . However, this uncertainty is generally high, and the best guess may be biased. To incorporate knowledge of real-world observations, statistical methods called observational constraints (OCs) or emergent constraints examine the distribution of the projected variable given real-world observations of an observable variable to obtain a constrained distribution. Such OC approaches are now used in the reports of the Intergovernmental Panel on Climate Change (IPCC) from 2021. They have huge implications for our society. The literature on OC methods is flourishing, but there are many difficulties in using them.

Firstly, the large number of existing OC methods makes it very difficult to choose one. Some methods average the projections of climate models, with weights that depend on the ability of the models to reproduce real-world observations of a given observable variable . Some methods use climate models to learn a relationship between the projected variable and a related observable variable and use this relationship and a real-world observation of that observable variable to predict the value of the projected variable. This relationship may be linear or non-linear . Other methods statistically provide the constrained distribution as the probability density function of the projected variable given the real-world observation of an observable variable . This diversity illustrates the lack of consensus on which approach to use. Methods are developed individually and need to be compared to better understand their differences and similarities, as done, for example, in .

Secondly, the approaches and assumptions used to compute the constrained distribution can vary widely between articles and are not always reported. For example, their calculation does not always take into account the instrumental error associated with the real-world observation. Some papers provide clarification, e.g. , which proposed a comprehensive review of the underlying assumptions and uncertainty calculation in OC methods based on linear regression. However, some elements are still missing from the literature. For example, the terms “very likely”, “unlikely” etc. used by the IPCC come from an underlying statistical model that provides a confidence interval, i.e. an interval that contains the projected variable value with a given confidence level. OC methods rarely use or describe a confidence interval. There is therefore a need for a proper statistical description of the theoretical basis of OCs, including confidence intervals, and a full description of the underlying statistical assumptions .

Thirdly, OC articles often use a univariate framework, i.e. they constrain the projected variable using only one observable variable. This may be surprising given the complexity of the climate system, which suggests that the spread between climate model projections may be related to multiple processes. For example, constrained the equilibrium climate sensitivity (ECS) using a measure of temperature variability. A few studies, particularly those using non-linear regression, employed a multivariate framework, but these are still rare. For example, constrained future spatio-temporal gross primary production (GPP) by past spatio-temporal GPP and temperature.

To address these challenges, this article proposes a statistical model called ClimLoco1.0, which stands for “CLimate variable confidence Interval of Multivariate Linear Observational COnstraint”. ClimLoco1.0 expresses the confidence interval of a climate variable constrained using a linear multivariate observational constraint that takes into account observational noise. ClimLoco1.0 can also be used in univariate as well as multivariate form. This is the first version, 1.0, calling for further improvements to better account for all uncertainties. This article builds ClimLoco1.0 in progressively increasing complexity by expressing the confidence interval of the projected variable as (Sect. ) unconstrained, (Sect. ) as constrained by noiseless real-world observations, and (Sect. ) as constrained by a noisy real-world observation, as represented in Fig. 1. The latter corresponds to ClimLoco1.0. Since the devil can be hidden in the details, the article presents the statistical procedure in a rigorous and clear manner, based on mathematical demonstrations. Moreover, the use of this complex statistical procedure is justified by illustrations of the underestimation of the uncertainty usually made in the literature by not using rigorous CIs. These results are then compared with some of the most widely used methods in the literature (Sect. ): statistical methods as in or and methods based on linear regression as in . Finally, the assumptions are summarised and discussed (Sect. ).

In addition to providing the mathematical demonstrations, the appendices supply three valuable pieces of information. (i) A summary of the mathematical results in Tables  and . (ii) A section that explains all the key statistical concepts useful for understanding all the details of the article (Appendix ). (iii) A case study, illustrating the use of ClimLoco1.0 and testing its sensitivity to some parameters (Appendix ). The (Python) code and data that accompany the article are provided, as well as a user-friendly, simple example to replicate ClimLoco1.0.

In order to anticipate society's adaptation and mitigation plans in response to climate change, it is necessary to estimate the value of a future variable called the projected variable and denoted Y, e.g. the global temperature in 2100. A common approach is to use an ensemble of climate model projections, e.g. CMIP6 (Coupled Model Intercomparison project version 6), which give different values of Y. Y is therefore a random variable; the dispersion between the climate model projections results from the randomness of Y.

To properly estimate the value of Y, this section defines the confidence interval (CI) of Y. It provides a best guess of Y value (centre of the interval) and an associated uncertainty (width of the interval) at a given confidence level. This section gradually builds up the CI of Y in increasing complexity. Firstly, it defines the probability interval (PI) of Y obtained assuming that the theoretical distribution of Y is known. Secondly, it defines the CI of Y obtained using this distribution estimated based on an ensemble of climate models. These two types of intervals are illustrated and interpreted using a synthetic example.

As stated above, the PI of Y is built using the theoretical distribution of Y. Here, this distribution is assumed to be Gaussian: Y∼N(μY,σY2), where μY and σY2 are respectively the expectation and variance of Y. The PI of Y is the interval that contains Y values with a probability of 1-α: 1IPμY-zσY≤Y≤μY+zσY=1-α, where z is the quantile of order 1-α/2 of a distribution N(0,1). For example, the 90 % PI (α=0.1) is obtained with z=1.65. In the IPCC, this 90 % probability corresponds to the term “very likely”, while “likely” stands for the 66 % probability, etc. The level of probability, i.e. 1-α, is a choice of the user. In the following, the PI of Y associated with a probability of 1-α described in Eq. () is denoted: 2PI1-α(Y)=μY±zσY.

In fact, the expectation μY and the standard deviation σY are unknown. The PI described by Eq. () is therefore unknown. However, μY and σY can be estimated from an ensemble of climate model projections chosen by the user, for example, from CMIP6. This ensemble of M climate model projections yields a sample of M random variables, denoted (Y1, …, YM). These random variables are assumed to be independent and to follow the same law as Y, which is assumed to be N(μY,σY2). As a reminder, all the assumptions used in the article are summarised and discussed later in a dedicated section (Sect. ). The classical estimators of the expectation μY and variance σY2 are: 3μ^Y=1M∑i=1MYi,4σ^Y2=1M-1∑i=1MYi-μ^Y2.

The literature usually replaces μY and σY with their estimators μ^Y and σ^Y to estimate the PI [μY±zσY], which gives the interval [μ^Y±zσ^Y]. This interval has no clear statistical meaning. In fact, μ^Y and σ^Y are random variables that depend on M, the number of climate models used. The quality of these two estimators affects the quality of the interval. It can be shown (see Appendix ) that using these estimators μ^Y and σ^Y, the values of Y are contained in the following interval with a probability of 1-α: 5IPμ^Y-tM-1σ^Y1+1M≤Y≤μ^Y+tM-1σ^Y1+1M=1-α, where tM-1 is the quantile of a Student distribution with M-1 degrees of freedom associated with the probability 1-α. For example, with a confidence level of 90 % (α=0.1), t30=1.70 and t5=2.02.

A subtle point is that this interval described in Eq. () is not a probability interval (PI) but a so-called confidence interval (CI). For example, the 90 % PI of Y is an interval that has a 90 % probability of containing Y values. It has deterministic bounds that frame a random variable. The CI of Y has random bounds, which also frame a random variable. In fact, the CI of Y described in Eq. () has random bounds because μ^Y and σ^Y are random variables. Thus, different sample realisations, e.g. from different ensembles of climate models, will lead to different realisations of this CI. There is a confidence of 90 % that one realisation of the CI contains Y. In other words, out of 100 realisations of the 90 % CI, 90 should contain the value of the random variable Y. This is illustrated in Fig. , which shows 100 realisations of this CI of Y (error bars) at a 90 % confidence level, as well as 100 realisations of Y (red dots). This specific type of CI is also often called a prediction interval.

The CI of Y associated with a confidence level of 1-α is then denoted: 6CI1-α(Y)=μ^Y±tM-1σ^Y1+1M.

Now that the CI of Y is defined, it allows us to study the effect of the number of climate models considered (M) on this CI. Throughout the paper, the same synthetic example is used, defined in Appendix . It provides the theoretical PI of Y, which is unknown in reality and estimated by the CI of Y. It also provides one realisation of the CI of Y using a small (M=5) sample (Y1, …, YM), and another using a large sample (M=30). These different samples can, for example, represent different ensembles of climate models (CMIP5, CMIP6, HighResMIP, …). Figure  shows the probability interval of Y, PI1-α(Y) defined in Eq. (), and the realisations of the two CI of Y (M=5 and M=30), CI1-α(Y) defined in Eq. (). In reality, the PI is unknown. This synthetic example allows us to compare the estimates with the truth.

There are two important remarks about the CI of Y described in Eq. (). Firstly, it converges in probability to the PI of Y described in Eq. () as M, the number of climate models considered, increases. Indeed, as M becomes large, the estimates μ^Y and σ^Y (Eqs.  and ) converge (in probability) to μY and σY, and the Student quantile converges to a Gaussian quantile. This is illustrated in Fig. , by comparing the PI of Y (left error bar) with the realisations of the CI of Y (middle and right error bars). Indeed, the large sample gives a CI of Y (right interval, at CI=[0.1±1.9]) closer to the PI of Y (left interval, at PI=[0±1.6]) than the small sample (middle interval, at CI=[-0.8±2.2]). To accurately estimate both the centre and the width of the CI of Y, which represent the best guess and the uncertainty, respectively, it is therefore necessary to have as large a sample as possible. Secondly, the fewer the models, the larger the CI of Y. It is intuitive that estimating the CI of Y with less data will give a more uncertain result. Indeed, in Eq. (), the terms tM-1 and 1+1M are larger when M is smaller. These two aspects highlight the importance of having as many climate models as possible. However, the climate models considered must be independent, and the simulated variables must follow the same distribution as the real variables – two assumptions necessary for the calculation that are not fully satisfied .

In the literature, the PI of Y, i.e. [μY±zσY], is often estimated as the empirical interval [μ^Y±zσ^Y]. However, as seen previously, this interval has no statistical basis, whereas CI1-α(Y)=μ^Y±tM-1σ^Y1+1M contains Y values with a given probability. The relative error of the interval width caused by using the wrong interval [μ^Y±zσ^Y] instead of the CI is therefore quantified as relative error in the width of this wrong interval (zσY) compared to the width of the CI tM-1σ^Y1+1M: 7E1=z-tM-11+1MtM-11+1M. This relative error, which depends on the sample size (M) and the confidence level (1-α) controlling z and t, is plotted as a function of these two parameters in Fig. . For typical sample sizes of ensembles of climate models, between 5 and 50, the relative error is between 3 % and 30 %. For example, with a confidence level of 68 % (z=1, “likely” in IPCC language) and a sample size of 20 climate models, the relative error is 5 %. Since the width of the interval represents the uncertainty, this means that the uncertainty is underestimated by 5 %, which is even higher for smaller sample sizes or higher confidence levels. This highlights the importance of using the rigorous formula provided in this article to express uncertainty more accurately.

Without even mentioning observational constraints, this section provides statistically sound formulae for estimating an interval that contains the value of a future variable from the projections of an ensemble of M climate models at a given confidence level, using confidence intervals. This brings a rigorous insight to climate science, where the simple mean and standard deviation are commonly used. The next part applies the same methodology to a linear observational constraint.

Observational constraint (OC) methods have been developed to estimate more accurately the value of a projected variable Y. These methods “constrain” the distribution of Y by a “real-world” observation, denoted x0, of an observable variable X. X has to be chosen by the user, as well as the observational dataset providing x0. In this section, the real-world observation is assumed to be noiseless (no observational noise, e.g. due to instrumental errors). This assumption is relaxed in the next section which defines ClimLoco1.0. The general formulation presented in this article can be applied to the choice of any arbitrary variables X and Y. The variable Y constrained by the observation x0 of X is written as Y|X=x0.

As mentioned in the Introduction, many articles in the literature use univariate OCs, i.e. only one observable variable X is used to constrain Y. This can be very limiting, especially when Y depends on many processes, which is often the case in climate science. Therefore, an important contribution of this article is to give the formulation in a multivariate form, i.e. X∈Rp with p the number of observable variables. However, for the sake of clarity, only the results for the univariate formulations are presented in the main part of the article. The multivariate formulations are given in Table .

This section gradually builds up the CI of Y|X=x0 in increasing complexity. Firstly, it defines the probability interval (PI) of Y built using the theoretical distribution of Y|X=x0 by assuming that this distribution is known. Secondly, it defines the CI of Y|X=x0 obtained using this distribution estimated based on an ensemble of climate models. These two types of intervals are illustrated and interpreted using a synthetic example.

As stated above, the PI of Y|X=x0 is build using the theoretical distribution of Y|X=x0. Here, this distribution is assumed to be Gaussian: Y|X=x0∼N(μY|X=x0,σY|X=x02), where μY|X=x0 and σY|X=x02 are, respectively, the expectation and variance of Y|X=x0. In the following, the PI of Y|X=x0 associated with a probability of 1-α is denoted as: 8PI1-αY|X=x0=μY|X=x0±zσY|X=x0, where z is the quantile of order 1-α/2 of a distribution N(0,1). In order to express the terms μY|X=x0 and σY|X=x02 in Eq. (), a linear regression framework is used: 9Y=E[Y|X]+ε,whereE[Y|X]=a0+a1X,ε∼N0,σε. The coefficients a0 and a1 are the intercept and the slope of the linear regression of Y on X, respectively, and ε is a random variable representing the regression error with σε its standard deviation. Using this linear regression model, it can be shown (see the proof in Appendix ) that the terms μY|X=x0 and σY|X=x02 can be expressed as: 10μY|X=x0=a0+a1x011=μY+ρσYσXx0-μX,12σY|X=x02=σε213=1-ρ2σY2, where μX and μY are the expectations of X and Y, σX and σY are the standard deviations of X and Y, and ρ is the linear correlation between X and Y. The PI of Y constrained by X=x0 described by Eq. () can then be rewritten as: 14PI1-αY|X=x0=a0+a1x0±zσε. To illustrate this, the same synthetic case study as before is used, detailed in Appendix . The PI of Y constrained is shown in Fig. a and b (in red) and is compared with the PI of Y unconstrained (in black) in Fig. b. The constraint on Y has two effects: (a) it changes the best guess (centre of the interval) and (b) it reduces the uncertainty (width of the interval).

a.

When Y is constrained (PI(Y|X=x0)), it has a different best guess (centre of interval) than when it is unconstrained (PI(Y)). We interpret this in two different ways, using Eqs. () and (). The first equation gives a graphical interpretation: the constrained expectation of Y is directly the real-world observation fed into the regression. This is illustrated in Fig. a. The second equation is useful to understand the correction between the best guess of Y constrained and unconstrained: μY|X=x0-μY=ρσYσX(x0-μX). It depends on two terms: the regression slope ρσYσX, which depends in particular on the correlation between X and Y, and the difference between the real-world observation and the theoretical centre of the climate models distribution on X. It is called here (x0-μX) the theoretical multi-model bias. In other words, the constrained best guess of Y (μY|X=x0) is a corrected version on the unconstrained best guess of Y (μY), knowing the theoretical multi-model bias of X (x0-μX) and the relationship between X and Y ρσYσX. In the example of Fig. a, there is a positive theoretical multi-model bias associated with a positive relationship between X and Y, thus a correction to a higher best guess (μY|X=x0>μY). Observational constraints are generally used to reduce uncertainty, but the correction of the best guess between constrained and unconstrained is very important and should not be forgotten, as it allows for correcting the multi-model bias.

The previous results were obtained under the assumption that the real-world observation x0 is not noisy. In reality, x0 is affected by observational noise, which is taken into account in this section, inspired by the theory of measurement error models . Some papers define observational noise as internal variability e.g., others as measurement error e.g., and others as both e.g.. We argue here that both internal variability and measurement error should be taken into account, as both affect the real-world observation. Let XN be the noisy version of X, linked by the noise model defined in : 17XN=X+N,withN∼N0,σN2andN⟂⟂X, where N is a random variable representing the observation noise, assumed to be Gaussian, centred, and independent of X. Its variance σN2 is assumed to be known. The projected variable Y constrained by the observation x0N of XN affected by the observation noise is denoted Y|XN=x0N.

This section constructs ClimLoco1.0, which is the confidence interval of Y|XN=x0N, in increasing complexity, following the same steps as in the previous two sections. Firstly, it defines the probability interval (PI) of Y|XN=x0N obtained using the theoretical distribution of Y|XN=x0N by assuming that the distribution of Y|XN=x0N is known. Secondly, it defines the CI of Y|XN=x0N obtained using this distribution estimated based on an ensemble of climate models. These two types of intervals are illustrated and interpreted using a synthetic example. These different steps that construct ClimLoco1.0 are crucial to define and understand with rigour the best guess and uncertainty of any variable constrained by a noisy observation.

As stated above, the PI of Y|XN=x0N is built using the theoretical distribution of Y|XN=x0N. Here, this distribution is assumed to be Gaussian: Y|XN=x0N∼N(μY|XN=x0N, σY|XN=x0N2), where μY|XN=x0N and σY|XN=x0N2 are, respectively, the expectation and variance of Y|XN=x0N. The following interval is the PI of Y|XN=x0N, i.e. it contains Y|XN=x0N values with a probability of 1-α: 18PI1-αY|XN=x0N=μY|XN=x0N±zσY|XN=x0N, where z is the quantile of order 1-α/2 of a distribution N(0,1). This interval contains realisations of Y with a given confidence 1-α controlling z. To express the parameters μY|XN=x0N and σY|XN=x0N2, a linear regression framework is used, as in the previous section: 19Y=E[Y|XN]+εN,whereE[Y|XN]=b0+b1XN,εN∼N0,σεN, and where b0 and b1 are the intercept and slope of the linear model, respectively, and εN is a random variable representing the regression error with σεN its standard deviation. This linear regression is the regression of Y on XN. However, climate models do not suffer from observational noise (instrumental error and internal variability): they provide realisations of X, not XN. In fact, climate models do not suffer from instrumental error, but they can be affected by internal variability. However, the impact of internal variability can be reduced, for example, by averaging the members of a given climate model (different realisations run from different initial conditions). As climate models provide realisations of X and not XN, it can be difficult to express the linear coefficients b0 and b1. However, using the model noise described by Eq. (), which relates XN to X, it is possible to obtain the expressions of b0 and b1 and hence the expression of μY|XN=x0N and σY|XN=x0N2. In fact, it can be shown (see Appendix ) that: 20μY|XN=x0N=b0+b1x0N21=μY+ρσYσX11+1/SNR2x0N-μX,22σY|XN=x0N2=σεN223=1-ρ21+1/SNR2σY2, where ρ is the correlation between X and Y and SNR=σX/σN is the signal-to-noise ratio (where X is the signal and N is the noise). Using correlation and signal-to-noise ratio to express the equations is inspired by . Equations () and () are very useful because they use parameters related to X, not XN. The parameters b0, b1, and σεN2 can thus be computed using the sample of X, which is noiseless. This formalisation is possible because of the theory of measurement error models developed by . Using Eqs. () and (), the PI of Y|XN=x0N described Eq. () can be rewritten: 24PI1-αY|XN=x0N=b0+b1x0N±zσεN. These results are interpreted mathematically and graphically using the same synthetic case study as before, detailed in Appendix . Figure a shows the PI of Y constrained by a noisy observation, PI1-α(Y|XN=x0N) (green interval). It also shows how it is constructed by plotting the linear regression y=b0+b1x (green line) and its error σεN (green shading). For comparison, it also shows the linear regression and its error obtained when the observational noise is neglected, as in the previous section, in red. Figure b compares PI1-α(Y|XN=x0N) (green) with the PI of Y constrained by a noiseless observation (red) and unconstrained (black), respectively, PI1-α(Y|X=x0) and PI1-α(Y).

The expression of the PI of Y constrained by a noisy observation (PI(Y|XN=x0N)) has a form close to that in which the observational noise was neglected, as in the previous section (PI(Y|X=x0)). As before, the expectation of Y constrained is directly the real-world observation fed into the regression (see Eq. ), and the variance of Y constrained is the variance of the regression error (see Eq. ). The constraint corrects the expectation (see Eq. ) and reduces the variance (see Eq. ). However, the difference between including or not including the observational noise (difference between green and red in Fig. ) lies in a term called here the attenuation coefficient: 1/(1+1/SNR2). The slope considering the observational noise, b1, is attenuated compared to the slope neglecting the observational noise, a1: b1=a1/(1+1/SNR2). The larger the observational noise, the greater the attenuation. This is illustrated in Fig. a, where the linear relationship is stronger when the observational noise is neglected (red) than when it is included (green). In this example, there is as much signal as noise (SNR=1). The attenuation coefficient is therefore 50 %. In reality, depending on the application, the observational noise can be very small, although it is difficult to estimate, especially for low-frequency internal variability, which can lead to serious overconfidence . This attenuation coefficient, 1/(1+1/SNR2), weakens both the expectation correction and the variance reduction, as described in Eqs. () and (), respectively. This highlights the need to account for observational noise; otherwise the PI of Y constrained will be overconfident, with too strong an expectation correction.

The use of the PI of Y|XN=x0N described by Eq. () requires knowledge of the parameters b0, b1, and σεN, which are unknown in reality. As in the previous section, an ensemble of M climate model projections is used to estimate them. The estimators of a0, a1, and σεN are given in Table . Using them, it is shown in Appendix  that the confidence interval (CI) of Y constrained by a noisy observation is: 25CI1-αY|XN=x0N=b^0+b^1x0N±tM-2σ^εN1+1M+x0N-μ^X2Mσ^X2+σN2.

When X is multivariate, its expression is: 26CI1-αY|XN=x0N=b^0+b^1Tx0±tM-1-pσ^εN1+1M+x0-μ^XTΣ^X+ΣN-1Mx0-μ^X, where p is the number of features in X and ΣX and ΣN are the variance-covariance matrices of X and N, respectively. The confidence interval of Y constrained by a noisy observation (CI1-α(Y|XN=x0N)) described in Eq. () is the statistical model called “CLimate variable confidence Interval of Multivariate Linear Observational COnstraint” (ClimLoco1.0). To illustrate these mathematical results, Fig.  shows two realisations of ClimLoco1.0: one realisation from the sample (X1, Y1), …, (XM, YM) of size M=5 and one realisation from the sample of size M=30, taken from the same synthetic example as before. In Fig. a, each sample realisation gives a different realisation of the linear relationship y=b^0+b^1x (green line) and a realisation of the confidence interval constrained by a noisy observation described in Eq. () (green interval). The green shading represents the CI90%(Y|XN=x0N) obtained for different positions of the observation x0N. For comparison, this Fig. a shows in red the CI90%(Y|X=x0) for different positions of x0 (red shading). This enables us to compare the difference in the width and centre of the intervals, whether the observational noise is considered or neglected. Fig. b compares the CI of Y: unconstrained (CI90%(Y), in black), constrained by a noiseless observation (CI90%(Y|X=x0), in red), and constrained by a noisy observation (ClimLoco1.0, CI90%(Y|XN=x0N), in green).

When comparing the CI of Y constrained by a noisy vs. noiseless observation (green vs. red in Fig. b), the same conclusions are reached as when comparing PI of Y constrained by noisy vs. noiseless observation; there is a decrease in the reduction of uncertainty (interval width) and in the correction of the best guess (interval centre). In other words, observational noise weakens the constraint. When comparing the two rows of Fig. , corresponding to a small (first row) and a large sample (second row), the large sample leads to narrower CI. The CI is more precise when estimated from more data. This is visible in all three expressions of the CI discussed in this article through the effect of the term M. Moreover, this synthetic example uses strong observational noise (SNR=1). Combined with a small sample (first row of Fig. ) this tends to make the CI(Y|XN=x0N) large, meaning that the uncertainty is large. Therefore, the CI(Y|XN=x0N) is larger than the CI(Y) in the second row: the constraint has not reduced the uncertainty, which is surprising. However, this is an extreme case, combining both high observational noise and small sample size. In summary, low observational noise combined with a high correlation between X and Y leads to a strong constraint, which means a strong best guess correction (centre of the confidence interval) and a strong uncertainty reduction (width of the confidence interval). Uncertainty is also affected by the sample size: the larger the sample size, the greater the uncertainty reduction. The best guess correction is also affected by the distance between the observation and the multi-model mean (x0-μ^X), which is called in this article the “multi-model bias”. The larger the bias, the greater the correction.

The main contributions of this section are to provide the statistical model ClimLoco1.0, the confidence interval of the projected variable constrained by a noisy observation, to express and illustrate it graphically as an attenuated linear regression, and to highlight the need to take this observational noise into account and to have a sample size as large as possible. Figure  is proposed as an illustrative summary of a comparing the CI of Y unconstrained, of Y constrained by a noiseless observation, and of Y constrained by a noisy observation. This figure is built using synthetic data detailed in Appendix .

In this section, the results of this article are compared with those of some of the most widely used approaches in the observational constraint literature: (a)  and and (b) .

a.

Both and use statistical approaches to constrain Y by real-world observations. One can demonstrate (not shown here) that these two articles yield equivalent expressions for the expectation and variance of Y|XN=x0N. The main difference between them is that the first article considers the variables X and Y as univariate and the second as multivariate, respectively. It can be seen that these articles have the same expressions for the expectation and variance of Y|XN=x0N as those obtained in Eqs. () and (). This means that the approaches of both and are equivalent to using a linear regression model (multivariate and univariate, respectively). As seen in the previous section, this regression is corrected for the observational noise. This is an important result for interpreting these methods using linear regression, as is done in our article. Furthermore, an important caveat to this equivalence is that there is a well-known risk of overfitting when using multivariate linear regression, i.e. learning incorrect relationships between features by over-fitting the data. This risk is greater when the number of variables is large and the number of climate models used to learn the regression is small. The multivariate method developed by therefore presents a risk of overlearning.

Furthermore, the articles by and only gave the theoretical expressions for the expectation and variance of Y|XN=x0N. These theoretical values are in reality unknown. They did not provide details of the exact expression of the estimates, which – as previously seen using confidence intervals – leads to a higher uncertainty due to the limited sample size. This can be neglected when the sample size is very large, but it becomes very important when the sample size is small, as shown in the previous section (see Eq. ). In climate science, sample sizes are usually small (especially when considering high-resolution models; ), so that we argue here that this uncertainty must be included in the estimates.

We suggest that there are two main problems with this approach. Firstly, use two different distributions of the same variable X: one from the climate models to learn the linear relationship Y=a0+a1X+ε and one from the noisy observation. However, the climate models have a different X distribution from the observation: E[X]=μX≠x0N and V(X)=σX2≠σN2. The variable X cannot have two different expectations or two different variances; therefore, Eqs. () and () written above are incorrect from our point of view. It is necessary to distinguish the variable X, whose distribution is given by the climate models, and the variable XN, which is observed from the real world with observational noise, as is done in this article. Secondly, the constrained variable should be denoted Y|XN=x0N, not just Y. This has a major effect on the resulting equations. Indeed, Eqs. () and () are correct, but they do not constrain either the expectation or the variance of Y.

This conclusion is consistent with , who states that “care must be taken to characterise the uncertainty in the observational values of the X variable. The translation of observed X-values into predicted Y-values is not trivial. It is certainly not as simple as finding the intersection of the most likely value of observed X and the regression line relating Y to X and `reading' the predicted Y value. Instead, both observed X and predicted Y must be treated statistically”. The clarification proposed here may help to move in this direction.

A key feature of this article is the use of multiple observable variables simultaneously (i.e. a multivariate approach). Most approaches in the literature are univariate. However, we identified some multivariate approaches. As mentioned in this section, use a linear multivariate approach. The other approaches in the literature are mainly non-linear, making use of the large amount of information provided by multiple variables in more complex regression models. However, due to the non-linearity, it is not possible to formulate an analytical expression for the confidence interval. For instance, constrain future gross primary production (GPP) using a regression model based on random forest (gradient-boosted regression trees) that takes into account past GPP, temperature, precipitation, etc. To estimate uncertainty, the non-linear model is locally approximated by a linear one.

The different assumptions used to obtain ClimLoco1.0 are all compiled in the following list. Climate models are supposed to be (i) random and independent realisations of the same (ii) Gaussian distributions as (iii) reality. (iv) The observations are noisy realisations of reality, with additive Gaussian noise that is independent of X. Its covariance is assumed to be known. Each assumption is detailed below. If possible, we evaluate its impact on the results and provide insights into how it could be addressed in the next version of ClimLoco1.0. i.

ClimLoco1.0 assumes that climate models are independent and equally plausible. This assumption is used by most OC methods, except those that assign weights to climate models e.g.. However, defining confidence intervals using weighted samples without these assumptions is challenging, and this is an area in which ClimLoco1.0 could be improved. In ClimLoco1.0, these assumptions virtually give too much importance to dependent climate models, as if there were duplicates in the data. Groups of dependent climate models are closer together in the (X, Y) space, as observed for climate models belonging to the same institute, for example. Groups of models that are close together pull the best guess towards them and reduce the inter-model spread, leading an underestimation of the uncertainty.

A confidence interval for future climate, i.e. a best guess of the future climate with uncertainty given at a confidence level, can be obtained from an ensemble of climate model projections. However, the large dispersion between climate model projections makes this interval large, and consequently the future climate very uncertain. To refine it, methods called observational constraints (OCs) combine climate model projections with some real-world observations (cf. ). These methods are now increasingly used , even by potential stakeholders at the national level e.g.. They therefore deserve to be rigorously described in their assumptions and mathematical description. However, there are many challenges in dealing with the literature of OC. There is a wide variety of OC methods, which are sometimes difficult to reproduce and may lack mathematical detail, which are usually limited to the use of a single observable variable to constrain, and which do not strictly use confidence intervals, which are essential to correctly define uncertainty.

To address these challenges, this article proposes a new (1.0) statistical method, ClimLoco, which stands for “CLimate variable confidence Interval of Multivariate Linear Observational COnstraint”. ClimLoco1.0 describes the confidence interval of a projected variable constrained by a noisy observation using a linear multivariate framework. It is inspired by the theory of measurement error models from . We found that constraining a projected variable has two effects: it corrects the best guess of the projected variable according to the multi-model bias (the difference between the multi-model mean and the real-world observation) and reduces the associated uncertainty.

Compared with the existing literature, ClimLoco1.0 provides a more rigorous expression of uncertainty through the use of confidence intervals. This takes into account the quality of the estimators of the best guess and the uncertainty of a projected variable, which depends in particular on the number of climate models used. We have therefore emphasised the need to have as large an ensemble of models as possible, in order to obtain the most accurate estimates. In addition, ClimLoco1.0 takes into account the observational noise in a rigorous framework, which is important to correctly estimate the uncertainty. We find a new graphical interpretation (cf. Fig. ), of the effect of observational noise, which weakens the constraint (less reduction of uncertainty and a smaller change in the best guess). This article is intended to be didactic, building the statistical model ClimLoco1.0 step by step, from the unconstrained case to the case constrained by noisy observations, and illustrating each step with univariate examples.

In addition, the results are compared with some of the most commonly used methods in the literature: “statistical” methods e.g. and “linear regression” methods e.g.. There are strong similarities between the statistical methods of and and the multivariate linear regression OC developed in this article. We argue that there is an equivalence between their methods and a multiple linear regression. This implies that the methods are subject to the risk of overfitting (i.e. learning incorrect relationships between features by over-fitting the data). The use of methods to limit overfitting, such as ridge regression, seems to be a promising perspective in this respect. However, since and did not use confidence intervals, they neglect the quality of the estimators, which depends on the number of climate models considered. They therefore underestimate the uncertainty. There is a major discrepancy between our method and that of , which is now widely used for linear regression OCs. We highlight problems in the underlying mathematics and propose a new figure (Fig. ), which may be more appropriate than Fig. 1 from , to describe exactly how linear OC works in a geometric sense.

The statistical model ClimLoco1.0 is an effort to better account for uncertainties and bring more clarity to OC methods. However, there are still some challenges to overcome, for example, considering non-Gaussian distributions, dependence between climate models, non-linear regression, etc. These are interesting perspectives to build more advanced versions of ClimLoco1.0. Finally, finding equivalences between OC methods, as performed here, can be very useful to bring more clarity to the large literature of OC methods. For example, succeeded in converting a linear regression into climate model weights, but neglected the observational noise.

As an extension of this article, Appendix  illustrates the use of ClimLoco1.0 in a real case study and performs some sensitivity tests. The Python code and data, along with a simple, user-friendly example to replicate ClimLoco1.0, are provided via the “Code and data availability” section.