A Gaussian process is a stochastic process, which can be thought of as an infinite-dimensional Gaussian distribution in that the joint distributions of the process at any finite set of space–time points are multivariate normal. We denote points in the spatiotemporal domain by x∈Rq. In this work q=3, even though this restriction can be overcome if needed, and satGP does have limited support for space-only problems.

The Gaussian process, or Gaussian random field, is denoted by 1Ψ∼GP(m(x;β),k(x,x′;θ)), where m:Rq→R and k:Rq×q→R are the mean and covariance functions of the process parameterized by hyperparameter vectors β∈Rnβ and θ∈Rnθ. The infinite-dimensional (since the domain of x is typically infinite) description in Eq. () is reduced below to a finite-dimensional problem, in which case k(x,x′) describes an entry of the covariance matrix of the joint distribution of random variables Ψ(x) over all x that one is interested in.

The function m above is called drift in kriging literature, and the expected value of the process in regions with no data will tend to the value of this mean function. It is chosen to reflect the deterministic patterns in the data, and the particular form picked to model m will also affect how the function k and parameters θ in Eq. () need to be specified. With inadequate modeling of the mean function, the uncertainty estimates obtained with Gaussian process regression may end up being unnecessarily large. For instance linear trends, constant factors, and seasonal and other periodic fluctuations should be included in m if they are known. An example of what is used with the OCO-2 data is shown later in Eq. ().

In what follows, the domain Rq∋x is divided into two disjoint parts, one of which, Xtrain⊂Rq, is the set of coordinates xiobs, where observation data (training data) were measured, and another one, Xtest≜Rq\Xtrain, denotes its complement. Points in Xtest are denoted by x* and called test inputs as is often done in Gaussian process literature. Observations at locations {xiobs:i=1,…,n}, both real and synthetic ones generated by the Gaussian process, are denoted by ψiobs∈R, and the vector of all ψiobs is written as ψobs.

For the mean function m in Eq. (), a specific form, 2m(x;β,δ)=f(x;δ)Tβ≡f̃(xt;δ(xs))Tβ(xs), is used in this work. The superindexes s and t refer to the spatial and temporal parts of the generic coordinate x, and δ values are auxiliary parameters which are space dependent. The purpose of the righthand side with the function f̃ is to underline that f depends on the spatial part of x only via the space-dependent δ parameters and that the β parameters do not depend on xt, the temporal part of x. The temporal evolution of the mean function is in this particular form determined only by the function f(x;δ)≜[f1(x;δ),…,fnβ(x;δ)]T, and for each fi there is a space-dependent regression coefficient βi.

The parameter vectors δ contains space-dependent parameters that affect the form of any of the fi in a way that cannot be modeled with the β coefficients in the functional form of Eq. (). The length of these space-dependent δ vectors is nδ. Given the parameters δ for all the inputs in xobs and a set of functions fi for constructing the mean function, we define matrix F∈Rn×nβ with elements Fij=fi(xjobs;δ), where the δ is now specific to the location xiobs.

The definition of m above is very general and can describe in practice a large number of realistic scenarios. Nonetheless, the form of Eq. () imposes the strong assumption of separation of space and time in that the β and δ parameters do not depend on time. The explicit form of functions fi used to model the OCO-2 data are given below in Sect. .

The covariance function k(x,x′;θ) controls the smoothness of the draws ψ from Ψ. It outputs the prior covariance of the random variables Ψ(x) and Ψ(x′) at x and x′. The parameter vector θ typically contains at least one scale parameter ℓ and a parameter τ controlling the maximum covariance, τ2. The ℓ parameters correspond to the length scales of the random fluctuations of the realizations around the mean function, and the τ parameters describe the amplitude of that fluctuation. By defining the covariance matrix K∈Rn×n with elements Ki,j=k(xiobs,xjobs;θ), the joint distribution of the field at observed locations is given by 3Ψobs∼NFβ,K). Explicit forms of functions m and k are described in Sect.  and , respectively. Additional practical guidelines are given in Appendix .

The paradigm of Bayesian statistics is standard for analyzing data and uncertainties, and it is also widely used in geosciences . Given the observed data Ψobs=ψobs at some finite set of points xobs, the object of interest of the Bayesian inference problem in this work is the joint posterior distribution of the Gaussian process and the parameters, 4p(ψ,β,δ,θ|ψobs)=p(ψobs|ψ,β,δ,θ)p(ψ|β,δ,θ)p(β,δ,θ)p(ψobs), where p(ψ|β,δ,θ) is the Gaussian process prior, and p(β,δ,θ) is a prior on the Gaussian process hyperparameters. In this particular equation β and δ actually denote spatially varying hyperparameter fields. The calculation in Eq. () is not generally tractable for a huge number of inputs x, but posterior estimates of the GP, p(ψ|ψobs,β^,δ^,θ^), can be calculated for a finite set of inputs by conditioning on parameter point estimates θ^, β^, and δ^. The covariance parameter estimate θ^ may be found by minimizing some loss function L, 5θ^=argminθL(θ), described explicitly below in Sect. . Given a point estimate of the parameters θ and δ, along with a Gaussian prior for the β parameters with mean μβ and covariance Σβ, the posterior distribution of the β parameters can be computed with, 6Eβ|Ψobs=ψobs,θ,δ=FTK-1F+Σβ-1-1FTK-1ψobs-Fμβ+μβ7Covβ|Ψobs=ψobs,θ,δ=FTK-1F+Σβ-1-1. The matrix K is generally a dense matrix of size n×n, where n is the number of observations, and as n may be extremely large, direct inversion of this matrix is in practice impossible. However, in this work inverting the full K is not necessary; we want to find parameters β that vary locally, which is done by splitting the full problem into many smaller subproblems, solving the β parameters in a grid, as described in Sect. . This grid is then used to construct matrix F by interpolating the values of β and δ obtained.

The δ parameters are found approximately in this work by a three-step process: first a point estimate of parameters β and δ is computed using an optimization algorithm; second, parameters β are recomputed by Eq. () given the estimate of δ from the first stage; and third, the δ parameters alone are recalibrated by optimization using the newly found β parameters. In practice this procedure produces stable results with the OCO-2 data, and for pathological data sets repeated alternating optimization of the parameters may be performed. The calibration process is described in more detail in Sect. .

Even though a full posterior distribution of the parameters is not obtained this way, the solution of the Gaussian process itself is Bayesian in that the posterior marginals at each x* are found by conditioning on the observations. In the satGP software, the space-dependent β and δ parameters are fitted first, and any learning of the covariance parameters is done only after that.

For prediction in the context of Gaussian random functions, the properties of multivariate normal distributions are exploited for calculating marginals of the random field Ψ at any set of inputs. The posterior distribution p(ψ*|ψobs,θ^,β^) of the Gaussian process at some test input x* can, given point estimates β^ and θ^, be modeled according to Eq. () with 8Ψ*Ψobs∼Nf(x*)TFβ^,K(x*,x*)K(x*,xobs)K(xobs,x*)K(xobs,xobs), where the vector of inputs has been divided into two parts – one for the test input x* and the other one for the observations xobs. The notation K(x*,xobs) refers to the first row (minus the first element) of the covariance matrix with elements K(x*,xobs)j=k(x*,xjobs), and the matrix in the lower-right corner, K(xobs,xobs) is the same as matrix K in, e.g., Eq. (). The random variable at x* can then be written as Ψ*|β^,θ^∼N(μ*,Σ*), where its mean and covariance are given by 9μ*=fx*Tβ^+Kx*,xobsKxobs,xobs-1ψobs-Fβ^ and 10Σ*=Kx*,x*-Kx*,xobsKxobs,xobs-1Kxobs,x*, and where the covariance Σ* is the Schur complement of K(x*,x*). The formulas in Eqs. ()–() work equally well when the x* contains more than one test input. However, as of now, in satGP these equations are solved for a single test input at a time. When computing Ψ* with these formulas, satGP uses observations close to x* (see Sect. ) and the values of β and δ calibrated at x*s.

Equation () gives the most general mean function form available in satGP. The functions fi above are user defined, and, for ease of use, satGP includes functionality for using a zero-mean function, a spatially independent mean function, and an arbitrary gridded array of values. The specific forms of fi used for the OCO-2 experiments in Sect.  are 11f1(x)=sin⁡2πxtΔperiod-1+δf2(x)=cos⁡4πxtΔperiod-1+δf3(x)=1f4(x)=xt, where Δperiod is for OCO-2 the duration of 1 year, and δ is a space-dependent phase shift. The function f1 fits the summer–winter cycle, and f2 fits the semiannual cycle. It is assumed that for any given x, f1 and f2 can be modeled with the same δ parameters. The constant term is given by f3, and f4 gives the slow global trend. As an example of the local behavior, Fig.  shows the mean function fit to the observed local daily mean values of XCO2 from OCO-2 for several locations. The WACCM4 ozone study in Sect.  added two more functions f5 and f6 similar to f1 and f2 but with different Δperiod parameters.

When satGP is not used for learning GP covariance parameters or generating synthetic training sets, the finite set of test inputs x* for GP calculation is a grid with predefined geographical and temporal extents and resolution. Solving the GP marginalization and sampling problems then amounts to solving Eqs. () and () at each corresponding space–time point. Since, e.g., sources, sinks, and timing of seasons are local, the mean function should be different from one spatial grid point to another. This is achieved by modeling the β parameters as a Markov random field. The MRF imposes the condition that neighboring grid cells should not be too different from each other. How different they are allowed to be is a modeling choice; see Appendix . This section describes how the spatial dependence is resolved in satGP using computational statistics.

In addition to solving this spatial problem, the marginal distributions of the β parameters need to be solved for each individual vertex. Point estimates of the δ parameters, mentioned in Sect. , are found at the same time with the β parameters. The intimately connected spatial and local problems are described in the subsections below.

The smoothness, amplitude, and length scales of the Gaussian process realizations are determined by the covariance kernel used. The satGP program supports several different types of covariance function components for forming the full covariance function k in Eq. (). The options available reflect the properties that can be expected in remote sensing data – varying smoothness and meridional and zonal length scales, potential periodicity, and changing the orientation of the data-informed and uninformed axes according to wind speed and direction. This section lists the available covariance function formulations, and other forms may be easily added in the code.

For convenience, let 13ξIγ(x,x′)≜∑c∈Ixc-x′cℓcγ=‖PI(x)-PI(x′)‖Γγ, where γ>0 is the exponent, I is a (sub)set of the dimensions of the inputs x and x′, ℓc values are length-scale parameters corresponding to the different dimensions in I, e.g., temporal (ℓt), zonal (ℓlon), or meridional (ℓlat) directions, and xc values are different components of the inputs, e.g., xt, xlat, and xlon. The spatial length-scale parameters ℓlat and ℓlon are in units of distance on the surface of the unit sphere, corresponding to radians at the Equator. The PI matrix projects x onto indices/dimensions in I; Γ is a diagonal covariance matrix with diagonal elements ℓc2, and the notation ‖r‖Γ stands for rTΓ-1r, where r is an arbitrary vector of the appropriate size. For remote sensing data used in this work, space-only variables form the set IS≜{lat,lon}, and for spatial and temporal variables together the notation IST≜{lat,lon,t} is used. Notation lat and lon refer to the spatial components of x, collectively earlier referred to as xs, and t refers to the temporal component. The form of ξ in Eq. () implies that the different dimensions have separate length-scale parameters ℓ. The exponent γ in ξ is, however, shared between the dimensions. For the set of all ℓ parameters over a set I′ of dimensions we write ℓI′. All the covariance functions below depend on a parameter τ, square of which determines the maximum covariance that is attained when x=x′.

The exponential family of covariance functions with parameters θexp≜[τ,ℓIST,γ]T is defined by the covariance function 14kexp(x,x′;θexp)≜τ2exp⁡-ξISTγ(x,x′). The exponent γ controls the smoothness of the samples from the Gaussian process, with γ=2 yielding infinitely differentiable realizations.

The Matérn family of covariance functions, with θM≜[τ,ℓIST,ν]T is given by the covariance 15kM(x,x′;θM)≜τ2sνΓ(ν)2ν-1Kν(s), where s=2νξIST1(x,x′) and ν controls the smoothness parameter usually denoted by α via α=ν+q2. The function Kν is the modified Bessel function of the second kind of order ν. With q=1, the value ν=∞ corresponds to the squared exponential kernel and ν=0.5 to the exponential kernel with γ=1. Despite this similarity between the Matérn and exponential kernels, the realizations of the random function from the processes with values 12<ν<∞ do not correspond to those with the kernel kexp with any value of γ.

A periodic kernel with θper≜[τ,ℓIS,ℓper]T is defined in satGP by 16kper(x,x′;θper)≜τ2exp-2ℓper2sin⁡2πxt-x′tΔperiod-ξIS2(x,x′). The parameter Δperiod is the period length, which is assumed to be well known a priori and therefore is not among the parameters that are calibrated. The second term in the exponent controls the spatial dependence via length-scale parameters in ℓIS, and ℓper determines how far the temporal covariance extends, modulo Δperiod.

satGP contains an additional covariance function that utilizes local wind information when computing the covariances. The underlying rationale is that winds affect how quantities of interest such as gases in the atmosphere or algae blooms in surface water spread. For this reason, if wind data are available, it is natural to try to use them for inference with the Gaussian process. We define the wind-informed covariance kernel with parameters θw≜[τ,ℓ,ℓt,ρ,w*]T by 17kw(x,x′;θw)≜kexp(xw,xw′;τ,{ℓ‖,ℓ⟂,ℓt},2). The parameter ρ in θw defines how strongly the magnitude of the wind vector at the test input, w*≜[wlat*,wlon*]T (the last parameter in θw), affects the shape of the covariance. The kernel itself is an exponential kernel, where the spatial components of the vectors x and x′ are transformed by wind data, and the covariance lengths are transformed by wind speed. A spatiotemporal vector x=[xlat,xlon,xt] is transformed by wind to the vector xw in a new coordinate system according to 18xw≜(xs-x*s)Tw‖(xs-x*s)Tw⟂xt, where xs and x*s are the spatial components of vectors x and x*, and w‖ and w⟂ are the unit vectors in the lat–long coordinates along and perpendicular to wind direction at the test input x*.

The spatial scaling (ℓ) parameters for kw, corresponding now to the covariance scales parallel and perpendicular to the wind direction, are given by 19ℓ‖≜ℓ1+|w*|ρ4,ℓ⟂≜ℓ. The parameter vector for the exponential kernel then becomes θexp←[τ,ℓ‖,ℓ⟂,ℓt,2]T, where the last element denotes the exponent γ used by the exponential kernel. A number of possible covariance ellipses resulting from the transformation procedure are shown in Fig. . Some data sets, like OCO-2, incorporate wind information, and satGP does have the capability of gridding that data using another Gaussian process. Reading in gridded wind data from other sources is also a possibility. Using kw requires that wind data are available at each x*.

The covariance functions used in this work to model Ψ are sums of several kernels – sums of valid Gaussian process kernels remain valid kernels. The general form of the multi-scale kernel used in satGP is given by 20kx,x′;θ=δx,x′σx2+∑i=1nkerkkerix,x′;θkeri, where the first term, which in kriging is called the nugget, contains the observation error variances, and each keri∈{exp,M,per,w}.

The kernel components of a multi-scale kernel are in this work called subkernels. The combined set of parameters is denoted by θ=[θker1T,…,θkernkerT]T. Not all subkernel types are included in all experiments – rather, the simulations in Sect.  utilize kernels with one to three components. What those components should be depends on what fields are being modeled and what kinds of correlation structures the user expects to find in the data. Section  discusses identifiability of the different subkernel parameters of the multi-scale kernel.

Instead of calling k(x,x′;θ) in Eq. () a multi-scale kernel, the term multi-component kernel could also be used to describe the form. The term “multi-scale” underlines that the purpose of the combined kernel is to model data well, which contains several natural length scales, as remote sensing products often do. Furthermore, we believe that combining several kernels with identical length-scale parameters does not represent a common use case.

Using a large number of observations makes solving the Gaussian process Eqs. () and () intractable as the cost of inverting the covariance matrix scales as O(nobs3). This creates a need for finding approximate solutions while introducing as little error as possible. In satGP, covariance localization is used to utilize only a subset of observations for computing Eqs. () and (). To control the localization behavior the user needs to set two parameters: the maximum subkernel covariance matrix size κ and the minimum covariance parameter σmin2.

Assume that the multi-scale kernel defined by the user contains nker subkernels. For each test input x* and for each subkernel kl, the set of observations feasible for inclusion in K in Eqs. () and () is 21A*,lobs≜{ψi∈ψobs|kl(xiobs,x*)>σmin2,ψi∉A*,jobs∀j<l}, where the last condition prevents observations from being added by several subkernels. In the end we select a single set of observations A*obs for each test input by combining some or all of the observations included in each A*,lobs. The observation selection proceeds sequentially through the list of subkernels according to the procedure presented in Fig. . Recomputing the κ′ for each subkernel on line 3 of the algorithm allows the selection of more than κ observations by subkernels if the previous subkernels did not have κ feasible observations available. This is done to allow the full kernel size to grow to nkerκ when possible. On line 4, the observation selection operator S(A*,lobs,κ′) chooses κ′ observations from each A*,lobs either greedily by picking the observations with highest covariance with x* or randomly by sampling uniformly without replacement from A*,lobs. Out of these two methods random selection avoids observation sorting and is therefore faster, especially if a huge number of data are near the test input x*. This comes at the cost of producing noisier fields of marginal posterior means. For covariance parameter estimation random selection works well. See Appendix  for additional details.

Since the subkernels are handled sequentially, their order may affect which observations are selected due to the exclusion in Eq. (), and to grow the full kernel to size nkerκ as often as possible, it is recommended to specify the subkernel with the largest ℓ parameters as the last one. After constructing A*obs, the covariance matrix K is constructed by evaluating the full covariance function k according to Eq. () for all pairs of selected observations.

For learning the spatially varying β and δ parameters for grid index (i,j) in the mean function with the methods in Sect. , the observation selection is performed by disregarding the time component on the inputs, i.e., by setting xiobst←xijt for all xiobs in the training data. The reason for this is that, since learning the mean function amounts to fitting spatially varying parameter vectors β and δ, the data to perform the fit should not be selected based on covariance in the time direction, as the mean function should be equally valid at all times.

Selecting the observations could also be done based on values of k instead of each kl individually or by other approaches, such as the one presented by . However, even though the method of observation selection does have an effect on the inferred posterior marginals, the screening property of Gaussian processes ensures that this effect is not major as long as observational noise is small and the nearest observations are included in all directions. The parameter identifiability results in Sect.  and the WACCM4 results in Sect.  verify that the current nearest-neighbor-in-covariance approach works as intended.

From Sect.  the log marginal likelihood of observations ψobs given a set of parameters θ, β, and δ is given by 222log⁡pψobs|β,δ,θ=-‖ψobs-Fβ‖K2-log⁡|K|-nobslog⁡2π, where the covariance function parameters θ implicitly determine K, and the nonlinear space-dependent mean function parameters δ affect the values in F. The maximum (marginal) likelihood estimate (MLE) θ^ of θ can be found via minimizing 23Lθ=‖ψobs-Fβ^‖K2+log⁡|K|+nobslog⁡2π as stated in context of Eq. ().

In the presence of a huge number of observations, calculating the determinant of the full covariance |K| is not feasible, and maximizing the log-likelihood is approximated by 24θ^=arg minθ∑xi*∈Eref‖ψlocaliobs-Fiβ^‖K̃i2+log⁡|K̃i|, where Eref is a set of randomly sampled points from the spatiotemporal domain specified for the experiment, determined by the parameters area and ndays in Table . The β^ and δ^ parameters, the latter of which is embedded in F, are the point estimates corresponding to each xi*, interpolated from the values obtained for the full grid. The optimization in Eq. () is carried out over all subkernel parameters with some caveats: currently the smoothness-related parameter ν for the Matérn kernel and the exponent γ for the exponential kernel are not calibrated, and naturally neither are the wind data w* listed as a parameter for the wind-informed covariance – however, the parameter ρ affecting that kernel can be learned.

While the selection of inputs included in Eref has an effect on the obtained parameter estimate, that effect has proven in simulations to be small. The vectors ψlocaliobs∈Rdi, where di is the number of observations chosen by the observation selection method of Sect.  for test input xi*, contain observations closest in covariance to xi*, each of which is a reference point included in Eref. The matrices Fi are the corresponding F-matrices, as described in Sect. . The last term in Eq. () is dropped, since while varying θ in Eq. () changes di, the number of total observations in the problem should fundamentally stay the same.

The maximum likelihood estimate approximation in Eq. () contains a sum over blocks of observations, which can together be thought of as a block-diagonal approximation of the full dense covariance for all observations in all ψlocaliobs. The blocks in this approximation are the dense covariance matrices Kĩ, and in contrast to a full dense K, in this approximation the cross-covariances between observations in ψlocaliobs and ψlocaljobs, i≠j, are set to 0. This is done even if the randomly selected corresponding inputs xi* and xj* are close to each other. Due to the O(n3) cost of inverting the covariance matrix, which is needed for finding the maximum likelihood estimate, using the block approximation provides a critical efficiency improvement without which learning the covariance function parameters would not be feasible.

While this method is suitable for finding point estimates for the parameters θ, the computed approximated log-likelihood has an unknown scaling factor resulting in an unknown multiplicative factor for the variance term in the exponent of the Gaussian distribution, and hence information about the true size of the posterior distribution of the covariance parameters p(θ|ψobs,β^V,δ^V) is lost.

By default the scaled posterior p(θ|ψobs,β^V,δ^V) is explored by using the adaptive Metropolis (AM) Markov chain Monte Carlo (MCMC) algorithm , an implementation of which is included in the satGP source code. MCMC methods are used to draw samples from probability distributions when direct sampling is not possible, but the likelihood function can still be evaluated. The samples are drawn by generating a Markov chain of parameter values, which is an autocorrelated sample from the posterior. The AM algorithm is an adaptive method that is efficient for many real-world sampling situations. The observation selection procedure in Sect.  introduces discontinuities to the posterior distribution due to selected observations changing when the covariance function parameters are modified. Computing θ^←E[θ|ψobs,β^V,δ^V] with MCMC – i.e., using the posterior mean of a Monte Carlo sample – usually works around this noisiness in the likelihood. On the downside, MCMC is computationally much more demanding than finding the maximum a posteriori estimate with optimization, since MCMC may require computing up to millions of likelihood evaluations. In the satGP context using MCMC is feasible since the forward model simply amounts to sampling from a multivariate normal distribution, which is very fast. Furthermore, the parameter dimension is moderate, even with multiple subkernels, limiting the need to generate extremely long chains. The current version of satGP uses a flat prior distribution for the covariance parameters, with hard limits on the parameter ranges.

The software also includes a capability to learn the covariance parameters using optimization algorithms such as COBYLA or SBPLEX available in NLopt. These methods are much faster than MCMC but have the tendency of getting stuck in local minima, limiting their usefulness.

We performed a synthetic study to confirm the identifiability of the multi-scale covariance function parameters. The synthetic data were generated by satGP by sampling from zero-mean processes with known covariance parameters and with a random spatial pattern from the prior, adding 1 % noise. The parameters were then estimated by computing the posterior mean estimates using adaptive Metropolis.

The identifiability experiment was performed with various kernels, and recovering the true parameters was the more difficult the more complex the kernel was. With a single Matérn, exponential, or periodic kernel, the parameters could be recovered very easily. This was also true for a combination of exponential and Matérn kernels with a relatively small κ parameter.

The covariance kernel parameters were still recoverable with a combination of three kernels, Matérn with ν=52, exponential, and periodic. This setup required using a larger κ=256. With small κ, some of the parameters had a tendency to end up at the lower boundary, possibly due to effects of the covariance cutoff on the determinant of the covariance matrix in Eq. (). Optimization using minimization algorithms such as Nelder–Mead, COBYLA (Constrained Optimization BY Linear Approximation), or BOBYQA (Bound Optimization BY Quadratic Approximation) tended to often end up in local minima, and for this reason MCMC was used instead. The number of random reference points in Eref in Eq. () was set to 12, which was enough to reliably recover parameters close to the true value.

The parameter limits, true values, and posterior means of the synthetic experiment with three kernels are given in Table . In total 200 000 observations were created in the region between -10 and 10∘ latitude and -10 and 10∘ longitude over a period of 4 years according to the true values reported in Table . A total of 10 million MCMC iterations were computed to make sure that the posterior covariance stabilized. The posterior, with first 50 % of the chain discarded as burn-in, is shown in Fig. 

How well parameters can be learned from data depends always on the data and the exact Gaussian process form chosen. While the identifiability studies presented here show that the parameter calibration procedure works and that covariance parameters are recoverable in a synthetic settings, identifiability cannot be always expected. Still, even in these cases, the MAP and/or posterior mean estimates of the covariance parameters should provide good point estimates for θ.

A synthetic study using WACCM4-generated ozone data was conducted to verify and to illustrate that the methods to learn the model parameters β, δ, and θ produce a realistic GP regression model that then produces credible posterior predictive fields. In a synthetic setting the mean values of the posterior predictive distributions should be close to the true fields, and the discrepancies between the ground truth and the predicted fields need to be explainable by the predicted marginal uncertainties. The role of this part in the study is to give an example of how a Gaussian process predictive posterior field produced with satGP compares with the underlying true field.

The WACCM4 model is an atmospheric component of the Community Earth System Model from NCAR , capable of comprehensively representing atmospheric chemistry and modeling the atmosphere up to thermosphere. WACCM4-generated ozone data for the years 2002–2003, with a latitude–longitude grid resolution of 1.9∘×2.5∘, 88 vertical levels going up to roughly 140 km, and an internal time step of 30 min, were used as ground truth and to generate synthetic observations. Since the model was used for generating synthetic two-dimensional data, a specific atmospheric sigma hybrid pressure level of 3.7 kPa was selected.

Ozone data at approximately 400 locations were sampled daily over a two-year period in a random pattern from the domain of the experiment to learn the parameters of the mean and covariance functions. The training data set was then generated by interpolating to these points from the simulated WACCM4 data. This sampling procedure corresponds to creating on average one observation daily for each 12.5∘×12.5∘ longitude–latitude square.

Using these data, the mean function parameters were fitted locally using the method in Sect. , utilizing the functions fi in Eq. (), but with two additional terms f5 and f6, which were similar to the f1 and f2 except for different Δperiod parameters and phase-shift parameters δ that were shared between these f5 and f6 only. These functions were used to model periodical behavior with 2 and 1.5 year period lengths. The covariance function parameters of a kernel consisting of a single Matérn kernel, Eq. (), were learned using the approximate maximum likelihood technique described in Sect.  with data from the first year. The parameter ν used for the kernel was 52. The optimization was carried out with MCMC, and the posterior mean estimate of the covariance parameters was selected for θ^. The values of the covariance parameters obtained were τ=0.589, ℓlat=0.143, ℓlon=0.225, and ℓt=2d16h15min. That ℓlon is larger than ℓlat echoes the OCO-2 results presented later in Table .

For computing the posterior predictive distributions, the observational data ψobs were sampled from the WACCM4 simulations at locations closest in space and time to where the GOMOS instrument made measurements during its first year of operation. No noise was added to these observations. The posterior predictive distribution was computed for 1 full year, and the total number of observations used was 39 538. The reason for using different spatial patterns for learning the model parameters and for running the Gaussian process regression was that with this choice, the quality of the fit of the mean and covariance functions was not dependent on the spatial location, and therefore, if major spatial discrepancies between the ground truth and the posterior predictive fields had emerged, those could then have been attributed to the GOMOS sampling pattern used to generate the synthetic observations ψobs.

The marginal posterior predictive distributions were computed globally in a uniform grid with the resolution of 2.5∘ in the east–west direction and 1.9∘ in the north–south direction between 78.63∘ S and 78.63∘ N and daily over the period from 6 January 2002 to 5 January 2003, totaling around 4.384 million marginals in the predictive posterior. The 1-year-long computation took 19 min 18 s on a relatively fast Intel i7-8850H laptop CPU.

Figure shows the ground truth from WACCM4 with the mean field and corresponding marginal uncertainties obtained from satGP for 2 December 2002. The ground truth and the estimated fields are very similar, and the uncertainty is higher when there are no observations nearby. The posterior mean field retains a lot of fine detail from the ground truth and is not overly smoothed or sharp, suggesting that the covariance parameter calibration procedure has found a well-performing estimate for the covariance parameters θ. The smallest reported uncertainties are close to 0, as they should, due to lack of observation error.

The simulations with non-synthetic remote sensing data use the V9 data from the OCO-2 satellite. OCO-2 was launched in 2014, and it orbits the Earth on a Sun-synchronous orbit , completing 14.57 revolutions around Earth in 1 d. The footprint area of each measurement is roughly 1.29km×2.25km, but the data are very sparse in time and in space. In the presence of clouds, the satellite is not able to produce measurements, and this poses a challenge for areas with persistent cloud covers, such as northern Europe in the winter.

The present work uses the XCO2 data, their related reported uncertainties, associated coordinate information, and zonal and meridional wind speeds that are contained in the data files. The time period considered is from 6 September 2014 to 10 November 2018, and we use only observations flagged as good, of which there are in total 116 489 342.

Calibrating the mean function from OCO-2 V9 XCO2 data as described in Sect.  produces the estimates for the β and δ coefficients shown in Fig. . The βi parameters are the coefficients of the functions fi in Eq. (), and δ is the phase-shift parameter in f1 and f2. The upper-left quadrant of Fig.  shows the semiannual variability in the XCO2 concentration. The timing of winter and summer in the Northern Hemisphere and Southern Hemisphere explains the color shift along the Equator. The lower-left quadrant shows the amplitude of the 2-times-faster oscillations, and like β1, β2 also shows the highest amplitude oscillations in the boreal region.

The constant term β3 in the upper-right quadrant shows the background concentration. Some of the reddest areas such as East China, both coasts of the United States, central Europe, and the Persian Gulf stand out, and they are also areas where major emission sources are known to exist. Finding local elevated concentrations compared to surrounding areas echoes the observations made by , where empirically defined time-integrated local XCO2 anomalies were interpreted as possible emission sources. The trend component β4 varies only a little spatially, due to CO2 mixing in the atmosphere over time, and for this reason it is not shown here. The phase-shift parameter δ is modeled separately, and the field in the lower-right quadrant is obtained by optimization, conditioning on the β factors. This partly explains the different spatial pattern. Figure  shows how the phases of the annual XCO2 cycles differ between regions, but the δ values need to be interpreted together with the β1 and β2 coefficients.

At high latitudes XCO2 observations from OCO-2 are available only for a short period every year, and the quality of these measurements is often poorer. For this reason the calibration procedure may yield unrealistic and noisy values close to the poles, especially for parameters β1 and β2. The obtained parameter values closer than 20∘ to the northern and southern edges of the domain were averaged by setting the parameter values at each xij to βij←β^ijd20+(20-d)β‾20, where d is the distance to the edge of the domain in degrees, β^ij is the calibrated parameter vector at xij, and β‾ is the average value of the parameters over the area where xij is located and where averaging is performed. The δ parameter was treated similarly. This adjustment was done as a postprocessing step after finding the mean function coefficients. The main benefit of performing this adjustment is that the posterior predictive distributions become more realistic in winter at high latitudes when the mean function dominates.

Figure  shows time series of the mean function for a variety of locations, verifying that the exact form chosen is able to describe much of the local variability in XCO2.

The OCO-2 data have several natural spatial and temporal length scales. The distance between adjacent observations is only 1 to 2 km in space and some hundredths of a second in time, but the distance between consecutive orbits is thousands of kilometers in space and several hours in time. On consecutive days the satellite passes close to the trajectory of the previous day at a distance of tens to 300 km depending on the latitude. The Earth has natural temporal diurnal and annual cycles, but since OCO-2 is Sun-synchronous, only the latter matters with OCO-2 data. Since the annual cycle is already fitted by finding the mean function coefficients β1 and β2 in Eq. () corresponding to the periodic functions f1 and f2, a periodic covariance kernel component is not included. The OCO-2 data are therefore modeled with a kernel consisting of a larger-scale exponential and a smaller-scale Matérn subkernel.

The covariance parameters for the two-component kernel are given in Table . The values used were the median values from sampling the posterior with MCMC. When learning the parameters from a data set with several natural length scales, the posterior may appear multimodal, with some of the modes only having relatively little mass. In such a case, the median provides a more robust estimate for the parameters than the mean. The ℓlon(⋅) and ℓt(⋅) parameters of the posterior mean were slightly larger, which would result in slower computation. Selecting the median is further justified by the slight overestimation of some parameters in the synthetic study in Sect. .

Learning the covariance parameters from OCO-2 V9 data used the following configuration parameters for satGP: ζtrain=0, κ=256, and nref=12. A total of 1.1184 million MCMC iterations were completed, with the first 50 % discarded as burn-in to produce statistics. The reference points were randomly picked from a rectangle with corners at (0∘ S, 65∘ E) and (60∘ N, 145∘ E). While using the whole globe would have been a principled choice, MCMC requires lots of iterations, and for any claim of global coverage, nref would have needed to be much larger.

The marginal posterior predictive distribution at test points x*, given by Eqs. () and (), were calculated globally in a 0.5∘ grid between 80∘ S and 80∘ N at daily time resolution. The first day of simulation was 6 September 2014, and the last day was 10 November 2018, spanning in total 1526 d. For each day, 230 400 marginals were computed, resulting in a collective 351 million inverted covariance matrices. The satGP parameters used were ζsample=0 and κ=256, and the covariance kernel used was the one learned in Sect. , with parameters given in Table . The simulation wall time was 26 d on a moderately fast Intel i7-8700K CPU utilizing the available 12 CPU threads and 32 GiB memory.

Figures  and present global fields of the mean values and marginal uncertainties, with a subset (to avoid excessive overdrawing) of observations shown as a scatter plot in the (a) panels. The (b) panels show how uncertainty is reduced with the overpass of OCO-2. This uncertainty reduction diminishes fast due to the Matérn component of the multi-scale kernel having a very short length-scale parameter in the time dimension. In Figs. a and a, the background color (mean of the Gaussian process posterior) usually matches the observations, but due to observational noise, the posterior mean is not everywhere an interpolated field.

Data from the OCO-2 can be used to demonstrate how the multi-scale kernel formulation affects the predictive posterior distributions. Figure  shows posterior marginals from 15 September 2014. Figure a and b contain results from the multi-scale kernel described in Sect. , and the second row (c and d) shows fields from only the exponential part of the multi-scale kernel. The parameters of the multi-scale kernel are shown in Table . Figure e and f contain the difference fields between the first and the second rows. The single-kernel uncertainty is very low in Fig. d since lots of observations fall into regions of high covariance with almost any test input, with the exception of the northern side of Ireland, which does not have any observations nearby. Since the covariance kernel parameters were trained for the multi-scale kernel, the parameters used for the single kernel are not the ones describing the XCO2 field best.

Figure a shows that as intended, the multi-scale approach leads to local enhancements of the XCO2 mean field. Far from the measurements, the smaller Matérn kernel no longer reduces the predicted marginal uncertainties, and this leads to an increase in uncertainty in these areas. Figure e shows additional enhancements of the XCO2 mean fields, which are in this case due to the different maximum covariances between the multi-scale and single-scale kernels.

The total kernel size was kept at 1024 (κ=512 for a and b; κ=1024 for c and d) in both experiments, and thinning and grid resolution parameter values were ζsample=5 and ω=0.5∘. The very same observations were used for both simulations.

The wind-informed kernel, Eq. (), lets local wind data at test input x* rotate and scale the axes along which the covariance between two points is computed. Modeled winds are included with OCO-2 data, and they can be used to produce gridded winds that can then be used locally with the computation of each marginal posterior predictive distribution.

The covariance parameters for a single wind kernel were learned by taking the median of an MCMC posterior, similarly as was done in Sect. . The resulting parameters were τ=2.07, ℓ=0.038, and ρ=56.7. The variance in ρ was high, possibly due to the square root in the current formulation in Eq. (). For this simulation, ζ=1, κ=1024, and ω=0.7, and the simulation time for the area from 27∘ N, 115∘ E to 40∘ N, 145∘ E for the single day was 2.652 s (wall time) on the i7-8750H laptop CPU.