Dynamics encountered in geosciences are given as a system of partial differential equations (PDEs): 1∂tX=M(t,∂X), where X(t,x) is the state of the system and denotes either a scalar field or multivariate fields in a coordinate system x=(xi)i∈[1,d], where d is the dimension the geographical space, ∂X represents the partial derivatives with respect to the coordinate system at any order, with the convention that order zero denotes the field X itself, and M denotes the trend of the dynamics. A spatial discretization (e.g., by using finite differences, finite elements, finite volumes, spectral decomposition) transforms Eq. () into 2∂tX=M(t,X), where, this time, X(t) is a vector, and M denotes the discretization of the trend in Eq. (). Thereafter, X can be seen either as a collection of continuous fields with dynamics given by Eq. () or a discrete vector of dynamics as in Eq. ().

Because of the sparsity and the error of the observations, the forecast Xf is only an estimation of the true state Xt, which is known to within a forecast error defined by ef=Xf-Xt. This error is often modeled as an unbiased random variable, Eef=0. In the discrete formulation of the dynamics in Eq. (), the forecast error covariance matrix is given by Pf=Eef(ef)T, where the superscript T denotes the transpose operator. Since this contribution is focused on the forecast step, hereafter the upper script f is removed for the sake of simplicity.

We now detail how the error covariance matrix evolves during the forecast by considering the formalism of the second-order nonlinear Kalman filter.

A second-order nonlinear Kalman filter (KF2) is a filter that extends the Kalman filter (KF) to nonlinear situations in which the error covariance matrix evolves tangent-linearly along the trajectory of the mean state and the dynamics of this mean are governed by the fluctuation–mean interacting dynamics . Hence, we first state the dynamics of the mean under the fluctuation–mean interaction, then the dynamics of the error covariance. Note that the choice of the following presentation is motivated by the perspective of using a computer algebra system to perform the computation.

The parametric formulation of covariance evolution stands as follows. If P(P) denotes a covariance model featured by a set of parameters P=(pi)i∈I, then there is a set Ptf featuring the forecast error covariance matrix so that P(Ptf) approximates the forecast error covariance Ptf, i.e., P(Ptf)≈Ptf. Note that a parameter pi can be a scalar or a field, e.g., a variance field.

Hence, starting from the initial condition Pf=P0f so that P(P0f)≈P0f, if the dynamics of the parameters Ptf are known, then it is possible to approximately determine Ptf by P(Ptf) without solving Eq. (8) explicitly. This approach constitutes the so-called parametric Kalman filter (PKF) approximation introduced by (P16, P18).

In practice, the parametric covariance models considered in the PKF are such that the number of parameters in P is much lower than the number of coefficients required to represent the full covariance P(P). For instance, for the dynamics of a scalar field discretized with n grid points, as introduced in Sect. , the total number of parameters in P should be of same order as n, e.g., 2n or 3n, so that the cost to predict the evolution of the parameters would represent 2 or 3 times the cost to predict the evolution of the scalar field. Said differently, the cost to predict the parameters should scale in O(n), which is much lower than the computation of Eq. (8) in O(n2) - O(n3).

The cost of the PKF can be compared with other low-rank methods such as the reduced-rank Kalman filter or the ensemble Kalman filter , for which an ensemble size of 100 members is often encountered depending on the dimension of the unstable subspace and to limit the amount of sampling noise. Hence, when each forecast is made at full resolution the cost of these approaches is 100 times the cost of a prediction, which is larger than the cost we expect for the PKF (see P16, P18). Note that low-rank and ensemble methods often consider the computation of the dynamics at a lower resolution, which leads to a lower cost than the 100 forecast at full resolution. The PKF is computed at the full resolution and is free from sampling noise.

But the frugality of the covariance model is not the only criterion. For instance, the first variational data assimilation systems considered a covariance model based on the spectral diagonal assumption in spectral space . This covariance model reads as Ps(Ps)=ΣS-1DS-1*Σ*, where S denotes the spectral transform with * the conjugate transpose operator, and Σ is the diagonal matrix of standard deviation, i.e., the square root of the variance field V. In this model, the set of parameters Ps is given by the grid points and the spectral variances, Ps=V,diag(D). If the shape of Ps(Ps) is n2, the number of parameters Ps for this covariance model is 2n (n variances in grid points, stored as the standard deviation in the diagonal of Σ; n variances in spectral, stored in the diagonal of D), which is quite economical. However, the resulting correlation functions are homogeneous (there is the same correlation function in each point), which is enough to represent climatologically stationary background error statistics but not the flow-dependent statistics as existing in the KF. While it is possible to write the equations for the dynamics of the spectral variances (e.g., for the linear waves), the limitation that the spectral diagonal approach can only model homogeneous correlations motivated the introduction of other covariance models. For example, the covariance model based on the diagonal assumption in wavelet space can model heterogeneous correlations at a low memory cost. However, the dynamics of the wavelet variances are much more difficult to develop because of the redundancy of the wavelet transform on the sphere.

Hence, a covariance model adapted for the PKF should be able to represent realistic correlations and be such that the dynamics of the parameters can be computed, e.g., a covariance model defined by parameters in grid points. To do so, we now focus on the PKF applied to a particular family of covariance models, whose parameters are defined in grid points by the variance and the anisotropy fields: P=(V,g), where g will denotes the local anisotropy tensor of the local correlation function.

From now, we will focus on the forecast error statistics, so the upper script f is removed for the sake of simplicity. Moreover, for a function f, when there is no confusion, the value of f at a point x is written either as f(x) or as fx.

The forecast error being unbiased, E[e]=0, its variance at a point x is defined as 9V(x)=E[e(x)2]. When the error is a random differentiable field, the anisotropy of the two-point correlation function ρ(x,y)=1VxVyE[e(x)e(y)] is featured from the second-order expansion, 10ρ(x,x+δx)≈1-12||δx||gx2, by the local metric tensor g(x) and defined as 11g(x)=-∇∇Tρx, where ρx(y)=ρ(x,y), e.g., gij(x)=-∂yiyj2ρx(y)y=x. The metric tensor is a symmetric positive definite matrix, and it is a 2×2 (3×3) matrix in a 2D (3D) domain.

Note that it is useful to introduce the local aspect tensor , defined as the inverse of the metric tensor: 12s(x)=g(x)-1, where the superscript -1 denotes the matrix inverse. The aspect tensor at the point x is geometrically interpreted as an ellipse whose shape coincides with that of the local correlation function.

What makes the metric tensor attractive, either at a theoretical or at a practical level, is that it is closely related to the normalized error ε=eV by 13gij(x)=E(∂xiε)(∂xjε) (see, e.g., , for details).

Hence, using the notation introduced in Sect. , a VLATcov model is a covariance model, P(P), characterized by the set of two parameter fields, P=(p1,p2), given by the variance field and by the anisotropy field – the latter being defined either by the metric tensor field g or by the aspect tensor field s – i.e., P=(V,g) or P=(V,s). Said differently, any VLATcov model reads as P(V,g) or P(V,s).

To put some flesh on the bones, two examples of VLATcov models are now presented.

We first consider the covariance model based on the heterogeneous diffusion operator of , which is used in variational data assimilation to model heterogeneous correlation functions, e.g., for the ocean or for air quality. This model has the property that, under the local homogenous assumption (when the spatial derivatives are negligible), the local aspect tensors of the correlation functions are twice the local diffusion tensors . Hence, by defining the local diffusion tensors as half the local aspect tensors, the covariance model based on the heterogeneous diffusion equation is a VLATcov model.

Another example of a heterogeneous covariance model is the heterogeneous Gaussian covariance model: 14Phe.g(V,ν)(x,y)=V(x)V(y)νx1/4νy1/412(νx+νy)1/2×exp⁡-||x-y||(νx+νy)-12, where ν is a field of symmetric positive definite matrices, and |ν| denotes the matrix determinant. Phe.g(V,ν) is a particular case of the class of covariance models deduced from Theorem 1 of . Again, this covariance model has the property that, under local homogenous assumptions, the local aspect tensor is approximately given by ν, i.e., for any point x, 15sx≈νx. Hence, as for the covariance model based on the diffusion equation, by defining the field ν as the aspect tensor field, the heterogeneous Gaussian covariance model is a VLATcov model .

At this stage, all the pieces of the puzzle are put together to build the PKF dynamics. We have covariance models parameterized from the variance and the local anisotropy, which are both related to the error field: knowing the dynamics of the error leads to the dynamics of the VLATcov parameters. This is now detailed.

When the dynamics of the error e are well approximated from the tangent-linear evolution in Eq. (), the connection between the covariance parameters and the error, represented in Eqs. () and (), makes it possible to establish the prediction step of the PKF , which reads as the dynamics of the ensemble average (at the second-order closure), 16a∂tE[X]=M(t,∂E[X])+M′′(t,∂E[X])(E[∂e⊗∂e]), coupled with the dynamics of the variance and the metric, 16b∂tV(t,x)=2Ee∂te,16c∂tgij(t,x)=E[∂t(∂xiε)(∂xjε)], where it remains to replace the dynamics of the error (and its normalized version ε=e/V) from Eq. () and where the property that the expectation operator and the temporal derivative commutes, ∂tE[⋅]=E[∂t⋅], has been used to obtain Eq. () and ().

Following the discussion in Sect. , the set of Eq. (16) is at the heart of the numerical sobriety of the parametric approach since the cost of the prediction of the parameter scales like O(n). In contrast to the matrix dynamics of the KF, the PKF approach is designed for the continuous world, leading to PDEs for the parameter dynamics in place of ODEs in Eq. (8) for the full matrix dynamics. Moreover, the dynamics of the parameters shed light on the nature of the processes governing the dynamics of covariances, and it does not require any adjoint of the dynamics .

Note that Eq. (16) can be formulated in terms of aspect tensors thanks to the definition in Eq. (): since sg=I, its time derivative (∂ts)g+s(∂tg)=0 leads to the dynamics ∂ts=-g-1(∂tg)s, and then 17∂ts=-s(∂tg)s, where it remains to replace occurrences of g by s-1 in the resulting dynamics of the mean, the variance, and the aspect tensor.

Hence, the PKF forecast step for a VLATcov model is given by either the system in Eq. (16) (in metric) or by its aspect tensor formulation thanks to Eq. (). Whatever the formulation considered, it is possible to carry out the calculations using a formal calculation language. However, even for simple physical processes, the number of terms in formal expressions can become very large; e.g., it is common to have to manipulate expressions with more than 100 terms. Thus, any strategy that simplifies the assessment of PKF systems in advance can quickly become a significant advantage.

In the following section, we present the splitting method that allows the PKF dynamics to be expressed by bringing together the dynamics of each of the physical processes, calculated individually.

When there are several processes in the dynamics in Eq. (), the calculation of the parametric dynamics can be tedious even when using a computer algebra system. To better use digital resources, a splitting strategy can be introduced .

While the theoretical background is provided by the Lie–Trotter formula for Lie derivatives, the well-known idea of time splitting is easily taken from a first-order Taylor expansion of an Euler numerical scheme.

The computation of dynamics, 18∂tX=f1(X)+f2(X), over a single time step δt can be done in two steps following the numerical scheme 19X⋆=X(t)+δtf1(X(t)),X(t+δt)=X⋆+δtf2(X⋆), where at order δt, this scheme is equivalent to X(t+δt)=X(t)+δtf1(X(t))+f2(X(t)), which is the Euler step of Eq. (). Because f1 and f2 can be viewed as vector fields, the fractional scheme, joining the starting point (at t) to the end point (at t+δt), remains to go through the parallelogram formed by the sum of the two vectors along its sides. Since there are two paths joining the extreme points, starting the computation by f2 is equivalent to starting by f1 (at order δt); this corresponds to the commutativity of the diagram formed by the parallelogram.

Appendix  shows that the dynamics given by Eq. () imply dynamics of the error, the variance, the metric, and the aspect written as a sum of trends. Hence, it is possible to apply a splitting for all these dynamics.

As a consequence of the calculation of the parametric dynamics, calculating the parametric dynamics of Eq. () is equivalent to separately calculating the parametric dynamics of ∂tX=f1(X) and ∂tX=f2(X), then bringing together the two parametric dynamics into a single one by summing the trends for the mean, the variance, the metric, or the aspect dynamics. This splitting applies when there are more than two processes and appears to be a general method to reduce the complexity of the calculation.

However, although the calculation of the system in Eq. (16) is straightforward, as it is similar to the calculation of Reynolds equations , it is tedious because of the many terms involved, and there is a risk of introducing errors during the calculation by hand.

Then, once the dynamics of the parameters are established, it remains to design a numerical code to test whether the uncertainty is effectively well represented by the PKF dynamics. Again, the design of a numerical code is not necessarily difficult, but with numerous terms the risk of introducing an error is important.

To facilitate the design of the PKF dynamics and the numerical evaluation, the package SymPKF has been introduced to perform the VLATcov parameter dynamics and to generate a numerical code used for the investigations . The next section introduces and details this tool.

The definition of the dynamics relies on the formalism of SymPy as shown in Fig. . The coordinate system is first defined as instances of the class Symbols. Note that the time is defined as sympkf.t, while the spatial coordinate is left to the choice of the user, here x. Then, the function u is defined as an instance of the class Function as a function of (t,x).

In this example, the dynamics consist of a single equation defined as an instance of the class Eq, but in the general situation in which the dynamics are given as a system of equations, the dynamics have to be represented as a Python list of equations.

A preprocessing of the dynamics is then performed to determine several important quantities to handle the dynamics: the prognostic fields (functions for which a time derivative is present), the diagnostic fields (functions for which there is no time derivative in the dynamics), the constant functions (functions that only depend on the spatial coordinates), and the constants (pure scalar terms that are not a function of any coordinate). This preprocessing is performed when transforming the dynamics as an instance of the class PDESystem and whose default string output delivers a summary of the dynamics: for the Burgers equation, there is only one prognostic function, u(t,x), and one constant, κ.

The prognostic quantities being known, it is then possible to perform the computation of the PKF dynamics, as discussed now.

Thanks to the preprocessing, we are able to determine the VLATcov parameters needed to compute the PKF dynamics, which are the variance and the anisotropy tensor associated with the prognostic fields. For the Burgers equation, the VLATcov parameters are the variance Vu and the metric tensor gu=(gu,xx) or its associated aspect tensor su=(su,xx). Note that, in SymPKF, the VLATcov parameters are labeled by their corresponding prognostic fields to facilitate their identification. This labeling is achieved when the dynamics are transformed as an instance of the class SymbolicPKF. This class is at the core of the computation of the PKF dynamics from Eq. (16).

As discussed in Sect. , the PKF dynamics rely on the second-order fluctuation–mean interaction dynamics wherein each prognostic function is replaced by a stochastic counterpart. Hence, the constructor of SymbolicPKF converts each prognostic function as a function of an additional coordinate, ω∈Ω. For the Burgers equation, u(t,x) becomes u(t,x,ω).

Since the computation of the second-order fluctuation–mean interaction dynamics relies on the expectation operator, an implementation of this expectation operator has been introduced in SymPKF: it is defined as the class Expectation built by inheritance from the class sympy.Function to leverage the computational facilities of SymPy. The implementation of the class Expectation is based on the linearity of the mathematical expectation operator with respect to deterministic quantities and its commutativity with partial derivatives and integrals with respect to coordinates different from ω, e.g., for the Burgers equation E[∂xu(t,x,ω)]=∂xE[u(t,x,ω)]. Note that E[u(t,x,ω)] is a function of (t,x) only: the expectation operator converts a random variable into a deterministic variable.

Then, the symbolic computation of the second-order fluctuation–mean interaction dynamics in Eq. () is performed, thanks to SymPy, by following the steps as described in Sect. . In particular, the computation also leads to the tangent-linear dynamics of the error in Eq. (), from which it is possible to compute the dynamics of the variance in Eq. () and of the metric tensor in Eq. () (or its associated aspect tensor version). Applying these steps and the appropriate substitutions, this is achieved when calling the in_metric or in_aspect Python property of an instance of the class SymbolicPKF. This is shown for the Burgers equation in Fig. , where the background computation of the PKF dynamics leads to a list of the three coupled equations corresponding to the mean, the variance, and the aspect tensor, similar to the system in Eq. (22) first obtained by .

Hence, from SymPKF, for the Burgers equation, the VLATcov PKF dynamics given in the aspect tensor read as 21∂tu=κ∂x2u-u∂xu-∂xVu2∂tVu=-2κVusu,xx+κ∂x2Vu-κ∂xVu22Vu-u∂xVu-2Vu∂xu∂tsu,xx=2κsu,xx2Eεu∂x4εu-3κ∂x2su,xx-2κ+6κ∂xsu,xx2su,xx-2κsu,xx∂x2VuVu+κ∂xVu∂xsu,xxVu+2κsu,xx∂xVu2Vu2-u∂xsu,xx+2su,xx∂xu, where su,xx is the single component of the aspect tensor su in 1D domains. Note that in the output of the PKF equations, as reproduced in Eq. (), the expectation in the dynamics of the mean is replaced by the prognostic field; for the Burgers equation, E[u](t,x) is simply denoted by u(t,x).

While the Burgers equation only contains two physical processes, i.e., the nonlinear advection and the diffusion, the resulting PKF dynamics in Eq. () make numerous terms appear, which justifies the use of symbolic computation, as mentioned above. The computation of the PKF dynamics leading to the metric and to the aspect tensor formulation takes about 1 s of computation (Intel Core i7-7820HQ CPU at 2.90 GHz × 8).

In this example, the splitting strategy has not been considered to simplify the computation of the PKF dynamics. However, it can be done by considering the PKF dynamics for the advection ∂tu=-u∂xu and the diffusion ∂tu=κ∂x2u, and computed separately, then merged to find the PKF dynamics of the full Burgers equation. For instance, Fig.  shows the PKF dynamics for the advection (first cell) and for the diffusion (second cell); the output can be traced back in Eq. (), e.g., by the terms in κ for the diffusion.

Thanks to the symbolic computation using the expectation operator, as implemented by the class Expectation, it is possible to handle terms such as E[εu∂x4εu] during the computation of the PKF dynamics. The next section details how these terms are handled during the computation and the closure issue they bring.

While compiled language with appropriate optimization should be important for industrial applications, we chose to implement a pure Python code generator, which offers a simple research framework for exploring the design of PKF dynamics. It would have been possible to use a code generator already based on SymPy (see, e.g., ), but with such code generators being domain-specific, it was less adapted to the investigation of the PKF for arbitrary dynamics. Instead, we consider a finite-difference implementation of partial derivatives with respect to spatial coordinates. The default domain to perform the computation is the periodic unit square with the dimension of the number of spatial coordinates. The length of the domain can be specified along each direction. The domain is regularly discretized along each direction, while the number of grid points can be specified for each direction.

The finite difference takes the form of an operator F that approximates any partial derivate at a second order of consistency: for any multi-index α, Fαu=0∂αu+O(|δx|2), where O is Landau's big O notation. For any f, the notation f(δx)=0O(δx2) means that lim⁡δx→0f(δx)δx2 is finite. The operator F computed with respect to independent coordinates commutes, e.g., Fxy=Fx∘Fy=Fy∘Fx, where ∘ denotes the composition, but it does not commute for dependent coordinates, e.g., Fx2≠Fx∘Fx. The finite difference of the partial derivative with respect to the multi-index is computed sequentially, e.g., Fxxy=Fx2∘Fy=Fy∘Fx2. The finite difference of order α with respect to a single spatial coordinate is the centered finite difference based on α+1 points.

For instance, Fig.  shows how to close the PKF dynamics for the Burgers equation following P18 and how to build a code from an instance of the class sympkf.FDModelBuilder: it creates the class ClosedPKFBurgers. In this example, the code is rendered from templates thanks to Jinja

https://jinja.palletsprojects.com/en/2.11.x/, last access: 22 September 2021.​​​​​​​

In order to illustrate the ability of SymPKF to apply in a 2D or 3D domain, we consider the linear advection of a scalar field c(t,x,y) by a stationary velocity field u=(u(x,y),v(x,y)), which reads as the partial differential equation 29∂tc+u∇c=0. As for the Burgers equation, the definition of the dynamics relies on SymPy (not shown but similar to the definition of the Burgers equation as given in Fig. ). This leads to preprocessing the dynamics by creating the instance advection of the class PDESystem, which transforms the equation into a system of partial differential equations. In particular, the procedure will diagnose the prognostic functions of dynamics, here the function c. Then it identifies the constant functions, which can depend on space but not on time: here, these are the components of the velocity (u, v). The process also identifies exogenous functions and constants, of which there are none here.

The calculation of the parametric dynamics is handled by the class SymbolicPKF as shown in the first cell in Fig. . The parametric dynamics are a property of the instance pkf_advection of the class SymbolicPKF, and when it is called, the parametric dynamics are computed once and for all. The parametric dynamics formulated in terms of metric are first computed; see the second cell. For the 2D linear advection, the parametric dynamics are a system of five partial differential equations, as is shown in the output of the second cell: the dynamics of the ensemble average E[c], which outputs as c for the sake of simplicity (first equation), the dynamics of the variance (second equation), and the dynamics of the local metric tensor (last three equations). In compact form, the dynamics are given by the system 30a∂tc+u∇c=0,30b∂tVc+u∇Vc=0,30c∂tgc+u∇gc=-gc∇u-∇uTgc, which corresponds to the 2D extension of the 1D dynamics first found by and validates the computation performed in SymPKF. Due to the linearity of the linear advection in Eq. (), the ensemble average in Eq. () is governed by the same dynamics as in Eq. (). While both the variance in Eq. () and the metric are advected by the flow, the metric is also deformed by the shear in Eq. (). This deformation more commonly appears in the dynamics written in aspect tensor form, which is given by 31a∂tc+u∇c=0,31b∂tVc+u∇Vc=0,31c∂tsc+u∇sc=∇usc+sc∇uT, where Eq. () is similar to the dynamics of the conformation tensor in viscoelastic flow .

We do not introduce any numerical simulation of the PKF dynamics in Eq. () or Eq. (), but interested readers are referred to the 2D numerical PKF assimilation cycles of , which have been made thanks to SymPKF.

This example illustrates a 2D situation and shows the multidimensional capabilities of SymPKF. Similarly to the simulation conducted for the Burgers equation, it is possible to automatically generate a numerical code able to perform numerical simulations of the dynamics in Eq. () (not shown here). Hence, this 2D domain example showed the ability of SymPKF to apply in dimensions larger than 1D.

Before concluding, we would like to present a preliminary application of SymPKF in a multivariate situation.

SymPKF can be used to compute the prediction of the variance and the anisotropy in a multivariate situation.

Note that one of the difficulties with the multivariate situation is that the number of equations increases linearly with the number of fields and the dimension of the domain; e.g., for a 1D (2D) domain and two multivariate physical fields, there are two ensemble-averaged fields, two variance fields, and two (six) metric fields. Of course this is no not a problem when using a computer algebra system as done in SymPKF.

To illustrate the multivariate situation, only a very simple example is introduced. Inspired from chemical transport models encountered in air quality, we consider the transport over a 1D domain of two chemical species, whose concentrations are denoted by A(t,x) and (B(t,x), advected by the wind u(x). For the sake of simplicity, the two species interact following periodic dynamics as defined by the coupled system 32a∂tA+u∂xA=B,32b∂tB+u∂xB=-A. Thanks to the splitting strategy, the PKF dynamics due to the advection have already been detailed in the previous section (see Sect. ), so we can focus on the chemical part of the dynamics, which is given by the processes on the right-hand side of Eq. (). The PKF of the chemical part is computed thanks to SymPKF and shown in Fig. . This time, and as expected, multivariate statistics appear in the dynamics. Here, the dynamics of the cross-covariance VAB=E[eAeB] are given by the fifth equation. The coupling brings up unknown terms, e.g., the term E[∂xεA∂xεB] in the sixth equation of the output shown in Fig. . Note that by taking into account the multivariate situation with the dynamics of the cross-covariance, the multivariate PKF hybridizes the continuous nature of the multivariate fields with the matrix form in Eq. (), which corresponds here to the dynamics of the variances (VA,VB) and the cross-covariance VAB.

To go further, some research is still needed to explore the dynamics and the modeling of the multivariate cross-covariances. A possible direction is to take advantage of the multivariate covariance model based on the balance operator as often introduced in variational data assimilation . Note that such multivariate covariance models have recently been considered for the design of the multivariate PKF analysis step . Another way is to consider a data-driven strategy to learn the physics of the unknown terms from a training based on ensembles of forecasts .

To conclude, this example shows the potential of SymPKF to tackle the multivariate situation. Moreover, the example also shows that SymPKF is able to perform the PKF computation for a system of partial differential equations. However, all the equations should be prognostic; SymPKF is not able to handle diagnostic equations.