Paleoclimate reconstruction based on assimilation of proxy observations requires specification of the control variables and their background statistics. As opposed to numerical weather prediction (NWP), which is mostly an initial condition problem, the main source of error growth in deterministic Earth system models (ESMs) regarding the model low-frequency response comes from errors in other inputs: parameters for the small-scale physics, as well as forcing and boundary conditions. Also, comprehensive ESMs are non-linear and only a few ensemble members can be run in current high-performance computers. Under these conditions we evaluate two assimilation schemes, which (a) count on iterations to deal with non-linearity and (b) are based on low-dimensional control vectors to reduce the computational need. The practical implementation would assume that the ESM has been previously globally tuned with current observations and that for a given situation there is previous knowledge of the most sensitive inputs (given corresponding uncertainties), which should be selected as control variables. The low dimension of the control vector allows for using full-rank covariances and resorting to finite-difference sensitivities (FDSs). The schemes are then an FDS implementation of the iterative Kalman smoother (FDS-IKS, a Gauss–Newton scheme) and a so-called FDS-multistep Kalman smoother (FDS-MKS, based on repeated assimilation of the observations). We describe the schemes and evaluate the analysis step for a data assimilation window in two numerical experiments: (a) a simple 1-D energy balance model (Ebm1D; which has an adjoint code) with present-day surface air temperature from the NCEP/NCAR reanalysis data as a target and (b) a multi-decadal synthetic case with the Community Earth System Model (CESM v1.2, with no adjoint). In the Ebm1D experiment, the FDS-IKS converges to the same parameters and cost function values as a 4D-Var scheme. For similar iterations to the FDS-IKS, the FDS-MKS results in slightly higher cost function values, which are still substantially lower than those of an ensemble transform Kalman filter (ETKF). In the CESM experiment, we include an ETKF with Gaussian anamorphosis (ETKF-GA) implementation as a potential non-linear assimilation alternative. For three iterations, both FDS schemes obtain cost functions values that are close between them and (with about half the computational cost) lower than those of the ETKF and ETKF-GA (with similar cost function values). Overall, the FDS-IKS seems more adequate for the problem, with the FDS-MKS potentially more useful to damp increments in early iterations of the FDS-IKS.

Earth system models (ESMs) to simulate the Earth system and global climate
are usually developed using the present and recent historical climates as
references, but climate projections indicate that future climates will lie
outside these conditions. Paleoclimates very different from these reference
states therefore provide a way to assess whether the ESM sensitivity to
forcings is compatible with the evidence given by paleoclimatic records

The issue of fusing data into models arises in scientific areas that enjoy a
profusion of data and use costly models. In the geophysical community this is
referred to as inverse methods and data assimilation (DA), whose aim is
finding the best estimate of the state (the analysis) by combining
information from the observations and from the numerical and theoretical
knowledge of the underlying governing dynamical laws. Most known DA methods
stem from Bayes' theorem

DA has been used as a technique for low-frequency past climate field
reconstruction (CFR) with real case studies, such as the assimilation of marine
sediment proxies of sea surface temperature (SST) in a regional ocean model
of the North Atlantic at the termination of the Younger Dryas (YD) cold interval

Questions remain about how one should choose the control vector for the
assimilation. Regarding its dimension, one possibility is to select a
relatively high-dimensional control vector and to resort to ensemble methods,
which involve a low-rank representation of covariances. An example is the (adjoint-free) iterative ensemble Kalman smoother
(IEnKS) in

In any case, in a practical application of such a low-dimensional control
vector approach (whose dimension would be imposed by computational
constraints), the selection of the control variables should be carefully
done. From all available model inputs, the selected control variables (given
their respective background uncertainties) and model should try to explain
most of the observed variability. In turn, this assumes (a) a general need to
perform sensitivity analysis beforehand and (b) that the model has been
previously comprehensively tuned. The exclusion of relatively less sensitive
inputs from the control vector and the previous tuning would reduce possible
compensation effects (i.e. that increments in the control vector due to the
assimilation take partial responsibility for errors elsewhere). Nonetheless,
some error compensation will always be present (for example, this is
intrinsic to the common tuning of the coupled ESM, which follows tuning of
individual components) and very difficult to deal with. A striking example is
given by

Also, regarding initial conditions in a sequence of multi-decadal and longer
data assimilation windows with transient forcings, there is no clear
consensus about how one should approach the initialization at each DAW. For
example,

Throughout this study, all observations available during a data assimilation
window (DAW) are assimilated in parallel. This has been termed
four-dimensional data assimilation or asynchronous data assimilation and is
also commonly referred to as the smoothing problem

The rest of this article is organized as follows. In Sect.

The problem is to estimate the mean state (seasonal and annual means) of a
past climate state along a time window for multi-decadal and longer
timescales. From a variational perspective, in NWP this would be referred to
as a four-dimensional variational data assimilation (4D-Var) problem, in
which the initial conditions of a model integration are estimated subject to
model dynamics and according to background and observation uncertainties
within a data assimilation window (DAW). In NWP, the background (or prior)
is normally given by a previous model forecast. In this article, time

Then, we consider an augmented state vector

Observations at time

In general, an exact solution cannot be found. In the incremental formulation
of 4D-Var, the solution to Eq. (

Then, incremental 4D-Var has an inner loop, for which two approximations are
conducted. The first is

The minimization of

In our context in this paper, we assume Gaussian statistics and a
perfect-model framework except for the sources of model uncertainty in

So, the inner loop is omitted and the state vector is explicitly updated as

Thus, like incremental 4D-Var, the iterative approach described by Eq. (

Now, it remains to be seen how one would apply a scheme such as Eq. (

In general, ensemble methods rely on perturbations

For the more general transient forcing situation, in a current DAW, the
effects from a perturbed control vector and from transient forcing are
entangled. As opposed to the equilibrium simulations, integration times here
match physical forcing times. Observations earlier than a specified

In both cases, it is unlikely that errors in initial conditions are among the
most sensitive ones out of all possible input errors for the evaluated
integration times after quasi-equilibrium. Thus, for low-frequency past
climate analysis, it should be generally acceptable to exclude

After the assimilation, a forward integration with the updated

The current implementation of variational assimilation is different in each
operational NWP centre. A recent review of operational methods of variational
and ensemble-variational data assimilation is given by

While the relation between

Alternatively, finite-difference sensitivity (FDS) directly samples from the
conditional probability density function (CPDF) of the perturbed variable, as
the remaining control variables are kept to their current estimate. However,
the computing requirements in FDS are linearly proportional to the size of
the input vector, its numerical estimation of derivatives is inaccurate,
and the associated error can be unacceptably large due to inadequate choice
of the finite-differencing step size. High perturbations increase the
truncation error, which increases linearly with the perturbation magnitude,
while as the magnitude of the perturbation gets smaller the accuracy of the
differentiation degrades by the loss of computer precision

The algorithm we describe here, denoted as finite-difference
sensitivity iterative Kalman smoother (FDS-IKS), is a Gauss–Newton scheme
akin to the IKF and the IKS. The “FDS” acronym clarifies that the scheme
is (a) expressed in terms of explicit sensitivities to all variables in the
control vector, and (b) these local sensitivities are estimated numerically by
individual perturbation experiments for each variable in the control vector.
The scheme then uses a full-rank representation of the background-error
covariance matrix (hence, it is not called an

For any natural number

The sequences

Equations (

Here, a multistep approach is conducted by inflating the observation-error
covariance matrix

The multistep strategy was termed multiple data assimilation (MDA) by

The scheme considers the total increment in the state vector that would
result from the linear assimilation of one specific observation and
alternatively conducts a recursive sequence of assimilations of the same
observation whose sum of fractional increments equals the total increment.
This is achieved by considering the observation-error variance at loop

Here, the sensitivity matrix

The sequences

Regarding

The computational cost of the ESM integrations is much higher than that of
the assimilation steps as considered in the FDS-MKS for a low-dimensional
control vector. In this study, we do not evaluate adaptive strategies for the
planning of the weights in the FDS-MKS. However, the evolution of the
increments in the control variables along the iterations could potentially be
used to guide the size of

Comparison of the sequence of increments given by the fractional steps of the FDS-MKS with those with simultaneous early-stopped solutions may be used to support replanning weights and even to decide on an early stopping of the iterations using the update given by using the completion weight as a final solution.

The ensemble Kalman filter (EnKF) was introduced by

Still, for the (En)KF to be optimal, three special conditions need to apply:
(1) Gaussianity in the prior, (2) linearity of the observation operator, and
(3) Gaussianity in the additive observational error density. In order to
better deal with non-linearity, a number of studies have addressed the use of
transformation of the model background and observation to obtain a Gaussian
distribution such that the (En)KF can be applied under optimal
conditions. This pre-processing transformation step is known as Gaussian
anamorphosis (GA)

It is not standard, however, how the GA should be applied in the context of
DA

In a theoretical framework and with simple experiments,

In our implementation of the ETKF we augmented the state vector with the
model equivalent of the observations. We evaluated transformations of the
control variables as well as transformations in sea surface temperature
(SST) as observed variables. We transformed the control variables marginally.
Regarding SST, due to sparsity and heterogeneity, we consider it not
possible to estimate the marginal distribution of the low-frequency
paleoclimate observations with enough confidence to support a
transformation. Thus, in our experiments we estimated the marginal
distribution of the model equivalent of the SST observations, as derived from
the background ensemble, and also used the same transformation for the SST
observations. The transformation then operates in the marginals in
an independent way at each grid point:

As indicated, here we use empirical cumulative density functions (CDFs) for
the anamorphosis based on the background ensemble. The risk of using the
tails of the transformation function during the anamorphosis of the ensemble
is significant, and tail estimation can highly impact the analysis. Here, we
obtained linear tails following

This experiment is based on a conceptual one-dimensional, south–north,
energy balance model (Ebm1D;

1-D energy balance model. PD1 tests. Parameter definition and first-guess values.

PL2012 evaluate several climate conditions and uncertain parameter scenarios,
including present-day and Last Glacial Maximum (LGM) climate states. Then,
with the model constrained by the present-day and LGM parameter estimates,
they conduct climate projections under several

As observations, we took surface air temperature (SAT) derived from the
NCEP/NCAR reanalysis data

Like PL2012, we set the grid resolution to 10

For the PD1 tests, we made the observation weights

We conducted a number of additional tests to compare the convergence of the
FDS-IKS versus the FDS-MKS as higher weight is given to the observations.
These were named PD2 and PD3, corresponding to

Here we provide a succinct summary of the estimation process. Broader
explanation of the model climatology in relation to the control variables
is given in PL2012. The background sensitivity of the 10-year mean surface
temperature

Ebm1D experiment. Background sensitivity of winter surface air
temperature to the control variables estimated as

For the PD1 scenario, Fig.

Ebm1D PD1 tests. Parameter estimation and cost function values

Ebm1D experiment. Convergence as a function of the number of simulations for the
scenario PD1 (

Table

Experiment 2 is a synthetic test with the Community Earth System Model
(CESM1.2), a deterministic ESM. The CESM component set used here comprises
the Community Atmosphere Model version 4 (CAM4), the Parallel Ocean Program
version 2 (POP2), the Community Land Model (CLM4.0), the Community Ice CodE
(CICE 4) as a sea ice component, the River Transport Model (RTM), and the CESM
flux coupler CPL7. The coupler computes interfacial fluxes between the
various component models (based on state variables) and distributes these
fluxes to all component models while ensuring the conservation of fluxed
quantities. Land ice is set as a boundary condition, and the wave component is
not active. The configuration uses pre-industrial forcings and it is a
standard component set named B1850CN in the CESM1.2 list of

Here we focus on the analysis for a single DAW and equilibrium forcing and,
as adequate, introduce some comments regarding practical implementations for
real cases, including the case of transient forcings. The identical twin
assimilation experiment is designed to approach a case of past climate
reconstruction with sparse observations, as usual in pre-instrumental climate
analysis. Specifically, we use the features of available observations of near
sea surface temperature for the Last Glacial Maximum (LGM) from the
MARGO database

To create the background ensemble we perturbed a number of parameters for the (deterministic) physics in both the ocean and the atmosphere components, as well as input greenhouse gases and an additional influx of water into the North Atlantic Ocean. As indicated in the Introduction, the selected control variables have the responsibility of creating all the background uncertainty in a perfect-model scenario, and through the assimilation they will try to compensate for any unaccounted model error elsewhere. In a step-by-step approach, here all perturbed model parameters and forcings were included as control variables in the assimilation. An obvious (still synthetic) and very useful extension would be to perturb a wider set of model parameters and/or forcings and boundary conditions (e.g. various ice sheet configurations or alternative freshwater influx) and explicitly evaluate the compensation effect and climate reconstruction results by using subsets of the perturbed inputs as control vectors. Here, the selected parameters for model physics and radiative constituents are relevant to the global energy budget of the Earth system, but not necessarily the most sensitive model inputs for multi-decadal and longer scales. In real cases, the selection of control variables (if the control vector is to be kept low-dimensional) should be done carefully and generally based on previous global sensitivity analyses.

We included an influx of water into the North Atlantic from melting in the
Greenland ice sheet (GIS) to the true run and as a control variable. This flux
was homogeneously distributed along the coast of Greenland and at the ocean
surface, and it is appealing to explore as a control variable because the
Atlantic meridional overturning circulation (AMOC) plays a critical role in
maintaining the global ocean heat and freshwater balance. It is commonly
acknowledged that North Atlantic deep water (NADW) formation is key in
sustaining the AMOC

We initiated the background with biased control variables with respect to the
truth and a zero-mean Greenland ice sheet freshwater flux. We used reasonable
uncertainties in the control variables derived from previous publications.
Separate analyses (weakly coupled assimilation) for different model
components (atmosphere, ocean, land) may be inconsistent. In our set-up, all
observations are allowed to directly impact model parameters from any
component in the Earth system model. This is known as strongly coupled data
assimilation. Both truth and background simulations were branched from the
same initial conditions, which allowed us to use relatively short integration
times (60 years) in the experiment. In a real case with steady-state forcings
(e.g. estimation of real LGM climate state by assimilating the MARGO
database), the model should be integrated even longer towards
quasi-equilibrium to ensure that errors in the initial conditions will
not affect the analysis (or they should be accounted for). Also, each model
equivalent of the observations has to be mapped into the corresponding
spatio-temporal domain of each paleoclimate proxy observation. Similar to the
previous experiment, for the FDS schemes, we set the perturbations for each
control variable as equal to their standard deviation multiplied by a
perturbation factor SDfac. For computational reasons we only tested

The cost function was as in Eq. (

We used the Community Atmosphere Model version 4 (CAM4) as an atmospheric global
circulation model (AGCM) component. A comprehensive description of CAM4 can
be found in

Here we used CAM4 with the

In this study, as perturbed parameters and control variables we selected
parameters related to the ZM deep convection scheme and the relative humidity
thresholds for low and high stable cloud formation. Also, within the
radiative constituents, we included invariant surface values of CO

CESM definition of control variables.

This experiment is based on equilibrium simulations. Regarding real cases in
transient conditions,

As an ocean component, we used POP2

As control variables in POP2 we chose the Gent–McWilliams isopycnic tracer diffusion parameter and the (constant with depth) KPP background viscosity, both with default values for the truth. A third control variable in POP2 was the total freshwater influx from the Greenland ice sheet, which we distributed homogeneously along the coast of Greenland and only at the ocean surface.

The observational dataset is composed of point samples of climate averages
for the last 20 years out of a total 60 years of integration time in a true
simulation. The synthetic observations were located at the horizontal
locations and 10 m of depth of the MARGO database, and the sampling
characteristics reproduce those of MARGO. The MARGO database is a
synthesis of six different proxies and is considered to represent the
combined expertise of at least a sizeable fraction of the LGM paleocommunity.
The observational uncertainty was taken from the MARGO database as input to
the assimilation, but we did not add any error to the synthetic observations.
MARGO provides observations (or reconstructions) of near sea surface
temperature (SST) for the Last Glacial Maximum (LGM). The proxy types on
which the SST estimates are based are (a) microfossil based (planktonic
foraminifera, diatom, dinoflagellate cyst, and radiolarian abundances) and (b)
geochemical paleothermometers (alkenone unsaturation ratios (

In summary, MARGO provides seasonal means for Northern Hemisphere winter
(January, February, and March; JFM) and summer (July, August, and September;
JAS), as well as annual means. However, the data availability for each of the
three temporal means (winter, summer, and annual) is different for each proxy
type. Specifically, diatoms are just available for Southern Hemisphere
summer; dinoflagellates, foraminifera, and Mg

MARGO data coverage

The need for non-linear estimation is justified based on the assumed non-linear
relationship between the control variables and the observation space. In this
experiment, non-linearity is imposed by the Earth system model, which directly
generates SST as the observed variable. In a more general case of past climate
analysis, the forward operator (proxy system model) can impose further
non-linearity when the observed variables are direct proxy records (e.g.
foraminiferal counts, tree-ring widths, speleothems, etc.). In addition to
model and forward operator non-linearity, non-Gaussianity in the control
variables also renders (En)KF non-optimal. Here we conducted a test with ETKF
including the Gaussian anamorphosis (GA) transformation (ETKF-GA). The test
transforms the control variables, whose background deviations from
Gaussianity here derive from imposed bounds, and the SST in both the model
equivalent of the observations and the observations themselves with the
strategy explained in Sect.

In the ETKF-GA test, we conducted the marginal Gaussian anamorphosis for all
the variables in the control vector. Table

CESM experiment.

The minimum relative humidity for high stable cloud formation
(CAM.cldfrc_rhminl parameter), given its background uncertainty, was shown
to have a strong effect on SST in the experiment. In most locations of the
global ocean the marginal background SST had a non-Gaussian but still
unimodal probability density function. Especially complicated was the North
Atlantic, with strongly bimodal background distributions in some locations.
One of these cases is depicted in Fig.

CESM experiment. Example of Gaussian anamorphosis transformation in a control
variable (minimum relative humidity for high stable cloud formation) and the sea surface
temperature (SST) for both the model equivalent of the observations and the observation
(transformation details in text) at a location (343.17

Table

CESM control vector estimation

All schemes obtained a substantial reduction in the value of the cost
function with respect to the background, which had a

As seen, regarding the cost function, the Gaussian anamorphosis (as
implemented here) did not improve the minimization with respect to the ETKF,
although the transformation served to obtain a lower value for the
observational term

CESM experiment. Absolute bias reduction for SST and SSS as a result
of a new integration with the parameters estimated with the ETKF and the
FDS-MKS. The statistics are the absolute bias between the background and the
truth minus the absolute bias between the analysis and the truth. Thus,
positive values are a net bias reduction. Isolines at value 0 shown in grey.
The two triangles indicate the locations in the North Atlantic and South
Pacific connected with Figs.

CESM experiment. Sensitivity of SST (

The lower cost function values of the FDS schemes with respect to the ETKF (with and without GA) suggest a benefit in the more limited (and noisier) but iterated local sensitivity estimation. Also, the computational cost in the FDS tests was about half of the ETKF (and ETKG-GA). Regarding the estimation of specific control variables, all of the evaluated schemes had some variables for which the estimation, starting from the background, went in the wrong direction with respect to the true values. For example, the closest estimate to the true value of the relative humidity threshold for high stable cloud formation (cldfrc_rhminh) was given by the FDS-MKS, with a slight overshooting (0.81 versus 0.80 for the truth). It may have been that this slight overshooting has partially compensated for the effect of other control variables. Thus, the FDS-MKS estimates of the freshwater flux from the Greenland influx went in the wrong directions, as did the estimates for the autoconversion coefficients in the Zhang–McFarlane deep convection scheme. On the other hand, the FDS-IKS had a total increment in cldfrc_rhminh in the wrong direction, but had the Greenland influx total increment in the right direction. The ETKF did not show any overshooting, but had some control variable increments going in the wrong direction. For the ocean background vertical diffusivity, the only two schemes for which there was some, albeit minor, improvement in the estimate were the ETKF and the FDS-IKS. Still, the improvement is so slight in these cases that it could be a random effect.

The perturbations in the FDS may have been far from optimal for sensitivity
estimation regarding their effect on the model SST at the locations
(including depth) of the observations for the integration times. Table

In sensitivity studies, the conditional sensitivity exploration of the FDS
has also been termed one-at-a-time (OAT) sensitivity analysis. The difference
between the local sensitivities of the FDS and the mean sensitivities of the
ensemble for the ETKF may affect the estimation of the various parameters to
different degrees. For comparison, with a prescribed SST,

This study does not attempt to give an in-depth analysis of the assimilation
results for the corresponding climate field reconstructions. However, we
summarize some results of the spatial patterns shown in the climate
reconstructions and give examples of sensitivities as estimated by the FDS
schemes and the ETKF. Figure

As an example of estimated sensitivities in the ocean, Fig.

Vertical diffusion in the ocean determines ocean heat uptake and in turn the
air–sea heat flux and atmospheric heat transport, but also sea surface
temperature, evaporation, and atmospheric moisture transport. Regions more
sensitive to ocean vertical diffusion would be coastal upwelling systems
(e.g. Namibia), the equatorial oceans, and the Southern Ocean in the case of
upwelling, but also the North Atlantic Ocean in the case of downwelling
(deep water formation), which is key in sustaining the Atlantic meridional
overturning circulation (AMOC)

CESM experiment. Sensitivity of the Atlantic meridional overturning circulation (AMOC) to the ocean background vertical
diffusion parameter (

As a last related example, Fig.

CESM experiment. Sensitivity of several atmospheric variables to the
ocean background vertical diffusion parameter (

An important last consideration is that the assimilation will just attempt to
minimize (or get the first moments of) the cost function. So the assumed
background statistics in the cost function are instrumental in controlling
the control vector increments in the assimilation and the resulting climate
field reconstruction (CFR). In this synthetic experiment the source of errors
is known, and we assume a perfect-model framework except for the assumed
uncertainties in the chosen control vector. However, in a real applied
situation the real model errors are unknown. As described in the
Introduction, the control vector increments will compensate for non-accounted
errors. Although the minimization can highly reduce the value of a cost
function and improve the corresponding CFR, it does not necessarily imply
that updated parameters for the model physics (or their moments), as part of
the control vector, actually correspond to improved (extrapolable) model
physics. For example, the use of the posterior model parameters can
potentially lead to improved climate simulations for other prospective
climatic conditions, but not necessarily. Thus, it is important to
distinguish between the use of the assimilation methods for CFR including
model parameters as control variables and the trust one can have in the
estimated model parameters for future climate projections under very
different climatic conditions. A fair caveat was recently given by

This study focuses on low-frequency climate field reconstruction (multi-decadal and longer timescales) with comprehensive deterministic Earth system models (ESMs). Given the enormous computational requirements for this class of models, we evaluate two iterative schemes based on reduced-order control vectors and the Kalman filter as assimilation approaches for climate field reconstruction. The schemes use an explicit representation of the background-error covariance matrix, and the Kalman gain is based on finite-difference sensitivity (FDS) experiments. As such, the schemes are computationally limited to the estimation of a low-dimensional control vector. The underlying assumption is so that a low-dimensional control vector and its background uncertainty, containing the most sensitive variables for a given climate, can encapsulate most of the modelled internal and external climate variability. The control vector can contain parameterized errors in initial conditions and parameters for the small-scale physics, as well as parameters for forcing and boundary condition errors (e.g. a bias in a time-varying radiative constituent). In general, it is expected that errors in initial conditions are a low sensitive input for the low-frequency model climate response. Thus, these would be generally excluded from the control vector, which makes it relatively easier to keep its low dimensionality.

The evaluated schemes are an FDS implementation of the iterative Kalman smoother (FDS-IKS, a Gauss–Newton scheme) and a so-called FDS-multistep Kalman smoother (FDS-MKS, based on repeated assimilation of the observations). We have conducted two assimilation experiments: (a) a simple 1-D energy balance model (Ebm1D; which has an adjoint code) with present-day surface air temperature from the NCEP/NCAR reanalysis data as a target and (b) a multi-decadal synthetic case with the Community Earth System Model (CESM v1.2, with no adjoint). The methodological description and the first experiment serve to show that, under a strong-constraint minimization and perfect-model framework, the FDS-IKS should converge to the same minimum as incremental 4D-Var. Actually, in this experiment, the FDS-IKS converges substantially faster than 4D-Var and to the same minimum. The FDS-MKS does not theoretically converge to the same minimum (except for linear cases), but it is more stable than the FDS-IKS for poorly regularized cost functions.

In a second experiment with CESM, given the lack of an adjoint code, we
included an ETKF (with

This study is a first attempt to use the described iterated schemes for
assimilation with comprehensive ESMs and multi-decadal or longer timescales.
It has provided the context of the problem, described the schemes, and
conducted preliminary experiments. The study is limited by the same
computational constraint that motivates it. Further study is clearly
needed before this type of scheme can be applied soundly for low-frequency
past climate analyses in real cases. This would at least include sensitivity
analyses for control vector design, error compensation analyses, and
model error

In this paper we used Ebm1D-ad v1.0.0, a version of the
Ebm1D model including the adjoint code available at

Here we give tabulated details and a summary of the convergence tests for the
finite-difference schemes in the experiment with the model Ebm1D. Table

SCN: weight scenario for the cost term

PD1: present-day scenario,

PD2: present-day scenario,

PD3: present-day scenario,

FFFF: assimilation scheme, with the following options:

pIKS: FDS-IKS

pMKS: FDS-MKS

IT: Number of iterations. Fixed for FDS-MKS. Maximum for FDS-IKS.

NN: number of perturbations

SSSS:

1-D energy balance model experiment. Cost function values for the
tests with the FDS schemes

The tests are considered to evaluate the resilience of the Gauss–Newton scheme (FDS-IKS) to high perturbations and decreasing regularization and how this affects the relative performance of the Gauss–Newton scheme versus the multistep scheme (FDS-MKS). Specific cost function values are to be compared quantitatively only within a specific weight scenario (PD1, PD2, or PD3). For higher weights (being PD3 the highest), the effect of regularization by the background term decreases, which increases the chances of the FDS-IKS not converging (STOPPED tests due to unstable model integrations). Further tests (not shown) with even higher observational weight than PD3 are more and more difficult for the FDS-IKS to converge.

Scheduling of computing resources is more uncertain with the FDS-IKS than
with the FDS-MKS. However, regarding this experiment and model, when the
FDS-IKS converges, the values of the cost function are lower than those of
the corresponding FDS-MKS test. This happens for FDS-MKS with either two or
three
iterations. Thus, with adequate regularization, the FDS-IKS is favoured. With
decreasing observation uncertainty (decreasing regularization), the FDS-MKS
stays more stable (see PD2 and PD3 tests). Still, for low regularization
(PD3) the FDS-MKS cost function values (

As an example, Fig.

Table

CESM parameter estimation: convergence of FDS-IKS cost function.

CESM posterior standard deviation in control variables

CESM experiment. Example of Gaussian anamorphosis transformation in a control variable
(minimum relative humidity for high stable cloud formation) and the sea surface temperature (SST)
for both the model equivalent of the observations and
the observation (transformation details in text) at a location (245.3

AP wrote the code for Ebm1D-ad. JGP wrote the code for rDAF, rdafEbm1d, and rdafCESM. Both authors were involved in the experiments and contributed to the writing of the paper.

The authors declare that they have no conflict of interest.

This work has been supported by the German Federal Ministry of Education and Research (BMBF) as a Research for Sustainability initiative (FONA) through the Palmod project (01LP1511D). The authors acknowledge the North-German Supercomputing Alliance (HLRN) for providing HPC resources that have contributed to the research results reported in this paper. We thank two anonymous reviewers for their positive comments, which have been very helpful in improving the paper. The article processing charges for this open-access publication were covered by the University of Bremen. Edited by: James Annan Reviewed by: two anonymous referees