We demonstrate the practicality and effectiveness of using a Green's functions estimation approach for adjusting uncertain parameters in an Earth system model (ESM). This estimation approach has previously been applied to an intermediate-complexity climate model and to individual ESM components, e.g., ocean, sea ice, or carbon cycle components. Here, the Green's functions approach is applied to a state-of-the-art ESM that comprises a global atmosphere/land configuration of the Goddard Earth Observing System (GEOS) coupled to an ocean and sea ice configuration of the Massachusetts Institute of Technology general circulation model (MITgcm). Horizontal grid spacing is approximately 110 km for GEOS and 37–110 km for MITgcm. In addition to the reference GEOS-MITgcm simulation, we carried out a series of model sensitivity experiments, in which 20 uncertain parameters are perturbed. These “control” parameters can be used to adjust sea ice, microphysics, turbulence, radiation, and surface schemes in the coupled simulation. We defined eight observational targets: sea ice fraction, net surface shortwave radiation, downward longwave radiation, near-surface temperature, sea surface temperature, sea surface salinity, and ocean temperature and salinity at 300 m. We applied the Green's functions approach to optimize the values of the 20 control parameters so as to minimize a weighted least-squares distance between the model and the eight observational targets. The new experiment with the optimized parameters resulted in a total cost reduction of 9 % relative to a simulation that had already been adjusted using other methods. The optimized experiment attained a balanced cost reduction over most of the observational targets. We also report on results from a set of sensitivity experiments that are not used in the final optimized simulation but helped explore options and guided the optimization process. These experiments include an assessment of sensitivity to the number of control parameters and to the selection of observational targets and weights in the cost function. Based on these sensitivity experiments, we selected a specific definition for the cost function. The sensitivity experiments also revealed a decreasing overall cost as the number of control variables was increased. In summary, we recommend using the Green's functions estimation approach as an additional fine-tuning step in the model development process. The method is not a replacement for modelers' experience in choosing and adjusting sensitive model parameters. Instead, it is an additional practical and effective tool for carrying out final adjustments of uncertain ESM parameters.

Earth system models (ESMs) include various parameters that govern the representation of unresolved, unrepresented, or underobserved processes in the models. The most sensitive parameters are typically adjusted during the last step of model development relative to observational targets. Currently, there is no agreed-upon methodology to adjust model parameters. As an illustration of the range of approaches and observational targets,

ESM tuning is often done in a heuristic, trial-and-error manner, wherein one or more parameters are perturbed to new values based on diagnosing the causes of systematic error with the aim of improving the model's overall behavior

There are three main drawbacks of heuristic optimization approaches. The first is that this method necessitates an analysis step between each sensitivity sweep, a time-consuming process. The second drawback is the large number of required simulations, since a new set of experiments is required for iteration of the process. The third drawback is the interdependence of each parameter's effect on the resulting ESM simulation. For example, both sea ice albedo and precipitation efficiency could affect the globally averaged radiation balance at the top of the atmosphere. The adjustment of one parameter or set of parameters at a time is suboptimal, because estimates of empirical parameters depend on each other and on model configuration, boundary conditions, etc.

Given unrestricted computer resources, one could randomly explore model parameter space exhaustively and choose the simulation that best fits all available observations. In practice, this type of exhaustive parameter exploration is not feasible, and we need tractable methods that combine objective methodologies with the model developer's experience to arrive at an optimized choice of model parameters.

To mitigate the drawbacks and computational cost of the above approaches, a number of more objective parameter calibration methods have been proposed for systematically choosing or fine-tuning the final set of optimized parameters. These methods include the use of Latin hypercube sampling techniques

Another alternative to heuristic estimation approaches is the adjoint method. Using the adjoint method, one can adjust many parameters simultaneously. A successful example is provided by the Estimating the Circulation and Climate of the Ocean (ECCO) consortium, which used the adjoint method to adjust Massachusetts Institute of Technology general circulation model (MITgcm) ocean model parameters, initial conditions, and boundary conditions

In this study, we will explore the applicability of the Green's functions approach of

The key drawback of the Green’s function approach is that computational cost increases linearly with the number of control parameters. Therefore, the method is only applicable to situations where a small number of control parameters need to be estimated. Despite these drawbacks, we will demonstrate that the Green's functions approach can be an invaluable addition to the repertoire of estimation tools used to adjust ESM model parameters.

The remainder of the paper is organized as follows: the mathematical formalism and the Green's functions method are presented in the next section. This presentation is followed by a description of the model, the experiments, and the used observational targets. Section

The model used for this study is the GEOS-MITgcm coupled model. We briefly describe here the particular configurations of these two models as they are used in our study. The dynamical core and suite of physical parameterizations of the GEOS Atmospheric General Circulation Model (AGCM), along with the land model and aerosol model, are described in

MITgcm has a finite-volume dynamical core

The GEOS-MITgcm atmosphere–ocean interface includes a skin layer

Algebraically, the ESM described in Sect. 2 can be expressed as a set of rules for time stepping a state vector,

To fit the ESM to the available observations, we aim to minimize an objective cost function

In practice, the Green's functions estimation method can be divided into the following six steps:

Run a reference ESM simulation using default values for model error parameters, i.e.,

Choose a set of observational targets, i.e., define operator

Estimate error covariance matrices

Run a set of model sensitivity experiments, where each element of vector

Calculate a set of optimized model error parameters

Run a new simulation using optimized parameters

In this section, we illustrate the application of the six steps listed above.

The 10-year reference experiment was configured with reference values of a set of parameters that we aimed to optimize. The model was configured to run in “perpetual year” mode, meaning that the external boundary conditions (solar insolation, greenhouse gas amount, and aerosol emissions) are kept to those of 2000. The “perpetual year” mode was chosen to optimize the model's equilibrium state relative to our climatology observational targets.

List of observational targets and covariance value.

The motivation of this study was to set up a simple, systematic, reproducible, extensible, and efficient framework for improving the climatology of the new coupled model both now and as it undergoes development in the future. To start, we chose eight key observational targets (Table

In this study,

List of perturbed parameters.

In addition to the reference experiment, 20 different 10-year sensitivity experiments were performed using the exact model configuration as the reference experiment but perturbing one of the uncertain parameters each time. The length of the simulations needed for the Green's functions optimization is related to the application of the model being optimized. For climate projection applications, where the long-term equilibrium of the model is paramount, it seems clear that longer simulations would be warranted in order to allow for model spin-up time. The primary application of the coupled model presented here is seasonal to decadal prediction, and so the optimization is done to include (ideally to minimize) the initial model drift.

The reference and perturbed parameter values and a short description of the parameters are listed in Table

The optimized parameters are given in the last column of Table

Once the Green's functions methodology provides the optimized parameters, a projected cost can be derived directly from the sensitivity experiments using the optimized parameters. This derivation can be done before actually running the optimized simulation with the optimized parameters. The projected cost provides a first indication of the expected cost reduction, assuming linearity. After a new optimized experiment is performed, the projected cost can be compared with the optimized experiment's cost to assess the correctness of the assumption of linearity.

Figure

The total cost of the reference experiment, the optimized experiment, and the 20 perturbed experiments. Projected cost is also indicated.

The cost of the reference experiment and the 20 perturbed experiments for each of the observational targets. Projected and optimized costs are also indicated.

Explained variance in percentage. Negative values represent reduction of explained variance. TS: near-surface temperature; FR: sea ice fraction; LW: downward longwave radiation; SW: net surface shortwave radiation; SST: sea surface temperature; SSS: sea surface salinity; T (300): THETA at 300 m; S (300): salt at 300 m.

The cost reduction was also found to be rather homogeneous across many regions around the globe. Figure

The net shortwave radiation cost for the reference

The actual cost reduction of the various variables seen in the optimized experiment is also generally consistent with the projected cost. Figure

Net shortwave radiation cost difference between the projected value and the reference experiment.

The surface temperature, which exhibits an overall 16 % increase in cost (Fig.

The 2 m temperature cost for the reference

The parameter estimation experiment presented in the previous section describes our current best practice, which was guided by a series of sensitivity experiments with different configurations, where different variations of the methodology were tested. Below, we discuss sensitivity to the choice of (1) prior covariance matrices, (2) observational targets, and (3) the number of control parameters. Many other sensitivity experiments can be designed. The goal of this section is not to cover all the options, but rather to provide an additional dimensional depth of the Green's functions methodology. In the end, the future choice of the Green's functions “flavor” should reflect the goals of the optimization exercise.

In general, the error covariance matrices

The sensitivity to the choice of covariance matrices

We tested four different options for the prior error variance, the diagonal elements of matrix

We calculated optimized parameters,

Table

Table

After running a new model simulation with the optimized parameters based on Eq. (

Optimized parameters of the three perturbed parameters experiments based on two different definitions of the parameters' covariance matrix

Cost of the three perturbed parameters experiments based on two different definitions of the parameter covariance matrix

Here, we evaluate the sensitivity of the cost to the number of optimized parameters. We calculated the projected cost reduction for four cases which used a subset of the first 5, 10, 15, and all 20 parameters from Table

We tested the projected cost reduction as a function of the optimized parameter number (Fig.

Cost relative to the reference experiment as a function of the number of optimized parameters based on annual means.

Here, we used nine sensitivity experiments, with nine optimized parameters, to look at the sensitivity of the optimized parameters to seasonality. Two sets of parameters were calculated based on multi-year means (annual, Table

The most noteworthy difference between the annual and seasonal results was found in the Charnock parameter that controls surface winds. Although the difference between parameters is smaller than 10 %, the projected cost differences are large, with the annually based optimization reducing the cost by about 15 % and the seasonally based optimization reducing the cost by about 9 %. This result is expected, as seasonally based data have more variability and we use the same number of parameters to optimize model results to a larger number of observations.

Optimizing for seasonal observational targets is expected to be essential for improving seasonal variability. Therefore, despite the less efficient cost reduction for the seasonal targets, we decided to continue to focus on experiments with seasonal observational targets. For these two sets of experiments, we did not choose to rerun the model with optimized parameters, but instead decided to increase the number of optimized parameters by performing additional simulations.

Optimized parameters of the nine perturbed parameters experiments configured with

Proj./ref. cost of the nine perturbed parameters experiment configured with

This sensitivity test was performed with the 20-parameter set of simulations. The appropriate length of the sensitivity experiments was chosen based on the intended application of our tuned model for seasonal and decadal prediction. It seems clear that for a different application of the model, say climate projection, the length of the sensitivity experiments, optimization, and observational targets must be long enough to reach a climate equilibrium. Within the scope of seasonal to decadal prediction, to examine the impact of the optimization period, we performed three Green's function calculations: one that optimizes the parameters based on the first 5 years of the runs (2000–2005), one that uses the last 5 years of the runs (2005–2010), and another that uses the whole 10 years (2000–2010) as in Sect.

This study demonstrates the applicability of the Green's functions approach – introduced in

The Green's functions approach assumes that the response of the model to small perturbations is approximately linear. This assumption does not rigorously hold for atmospheric, oceanic, or coupled ocean–atmosphere models because of nonlinear weather and climate phenomena such as synoptic storms and El Niño–Southern Oscillation (ENSO) variability. Therefore, a key consideration is that the perturbation experiments be sufficiently long to average out nonlinear weather and lower-frequency phenomena. This study shows that 10-year long simulations can be sufficient to enable successful application of the Green's functions approach.

The cost reduction was spread across most of the observational targets, though our configuration of observational targets may have been responsible for the cost increase in two amongst them – the targets associated with land areas that are underrepresented in our cost function. In the end, the choice of the observational targets should reflect the objective of the study and, probably, there is no single set of observational targets that is adequate for all applications a priori. Our proof-of-concept study and the experiments that show the impact of different choices in the details readily reflect the tradeoffs among data constraints that can be important considerations.

Increasing the number of optimized parameters reduced the overall cost, an effect which seems to be quasi-linear with the number of parameters, but further investigation into this is required. Increasing the temporal resolution of the observational targets from annual to seasonal increased the cost further – a result of the increasing number of observational targets. Adding seasonality into parameter adjustments is viewed as one of the next logical steps in this regard. We also tested four different methods to calculate the error covariance matrix, and found that using the error of the reference experiment produced the most balanced results in terms of accounting for all observational targets.

This study shows that the Green's functions methodology can benefit the earth system modeling community by providing a more structured yet practical method to tune the complex global models. The Green's functions methodology has several advantages that are worth emphasizing here: (a) it is simple to implement and it does not require internal amendments to the model code; (b) it is relatively cheap for a small number of parameters – one extra sensitivity experiment per parameter; (c) one can generate virtual predictions and calculate projected cost for a very large number of cost configurations without running new experiments; (d) it offers a systematic way to optimize parameters accounting for a possible dependent response of the model to a change of different parameters; (e) the optimization process can be extended with more sensitivity experiments and it can also be revisited with a different set of parameters; and (f) the sensitivity experiments can be done in parallel, suggesting potential scalability.

Identifying the most important uncertain parameters for applying Green's functions methodology still requires close familiarity with models and there is no replacement for the experience of the modelers when using this methodology. Therefore it is viewed as a valuable framework to further leverage modelers' expertise to fine-tune model parameters, standardize practices across the various modeling centers, and improve model products beyond the present state of the art.

Data leading to this publication were made available as part of a Zenodo repository at

ES ran the simulation and performed the experiments. AM, DB, AT, DM, and GF contributed to the design of the experiments and writing the manuscript.

The contact author has declared that neither they nor their coauthors have any competing interests.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The SRB data were obtained from the NASA Langley Research Center Atmospheric Sciences Data Center NASA/GEWEX SRB Project. This research was made possible by grants from the NASA Modeling, Analysis, and Prediction (MAP) and Physical Oceanography (PO) programs. GEOS development was funded under NASA MAP-supported GMAO “core” funding. Dimitris Menemenlis carried out research at the Jet Propulsion Laboratory, California Institute of Technology, under contract with NASA. High-end computing resources were provided by the NASA Center for Climate Simulation (NCCS).

This paper was edited by Fabien Maussion and reviewed by two anonymous referees.