Uncertainties in an output of interest that depends on the solution of a complex system (e.g., of partial differential equations with random inputs) are often, if not nearly ubiquitously, determined in practice using Monte Carlo (MC) estimation. While simple to implement, MC estimation fails to provide reliable information about statistical quantities (such as the expected value of the output of interest) in application settings such as climate modeling, for which obtaining a single realization of the output of interest is a costly endeavor. Specifically, the dilemma encountered is that many samples of the output of interest have to be collected in order to obtain an MC estimator that has sufficient accuracy – so many, in fact, that the available computational budget is not large enough to effect the number of samples needed. To circumvent this dilemma, we consider using multifidelity Monte Carlo (MFMC) estimation which leverages the use of less costly and less accurate surrogate models (such as coarser grids, reduced-order models, simplified physics, and/or interpolants) to achieve, for the same computational budget, higher accuracy compared to that obtained by an MC estimator – or, looking at it another way, an MFMC estimator obtains the same accuracy as the MC estimator at lower computational cost. The key to the efficacy of MFMC estimation is the fact that most of the required computational budget is loaded onto the less costly surrogate models so that very few samples are taken of the more expensive model of interest. We first provide a more detailed discussion about the need to consider an alternative to MC estimation for uncertainty quantification. Subsequently, we present a review, in an abstract setting, of the MFMC approach along with its application to three climate-related benchmark problems as a proof-of-concept exercise.

This article has been co-authored by an employee of National Technology & Engineering Solutions of Sandia, LLC under contract no. DE-NA0003525 with the U.S. Department of Energy (DOE). The employee owns right, title, and interest in and to the article and is responsible for its contents. The United States Government retains and the publisher, by accepting the article for publication, acknowledges that the United States Government retains a non-exclusive, paid-up, irrevocable, world-wide license to publish or reproduce the published form of this article or allow others to do so, for United States Government purposes. The DOE will provide public access to these results of federally sponsored research in accordance with the DOE Public Access Plan

In many application settings – climate modeling being a prominent one – large computational costs are incurred when solutions to a given model are approximated to within an acceptable accuracy tolerance. In fact, this cost can be prohibitively large when one has to obtain the results of multiple simulations, as is the case for, e.g., uncertainty quantification, control, and optimization, to name a few. Thus, there is often a need for compromise between the accuracy of simulation algorithms and the number of simulations needed to obtain, say, in the uncertainty quantification setting, accurate statistical information.

For example, consider the following case, which represents the focus of this paper. Suppose one has a complex system, say, a system of discretized partial differential equations, for which the input data depends on a vector of randomly distributed parameters

On the other hand, Monte Carlo estimation is not without its drawbacks. Consider the following scenario. Let

The cost CM can quickly get out of control when dealing with large-scale problems. For example, suppose that the approximate solution of the PDE system is second-order accurate, i.e.,

Given that the cost of MC estimation is at times prohibitively expensive, it comes as no surprise that many alternatives or run-arounds to such estimation have been proposed. One approach in this direction has led to the development of many different random-parameter sampling schemes (e.g., quasi-Monte Carlo sampling, sparse-grid sampling, importance sampling, Latin hypercube sampling, lattice sampling, and compressed sensing, to name just a few) for which the estimation error is guaranteed to be smaller than its Monte Carlo equivalent; see, e.g.,

A second approach towards reducing the cost of Monte Carlo (and, for that matter, for any type of) uncertainty estimation is to use approximate solutions of the PDE system that are less costly to obtain compared to the cost of obtaining the approximation of actual interest. For example, using simulations obtained using coarser grids or using reduced-order models such as reduced-basis or proper-orthogonal-decomposition methods are less costly, as are interpolation and support vector machine approximations; see, e.g.,

In this paper, we do not consider any of the possible alternate sampling schemes, nor do we exclusively consider using less costly and less accurate approximate solutions of the PDE system. Instead, because of the near ubiquity of its use in practice, our goal is to outperform traditional Monte Carlo estimation by using a nontraditional Monte Carlo sampling strategy and, in so doing, to refrain from incurring any loss of accuracy. To meet this goal, we invoke multifidelity Monte Carlo estimation which, in addition to the expensive and accurate PDE system approximation of interest (hereafter referred to as the “truth” approximation), also uses cheaper-to-obtain and less accurate approximations (which are referred to as the “surrogates”). The bottom line is that multifidelity Monte Carlo estimation meets our goal by leveraging increased sampling of the less accurate and less costly approximations alongside low sampling of the more expensive and more accurate truth approximation. The multifidelity Monte Carlo algorithm systematically determines the number of samples taken from each surrogate (i.e., there is no guess-work involved) and systematically (i.e., again, there is no guess-work involved) combines the samples of the surrogates to obtain the desired estimator. We note that multifidelity Monte Carlo estimation has already been shown to outperform Monte Carlo estimation in a variety of application settings; see, e.g.,

In Sect.

An abstraction of the specific settings considered in Sects.

It involves having in hand a (discretized) partial differential equation (PDE) system for which the
solution

Note that the input data to this PDE system (e.g., forcing terms, initial conditions, and coefficients) could depend on one or more of the components of the random vector

We are interested in situations whereby, for any

We define a scalar output of interest (OoI)

While they are not considered in this work, vector-valued outputs of interest can also be treated with multifidelity Monte Carlo techniques. OoIs could be, e.g., averages or extremal values of the energy associated with the solution

Having defined an OoI

Because the estimation of

Commonly, even ubiquitously, a Monte Carlo (MC) sampling method is used to (approximately) quantify the uncertainty in the chosen OoI

Specifically, MC sampling is used to estimate the quantity of interest

We are then faced with the following dilemma: on the one hand, obtaining an acceptably accurate MC estimator

Quantifying uncertainties in climate system settings are victimized by this two-headed dilemma to the extent that, e.g., accurate long-time integrations often cannot be realized in practice.

For the truth and the

Letting

The variance

Given a fixed computational budget

In

Because “small” computational budgets are of high interest for climate modeling, here, instead of using the MFMC method of

In Eq. (

Repeating the arguments made in

Optimal non-integer sampling numbers for the modified MFMC method; see

It is remarkable that, due to Eq. (11), the weights

Unfortunately, as was the case in

Practical near-optimal integer sampling numbers for the modified MFMC method; see

Consider the single-layer rotating shallow-water equations (RSWEs) posed on the domain

Note that the expression

The RSWEs represent a useful simplification of the primitive equations (

Spatial discretization of the system (Eq.

Temporal discretization of the system (Eq.

The specific setting considered here is a modification of the benchmark test case referred to as “simulating ocean mesoscale activity” (SOMA;

In Eq. (

Data corresponding to the three SCVT meshes used in the wind-driven gyre experiment.

For constructing the MC and MFMC estimators, we use three SCVT grids of the SOMA domain

The

The output of interest we consider is the maximum fluid layer thickness, which is a common benchmark in oceanic RSWE simulations and which is relevant to, e.g., the detection of phenomena such as flooding. In particular, we simulate the RSWE until the final time of

The quantity of interest we choose considers the effect that perturbations of the initial velocity

Note that in practical RSWE simulations, as is the case for more sophisticated models such as the primitive equations, an approximation of the initial data

For the setting of Sect.

The MC estimator

Naturally, to mitigate this double shortcoming, we turn to the MFMC estimator described in Box 2 of Sect.

From Eq. (

As already alluded to, the goal is to compare, for the same budget, the MSEs of the approximations

The results of this procedure are given in Table

Results of the wind-driven barotropic gyre test with perturbed initial velocities for budgets

Results for the wind-driven barotropic gyre test with output of interest (Eq.

We again consider the RSWE system (Eq.

Test Case 5 considers the flow over an isolated mountain centered at longitude

The initial tangential (to the sphere) velocity in the longitudinal and latitudinal directions is chosen to be

The initial thickness

For the simulation results given in Fig.

Data corresponding to the three SCVT meshes used in the Test Case 5 experiment.

Two global SCVT meshes of the sphere surface with different grid resolutions: 480 km

For the output of interest, we choose

Similarly to before, the goal is now to construct and compare, for the same computational budget, MC and MFMC estimators of the QoI defined in Eq. (

In this case, 4500 uniform i.i.d. realizations of

The results provided in Table

Results of Test Case 5 with perturbed initial velocities for budgets

Results over 250 runs of the RSWE system for the Test Case 5 experiment with quantity of interest given by Eq.(

The next experiment we consider illustrates the effectiveness of MFMC estimation on a QoI important for the realistic modeling of ice sheets such as those found near, e.g., Greenland, Antarctica, and various glaciers.

The dynamical behavior of ice sheets is commonly modeled by what is referred to as the first-order model or the Blatter–Pattyn model. Here, we provide a short review of that model; detailed descriptions are given in, e.g.,

Let

An

Also, let

The system (Eq.

Once the horizontal components

Of paramount interest in the modeling of ice sheets is the monitoring of the temporal evolution of the ice sheet domain

Thus, in a computational model of ice sheet dynamics, determination of the domain

The specific application setting we consider is based on Experiment C of the benchmark examples in

For constructing the MC and MFMC estimators, we set the length

Data corresponding to the three tetrahedral meshes used in the ice sheet experiment.

The first-order ice sheet model is discretized using the stabilized P1–P1 finite elements given in

A simulation of the ice sheet model with

Because the cracking and melting of ice sheets is an important indicator of climate change (

Two surrogates for the ice sheet model,

At this point, we proceed as was done in Sect.

The remaining experimental details are nearly identical to those of Sect.

Despite this, it is clear from Table

Results of the ice sheet experiment for budgets equivalent to

Results over 250 runs of the ice sheet experiment with quantity of interest (Eq.

This paper serves to introduce multifidelity Monte Carlo estimation as an alternative to standard Monte Carlo estimation for quantifying uncertainties in the outputs of climate system models, albeit in very simplified settings. Specifically, we consider benchmark problems for the single-layer shallow-water equations relevant to ocean and atmosphere dynamics, and we also consider a benchmark problem for the first-order model of ice sheet dynamics. The computational results presented here are promising in that they amply demonstrate the superiority of MFMC estimation when compared to MC estimation on these examples. Furthermore, the use of MFMC as an estimation method will surely be even more efficacious when quantifying uncertainties in more realistic climate-modeling settings for which the simulation costs are prohibitively large, e.g., for long-term climate simulations. Thus, our next goal is to apply MFMC estimation to more useful models of climate dynamics (such as the primitive equations for ocean and atmosphere and the Stokes model for ice sheets) that are also coupled to the dynamics of other climate system components and also to passive and active tracer equations.

The climate simulation data used in this work along with Python code for reproducing the relevant experiments can be found at the GitHub repository (

The idea for this work was conceived by MG, and the experiments presented were designed jointly by all authors. Software development, data collection, and figure generation were carried out by AG and RL, while post-experiment analysis was conducted by all authors. The resulting paper was written by MG and AG with editorial contributions from all authors.

The contact author has declared that none of the authors has any competing interests.

This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government.Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Sandia National Laboratories is a multi-mission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract no. DE-NA0003525.

This material is based upon work partially supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Mathematical Multifaceted Integrated Capability Centers (MMICCS) program, under Field Work Proposal no. 22025291 and the Multifaceted Mathematics for Predictive Digital Twins (M2dt) project. This work is further supported by U.S. Department of Energy Office of Science under grant nos. DE-SC0020270, DE-SC0022254, DE-SC0020418, and DE-SC0021077.

This paper was edited by Dan Lu and reviewed by Huai Zhang and one anonymous referee.