Introduction
Simple, mechanistically motivated Earth system models often play a pivotal
role in climate and flood risk management
(Hartin et al., 2015). For example, they are used for uncertainty quantification (Bakker et al., 2017; Grinsted et al.,
2010; Urban et al., 2014; Urban and Keller, 2010) and complex model emulation
(Applegate et al., 2012; Bakker et al., 2016; Hartin et al., 2015;
Meinshausen et al., 2011a), and are incorporated into integrated assessment
models (Hartin et al., 2015;
Meinshausen et al., 2011a).
Computational constraints often impose hard trade-offs between physical model complexity and statistical model
complexity. For example, a sizable allocation of computational time could be spent running a small number of simulations
using a high-complexity physical model. Highly detailed simulations are useful to better understand the complex system,
but with just a small number of simulations, only weak ensemble statistics can be drawn. In contrast, numerous
realizations of a less detailed physical model could be run. This would provide the opportunity for more advanced ensemble
statistical techniques including the characterization and quantification of uncertainties. It is important in
climate-related applications such as mitigation of greenhouse gas emissions or adaptation to sea-level rise that the
relevant uncertainties are explored and communicated clearly to policy-makers (e.g., Garner et al., 2016; Gauderis et al.,
2013; Goes et al., 2011; Hall et al., 2012; Lempert et al., 2004).
Several studies have broken important new ground in tackling these challenges. For example, Nauels et al. (2017) present
a platform of sea-level emulators (i.e., simple models of complex models)
that efficiently produces future projections and
characterizes key model structural uncertainties using statistical calibration methods. Semi-empirical modeling (SEM)
approaches trade detailed physics for a model that can efficiently project sea level using statistical, but
mechanistically motivated, relationships between sea-level changes and climate conditions such as temperature and
radiative forcing (Grinsted et al., 2010; Jevrejeva et al., 2010; Kopp et al., 2016; Rahmstorf, 2007). Recent work has
expanded upon the SEM approach to use simple models to resolve individual contributions to global sea level (Bakker
et al., 2017; Mengel et al., 2016; Nauels et al., 2017).
Studies based on simple, mechanistically motivated models have the potential
to be transparent and reproducible when
presented in open platforms and when the underlying data are readily available. Yet, although there is an increasing
tendency to share scientific code, it can be (perhaps surprisingly) hard to get the models running and to reproduce the
results. A likely cause of this is that not enough attention is given to the
scientific coding itself. Careful coding,
documentation, and review require a dedicated commitment of time, but scientific incentives to do so can be weak.
Here we describe in detail BRICK (Building blocks for Relevant Ice and
Climate Knowledge, Bakker et al., 2017) v0.2,
a model framework that focuses on accessibility, transparency, and flexibility while
maintaining, as much as possible, the computational efficiency that
makes simple models so appealing. As compared
to Bakker et al. (2017), BRICK v0.2 accounts for land water storage with the other components kept unchanged. There is
a wide range of potential applications for such a model. A simple framework enables uncertainty quantification via
statistical calibration approaches (Higdon et al., 2004; Kennedy and O'Hagan, 2001), which would be infeasible with more
computationally expensive models. A transparent modeling framework enables communication between scientists as well as
communication with stakeholders. This leads to potential application of the model framework in decision support and
education (Fischbach et al., 2012; Johnson et al., 2013; Weaver et al., 2013). The present work expands on previous
studies by (i) providing a platform of simple but mechanistically motivated
sea-level process models that resolve more
processes, (ii) providing a model framework that can facilitate model comparisons (for example, between our models and
those of Nauels et al., 2017), (iii) exploring combined effects of key structural and parametric uncertainties,
(iv) explicitly demonstrating the flexibility of our framework for interchanging model components, and (v) explicitly
demonstrating the utility of our model framework for informing decision analyses.
In this model framework, we present a set of existing, well-tested, and easy-to-couple simple models for climate and flood
risk management. They simulate global mean surface temperature and contributions to global mean sea-level rise. BRICK also
includes a regional sea-level rise module, which translates the global mean
sea-level contributions to regional sea level at a user-defined location. We
use these regional sea-level projections to demonstrate how the physical
model may be
linked to decision-making and impacts. We implement a Bayesian calibration approach with an aim to adequately resolve the
tails of the distribution of future sea levels because these low-probability areas represent high-risk events. In robust
decision-making approaches, it can be favorable to be underconfident as
opposed to overconfident, e.g., by applying
conservative estimates in the sense of being risk-averse (Herman et al., 2015). We hence include in our Bayesian approach
wide, mechanistically motivated prior parameter probability distributions
(Bakker et al., 2017). Yet, the flexibility of
the BRICK model framework also enables the implementation of other calibration schemes. This paper is intended to showcase
a useful model framework that is attractive for a sustainable approach to model development, for example by inspiring
fellow researchers to contribute to the framework, to rethink their coding practice, and maybe even to adopt some of the
demonstrated design objectives in their future research proposals.
The hindcast skill of the BRICK model has been previously demonstrated (Bakker et al., 2017). Thus, the present work
focuses on outlining a set of epistemic modeling values that we believe facilitates advances in the modeling
community. The remainder of this work is organized as follows. In Sect. 2, we describe these values and the ways in which
the BRICK model implementation strives to attain them. Section 3 contains an overview of the BRICK model components for
climate and the contributions to sea-level rise. Section 4 describes and presents the results of a set of model
experiments conducted to demonstrate how BRICK lives up to our epistemic modeling values. Section 5 summarizes the
findings of this work and provides conclusions and guidance for future work.
BRICK model structural diagram. Dashed connectors indicate couplings
that are non-essential for projections of global mean sea level. These dashed
couplings are required for projecting regional sea-level and climate impacts.
DOECLIM is the Diffusion-Ocean-Energy balance CLIMate model (Kriegler, 2005);
GIC-MAGICC is the Glaciers and Ice Caps module from the MAGICC climate
model (Meinshausen et al., 2011a);
TE is the Thermal Expansion model (Grinsted et al., 2010; Mengel et al.,
2016); SIMPLE is the Simple Ice-sheet Model for Projecting Large Ensembles
(Bakker et al., 2016); ANTO is the ANTarctic Ocean temperature model; DAIS is
the Danish Center for Earth System Science Antarctic Ice Sheet model
(Shaffer, 2014); regional sea-level fingerprinting downscales from global
sea-level contributions to regional (Slangen et al., 2014); and the model of
Van Dantzig (1956) assesses flood risk.
![]()
Model components
Global mean climate
We adopt DOECLIM (Diffusion Ocean Energy balance CLIMate model, Kriegler, 2005) as a starting point for a simple climate
model (Fig. 1). DOECLIM is a zero-dimensional energy balance model coupled to a three-layer, one-dimensional diffusive
ocean model. The DOECLIM physical model outputs are global mean surface temperature anomaly (∘C) and ocean
heat uptake (1022 J). Calibration data for DOECLIM include both global surface temperature (Morice et al.,
2012) and ocean heat uptake (Gouretski and Koltermann, 2007). We use a 1-year
time step for the DOECLIM model, and the
required input to drive the model is the radiative forcing time series (Wm-2). This forcing is partitioned
into aerosol and non-aerosol components, to enable a representation of the uncertainty associated with these forcings. The
BRICK model considers this as an uncertain model parameter denoted as the aerosol forcing scaling factor
(αDOECLIM). This aerosol scaling factor has been used elsewhere in the literature (Urban et al., 2014;
Urban and Keller, 2010) and accounts for some uncertainty in the radiative forcing of aerosols (Meinshausen et al.,
2011b). The interested reader is directed to Kriegler (2005) and Tanaka and Kriegler (2007) for more information about the
DOECLIM model.
We fit a first-order autoregressive (AR1) error model to the model–data
discrepancy between temperature and ocean heat
uptake model output and calibration data. We estimate the first-order lag autocorrelation parameters (ρT and
ρH) and homoscedastic component of the AR1 innovation variance (σT and σH) within the
calibration framework as statistical model parameters. We add the heteroscedastic observational error estimates for global
mean surface temperature from Morice et al. (2012) and for ocean heat uptake from Gouretski and Koltermann (2007) in
quadrature to σT and σH (respectively) for the complete heteroscedastic temperature and ocean heat
uptake error estimates. The model calibration approach implemented here
assumes normally distributed model–data residuals with mean 0 (Higdon
et al., 2004). The AR1 error model has the effect of “whitening” the
residuals to satisfy this
assumption. This type of calibration approach for DOECLIM has been implemented previously in the literature (Urban et al.,
2014; Urban and Keller, 2010), and we direct the interested reader to these studies for further details.
Sea-level components
The BRICK global mean sea-level module calculates global sea-level change as
the sum of four individual components: glaciers and ice caps (GIC), the
Greenland Ice Sheet (GIS), the Antarctic Ice Sheet (AIS), and thermal
expansion (TE). These
component sub-models are described in the following sections. BRICK accounts for land water storage contributions to
global mean sea level using mass balance trends from the International Panel on Climate Change (IPCC) Fifth Assessment
Report (AR5, Church et al., 2013) and from the work of Dieng et al. (2015). The differential equations for the GIC, GIS,
AIS, and TE contributions to global mean sea level are integrated into BRICK
using first-order numerical integration schemes with a 1-year time step.
Initial conditions are specified at a year dictated by the sub-model's
assumed reference point. This differs, in general, among the sub-models, and
some model parameters depend on preserving this reference
year. Starting from this initial condition, a first-order explicit numerical integration method integrates forward in time
to the end of the simulation and a first-order implicit (backward differentiation) method integrates backward in time to
the earliest year of the simulation. Preliminary experiments (not shown)
demonstrated that the 1-year time step is
sufficiently short to maintain numerical stability. The total global mean sea-level rise from the coupled BRICK model is
dSdt(t)=dSGICdt(t)+dSGISdt(t)+dSAISdt(t)+dSTEdt(t)+dSLWSdt(t),
where S(t) is the global mean sea level (m) in year t,
SGIC is the sea-level contribution from GIC (m),
SGIS is the sea-level contribution from the GIS (m),
SAIS is the sea-level contribution from the AIS (m),
STE is the sea-level contribution from thermal expansion (m),
and SLWS is the sea-level contribution from changes in land water
storage. We report projections of future sea
level relative to the 1986–2005 mean.
Glaciers and Ice Caps
We adopt a simple zero-dimensional sub-model for the contribution to global sea-level rise from Glaciers and Ice Caps
(GIC) from Wigley and Raper (2005). This same formulation is used in the MAGICC climate model (Meinshausen et al.,
2011a). The parameterization for the GIC contribution to global sea-level
rise is
dSGICdt(t)=β0(Tg(t)-Teq,GIC)1-SGIC(t)V0,GICn.
In Eq. (), SGIC is the cumulative sea-level contribution
from GIC (m), β0 is the
initial mass balance sensitivity to global temperatures (m∘C-1yr-1), Teq,GIC is
the theoretical equilibrium temperature at which the GIC mass balance is at steady state (∘C),
V0,GIC is the initial total volume of GIC available in 1990
(m sea-level equivalent (SLE)), and n is
an exponent parameter for area-to-volume scaling. An initial condition, S0,GIC, is provided as an uncertain
model parameter. Teq,GIC is taken equal to -0.15 ∘C (Wigley and Raper, 2005). Note
that in this formulation for GIC contribution to sea-level rise, whether the GIC mass is increasing or decreasing depends
only on Tg(t) relative to Teq,GIC; it is independent of the current state
SGIC(t). Within this model for the GIC sea-level contribution, Tg is relative to the 1850–1870 mean
global surface temperature (Wigley and Raper, 2005).
The uncertain physical model parameters for GIC-MAGICC (which will be tested in Sect. 4.2) are β0, V0,GIC,
S0,GIC, and n. We fit an AR1 model to the model–data
discrepancy between GIC model output and calibration data
(Dyurgerov and Meier, 2005) in the same manner as the temperature and ocean heat uptake calibration (Sect. 3.1). Uniform
prior distributions are used for the GIC-MAGICC physical and statistical model parameters. These prior distributions, as
well as calibrated posterior medians, 5, and 95 % quantiles, are given in Appendix A.
Greenland Ice Sheet
BRICK uses the mechanistically motivated, zero-dimensional SIMPLE (Simple
Ice-sheet Model for Projecting Large Ensembles) model as the parameterization
for the Greenland Ice Sheet (GIS) contribution to global mean sea-level
change (Bakker
et al., 2016). SIMPLE estimates the GIS response to changes in global mean surface temperature by first estimating an
equilibrium ice sheet volume (Veq, GIS, m SLE) at which the
sea-level contribution from the GIS is 0, and estimating the e-folding
timescale of GIS volume changes due to changes in global temperature
(τGIS,
yr-1).
Veq, GIS(t)=cGISTg(t)+bGIS1τGIS(t)=αGISTg(t)+βGIS
In Eqs. () and (), cGIS, bGIS, αGIS, and βGIS
are uncertain physical model parameters. cGIS is the sensitivity of the equilibrium volume to changes in
temperature (mSLE∘C-1); bGIS is the equilibrium volume Veq, GIS for
0
temperature anomaly (mSLE); αGIS is the sensitivity
to temperature of the timescale of GIS
volume response to changes in temperature (∘C-1yr-1); and βGIS is the equilibrium
(Tg=0∘C) timescale of GIS volume response to changes in
temperature (yr-1). Global mean surface temperature, Tg, is
taken relative to the 1961 to 1990 mean. The GIS volume changes can then be
written as
dVGISdt(t)=1τGIS(t)(Veq, GIS(t)-VGIS(t)).
The initial condition V0,GIS is provided as an uncertain model parameter (mSLE). Using this initial
condition, designated in the year 1961, the sea-level rise due to the GIS is calculated as the change from V0,GIS to
the current GIS volume, VGIS(t). This formulation, of course, assumes that all GIS volume lost makes its way into the
oceans. An AR1 model is fitted to the GIS model–data residuals. Due to poor
convergence, the first-order lag
autocorrelation parameter (ρGIS) is held constant at a value determined by a preliminary model simulation that is
optimized using a differential evolution optimization algorithm (Storn and Price, 1997). The GIS training data set does
not provide heteroscedastic error estimates, so the AR1 innovation variance is taken to be the estimated statistical
parameter σGIS added in quadrature to the provided error estimate (Sasgen et al., 2012). All GIS physical and
statistical model parameters are assigned uniform prior distributions. The ranges for these priors and posterior
distribution medians, 5, and 95 % quantiles are given in Appendix A.
Further details regarding SIMPLE area provided
in
Bakker et al. (2016).
Antarctic Ice Sheet
We employ the Danish Center for Earth System Science Antarctic Ice Sheet (DAIS) model to simulate the Antarctic Ice Sheet
contribution to global sea level (Shaffer, 2014). This is a two-dimensional model for the Antarctic Ice Sheet that assumes
an axisymmetric geometry, shown graphically in Shaffer (2014), his Fig. 2. The DAIS model tracks changes in Antarctic Ice
Sheet volume, considering contributions from (i) incident precipitation, (ii) runoff of ice melt, (iii) ice flow, and
(iv) ice sheet disintegration from rising and warming sea levels. Input forcings for the DAIS model include Antarctic
surface temperature reduced to sea level (TA,
∘C), high-latitude ocean subsurface temperature
(TANTO, ∘C), global mean sea level (m), and
the time rate of change in global mean sea
level (myr-1).
When calibrated as a stand-alone model, the DAIS forcings are provided based on temperature reconstructions (see Shaffer,
2014). When the DAIS model is run as a component in the coupled BRICK model, a separate sub-model is needed to convert the
global mean surface temperature from the climate model (DOECLIM) to the Antarctic surface and ocean subsurface
temperatures required by the DAIS model. The Antarctic surface temperature is estimated from a linear regression with
global mean surface temperature (Morice et al., 2012; Shaffer, 2014). The Antarctic Ocean temperatures (TANTO)
are modeled through a simple relation with the global mean surface
temperature, Tg (relative to the 1850–1870 mean).
TANTO is bounded below at the freezing point of salt water (Tf=-1.8∘C):
TANTO(t)=Tf+aANTO⋅Tg(t)+bANTO-Tf1+exp[-(aANTO⋅Tg(t)+bANTO-Tf)/aANTO].
Equation () is a modified linear regression between the global mean surface temperature Tg and the Antarctic
Ocean temperature TANTO, such that the Antarctic Ocean temperature is bounded below by the freezing
temperature of seawater, Tf. In Eq. (), aANTO is
the sensitivity of the Antarctic Ocean
temperature to global mean surface temperature (unitless), and bANTO (∘C) is the approximate
Antarctic Ocean temperature for Tg=0∘C. bANTO is the approximate ocean temperature
because the relationship in Eq. () is not a simple linear regression. aANTO and bANTO are
both estimated as uncertain model parameters. The DAIS model contains
11 physical parameters and 1 statistical parameter, for
a total of 14 Antarctic Ice Sheet parameters to be estimated. The heavily parameterized Antarctic Ice Sheet module
reflects our focus on including a broad range of model and observational
uncertainties and considering the critical
role of the Antarctic Ice Sheet in driving substantial uncertainty in future sea levels (Church et al., 2013).
Here, we use an updated and corrected version of the DAIS model (Ruckert et al., 2017; Shaffer, 2014). In the original
formulation of the DAIS model, the input forcing from year t is used to determine the AIS contribution to sea-level rise
in year t. This implicit numerical scheme assumes sea level and temperatures for the current year are known during that
model iteration. For this study, in which temperatures and sea level originate in other BRICK model components, the DAIS
model is re-cast using an explicit numerical scheme. The sea level and temperatures from the year t-1 are used to
calculate the AIS contribution in year t. Antarctic shore-average local mean sea level functions as the input to DAIS
when run as a sub-model of the coupled BRICK model. This is estimated as described in Sect. 3.3.
The dynamical core of the DAIS model is more detailed than the GIC, GIS, and TE emulators given above. For this reason, we
do not undertake a full review of the model equations here. The interested reader is directed to Shaffer (2014) and
Ruckert et al. (2017) for further details regarding the DAIS model and its
hindcast forcings. Eq. () of
Shaffer (2014) is the main equation of state for the Antarctic Ice Sheet volume (VAIS, m3):
dVAISdt(t)=Btot(TA,R)+F(S,R).
In Eq. (), Btot is the total rate of accumulation of mass on the Antarctic Ice Sheet
(m3yr-1), TA is the Antarctic surface temperature reduced to sea level (∘C), S is the
sea level (m), R is the Antarctic Ice Sheet radius (m), and
F is the ice flux at the grounding line (m3yr-1). Following
Shaffer (2014), we take the present sea-level equivalent Antarctic Ice Sheet
volume to
be 57 mSLE, and the initial ice sheet volume (V0,AIS, m3) to be consistent with an initial
ice sheet radius of 1.86 × 106 m. Thus, the Antarctic Ice Sheet contribution to global sea level may
be calculated as
dSAISdt(t)=-57m⋅dVAISdt(t)V0,AIS.
Thermal expansion
BRICK uses a simple parameterization for the contribution of thermal expansion (TE) of the Earth's oceans to sea-level
rise. We make the simplifying assumption that thermal expansion of the oceans occurs uniformly around the globe. While
this is, of course, not strictly true, the next obvious step forward in model
complexity would be to use a vertically and latitudinally resolved model for
thermal expansion, incorporating the DOECLIM model output for ocean heat
uptake. This two-dimensional ocean model is beyond the scope of the simple
model framework described presently, but is an excellent
subject for future work. Here, we employ a simple zero-dimensional thermal expansion emulator based on the
parameterizations of the sea-level rise sub-models of Mengel et al. (2016)
and that was originally used by Grinsted et al. (2010) to model the total
global mean sea-level changes. First, an equilibrium sea-level rise from
thermal expansion, due to changing global surface temperature
(Seq,TE, m), is calculated as
Seq,TE(t)=aTETg(t)+bTE.
In Eq. (), aTE is the sensitivity of the equilibrium sea-level rise from thermal expansion, due to
changing global surface temperatures (m ∘C-1), and bTE is the equilibrium sea-level rise
from thermal expansion with no temperature anomaly (m). The sea-level rise due to thermal expansion evolves with
time as
dSTEdt(t)=1τTE(Seq,TE(t)-STE(t)),
where the quantity τTE is the e-folding timescale with which
the current sea level adjusts to the equilibrium
state, and 1/τTE is taken as an uncertain model parameter. This parameter is assigned a gamma prior
distribution with shape 1.81 and scale 0.00275, which places the 5th and 95th quantiles for τTE at 82 and
1290 years (Mengel et al., 2016). This choice of prior distribution is motivated by the fact that τTE
functions similarly to the uncertain timescale associated with an
exponentially distributed random variable. A gamma
distribution is the conjugate prior for such a random variable. The initial condition S0,TE is provided as an
uncertain model parameter (m), designated in year 1850. To match this accounting for sea-level rise relative to
pre-industrial, forcing temperature is taken relative to its 1850–1870 mean. We calibrate the thermal expansion component
of sea-level rise using trends reported by the IPCC (Church et al., 2013).
Regional sea-level patterns
In order to link the projections of global mean sea-level change from BRICK to a local coastal adaptation, information on
regional sea-level change is needed. Thus, the global mean sea level from
BRICK is downscaled to regional sea level using
previously published maps of scaling factors for the glacier and ice sheet components of sea-level change (Slangen et al.,
2014). Any redistributions of mass between the cryosphere and the ocean
(e.g., ice melt) lead not only to a change in the
total mass of the ocean, but also to changes in regional sea level as a result of variations in the gravitational field of
the Earth, which in turn affect the solid Earth and the rotation of the Earth
(e.g., Mitrovica et al., 2001). This
typically (and counterintuitively) leads to a sea-level fall close to the source of mass loss and larger-than-average
sea-level rise at larger distances (> 6700 km) from the source. These so-called regional sea-level
“fingerprints” are constant for the timescales used in this study, as long
as the location of the ice mass change
remains the same. The fingerprints can therefore be used to relate global glacier and ice sheet contributions to sea
level (Sects. 3.2.1–3.2.3) to their regional sea-level contribution. We
couple changes in global sea level to the
Antarctic Ice Sheet model using an Antarctic shore-average fingerprint ratio of -1.0 for the AIS contribution to global
sea level, and Antarctic shore-average fingerprint factors of 1.0 for the other contributions to sea-level rise from all
BRICK sub-models (Slangen et al., 2014). Preliminary experiments indicated that our results are not sensitive to the
precise choices of these fingerprints.
The glacier fingerprint is based on projected changes in glacier mass in 2100 using a glacier model driven by temperature
and precipitation information from the Fifth Climate Model Intercomparison Project database (Taylor et al., 2012) under
the Representative Concentration Pathway 8.5 climate change scenario (RCP8.5, Moss et al., 2010), as presented in Slangen
et al. (2014). It is assumed that the mass change ratios between the different glacier regions on Earth remain the same
throughout the 20th and 21st centuries, which is a valid assumption as long
as none of the glacier regions “finish” (which is not expected to happen in
the next century). For the Greenland and Antarctic ice sheets, it is assumed
that ice
melt takes place uniformly over the ice sheet surface. Within the BRICK model structure, users may define a latitude and
longitude to obtain regional sea-level change.
Model experiments
Model calibration
We calibrate the model through a coupled physical–statistical calibration
framework. The relatively simple physical
modeling framework of BRICK is motivated by our epistemic modeling values (Sect. 2.1). This efficient model permits the
use of a sophisticated model calibration technique. The calibration uses a robust adaptive Markov chain Monte Carlo (MCMC)
approach (Vihola, 2012). The specifics of how it is applied to the BRICK model as well as a demonstration of calibrated
BRICK model hindcast skill are documented in Bakker et al. (2017).
The vastly different timescales and characterizations of uncertainty in the
Antarctic paleoclimatic calibration period
and the modern period (1850 to present) lead to two separate sets of calibration parameters: (i) DAIS parameters,
calibrated using paleoclimatic data, and (ii) DOECLIM, GIC, GIS, and TE parameters, jointly calibrated using modern
data. The paleoclimatic calibration is done using four parallel MCMC chains of 500 000 iterations each. The first
120 000 iterations of each chain are removed for burn-in. The paleoclimatic calibration requires about 10 h on a laptop
with a 2.9 GHz dual-core processor with 16 GB of RAM. The modern calibration is done using four parallel
MCMC chains of 1 × 106 iterations each. The first 500 000 iterations of each chain are removed for
burn-in. This requires less than 1 h on the same machine as the
paleoclimatic calibration. Convergence and burn-in
lengths are assessed using Gelman and Rubin diagnostics (Gelman and Rubin, 1992).
We combine these two disjoint sets of parameters to form concomitant full
BRICK model parameter sets, and calibrate these to global mean sea-level data
(Church and White, 2011) using rejection sampling (Votaw Jr. and Rafferty,
1951). Prior to
rejection sampling, contributions from land water storage are estimated using trends from the IPCC (Church et al., 2013)
and subtracted from global mean sea level. When projecting global mean sea-level rise, we estimate land water storage
contributions by extrapolating using the 2003–2013 trend of 0.30 ± 0.18 mmyr-1 found by Dieng
et al. (2015). This approximation may not hold in reality (Wada et al., 2012), but serves as a starting point for future
model developments. The use of rejection sampling and the estimation of land water storage contributions to sea level are
the two aspects in which our calibration approach differs from that of Bakker et al. (2017). In this rejection sampling
step, each full BRICK parameter set is constructed by parsing a random draw from the calibrated DAIS parameter sets with
a random draw from the DOECLIM-GIC-GIS-TE calibrated parameter sets. This full BRICK model has the major components of
global mean sea-level rise represented, so only at this stage is calibration
using global mean sea-level data possible. The calibration to global
sea-level data initially proposes 135 000 full BRICK model parameter sets.
We use a joint Gaussian normal likelihood function centered at the time
series of the global mean sea-level data, with standard deviation given by
the observational uncertainty of the sea-level data (corrected to account for
land water storage). For rejection sampling, the enveloping distribution is
this likelihood function evaluated at the observed sea-level time series
itself.
Thus, no model simulation can yield a realization of the likelihood function that exceeds this value. Rejection sampling
accepts each model simulation with probability equal to the ratio of the likelihood function evaluated at the selected
model simulation to the maximal value of the likelihood function;
10 589 ensemble members remain after the calibration to global mean
sea-level data. These model realizations serve as the control ensemble for
analysis. The entire analysis for
the control model, including paleoclimatic simulations and the risk assessment presented in Sect. 4.4, requires about 4 h
on a modern laptop, but constructing smaller ensembles is much faster (an ensemble of about 600 members requires less than
10 min).
In the spirit of our epistemic values, calibration routines are provided with the available BRICK source code. These
routines use modern methods readily available in R. It is our aim that the interested user can easily substitute their own
likelihood function (as physical scientific knowledge progresses), a new calibration method (as the statistical
state-of-the-art progresses), or both. To this end, we provide a sub-routinized likelihood function, called from an
R-packaged calibration method (Vihola, 2012). We also provide individual likelihood functions and calibration scripts for
each sub-model of BRICK individually, to enable interested users to perform experiments using stand-alone sub-models or
pre-calibration (Edwards et al., 2011).
In the interest of accessibility and transparency, with the available BRICK source code we also provide the sets of
calibrated model parameters for all experiments presented here. The purpose
of this is 2-fold. First, it greatly enhances
the reproducibility of these results. Second, these data sets enable users who would like to run their own ensembles and
make projections of local sea levels to do so. This supports our goal of accessibility. The calibrated parameter sets are
provided in netCDF format, with ensemble member as the “unlimited”
dimension. This permits concatenation of multiple data sets by using netCDF
operators (NCO) such as “ncrcat” (Zender, 2008). These are freely available
tools for manipulating
data stored in netCDF format.
Testing alternative model components: a sea-level rise module intercomparison
Experimental description
We achieve the accessibility, transparency, and computational efficiency of the BRICK modeling framework through use of
simple models written in a simple programming environment (R, R Core Team, 2016). It remains to be demonstrated that this
framework is flexible and efficient in post-processing.
We demonstrate BRICK's flexibility and efficiency by implementing and switching in an alternative formulation for the
global mean sea level, S(t). We exchange the more detailed model configuration for global mean sea level (the BRICK
control; see Fig. 1) for the simple emulator described by Rahmstorf (2007).
This is
dSdt(t)=aGMSL(Tg(t)-Teq,GMSL),
where t is time (years), S is the global mean sea level (m), aGMSL is a sensitivity constant
(m∘C-1yr-1), Tg is the global mean surface temperature anomaly (∘C), and
Teq,GMSL is the theoretical temperature at which the global sea level is steady
( ∘C). The parameters aGMSL and Teq,GMSL, as well as the statistical
parameters ρGMSL (the first-order lag) and σGMSL (the homoscedastic component of the
innovation variance), are calibrated using the same global mean sea-level
data set as the full BRICK sea-level rise module (Church and White, 2011).
The BRICK-GMSL model performance using Eq. () for the sea-level
rise module
(while still coupled to DOECLIM as the climate module) is compared against the full BRICK model configuration. This
BRICK-GMSL model configuration is calibrated using four parallel MCMC chains of 100 000 iterations each. The first
50 000 iterations are removed for burn-in, as determined using Gelman and Rubin diagnostics (Gelman and Rubin, 1992). We
randomly sample from the resulting posterior distribution to form an ensemble for analysis of 10 589 model
realizations. This ensemble size is chosen to be consistent with the BRICK control model ensemble size. The prior ranges
and posterior medians, 5 %, and 95 % quantiles for the BRICK-GMSL parameters are provided in
Appendix A.
Note that the Rahmstorf (2007) emulator is arguably not the state-of-the-art anymore (Grinsted et al., 2010; Kopp et al.,
2016). However, it serves here the purpose of demonstrating the ease with which alternative model formulations can be
tested. This greatly simplifies, for example, model intercomparisons and improvements. Some advantages of a simple
emulator such as this include fewer parameters to estimate and a transparent analysis. Disadvantages of such a model
include the inability to resolve individual contributions to global mean sea
level. This disables the use of sea-level fingerprinting to obtain regional
sea-level patterns. Thus, the choice of model should be motivated not only by
goodness-of-fit metrics, but also by applications.
Comparison of global mean sea-level rise hindcast skill relative to
sea-level data (Church and White, 2011), using (a) the full
sub-model approach (GIC, GIS, TE, and AIS) and (b) the model for
global mean sea-level rise of Rahmstorf (2007). Sea level is relative to
1961–1990 global mean sea level. Both model configurations use DOECLIM as
the climate module. Lower values of the Akaike information criterion (AIC),
the Bayesian information criterion (BIC), and root-mean-squared error (RMSE)
indicate a better model fit to the data. These error metrics are all
calculated using the maximum likelihood ensemble member, which is represented
by the solid blue line. Green highlighting indicates the model structure
suggested by each comparison metric.
![]()
Metrics for model–data comparison
Many goodness-of-fit metrics are available for the comparison of models and data. We focus on three metrics that are
motivated by the heavily parameterized full BRICK model framework. There are
39 free parameters in the coupled climate/sea-level rise model. By contrast,
BRICK-GMSL has 13 free parameters. We use the global mean sea-level time
series of Church and White (2011) for the model–data comparisons in skill
hindcasting global mean sea level.
Root-mean-squared error (RMSE) is a commonly used error metric, so
we employ it here. For consistency with other
error criteria defined below, we define the RMSE for a model as the RMSE of the model ensemble member that maximizes the
likelihood function.
The Akaike information criterion (AIC) is a measure of the relative goodness-of-fit between two potential models for
the same data (Akaike, 1974).
AIC=-2ln(Lmax)+2N
In Eq. (), Lmax is the maximum value of the likelihood function and N is the number of model
parameters. Lower values of the AIC provide a better match between model
output and data, and consider a penalty for
over-parameterizing a model.
The Bayesian information criterion (BIC) is formulated similarly to
the AIC, but enacts a different penalty for
over-parameterization (Schwarz, 1978).
BIC=-2ln(Lmax)+Nln(Nobs)
In Eq. (), Nobs is the number of observational data
points used in the model–data comparison. Thus, for
Nobs>e2, the BIC metric penalizes over-parameterization more
harshly than does the AIC.
Experimental results: sea-level rise module intercomparison
The full BRICK sea-level rise module (Fig. 1) performs better than the GMSL emulator (Eq. 11) according to RMSE; the full
sea-level rise module has an RMSE of 0.0057 m, which is about half the GMSL
emulator RMSE of 0.015 m (Fig. 2). These
hindcasts are presented as sea level relative to 1961–1990 global mean sea level. This is of course expected, because the
number of free model parameters in the full BRICK model is 39, while the GMSL emulator contains only 13 free
parameters. The BIC metric gives the expected result for this disparity in model complexity. The BIC for the full BRICK
model with respect to the sea-level data is 60.4 higher than the BIC for the
GMSL emulator. The AIC is actually lower (by
14.2) for the full BRICK model than for the BRICK-GMSL emulator. These mixed results for the model comparison metrics
indicate that the full BRICK sea-level rise module is not unreasonably over-parameterized; if the full BRICK model were
obviously over-parameterized, we would expect the AIC for the GMSL emulator experiment to be lower than for the full BRICK
model.
These results also show that the sea-level hindcast in the full BRICK model
smoothes much of the year-to-year variability
in sea-level rise. This can be seen by contrasting the full BRICK maximum likelihood ensemble member (solid blue line) in
Fig. 2a with the BRICK-GMSL emulator maximum likelihood ensemble member in Fig. 2b. The full BRICK simulation does not
capture the annual variation in global mean sea level that the BRICK-GMSL simulation successfully captures. This is
attributed to the smoothing effect of averaging over the model ensemble the four major contributions to global mean sea
level, as opposed to calibrating the BRICK-GMSL simulations directly to
global mean sea-level data. This does not affect
ensemble statistics, however, which can be seen from the shaded envelopes around the model simulations in Fig. 2. The
BRICK model has been developed with efficiency and large ensemble simulations in mind, so missing annual variability is of
little concern.
This demonstrates the ease with which model intercomparisons may be undertaken using BRICK. Deactivating the glaciers and
ice caps, thermal expansion, and Greenland and Antarctic Ice Sheet components and integrating the GMSL emulator into BRICK
involves low overhead in computer code. GMSL is the main output of the BRICK physical model. As such, it is our aim to
provide a framework in which users can easily integrate new processes and models into the climate and sea-level rise
modules as the scientific forefront progresses.
Interchanging BRICKs and sub-model intercomparisons
Experimental description
We conduct an experiment to demonstrate the flexibility of BRICK to permit easy exchanging of a single sub-model for one
component of global sea-level rise. In the control BRICK model setup, SIMPLE
is used to emulate the sea-level rise
contributions from the Greenland Ice Sheet (GIS) and GIC-MAGICC is used to emulate the contributions from glaciers and ice
caps (GIC). In this model intercomparison experiment, a second version of SIMPLE is calibrated to represent the GIC
component of sea-level rise. This experiment is motivated by potential structural shortcomings of the GIC-MAGICC model. In
Eq. (), the implied GIC volume equilibrium depends only on the current surface temperature relative to the fixed
parameter Teq,GIC. If the GIC volume is quite low (almost entirely melted), this structure potentially
enables unphysically fast growth of GIC volume. The SIMPLE model (Eqs. 3–5) contains an arguably more realistic
representation of the relaxation of ice sheet volume towards an equilibrium.
In this formulation, the timescale of the
relaxation and the equilibrium itself both depend on the surface temperature state. This type of potential disagreement
within the scientific community regarding model structure is precisely where the BRICK model framework can be useful. The
flexibility of BRICK enables easy exchange of one component sub-model (GIC-MAGICC) for another (GIC-SIMPLE). This enables
experiments examining the impacts of model structural choices.
This GIC-SIMPLE model configuration calibrates GIC-SIMPLE using the same observational data as the control GIC-MAGICC
setup. One key difference is that the prior distributions of the model
parameters for GIC-SIMPLE were modified to be
specific to the GIC conditions instead of the GIS. These prior distributions are given in Appendix A. The same calibration
method and likelihood functions are used for the GIC-SIMPLE experiment as in the GIC-MAGICC control model. We use the same
calibration approach as in the control ensemble, which yields an ensemble of 10 483 model realizations for analysis in
the GIC-SIMPLE experiment. As in Sect. 4.2, we focus on RMSE, AIC, and BIC as model goodness-of-fit metrics. The
GIC-MAGICC model has six model parameters (four physical parameters, two
statistical ones) and the GIC-SIMPLE model has seven parameters (five
physical parameters, two statistical ones).
Experimental results: glaciers and ice caps sub-model intercomparison
When the GIC-MAGICC model is used, RMSE, AIC, and BIC are all lower than when the GIC-SIMPLE model is used (Fig. 3). But
the AIC and BIC are not drastically lower for GIC-MAGICC than for GIC-SIMPLE. This indicates that the addition of a model
parameter (GIC-SIMPLE) may not be justified (Kass and Raftery, 1995). The GIC contribution to global sea level in Fig. 3
is presented relative to 1960 GIC sea-level rise. The median, 5 %, and
95 % quantiles of the calibrated GIC-SIMPLE
parameters are given in Appendix A.
Comparison of (a) GIC-MAGICC vs. (b) GIC-SIMPLE
model performance in hindcasting the glaciers and ice caps (GIC) contribution
to sea-level rise. GIC sea-level rise is presented relative to 1960 GIC
sea-level contribution. Lower values of the AIC, BIC, and RMSE indicate
a better model fit to the data (Dyurgerov and Meier, 2005). These error
metrics are all calculated using the maximum likelihood ensemble member,
which is represented by the solid blue line. Green highlighting indicates the
model structure suggested by each comparison metric.
![]()
Parameter descriptions and prior probability distributions for flood
protection cost–benefit analysis.
Parameter
Description
Distribution
p0
Initial flood frequency (yr-1) with 0 heightening
logN(logμ=log(0.0038),logσ=0.25)
α
Exponential flood frequency constant (m-1)
N(μ = 2.6, σ = 0.1)
V
Value of goods protected by dike ring (billion USD)
U(5, 30)
δ
Net discount rate (–)
U(0.02, 0.06)
Iunc
Investment uncertainty (–)
U(0.5, 2)
rsubs
Land subsidence rate (myr-1)
logN(logμ=log(0.0056),logσ=0.4)
The two models display similar levels of under-confidence, illustrated by the wide model ensemble envelope around the
narrower range of observational data (Fig. 3) (Dyurgerov and Meier, 2005). That both models show under-confidence is often
judged to be preferable to over-confidence, especially when physical models
are linked to applications-oriented decision-making frameworks (Herman
et al., 2015). This experiment demonstrates BRICK's flexibility and ability
to allow
the user to isolate and examine any source of uncertainty or dissatisfaction in the modeling framework. These results also
provide guidance for the use of the BRICK model framework for model intercomparison and selection experiments. At present
we do not make any recommendations regarding which GIC sub-model to use. The GIC-MAGICC component has both strengths
(e.g., fewer parameters and appropriate in melting regimes) and weaknesses (unphysical GIC growth, does not encourage
growth beyond V0,GIC, state-independent equilibrium).
Linking an impacts and decision-analysis module to BRICK
Experimental description
We demonstrate the ability of the BRICK framework to incorporate additional structure to link the physical model for
surface temperature and sea-level rise (climate and sea-level modules,
Fig. 1) to socioeconomic implications (impacts
module, Fig. 1). In this example application, we use the calibrated ensemble in the BRICK control configuration to obtain
local sea level projections for New Orleans, Louisiana (29∘57′ N, 90∘4′ W). We use
a common didactic model for coastal flood protection (Van Dantzig, 1956; Jonkman et al., 2009). In this flood risk model,
the policy lever available to decision-makers is the amount by which to heighten the dikes protecting the coastal
community. We consider a previously published simple analysis that focuses on the northern dike ring in central New
Orleans (Jonkman et al., 2009). We use this illustrative cost–benefit
approach to calculate an economically efficient
dike-heightening by weighing the decrease in probable losses due to flooding achieved by building taller dikes against the
increase in costs due to investments in construction.
The flood risk model implemented here follows a commonly used simple approach (Van Dantzig, 1956). The present
implementation considers the current year as 2015 and a time horizon of 85 years (to 2100). We consider discrete dike
heightenings in increments of 5 cm, between 0 and 10 m. The average annual flood probability is calculated from
the simulated local sea-level rise, the land subsidence rate (Dixon et al., 2006), and flood frequency parameters (Jonkman
et al., 2009). We calculate the expected losses (US dollars) for each proposed dike heightening from the flood
probabilities for each heightening, the value of goods protected by the dike ring, and the net discount rate (Jonkman
et al., 2009). The total expected costs are the sum of the expected losses and the expected investments. In this
simplified model, the investment costs only depend on dike heightening and are approximated by linear interpolation
between data points provided by Jonkman et al. (2009) (and linear extrapolation for dike heightenings outside this range),
and the expected losses are an exponentially decreasing function of dike height above mean sea level. The minimum total
expected cost then is the economically efficient dike heightening strategy in
the framework of this simple illustrative
model (Eq. 14 of Van Dantzig, 1956).
The uncertain parameters considered in this cost–benefit analysis include
the initial flood frequency with no heightening
(yr-1); the exponential flood frequency constant (m-1); the value of goods protected by the dike ring
(billion US dollars); the net discount rate (%); the uncertainty in investment costs (a unitless multiplicative
factor); and the land subsidence rate (myr-1) (Table 1). The central estimates for the exponential flood
frequency constant (α) and the initial flood frequency with no heightening (p0) are taken from Van Dantzig
(1956). The exponential flood frequency constant relates the increase in flood probability that results from an increase
in sea level relative to the dike height. We make the assumption that this factor should scale (to first order) relatively
well from the Dutch case considered by Van Dantzig (1956) to the test case of
New Orleans considered presently. The initial
flood frequency with no heightening (p0) may not translate directly between these two cases, but highlights our
intent for this experiment to serve as an example of future applications of the BRICK model to inform decision
analyses. The admittedly ad hoc distributions assumed for α and p0 were selected to sample tightly around the
central estimates from Jonkman et al. (2009). A more detailed treatment of this risk management problem would include
using methods from extreme value theory to address the risks posed by storm surges (Coles, 2001).
Regional projections of median sea-level changes under
Representative Concentration Pathways (RCPs) (a) 2.6,
(b) 4.5, and (c) 8.5 in the year 2100. Sea-level rise is
presented relative to 1986–2005 global mean sea level (meters).
![]()
The investment uncertainty considered in the sensitivity tests of Jonkman et al. (2009) included a base case, 50 %
lower, and 100 % higher than the base case. We use this range for the investment uncertainty, applied as
a multiplicative factor ranging from 0.5 to 2. The range for the value of
goods protected by the dike ring is taken from
Jonkman et al. (2009), where the lower bound is the lowest estimate of the
value of goods protected by the three dike rings
considered in that work (USD 5 billion), and the upper bound is the estimated combined value protected by all three dike
rings (USD 30 billion). The net discount rate range is centered at 4 %, the estimate from Jonkman et al. (2009)
accounting for inflation and interest rate. Those authors' net discount rate is decreased to 2 % due to economic
growth (1 %) and increased flooding probability due to sea-level rise (1 %). Our demonstrative example endogenizes
the effects of sea-level rise and accounts for parametric uncertainty in the
value of goods protected by the dike ring. Hence, we center our range for the
net discount rate at 4 % but allow for a ±2 % uncertainty range.
The
rate of land subsidence is based on the estimates of Dixon et al. (2006), with mean 5.6 mmyr-1 and standard deviation
2.5 mmyr-1. We transform this to a log-normal distribution to disallow negative rates of land subsidence.
We sample the uncertainty in these parameters via a Latin hypercube, where
the population size is given by the number of
sea-level rise ensemble members that are present (10 589 for the control BRICK ensemble). The distributions from which
the economic model parameters are drawn are given in Table 1. Each realization of regional sea level is assigned
a concomitant sample of flood risk model parameters. An economically
efficient dike heightening is calculated for each
ensemble member. “Return periods” (years) correspond to the frequency of storms with the potential to overtop dikes
with the corresponding dike height – essentially, the inverse of the annual flood probability. Return periods are
a convenient and intuitive way to view the probabilities of flooding in this economic analysis.
We present results for the flood risk management experiment using sea-level
projections under RCP8.5. We note that many factors are not incorporated into
this analysis, and this simple illustration is not designed to be used for
real decision-making. For example, storm surge non-stationarity and structural failure are not
considered (Grinsted et al., 2013; Moritz et al.,
2015). The purpose of this illustration is to demonstrate the flexibility and transparency of the BRICK model framework.
This experiment highlights the importance of transparency in particular when linking physical modeling results to the
impacts on socioeconomic modeling and policy decision-making.
Illustrative cost–benefit analysis for the economically efficient
dike heightening (lower horizontal axis) and return period (upper horizontal
axis) for the northern–central dike ring in New Orleans, Louisiana. The bold
dot denotes the economically efficient (i.e., cost-minimizing) solution. The
shaded region gives the 90 % ensemble range of trade-off
curves, and the bold line denotes
the ensemble mean trade-off curve.
![]()
Experimental results: regional sea-level changes
In order to link projections of sea-level rise to problems of local coastal adaptation, regional sea level is projected to
2100 under the climate change scenarios of RCP2.6, 4.5, and 8.5 (Fig. 4). These projections use the control configuration
of the model, with GIC-MAGICC and the full sea-level rise sub-model setup
depicted in Fig. 1. The ensemble median
projection is shown in Fig. 4. Sea level rises by 2100 globally by about 55 cm (43–72 cm), 74 cm (56–100 cm), and
130 cm (93–177 cm) under RCP2.6, 4.5, and 8.5, respectively (ensemble median and 5–95 % range in parentheses). The
Arctic Ocean is an obvious exception to the rest of the ocean. Due to the Greenland ice mass loss, Arctic regional sea
level will fall as a result of the loss of gravitational attraction. However, the addition of mass raises sea level in
other parts of the ocean farther away. Arctic sea level (median sea level of all latitudes higher than 60∘ N)
increases by 7 cm under RCP2.6, but falls by 2 cm under RCP4.5 and by 30 cm under RCP8.5. By contrast, the tropical sea
level (median of all latitudes between 30∘ S and 30∘ N) rises by 57, 82, and 147 cm under RCP2.6, 4.5,
and 8.5, respectively, which is greater than the global mean rise. Due to the asymptotically increasing gravitational
effects in proximity to the melting Greenland Ice Sheet, sea-level fall below -1.5 m is cut off at -1.5 m.
Experimental results: link to coastal defense strategies
We now focus on the regional sea-level projections for the grid cell
containing New Orleans, Louisiana, under RCP8.5
(Fig. 4c), to demonstrate the use of these sea-level projections in a common local flood risk management example. We find
the economically efficient (i.e., cost-minimizing) dike heightening to be
1.5 m (ensemble mean; the 90 % range is 0.75 to 1.95 m; Fig. 5). This
heightening corresponds to a return period of about 760 years (ensemble
mean; the 90 % range is
roughly 200–3000 years; Fig. 5). The simple analysis presented here should not be used to inform on-the-ground decisions
in New Orleans. This experiment is meant to demonstrate BRICK's ability to contribute in risk assessment applications.
Conclusions
We present BRICK v0.2, a modeling framework for global and regional sea-level change. BRICK has been designed with four
epistemic modeling goals: accessibility, transparency, efficiency, and
flexibility. BRICK can skillfully match observational data for
individual sea-level contributions in hindcasts
(Bakker et al., 2017). Here we focus on how BRICK achieves our epistemic values using a set of modeling experiments.
BRICK is coded in the widely available and simple coding language R (R Core Team, 2016), to achieve the goals of
accessibility and transparency. The main physics (global mean temperature and sea-level rise) codes are also (redundantly)
transcribed in Fortran 90, for more efficient simulations. BRICK is designed to be transparent, as well as efficient, by
coupling previously published simple, mechanistically motivated models for the major contributors to global sea level. The
efficient physical modeling approach provides the opportunity to incorporate a rigorous statistical calibration framework
as well, wherein various sources of uncertainty are incorporated into model
projections (see Bakker et al., 2017, for
a more detailed discussion of this). Finally, the model comparison experiments in Sect. 4.2 and 4.3 demonstrate the
flexibility of the BRICK modeling framework. These sections bring into focus the importance of these epistemic modeling
values. A modeling framework that is (in particular) transparent and accessible can help to streamline the process of
quantifying the local impacts of the physical model results, to link to decision-analytical models, and to communicate
these results to stakeholders and decision-makers.
We hope that the accessibility and transparency of BRICK are helpful to others, and will stimulate the continuous
peer-reviewing, challenging, and improving of the BRICK framework. Of course, although we tried to couple models that fit
our epistemic model values as closely as possible, we assume that others may
prefer other models and may have different
epistemic values. Our framework is designed in such a way that it is possible to plug in other model components to
reflect these different values. For example, it would be very interesting to
add the component models used for the
semi-empirical model frameworks of Mengel et al. (2016) and Nauels et al. (2017).
We demonstrated the flexibility and transparency of BRICK in connecting projections from the physical model to the impacts
on a local risk and decision-analysis problem. The simple probabilistic
calibration method and cost–benefit analysis that
we adopted for the simple demonstration can be expanded to incorporate aspects of deep uncertainties (Lempert et al.,
2004; Weaver et al., 2013) as well as more complex decision-making frameworks (e.g., considering multiple objectives,
beyond only expected total costs) (Kasprzyk et al., 2013; Lempert, 2014; Lempert and Collins, 2007). Climate change poses
decision problems where strong connections across academic disciplines are critical. Further, the study of climate
modeling relies on communal modeling efforts. The need for transparent communication among modelers and between
disciplines is where the BRICK framework and the epistemic modeling values presented here can facilitate future
developments. Above all, we hope that BRICK inspires the involved communities to pay careful attention to enhance
flexibility, transparency, and accessibility of modeling frameworks.