Simple models can play pivotal roles in the quantification and framing of uncertainties surrounding climate change and sea-level rise. They are computationally efficient, transparent, and easy to reproduce. These qualities also make simple models useful for the characterization of risk. Simple model codes are increasingly distributed as open source, as well as actively shared and guided. Alas, computer codes used in the geosciences can often be hard to access, run, modify (e.g., with regards to assumptions and model components), and review. Here, we describe the simple model framework BRICK (Building blocks for Relevant Ice and Climate Knowledge) v0.2 and its underlying design principles. The paper adds detail to an earlier published model setup and discusses the inclusion of a land water storage component. The framework largely builds on existing models and allows for projections of global mean temperature as well as regional sea levels and coastal flood risk. BRICK is written in R and Fortran. BRICK gives special attention to the model values of transparency, accessibility, and flexibility in order to mitigate the above-mentioned issues while maintaining a high degree of computational efficiency. We demonstrate the flexibility of this framework through simple model intercomparison experiments. Furthermore, we demonstrate that BRICK is suitable for risk assessment applications by using a didactic example in local flood risk management.

Simple, mechanistically motivated Earth system models often play a pivotal role in climate and flood risk management (Hartin et al., 2015). For example, they are used for uncertainty quantification (Bakker et al., 2017; Grinsted et al., 2010; Urban et al., 2014; Urban and Keller, 2010) and complex model emulation (Applegate et al., 2012; Bakker et al., 2016; Hartin et al., 2015; Meinshausen et al., 2011a), and are incorporated into integrated assessment models (Hartin et al., 2015; Meinshausen et al., 2011a).

Computational constraints often impose hard trade-offs between physical model complexity and statistical model complexity. For example, a sizable allocation of computational time could be spent running a small number of simulations using a high-complexity physical model. Highly detailed simulations are useful to better understand the complex system, but with just a small number of simulations, only weak ensemble statistics can be drawn. In contrast, numerous realizations of a less detailed physical model could be run. This would provide the opportunity for more advanced ensemble statistical techniques including the characterization and quantification of uncertainties. It is important in climate-related applications such as mitigation of greenhouse gas emissions or adaptation to sea-level rise that the relevant uncertainties are explored and communicated clearly to policy-makers (e.g., Garner et al., 2016; Gauderis et al., 2013; Goes et al., 2011; Hall et al., 2012; Lempert et al., 2004).

Several studies have broken important new ground in tackling these challenges. For example, Nauels et al. (2017) present a platform of sea-level emulators (i.e., simple models of complex models) that efficiently produces future projections and characterizes key model structural uncertainties using statistical calibration methods. Semi-empirical modeling (SEM) approaches trade detailed physics for a model that can efficiently project sea level using statistical, but mechanistically motivated, relationships between sea-level changes and climate conditions such as temperature and radiative forcing (Grinsted et al., 2010; Jevrejeva et al., 2010; Kopp et al., 2016; Rahmstorf, 2007). Recent work has expanded upon the SEM approach to use simple models to resolve individual contributions to global sea level (Bakker et al., 2017; Mengel et al., 2016; Nauels et al., 2017).

Studies based on simple, mechanistically motivated models have the potential to be transparent and reproducible when presented in open platforms and when the underlying data are readily available. Yet, although there is an increasing tendency to share scientific code, it can be (perhaps surprisingly) hard to get the models running and to reproduce the results. A likely cause of this is that not enough attention is given to the scientific coding itself. Careful coding, documentation, and review require a dedicated commitment of time, but scientific incentives to do so can be weak.

Here we describe in detail BRICK (Building blocks for Relevant Ice and
Climate Knowledge, Bakker et al., 2017) v0.2,
a model framework that focuses on

In this model framework, we present a set of existing, well-tested, and easy-to-couple simple models for climate and flood risk management. They simulate global mean surface temperature and contributions to global mean sea-level rise. BRICK also includes a regional sea-level rise module, which translates the global mean sea-level contributions to regional sea level at a user-defined location. We use these regional sea-level projections to demonstrate how the physical model may be linked to decision-making and impacts. We implement a Bayesian calibration approach with an aim to adequately resolve the tails of the distribution of future sea levels because these low-probability areas represent high-risk events. In robust decision-making approaches, it can be favorable to be underconfident as opposed to overconfident, e.g., by applying conservative estimates in the sense of being risk-averse (Herman et al., 2015). We hence include in our Bayesian approach wide, mechanistically motivated prior parameter probability distributions (Bakker et al., 2017). Yet, the flexibility of the BRICK model framework also enables the implementation of other calibration schemes. This paper is intended to showcase a useful model framework that is attractive for a sustainable approach to model development, for example by inspiring fellow researchers to contribute to the framework, to rethink their coding practice, and maybe even to adopt some of the demonstrated design objectives in their future research proposals.

The hindcast skill of the BRICK model has been previously demonstrated (Bakker et al., 2017). Thus, the present work focuses on outlining a set of epistemic modeling values that we believe facilitates advances in the modeling community. The remainder of this work is organized as follows. In Sect. 2, we describe these values and the ways in which the BRICK model implementation strives to attain them. Section 3 contains an overview of the BRICK model components for climate and the contributions to sea-level rise. Section 4 describes and presents the results of a set of model experiments conducted to demonstrate how BRICK lives up to our epistemic modeling values. Section 5 summarizes the findings of this work and provides conclusions and guidance for future work.

BRICK model structural diagram. Dashed connectors indicate couplings that are non-essential for projections of global mean sea level. These dashed couplings are required for projecting regional sea-level and climate impacts. DOECLIM is the Diffusion-Ocean-Energy balance CLIMate model (Kriegler, 2005); GIC-MAGICC is the Glaciers and Ice Caps module from the MAGICC climate model (Meinshausen et al., 2011a); TE is the Thermal Expansion model (Grinsted et al., 2010; Mengel et al., 2016); SIMPLE is the Simple Ice-sheet Model for Projecting Large Ensembles (Bakker et al., 2016); ANTO is the ANTarctic Ocean temperature model; DAIS is the Danish Center for Earth System Science Antarctic Ice Sheet model (Shaffer, 2014); regional sea-level fingerprinting downscales from global sea-level contributions to regional (Slangen et al., 2014); and the model of Van Dantzig (1956) assesses flood risk.

The essence of the BRICK physical model is to simulate changes in global mean surface temperature and sea level, in response to perturbations in radiative forcing. The socioeconomic impacts of the simulated temperature and sea-level changes may then be assessed. This is depicted in Fig. 1. The climate component, each individual contribution to global sea-level rise, and an impacts module are sub-models of BRICK, or “BRICKs.” We defer details of the specific sub-models to Sect. 3. The physical model (climate and sea-level rise) components of BRICK are intentionally simple. This choice is guided by the epistemic modeling values outlined below.

We selected R (R Core Team, 2016) as the base language for BRICK because it is (i) stable, (ii) freely available and open source, (iii) relatively easy to use, and (iv) easy to call subroutines written in faster languages. In the BRICK source code accompanying this study, the physical sub-models within the climate and sea-level rise modules are all provided as both R and Fortran 90 routines. It is our aim that the full physical–statistical model of BRICK will be accessible using a modern laptop. This means that sizable Monte Carlo simulations (on the order of a million samples) must be possible on a timescale of hours. This is made possible by calling Fortran 90 sub-models from the base code in R.

In addition to conceptual accessibility, it is our view that useful model codes should be physically accessible too. Openness with scientific codes is likely to lead to higher quality codes (Easterbrook, 2014). In an effort to be truly open source and freely available, all codes – including the physical model, statistical model, and processing and plotting scripts used for the results shown here – are available through a download server as well as the Github repository provided in the Code Availability section of this article. Providing all code and data necessary to recreate this study is a critical component of reproducible research (Murray-Rust and Murray-Rust, 2014) and can help to build trust between the general public and the scientific community (Easterbrook, 2014; Grubb and Easterbrook, 2011).

We aim to achieve transparency in two areas: the physical modeling, including the related model code, and the communication of scientific findings.

With regards
to transparent physical modeling, we use simple numerical integration schemes
whenever possible. We use as few
global variables as possible, in order to “write programs for people, not computers” (Wilson et al., 2014). The
essence of these authors' advice is that users should not be expected to remember more than a few pieces of information as
they read and develop code. To this end, in BRICK we aim to give appropriately suggestive names to our variables within
the code, such that a human intuitively understands what the quantity at hand represents. For example, when naming
a logical or Boolean variable, we prefer for its name to read as a question that the variable itself answers, and begin
the variable name with the letter “

Transparency also serves to link the findings of a physical model to decision-making and policy impacts. BRICK can be a useful tool to link climate changes (global temperature and sea-level rise) to decision-making frameworks through a clear outlet for coupling to socioeconomic models. Perhaps most importantly, the coupled physical–statistical framework in BRICK incorporates many sources of uncertainty into the physical findings on which the decisions will be based. It is important that these uncertainties in climate projections are represented in the decision-making framework (Lempert et al., 2004).

A modular programming approach is taken with BRICK, which allows each component sub-model to be exchanged for alternative models. In this way, as the scientific forefront progresses, the BRICK sub-models may advance as well. The flexible BRICK framework also permits a quantitative evaluation of model structural differences, which is valuable in the event that it is unclear which of two candidate models should be chosen. In these cases, the BRICK framework is valuable for model comparison and quantification of structural uncertainty. As new data sets for the calibration of the sub-models become available, these can also be incorporated instead of or in addition to the current data sets. We demonstrate the flexibility of the BRICK framework through a series of modeling experiments (Sect. 4).

Code efficiency is enabled primarily through (i) the use of simple models and (ii) using model versions written in R for
easy preliminary experimentation, and Fortran 90 versions for production simulations. This practice also follows the
advice of Wilson et al. (2014) for code developers to “write code in the highest-level language possible, and shift to
lower-level languages like C and Fortran only when they are sure the performance boost is needed.” This boost indeed
enables the generation of production simulations on most modern laptops. The
simulation of 1 million model iterations spanning from 1850 to the present,
performed on each of four CPUs (two cores and two threads per core), yields
an ensemble of 4 million model realizations. This procedure requires less
than an hour on a model year 2012 laptop with
a 2.9

Providing computationally efficient code simplifies its use. For example, there may be limitations on the computing resources allocated for a particular project, or an instructor might be interested in enhancing coursework by incorporating computer modeling exercises. In these cases, transparency is critical (as mentioned above), but the model must also be sufficiently efficient that it neither (i) expires the computational allotment for the experiment nor (ii) takes too long to be of any educational use. Our epistemic modeling values of accessibility, transparency, flexibility, and efficiency motivate the choice of a relatively simple physical modeling framework. Accordingly, a detailed statistical calibration framework is implemented. Within this framework, physical model and statistical model parameters are calibrated using observational data sets and mechanistically motivated prior ranges. The statistical model is reviewed at greater length by Bakker et al. (2017), so we provide only an overview in Sect. 4.1.

We invite the readers to download and test our code, as well as provide feedback on how best to further develop BRICK to fulfill the four epistemic values outlined above. Frequent and thorough code review by other team members as well as outside agents is another critical step towards good scientific coding practices (Wilson et al., 2014), and “peer review needs to be supplemented with a number of other mechanisms that help to establish the correctness and credibility of scientific research” (Grubb and Easterbrook, 2011). Wilson et al. (2014) also note that a number of high-profile research articles have been retracted or revised because of errors in the code. The likelihood of these errors may be greatly reduced by thoroughly testing other group members' codes. In our own experience conducting the experiments described in this study, we have anecdotal evidence for the value of testing one another's code. Some errors were corrected through this process, and many more pieces of code were modified for clarity. We continue to invite all comments and suggestions for improvements and modifications (to the corresponding author).

The use of a version control system greatly expands the accessibility of a code base, and also facilitates continuous improvement of the modeling framework itself. This is true and useful before, during, and after the peer-review process. Mistakes are inevitable and we assume that BRICK still contains some minor errors, ambiguities, and pieces of code that do not fully comply with our own standards. Openly sharing the code and documentation will help to address these issues. It is our hope that BRICK may be further developed as a community modeling tool, and that other users may contribute to the framework through added or revised models and data, or improved functionality. The use of a version control system facilitates this type of community effort (Wilson et al., 2014).

We adopt DOECLIM (Diffusion Ocean Energy balance CLIMate model, Kriegler, 2005) as a starting point for a simple climate
model (Fig. 1). DOECLIM is a zero-dimensional energy balance model coupled to a three-layer, one-dimensional diffusive
ocean model. The DOECLIM physical model outputs are global mean surface temperature anomaly (

We fit a first-order autoregressive (AR1) error model to the model–data
discrepancy between temperature and ocean heat
uptake model output and calibration data. We estimate the first-order lag autocorrelation parameters (

The BRICK global mean sea-level module calculates global sea-level change as
the sum of four individual components: glaciers and ice caps (GIC), the
Greenland Ice Sheet (GIS), the Antarctic Ice Sheet (AIS), and thermal
expansion (TE). These
component sub-models are described in the following sections. BRICK accounts for land water storage contributions to
global mean sea level using mass balance trends from the International Panel on Climate Change (IPCC) Fifth Assessment
Report (AR5, Church et al., 2013) and from the work of Dieng et al. (2015). The differential equations for the GIC, GIS,
AIS, and TE contributions to global mean sea level are integrated into BRICK
using first-order numerical integration schemes with a 1-year time step.
Initial conditions are specified at a year dictated by the sub-model's
assumed reference point. This differs, in general, among the sub-models, and
some model parameters depend on preserving this reference
year. Starting from this initial condition, a first-order explicit numerical integration method integrates forward in time
to the end of the simulation and a first-order implicit (backward differentiation) method integrates backward in time to
the earliest year of the simulation. Preliminary experiments (not shown)
demonstrated that the 1-year time step is
sufficiently short to maintain numerical stability. The total global mean sea-level rise from the coupled BRICK model is

We adopt a simple zero-dimensional sub-model for the contribution to global sea-level rise from Glaciers and Ice Caps
(GIC) from Wigley and Raper (2005). This same formulation is used in the MAGICC climate model (Meinshausen et al.,
2011a). The parameterization for the GIC contribution to global sea-level
rise is

The uncertain physical model parameters for GIC-MAGICC (which will be tested in Sect. 4.2) are

BRICK uses the mechanistically motivated, zero-dimensional SIMPLE (Simple
Ice-sheet Model for Projecting Large Ensembles) model as the parameterization
for the Greenland Ice Sheet (GIS) contribution to global mean sea-level
change (Bakker
et al., 2016). SIMPLE estimates the GIS response to changes in global mean surface temperature by first estimating an
equilibrium ice sheet volume (

We employ the Danish Center for Earth System Science Antarctic Ice Sheet (DAIS) model to simulate the Antarctic Ice Sheet
contribution to global sea level (Shaffer, 2014). This is a two-dimensional model for the Antarctic Ice Sheet that assumes
an axisymmetric geometry, shown graphically in Shaffer (2014), his Fig. 2. The DAIS model tracks changes in Antarctic Ice
Sheet volume, considering contributions from (i) incident precipitation, (ii) runoff of ice melt, (iii) ice flow, and
(iv) ice sheet disintegration from rising and warming sea levels. Input forcings for the DAIS model include Antarctic
surface temperature reduced to sea level (

When calibrated as a stand-alone model, the DAIS forcings are provided based on temperature reconstructions (see Shaffer,
2014). When the DAIS model is run as a component in the coupled BRICK model, a separate sub-model is needed to convert the
global mean surface temperature from the climate model (DOECLIM) to the Antarctic surface and ocean subsurface
temperatures required by the DAIS model. The Antarctic surface temperature is estimated from a linear regression with
global mean surface temperature (Morice et al., 2012; Shaffer, 2014). The Antarctic Ocean temperatures (

Here, we use an updated and corrected version of the DAIS model (Ruckert et al., 2017; Shaffer, 2014). In the original
formulation of the DAIS model, the input forcing from year

The dynamical core of the DAIS model is more detailed than the GIC, GIS, and TE emulators given above. For this reason, we
do not undertake a full review of the model equations here. The interested reader is directed to Shaffer (2014) and
Ruckert et al. (2017) for further details regarding the DAIS model and its
hindcast forcings. Eq. (

BRICK uses a simple parameterization for the contribution of thermal expansion (TE) of the Earth's oceans to sea-level
rise. We make the simplifying assumption that thermal expansion of the oceans occurs uniformly around the globe. While
this is, of course, not strictly true, the next obvious step forward in model
complexity would be to use a vertically and latitudinally resolved model for
thermal expansion, incorporating the DOECLIM model output for ocean heat
uptake. This two-dimensional ocean model is beyond the scope of the simple
model framework described presently, but is an excellent
subject for future work. Here, we employ a simple zero-dimensional thermal expansion emulator based on the
parameterizations of the sea-level rise sub-models of Mengel et al. (2016)
and that was originally used by Grinsted et al. (2010) to model the total
global mean sea-level changes. First, an equilibrium sea-level rise from
thermal expansion, due to changing global surface temperature
(

In order to link the projections of global mean sea-level change from BRICK to a local coastal adaptation, information on
regional sea-level change is needed. Thus, the global mean sea level from
BRICK is downscaled to regional sea level using
previously published maps of scaling factors for the glacier and ice sheet components of sea-level change (Slangen et al.,
2014). Any redistributions of mass between the cryosphere and the ocean
(e.g., ice melt) lead not only to a change in the
total mass of the ocean, but also to changes in regional sea level as a result of variations in the gravitational field of
the Earth, which in turn affect the solid Earth and the rotation of the Earth
(e.g., Mitrovica et al., 2001). This
typically (and counterintuitively) leads to a sea-level fall close to the source of mass loss and larger-than-average
sea-level rise at larger distances (

The glacier fingerprint is based on projected changes in glacier mass in 2100 using a glacier model driven by temperature and precipitation information from the Fifth Climate Model Intercomparison Project database (Taylor et al., 2012) under the Representative Concentration Pathway 8.5 climate change scenario (RCP8.5, Moss et al., 2010), as presented in Slangen et al. (2014). It is assumed that the mass change ratios between the different glacier regions on Earth remain the same throughout the 20th and 21st centuries, which is a valid assumption as long as none of the glacier regions “finish” (which is not expected to happen in the next century). For the Greenland and Antarctic ice sheets, it is assumed that ice melt takes place uniformly over the ice sheet surface. Within the BRICK model structure, users may define a latitude and longitude to obtain regional sea-level change.

We calibrate the model through a coupled physical–statistical calibration framework. The relatively simple physical modeling framework of BRICK is motivated by our epistemic modeling values (Sect. 2.1). This efficient model permits the use of a sophisticated model calibration technique. The calibration uses a robust adaptive Markov chain Monte Carlo (MCMC) approach (Vihola, 2012). The specifics of how it is applied to the BRICK model as well as a demonstration of calibrated BRICK model hindcast skill are documented in Bakker et al. (2017).

The vastly different timescales and characterizations of uncertainty in the
Antarctic paleoclimatic calibration period
and the modern period (1850 to present) lead to two separate sets of calibration parameters: (i) DAIS parameters,
calibrated using paleoclimatic data, and (ii) DOECLIM, GIC, GIS, and TE parameters, jointly calibrated using modern
data. The paleoclimatic calibration is done using four parallel MCMC chains of 500 000 iterations each. The first
120 000 iterations of each chain are removed for burn-in. The paleoclimatic calibration requires about 10 h on a laptop
with a 2.9

We combine these two disjoint sets of parameters to form concomitant full
BRICK model parameter sets, and calibrate these to global mean sea-level data
(Church and White, 2011) using rejection sampling (Votaw Jr. and Rafferty,
1951). Prior to
rejection sampling, contributions from land water storage are estimated using trends from the IPCC (Church et al., 2013)
and subtracted from global mean sea level. When projecting global mean sea-level rise, we estimate land water storage
contributions by extrapolating using the 2003–2013 trend of 0.30

In the spirit of our epistemic values, calibration routines are provided with the available BRICK source code. These routines use modern methods readily available in R. It is our aim that the interested user can easily substitute their own likelihood function (as physical scientific knowledge progresses), a new calibration method (as the statistical state-of-the-art progresses), or both. To this end, we provide a sub-routinized likelihood function, called from an R-packaged calibration method (Vihola, 2012). We also provide individual likelihood functions and calibration scripts for each sub-model of BRICK individually, to enable interested users to perform experiments using stand-alone sub-models or pre-calibration (Edwards et al., 2011).

In the interest of accessibility and transparency, with the available BRICK source code we also provide the sets of calibrated model parameters for all experiments presented here. The purpose of this is 2-fold. First, it greatly enhances the reproducibility of these results. Second, these data sets enable users who would like to run their own ensembles and make projections of local sea levels to do so. This supports our goal of accessibility. The calibrated parameter sets are provided in netCDF format, with ensemble member as the “unlimited” dimension. This permits concatenation of multiple data sets by using netCDF operators (NCO) such as “ncrcat” (Zender, 2008). These are freely available tools for manipulating data stored in netCDF format.

We achieve the accessibility, transparency, and computational efficiency of the BRICK modeling framework through use of simple models written in a simple programming environment (R, R Core Team, 2016). It remains to be demonstrated that this framework is flexible and efficient in post-processing.

We demonstrate BRICK's flexibility and efficiency by implementing and switching in an alternative formulation for the
global mean sea level,

Note that the Rahmstorf (2007) emulator is arguably not the state-of-the-art anymore (Grinsted et al., 2010; Kopp et al., 2016). However, it serves here the purpose of demonstrating the ease with which alternative model formulations can be tested. This greatly simplifies, for example, model intercomparisons and improvements. Some advantages of a simple emulator such as this include fewer parameters to estimate and a transparent analysis. Disadvantages of such a model include the inability to resolve individual contributions to global mean sea level. This disables the use of sea-level fingerprinting to obtain regional sea-level patterns. Thus, the choice of model should be motivated not only by goodness-of-fit metrics, but also by applications.

Comparison of global mean sea-level rise hindcast skill relative to
sea-level data (Church and White, 2011), using

Many goodness-of-fit metrics are available for the comparison of models and data. We focus on three metrics that are motivated by the heavily parameterized full BRICK model framework. There are 39 free parameters in the coupled climate/sea-level rise model. By contrast, BRICK-GMSL has 13 free parameters. We use the global mean sea-level time series of Church and White (2011) for the model–data comparisons in skill hindcasting global mean sea level.

The

The

The full BRICK sea-level rise module (Fig. 1) performs better than the GMSL emulator (Eq. 11) according to RMSE; the full sea-level rise module has an RMSE of 0.0057 m, which is about half the GMSL emulator RMSE of 0.015 m (Fig. 2). These hindcasts are presented as sea level relative to 1961–1990 global mean sea level. This is of course expected, because the number of free model parameters in the full BRICK model is 39, while the GMSL emulator contains only 13 free parameters. The BIC metric gives the expected result for this disparity in model complexity. The BIC for the full BRICK model with respect to the sea-level data is 60.4 higher than the BIC for the GMSL emulator. The AIC is actually lower (by 14.2) for the full BRICK model than for the BRICK-GMSL emulator. These mixed results for the model comparison metrics indicate that the full BRICK sea-level rise module is not unreasonably over-parameterized; if the full BRICK model were obviously over-parameterized, we would expect the AIC for the GMSL emulator experiment to be lower than for the full BRICK model.

These results also show that the sea-level hindcast in the full BRICK model smoothes much of the year-to-year variability in sea-level rise. This can be seen by contrasting the full BRICK maximum likelihood ensemble member (solid blue line) in Fig. 2a with the BRICK-GMSL emulator maximum likelihood ensemble member in Fig. 2b. The full BRICK simulation does not capture the annual variation in global mean sea level that the BRICK-GMSL simulation successfully captures. This is attributed to the smoothing effect of averaging over the model ensemble the four major contributions to global mean sea level, as opposed to calibrating the BRICK-GMSL simulations directly to global mean sea-level data. This does not affect ensemble statistics, however, which can be seen from the shaded envelopes around the model simulations in Fig. 2. The BRICK model has been developed with efficiency and large ensemble simulations in mind, so missing annual variability is of little concern.

This demonstrates the ease with which model intercomparisons may be undertaken using BRICK. Deactivating the glaciers and ice caps, thermal expansion, and Greenland and Antarctic Ice Sheet components and integrating the GMSL emulator into BRICK involves low overhead in computer code. GMSL is the main output of the BRICK physical model. As such, it is our aim to provide a framework in which users can easily integrate new processes and models into the climate and sea-level rise modules as the scientific forefront progresses.

We conduct an experiment to demonstrate the flexibility of BRICK to permit easy exchanging of a single sub-model for one
component of global sea-level rise. In the control BRICK model setup, SIMPLE
is used to emulate the sea-level rise
contributions from the Greenland Ice Sheet (GIS) and GIC-MAGICC is used to emulate the contributions from glaciers and ice
caps (GIC). In this model intercomparison experiment, a second version of SIMPLE is calibrated to represent the GIC
component of sea-level rise. This experiment is motivated by potential structural shortcomings of the GIC-MAGICC model. In
Eq. (

This GIC-SIMPLE model configuration calibrates GIC-SIMPLE using the same observational data as the control GIC-MAGICC setup. One key difference is that the prior distributions of the model parameters for GIC-SIMPLE were modified to be specific to the GIC conditions instead of the GIS. These prior distributions are given in Appendix A. The same calibration method and likelihood functions are used for the GIC-SIMPLE experiment as in the GIC-MAGICC control model. We use the same calibration approach as in the control ensemble, which yields an ensemble of 10 483 model realizations for analysis in the GIC-SIMPLE experiment. As in Sect. 4.2, we focus on RMSE, AIC, and BIC as model goodness-of-fit metrics. The GIC-MAGICC model has six model parameters (four physical parameters, two statistical ones) and the GIC-SIMPLE model has seven parameters (five physical parameters, two statistical ones).

When the GIC-MAGICC model is used, RMSE, AIC, and BIC are all lower than when the GIC-SIMPLE model is used (Fig. 3). But the AIC and BIC are not drastically lower for GIC-MAGICC than for GIC-SIMPLE. This indicates that the addition of a model parameter (GIC-SIMPLE) may not be justified (Kass and Raftery, 1995). The GIC contribution to global sea level in Fig. 3 is presented relative to 1960 GIC sea-level rise. The median, 5 %, and 95 % quantiles of the calibrated GIC-SIMPLE parameters are given in Appendix A.

Comparison of

Parameter descriptions and prior probability distributions for flood protection cost–benefit analysis.

The two models display similar levels of under-confidence, illustrated by the wide model ensemble envelope around the
narrower range of observational data (Fig. 3) (Dyurgerov and Meier, 2005). That both models show under-confidence is often
judged to be preferable to over-confidence, especially when physical models
are linked to applications-oriented decision-making frameworks (Herman
et al., 2015). This experiment demonstrates BRICK's flexibility and ability
to allow
the user to isolate and examine any source of uncertainty or dissatisfaction in the modeling framework. These results also
provide guidance for the use of the BRICK model framework for model intercomparison and selection experiments. At present
we do not make any recommendations regarding which GIC sub-model to use. The GIC-MAGICC component has both strengths
(e.g., fewer parameters and appropriate in melting regimes) and weaknesses (unphysical GIC growth, does not encourage
growth beyond

We demonstrate the ability of the BRICK framework to incorporate additional structure to link the physical model for
surface temperature and sea-level rise (climate and sea-level modules,
Fig. 1) to socioeconomic implications (impacts
module, Fig. 1). In this example application, we use the calibrated ensemble in the BRICK control configuration to obtain
local sea level projections for New Orleans, Louisiana (29

The flood risk model implemented here follows a commonly used simple approach (Van Dantzig, 1956). The present
implementation considers the current year as 2015 and a time horizon of 85 years (to 2100). We consider discrete dike
heightenings in increments of 5

The uncertain parameters considered in this cost–benefit analysis include
the initial flood frequency with no heightening
(

Regional projections of median sea-level changes under
Representative Concentration Pathways (RCPs)

The investment uncertainty considered in the sensitivity tests of Jonkman et al. (2009) included a base case, 50 %
lower, and 100 % higher than the base case. We use this range for the investment uncertainty, applied as
a multiplicative factor ranging from 0.5 to 2. The range for the value of
goods protected by the dike ring is taken from
Jonkman et al. (2009), where the lower bound is the lowest estimate of the
value of goods protected by the three dike rings
considered in that work (USD 5 billion), and the upper bound is the estimated combined value protected by all three dike
rings (USD 30 billion). The net discount rate range is centered at 4 %, the estimate from Jonkman et al. (2009)
accounting for inflation and interest rate. Those authors' net discount rate is decreased to 2 % due to economic
growth (1 %) and increased flooding probability due to sea-level rise (1 %). Our demonstrative example endogenizes
the effects of sea-level rise and accounts for parametric uncertainty in the
value of goods protected by the dike ring. Hence, we center our range for the
net discount rate at 4 % but allow for a

We sample the uncertainty in these parameters via a Latin hypercube, where the population size is given by the number of sea-level rise ensemble members that are present (10 589 for the control BRICK ensemble). The distributions from which the economic model parameters are drawn are given in Table 1. Each realization of regional sea level is assigned a concomitant sample of flood risk model parameters. An economically efficient dike heightening is calculated for each ensemble member. “Return periods” (years) correspond to the frequency of storms with the potential to overtop dikes with the corresponding dike height – essentially, the inverse of the annual flood probability. Return periods are a convenient and intuitive way to view the probabilities of flooding in this economic analysis.

We present results for the flood risk management experiment using sea-level projections under RCP8.5. We note that many factors are not incorporated into this analysis, and this simple illustration is not designed to be used for real decision-making. For example, storm surge non-stationarity and structural failure are not considered (Grinsted et al., 2013; Moritz et al., 2015). The purpose of this illustration is to demonstrate the flexibility and transparency of the BRICK model framework. This experiment highlights the importance of transparency in particular when linking physical modeling results to the impacts on socioeconomic modeling and policy decision-making.

Illustrative cost–benefit analysis for the economically efficient dike heightening (lower horizontal axis) and return period (upper horizontal axis) for the northern–central dike ring in New Orleans, Louisiana. The bold dot denotes the economically efficient (i.e., cost-minimizing) solution. The shaded region gives the 90 % ensemble range of trade-off curves, and the bold line denotes the ensemble mean trade-off curve.

In order to link projections of sea-level rise to problems of local coastal adaptation, regional sea level is projected to
2100 under the climate change scenarios of RCP2.6, 4.5, and 8.5 (Fig. 4). These projections use the control configuration
of the model, with GIC-MAGICC and the full sea-level rise sub-model setup
depicted in Fig. 1. The ensemble median
projection is shown in Fig. 4. Sea level rises by 2100 globally by about 55 cm (43–72 cm), 74 cm (56–100 cm), and
130 cm (93–177 cm) under RCP2.6, 4.5, and 8.5, respectively (ensemble median and 5–95 % range in parentheses). The
Arctic Ocean is an obvious exception to the rest of the ocean. Due to the Greenland ice mass loss, Arctic regional sea
level will fall as a result of the loss of gravitational attraction. However, the addition of mass raises sea level in
other parts of the ocean farther away. Arctic sea level (median sea level of all latitudes higher than 60

We now focus on the regional sea-level projections for the grid cell containing New Orleans, Louisiana, under RCP8.5 (Fig. 4c), to demonstrate the use of these sea-level projections in a common local flood risk management example. We find the economically efficient (i.e., cost-minimizing) dike heightening to be 1.5 m (ensemble mean; the 90 % range is 0.75 to 1.95 m; Fig. 5). This heightening corresponds to a return period of about 760 years (ensemble mean; the 90 % range is roughly 200–3000 years; Fig. 5). The simple analysis presented here should not be used to inform on-the-ground decisions in New Orleans. This experiment is meant to demonstrate BRICK's ability to contribute in risk assessment applications.

We present BRICK v0.2, a modeling framework for global and regional sea-level change. BRICK has been designed with four
epistemic modeling goals:

BRICK is coded in the widely available and simple coding language R (R Core Team, 2016), to achieve the goals of accessibility and transparency. The main physics (global mean temperature and sea-level rise) codes are also (redundantly) transcribed in Fortran 90, for more efficient simulations. BRICK is designed to be transparent, as well as efficient, by coupling previously published simple, mechanistically motivated models for the major contributors to global sea level. The efficient physical modeling approach provides the opportunity to incorporate a rigorous statistical calibration framework as well, wherein various sources of uncertainty are incorporated into model projections (see Bakker et al., 2017, for a more detailed discussion of this). Finally, the model comparison experiments in Sect. 4.2 and 4.3 demonstrate the flexibility of the BRICK modeling framework. These sections bring into focus the importance of these epistemic modeling values. A modeling framework that is (in particular) transparent and accessible can help to streamline the process of quantifying the local impacts of the physical model results, to link to decision-analytical models, and to communicate these results to stakeholders and decision-makers.

We hope that the accessibility and transparency of BRICK are helpful to others, and will stimulate the continuous peer-reviewing, challenging, and improving of the BRICK framework. Of course, although we tried to couple models that fit our epistemic model values as closely as possible, we assume that others may prefer other models and may have different epistemic values. Our framework is designed in such a way that it is possible to plug in other model components to reflect these different values. For example, it would be very interesting to add the component models used for the semi-empirical model frameworks of Mengel et al. (2016) and Nauels et al. (2017).

We demonstrated the flexibility and transparency of BRICK in connecting projections from the physical model to the impacts on a local risk and decision-analysis problem. The simple probabilistic calibration method and cost–benefit analysis that we adopted for the simple demonstration can be expanded to incorporate aspects of deep uncertainties (Lempert et al., 2004; Weaver et al., 2013) as well as more complex decision-making frameworks (e.g., considering multiple objectives, beyond only expected total costs) (Kasprzyk et al., 2013; Lempert, 2014; Lempert and Collins, 2007). Climate change poses decision problems where strong connections across academic disciplines are critical. Further, the study of climate modeling relies on communal modeling efforts. The need for transparent communication among modelers and between disciplines is where the BRICK framework and the epistemic modeling values presented here can facilitate future developments. Above all, we hope that BRICK inspires the involved communities to pay careful attention to enhance flexibility, transparency, and accessibility of modeling frameworks.

All BRICK model code is available at

Prior probability distribution ranges for the DOECLIM climate model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The priors are all uniformly distributed.

Prior probability distribution ranges for the thermal expansion model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The prior distribution for

Prior probability distribution ranges for the GIS-SIMPLE Greenland Ice Sheet model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The priors are all uniformly distributed. Due to convergence issues,

Prior probability distribution ranges for the DAIS Antarctic Ice Sheet model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. An inverse gamma prior distribution is used for

Prior probability distribution ranges for the GIC-MAGICC Glaciers and Ice Caps model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The priors are all uniformly distributed.

Prior probability distribution ranges for the GIC-SIMPLE model parameters, and median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The priors are all uniformly distributed.

Prior probability distribution ranges for the Rahmstorf (2007) global mean sea-level model parameters, and the median, 5th, and 95th quantiles of the calibrated posterior parameter distributions. The priors are all uniformly distributed.

KK initiated the study. AB and TW designed the general framework and research. TW and AB designed the initial figures and wrote the first draft. TW, AB, KR, and PA produced the major part of the coding and code testing. AS produced and interpreted the regional sea-level fingerprinting data. All contributed to the final text.

The authors declare that they have no conflict of interest.

We gratefully acknowledge Jared Oyler for guidance during code development. We thank Rob Nicholas, Chris and Bella Forest, Nancy Tuana, Robert Lempert, Gary Shaffer, and Ben Vega-Westhoff for helpful contributions. This work was partially supported by the National Science Foundation through the Network for Sustainable Climate Risk Management (SCRiM) under NSF cooperative agreement GEO-1240507 as well as the Penn State Center for Climate Risk Management. Any conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies. Any errors and opinions are, of course, those of the authors. Edited by: Olivier Marti no. 18933, olivier.marti@lsce.ipsl.fr. Reviewed by: three anonymous referees