Though the revised submission and softwared is an improvement, there
remains a core problem. As detailed below, there is a fundamental
contradiction in the paper that needs to be addressed. The authors
rigorously defend their inclusion of discussion on model and data
uncertainties but then ignore these uncertainties in their
metrics. For model calibration in the context inferring past ice sheet
evolution, a metric needs to take into account all uncertainties,
otherwise the resulting inferences will be invalid. And even for the
context of just data/reconstruction comparison, inferential
uncertainties (measurement, dating, downscaling). need to be accounted
for. The two RMSE metrics (equations 1 and 2) could easily be
modified to account for observational and downscaling uncertainties.
The user also needs to be provided with clear simple instructions for
modifying the metric to do so to insert their own estimates for
structural uncertainty. Without these corrections, I suspect ATAT will
foster invalid model/data based inferences and thereby do a
disservice to the community.
# detailed comments : responses and revised text
In reality, with a high-resolution ice-sheet model (say 5 km) it is
unlikely that two equally reliable dates will be contained within a
cell – radiocarbon in a core for example should just use the date that
is oldest, closest to the glacial contact.
# The expense of 5 km resolution paleo ice sheet models precludes
# their current useage for the large ensembles needed for paleomodel
# calibration. Also, in regions with high-topographic variance (eg
# most of the Greenland margin), there may easily be two relevantly
# valid dates within 5 km proximity at different elevations.
Tarasov et al. 2012 run ice sheet models that are not independent of
the dated chronology (there is a margin raster, their Fig 2, which
“nudges” the ice sheet into place based upon Dykes reconstruction).
# Not clear what you mean by "independent", the margin chronology
# (with uncertainties) is used for nudging the surface mass-balance
# within climate forcing uncertainties (nudging isn't unbounded), but
# the amount of nudging then goes into the cost function for the
# calibration.
>If this tool is meant to be used by those doing model calibration
>against paleo observations, then model uncertainty and downscaling
>error needs to be explicitly accounted for in the metric.
We have shortened the length of this discussion, but retain the
section as we think that understanding the uncertainty of the model is
important when comparing to data. Uncertainty handling will come with
ensemble design, the tool asks which ensemble member fits the data
best.
# The authors never respond to my concern above about model
# uncertainty (ie structural uncertainty). They thereby also
# contradict their own stated requirement in the text: "In order
# maximise the 68 benefit to all users, any comparisons between
# palaeo-ice sheet model output and empirical data should ideally 69
# consider the inherent uncertainties of both."
# Furthermore, if the user of ATAT is going to rely on either the
# original or current descriptions of dating and modelling uncertainty
# as their main source of understanding these critical issues, then
# the paper will be doing a disservice to the community. The main
# message should be that users need to invest the time to really
# understand these uncertainties or include a collaborator who does.
# The provision of a set of in depth appropriate references on this
# topic would therefore be of much more use than the current or past version
# cursory discussions.
Uncertainty handling will come with ensemble design, the tool asks
which ensemble member fits the data best.
# The first phrase is incorrect and the 2nd means the tool is useless
# for model calibration in the context of inferring past ice sheet evolution.
# Read Rougier, 2007 to understand what structural uncertainty means and
# why it has to be included in the likelihood function. The 2nd phrase
# what are the comparative fits of each ensemble member to the constraint
# data given all uncertainties.
It is also important keep the agree/disagree metric for the
following reason: you may do 100 simulations of a palaeo ice sheet and
keep getting the same sites that disagree.
# I don't follow the logic here. What does agree/disagree mean? Your
# responses seem to put a lot of emphasis on user choice, but here
# you are imposing a choice on what level of misfit constitutes disagreement
# instead of having the user apply their "expert judgement"...
Three classes of data are of particular use for constraining
palaeo-ice sheets; 46 (i) geomorphological data, (ii) relative sea
level history, and (iii) geochronological data
# This list is too limited. Present-day rates of vertical uplift are
# also a powerful constraint (cf my 2012 paper) for the North American
# and Eurasian ice sheets. So change RSL -> geophysical constraints
# (including RSL and present-day vertical velocities).
applying offsets derived from ice core records to contemporary climate
(Hubbard et al., 2009) and scaling 202 between present-day conditions
and uncoupled global-circulation-model simulations at maximum glacial
203 conditions (Gregoire et al., 2012; Gasson et al., 2016).
# The usual standard for example references is either a recent detailed
# review or first use. The above citations do not follow either logic.
Since all dating 390 techniques only record the absence of ice,
geochronological data provides only a one-way constraint on palaeo-391
ice sheet activity. For deglacial ages, deglaciation could occur any
time before the geochronological data provided 392 and within the
error of the date,
# Incorrect. As detailed:
Figure 3
# I am concerned about the metric indicated in this figure. It seems
# to indicate that a model with say last retreat for a given grid cell
# at say 30 ka, will be given the same score for this site as for a
# model that retreats ice at say 19 ka (for the given sample date of
# 18 +/- 1 ka). If my interpretation is correct, this needs to be
# remedied. This should be obvious for eg cosmogenic dates where
# inheritance will make the date if anything too old (and therefore
# cosmogenic dates may be maximum limiting depending on type of sample
# and location). But even for C14 dates, where the issues of sample
# availability, time required for in-migration of plants,.. mean that
# the dates are generally minimum limiting, I can't see anyone saying
# that a model with a 12 kyr misfit with a minimum limiting age should
# score the same as a model that fits the sample within sample age
# uncertainties.
# equations 1 and 2 are highly problematic given that observational
# and model uncertainties are ignored. And this again contradicts the
# authors own statements about the importance of these uncertainties. |