In order to demonstrate the validation features of LIVVkit, we analyze output from two Greenland ice sheet (GrIS) simulations from (1) a high-resolution ISM and (2) a global, fully coupled ESM from which ISM forcing is derived. Both have been presented in the literature and made available to us, which allows us to verify the software for use by others with confidence and complement existing community validation efforts.

For the high-resolution Greenland ISM simulation (1), data were generated from the Community Ice Sheet Model, version 2 (CISM2; ), coupled to the Albany/FELIX velocity solver , hereafter referred to as CISM-A. These simulations are described in detail by . This configuration is selected to highlight how LIVVkit could be used to validate a stand-alone simulation that can then be used as an initial condition for follow-on simulations. The simulation used here includes a dynamic ice sheet component forced by a time-varying surface mass balance (SMB) field. It spans the years 1991–2013, although these dates should be considered a generic present-day period because the data assimilation techniques used for the initialization used multiple datasets from the last 20 years. The initial state was generated through a multi-step procedure to produce balanced internal temperature and velocity fields, which was then forced by surface mass balance anomalies from the Regional Area Climate Model (RACMO, version 2; ). The goal in creating these simulations was to illustrate the use of the Cryospheric Model Comparison Tool (CmCt), an ice sheet model validation tool that focuses primarily on the preprocessing necessary to facilitate the comparison of satellite data to ISM output . That effort is complementary to the validation software presented here, which focuses on comparisons to in situ and remotely sensed data from other sources.

For the coupled ESM (2), we analyze output from the atmosphere and land surface components within a global, fully coupled simulation from the Community Earth System Model (CESM version 1.0.6; ). For this model, the ice sheet surface mass balance (accumulation of less runoff and sublimation) is calculated within the snowpack model of the land surface component and then downscaled to the relatively higher-resolution ice sheet grid in order to provide SMB forcing for the ISM component , as described in . Details about the simulation and the SMB are presented in (hereafter V13). Here, we focus on a historical, transient simulation spanning 1850–2005, although we restrict our analysis to 1960–2005 except where noted to avoid issues related to model spin-up. Hereafter, we refer to this simulation as CESM.

ISMs that use a data assimilation approach to most closely reproduce present-day conditions for their initial state and forcing (as compared to those that are spun up from pre-industrial or earlier states) currently use much of the most informative observational data for model initialization. As an example to move the community towards robust present-day validation, we apply LIVVkit to generate and present model–data comparisons using as many available datasets as possible and including those that are used for model initialization.

For validation of surface velocity and ice thickness, the data available for comparison are initialization data. These were obtained using a partial differential equation (PDE)-constrained optimization procedure . We include these in LIVVkit with the expectation that independent data will replace these data within the same workflow going forward. LIVVkit makes use of a GitHub repository of scripts and procedures that procures the surface velocity and thickness datasets from a host of publicly available sources. The workflow processes this dataset to create a number of fields of variables in an expected format, with all the masking and projections necessary for postprocessing in LIVVkit (explained in Sect. ). The velocity fields used for comparison originate from the NASA Making Earth System Data Records for Use in Research Environments (MEaSUREs) program, which provides annual ice-sheet-wide velocity maps for Greenland, derived using Interferometric Synthetic Aperture Radar (InSAR) data from the RADARSAT-1 satellite . The dataset currently contains ice velocity data for the winters of 2000–2001 and 2005–2006, 2006–2007, and 2007–2008 acquired from RADARSAT-1 InSAR data from the Alaska Satellite Facility (ASF), and a 2008–2009 mosaic derived from the Advanced Land Observation Satellite (ALOS) and TerraSAR-X data. For bed topography and ice thickness, we use the Greenland Ice Mapping Project digital elevation model GIMP DEM; for topography of the ice-free areas, and the bed topography and ice thickness estimates, which are derived from ice surface elevation data, airborne radar soundings from Operation IceBridge, and the GIMP DEM (as a reference surface). It is straightforward to update and/or augment these same data with new years and locations as they become available e.g.,.

In the case of coupled ESM validation for ISM development, modelers will benefit from more focused and quantitative evaluation in the vicinity of the GrIS, as compared to the global and regional model validation typically provided with the release of components that comprise ESM. Therefore, LIVVkit provides validation of the atmosphere and land surface components in ESMs specifically over the GrIS region for key variables that affect it. Validation of the atmosphere and land surface is still limited by the availability of observed data over the GrIS (and Antarctic ice sheet), whether it is used directly or within reanalysis products. As with the ice sheet data, additional regional in situ and remotely sensed data will improve LIVVkit and are a target for further development.

From LIVVkit, the examples presented in Sect.  include cloud fractions from an ESM, which are compared to the International Satellite Cloud Climatology Project (ISCCP) and the combined CloudSat radar and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar datasets (hereafter CLOUDSAT) reanalysis products. ISCCP resolves the diurnal cycle, within which monthly averages are created, and provides the longest available time record. CLOUDSAT covers only a short time record; however, the detection techniques are considered superior for the Arctic. The interested reader is referred to the Climate Data Guide for more details about the attributes and limitations of these datasets.

Given the dependence of ice sheet evolution on surface mass balance, LIVVkit tracks SMB for both coupled ESM and stand-alone ISM, even if it is provided as a forcing for a simulation for the ISM. For an ice-sheet-wide view of SMB behavior, LIVVkit presents metrics relative to the most recently available version of RACMO, version 2.3 (RACMO2.3), the characteristics of which are summarized in . The configuration of RACMO2.3 has been specifically designed and validated for its fidelity in capturing the extended Greenland region. As with initialization data, model-to-model comparisons of SMB data do not provide independent validation. This is especially true when applied to a stand-alone ISM that has been forced with model SMB, as is the case for CISM-A, which has been forced with RACMO2.0 (presented in Sect. ). We include these comparisons within LIVVkit because it is useful (1) when presented against ESM SMB and (2) to have available when monitoring other aspects of ice sheet evolution to attribute sources of bias. RACMO2.3 data are provided at approximately 11 km horizontal resolution with 40 vertical layers and include the GrIS and other nearby areas. LIVVkit is currently configured to compare years 1980–1999 of the RACMO2.3 simulation because it was forced with the more recent ERA-Interim reanalysis data from 1979 to 2014, and many of the model development runs for Earth system models cover the period spanning 1979–2000 as part of the Atmospheric Model Intercomparison Protocol (AMIP-II) .

To provide independent validation of an ESM SMB field, LIVVkit compares in situ SMB values using 623 core/pit/stake measurements from a diversity of sources. Many of these were included in Fig. 6 but have been updated in LIVVkit with new data and selection criteria. For a station to be included, it must contain a record of location, elevation, and observation start/end dates. Here, we use 387 locations out of a possible 450 from a collection of accumulation-zone (net specific SMB >0) estimates compiled by using data from , , and . In addition to the stations included from , 38 additional ice cores from the Program for Arctic Regional Climate Assessment (PARCA) have been added, with six of those as replacements to the earlier 2001 study. The PARCA compilation also includes reanalyses of time series from 20 coastal weather stations to estimate SMB; however, only estimates from ice cores and snow pits are currently included in LIVVkit. We also include data from the Programme for Monitoring of the Greenland Ice Sheet (PROMICE), a compilation of glacier explorations and in situ measurements taken since the 1960s from the GrIS ablation zone (SMB <0) . From 790 original locations, all but 198 stations were excluded based on the following criteria:

unknown elevation or start or end date of observation period;

Model results are also compared to accumulation estimates derived from 25 NASA Operation IceBridge radar airborne flights from the 2013 and 2014 seasons as detailed in and the associated supplementary data. The measurements capture internal reflecting horizons in the top few hundred meters of the ice sheet, which can be used to estimate the historical SMB record spanning the past three centuries. The IceBridge data from are provided as raw estimates seasonally for a given lat/long coordinate. Because the temporal record varies for each site, LIVVkit calculates annually averaged SMB at each location.

In order to provide basin-scale estimates of SMB, we use the drainage basin delineations as visualized by color in Fig.  (colors) and with numbering conventions preserved. This figure also illustrates the location and extent of basin-wide SMB data, including pit/core locations (filled shapes) and IceBridge altimetry transects (white lines). Accumulation-zone SMB estimates from and PARCA cores are shown as blue circles, while ablation-zone PROMICE data are shown as yellow triangles. The data points are sized by the length of their temporal record, with larger markers indicating that estimates were averaged from a greater number of annual SMB values.

Processing the IceBridge data from , which are provided as raw estimates seasonally for a given latitude/longitude coordinate, is a two-step process in LIVVkit. Each model cell is assigned to a Z12 basin. LIVVkit does this by converting the Z12 drainage basin outlines into polygons, and then decides which polygon contains which model cell centers. If a model cell center is not within any of the basin polygons, it is assigned basin “0”. Correspondingly, each IceBridge measurement at a lat/long location is averaged over seasons to obtain an annual SMB value. Some locations have older SMB records so the average annual SMB value used for comparison to the model is the mean over all available temporal records. Then, a kd-tree/nearest-neighbor method is used to find the model cell closest to the observed lat/long measurement. Thus, the Z12 basin at that location will be the same one as the corresponding model cell, found in the first step.

The present strategy to include both observational and model comparison data was chosen to maximize (1) clarity (the scientist understands the limitations of the information); (2) reproducibility (automation within LIVVkit); and (3) extensibility (users can add additional data for their own comparisons with minimal local adaption of the code). For provenance, changes to LIVVkit's inclusion of new data are controlled through releases so users can be certain of the data they are comparing for a given tag of the data repository. LIVVkit can be altered at will once the code is forked or downloaded locally to suit the user's needs. The step to process comparison data is a separate step explained in Sect. , but it is automated and fully documented. A discussion about potential new candidate data to add to the current collection is provided in Sect. . Additional details and references to the comparison data can be found on the GitHub site .

Using GrIS simulation data, processed as described in Sect. , LIVVkit produces a website with metrics and plots for validation. As an overall check of model behavior and to identify potential large-scales biases, there is a table of annualized and areal average values over the selected time record for relevant variables, which includes the SMB forcing, dimensional velocity (e.g., zonal (U) and meridional (V) velocity components, respectively), and velocity norm. Table  displays a similar version of the table produced by LIVVkit as applied to the CISM-A simulation described in Sect. .

LIVVkit displays a suite of two-dimensional contour plots of climatological averages over the selected time record. Figure  presents the surface mass balance from CISM-A, RACMO2.3, and their difference. Because the CISM-A simulation was forced with RACMO2.0 and not RACMO2.3 data, there are differences between CISM-A and RACMO2.3 that mirror the differences between the RACMO versions, in particular the southwestern and northern melt regions and over strong accumulation regions . Although for this example of SMB within CISM-A, the comparison is somewhat contrived to illustrate capability, SMB forcings can be monitored along with output variables to track their impact on the simulation.

Scatter plots comparing the same data are also displayed, whereby areas with close correspondence between the two datasets are clustered along the red identity line. As applied to CISM-A vs. RACMO2.3 for the SMB (Fig. ), generally the differences between the model and RACMO2.3 surface mass balance values are small (as expected given the origin of the SMB forcing data).

LIVVkit also presents contour figures for the GrIS climatology of top-level temperature (without comparison data; not shown), and velocity L2 norm for a stand-alone model. The norms are created for the entire GrIS and several regions of interest, which are those that highlight the Jakobshavn, Zachariae, and Petermann glaciers. Figure  is focused around drainage basins 7 and 8 (see Fig.  for reference) to include the Jakobshavn glacier.

Land ice behavior is critically dependent on various forcings from the atmosphere and land surface, so analyses of these fields within a coupled model comprise a significant portion of LIVVkit validation. LIVVkit also targets the land ice component within a coupled simulation using the same analysis procedures described in Sect.  for a stand-alone land ice model, if that component is active. As with the stand-alone analysis, LIVVkit presents area-averaged climatological fields in tabular form for a “quick look” overview applied to the CESM simulation and RACMO2.3 comparison data, as shown in Table . When applying the GrIS mask for averaging, the variables are multiplied by the percent of grid cell that is land ice, which is constant for the CESM simulation. Within the land model component, the downwelling and net shortwave (SWd and SWnet) and longwave (LWd and LWnet) radiation, respectively, the net surface radiation, Rnet, the turbulent sensible and latent heat fluxes (SHFs and LHFs), and the SMB are all summarized. From the atmosphere component, the 2 m air temperature (T2m) is presented. Here, LIVVkit produces identical results for CESM output as in V13 except for the SMB, because LIVVkit uses monthly averaged values of the ice growth/melt (QICE) field.

The variables summarized in Table  are also presented as contour plots over GrIS to highlight regional differences from RACMO2.3. Annualized and/or seasonal fields (as appropriate) from land surface and atmosphere are provided for the model, RACMO2.3, and the difference, where RACMO2.3 is interpolated to the CESM grid. The summer LWnet in CESM is presented in Fig.  as an example and shows that although the model is able to capture the general spatial variation and minimum values near the NE coast, there are overly large values (less heat loss) near the center of the land ice.

Contour and scatter plots of climatological atmospheric 2 m height temperature are also provided for polar summer (JJA) and winter (DJF) in LIVVkit; a contour plot for JJA is shown in Fig. . The near-surface temperature is critical to track in a coupled model because it drives the processes of surface melt and refreezing.

The representation of clouds over GrIS in the coupled model is also evaluated in LIVVkit because they contribute to the surface energy budget and related processes such as surface melting and refreezing . The comparisons provided by LIVVkit include the annual cycle of monthly averaged low, high, and total clouds for both the model and ISCCP and CLOUDSAT observationally derived datasets and contour plots of annual and seasonal averages of low, high, and total cloud amount. For the annual cycles, the data are area averaged over the GrIS region for each month of the climatology. Several examples of the cloud analysis as applied to CESM are displayed in Figs.  and . The ISCCP and CLOUDSAT monthly averages over GrIS in Fig.  exhibit a seasonal cycle, which is consistent with findings over the entire Arctic . However, the CESM produces total clouds that are considerably too few and capture no summer minimum, also consistent with noted CESM biases observed for the whole Arctic region . This bias is not universal for all cloud levels; Fig.  shows that the annual values of low clouds more closely match CLOUDSAT than ISCCP (although the seasonal cycle is opposite that from observations, with a summer maximum; not shown). These plots indicate the need for a deeper investigation. Recent efforts to understand the limitations of both models and observed and reanalysis datasets in representing clouds over the Arctic (e.g., ), and focused over Greenland , provide a good starting point.

Similar to Fig.  for CISM-A, LIVVkit produces a contour plot of ice-sheet-wide SMB compared to RACMO2.3 (not shown). For the coupled simulation, a times series for the area-averaged values over GrIS covering the entire range of a simulation is provided by LIVVkit, as in Fig.  for CESM. The values are calculated with the same methodology as with the SMB in Table , except for the time averaging.

To break down climatological behavior of the time series and quantify the variability, a box plot of the SMB time series is provided, as shown in Fig.  for CESM and RACMO2.3. In this plot, the rectangle spans the 25 % quartile to the 75 % quartile (the interquartile range, or IQR) of the time series with the median shown as the red line in the rectangle. The two whiskers represent 1.5× IQR above the 75 % quartile and below the 25 % quartile, respectively. The diamonds outside the whiskers are suspected outliers. In Fig. , the extent of the whiskers is similar for CESM and RACMO23, so the model data have similar variability compared to RACMO2.3. However, the slightly smaller and lower IQR box for the RACMO2.3 data indicates that its data are slightly less variable and skewed low. Given the smaller dataset size of RACMO2.3, this result is not surprising.

The increasing availability of SMB observational data outlined in Sect.  presents an opportunity to delve into the quality of simulated SMB. With this in mind, LIVVkit provides analyses of the SMB by basin and elevation using these data, applied here to the CESM SMB values that have been downscaled to 5 km within the land ice component. A LIVVkit contour plot of the CESM SMB, overlain by circles representing the observed data locations (as in Fig. ), is shown in Fig. . This figure is similar to Fig. 7 of V13 but includes more recently available data. The colors within the circles represent SMB estimates based on snow pit and/or firn/ice core studies.

To provide more quantitative comparisons, these data are also presented as a histogram of differences between modeled and observed annual surface mass balance values at pit/core locations where data exist. It is shown for the entire GrIS in Fig.  and for each of the eight basins delineated in Figs.  and . Observational data for Figs.  and  were compiled and processed by LIVVkit from the pit/core data as described in Sect. . The vertical light blue line denotes the 0 difference line for model vs. observed data; values above (below) this line indicate that the model overestimates (underestimates) SMB in comparison to altimetry observations. High frequencies near zero imply greater model agreement. Note that because this plot compares nearest-neighbor values without any spatial interpolation, one coarse model cell may be compared to multiple (if not many) observed SMB estimates, because the data collection sites are often located in clusters. The correspondence of model to observational data varies significantly by basin and shows better agreement in the central latitude regions as compared to the southern and northern regions. V13 showed lower values of SMB over region 4 for CESM relative to RACMO2.0, but in fact CESM is rather close to observations in this region. Consistent with this, RACMO version 2.3 exhibits a significant decrease in precipitation , apparently bringing its SMB closer to observations. LIVVkit also provides plots similar to those in Figs.  and  but with comparison to the IceBridge data.

Scatter plots comparing CESM to the SMB estimates from pit/core data and IceBridge areal estimates, separating accumulation and ablation zones and colored by basin, are also provided by LIVVkit. Figure , which shows pit/core data, demonstrates that CESM is currently not able to capture the SMB well in the ablation areas of basin 6 (gray, the southwest GrIS). Figure  shows this bias as well. Separating ablation from accumulation provides the source of bias: the overly positive SMB in region 6, and overall, is due to too little ablation relative to observations, not too much accumulation.

While SMB comparisons by basin are helpful, model biases in elevation can provide clues as to the source. Figure  presents CESM and observed SMB over accumulation and ablation regions vs. their elevation, and colored by basin, so that model developers are able to identify areas where there is an elevation mismatch. Because the model and observed data are not co-located, their horizontal locations are different. The observed data show a strong positive linear relationship between SMB and elevation up to about 1500 m, above which almost all points (except several in basin 1) have a positive SMB. The model is also able to capture this characteristic; however, the linear relationship is more diffuse, and the linear increase in ablation with decreasing elevation is too weak in CESM in basin 6 of the GrIS. Referring back to Fig. , the temperature gradient from the higher central part of GrIS to the edges is slightly weaker in CESM than RACMO2.3 in summer (and even more so in winter; not shown), with colder temperatures at the edges, which is consistent with the slope of elevation vs. SMB shown in Fig. .

LIVVkit breaks down the model vs. elevation data comparison along several key transects in the ablation zone, and Fig.  shows that for CESM, all four transects contain some mismatch, but the Qammanarssup Sermia and Kangerlussuaq (K) transects (basin 6) echo the overly weak modeled slope of elevation vs. SMB ablation as seen in Fig. . As for the accumulation zone, LIVVkit's processed IceBridge transect data are presented as contours by location in the GrIS along with model vs. observed data in Fig. . The CESM bias has a latitudinal gradient; SMB is overestimated (e.g., basin 2) compared to annual IceBridge estimates in the northern half of the GrIS and the bias decreases going south, becoming too low relative to the observed data in the southern half. The source of this bias could be analyzed within the context of temperature, cloud cover, precipitation, and latent and sensible heat fluxes using figures such as Figs.  and , but covering the relevant seasons, but that is beyond the scope of this survey.