CLASSIC is the open-source community model successor to the CLASS-CTEM modeling framework, which consists of the Canadian Land Surface Scheme (CLASS) and the Canadian Terrestrial Ecosystem Model (CTEM). CLASS and CTEM are two fully integrated model components that simulate physical and biogeochemical land surface processes, respectively. A brief outline of these two components is provided next.

CLASSIC's physical component (CLASS) divides the land surface into four possible subareas, namely bare ground, ground covered by snow, ground covered by canopy, and snow covered by canopy. The model solves the energy balance for the canopy layer and underlying ground, separately. The canopy consists of a single layer and is for the results presented here composed of four plant functional types (PFTs), namely needleleaf trees, broadleaf trees, grasses, and crops. The model output evaluated here is from the offline mode in which we use 20 soil layers ranging from 10 layers of 0.1 m thickness, gradually increasing to a 30 m thick layer for a total ground depth of over 61 m. Water fluxes are calculated for soil layers within the permeable soil depth of the ground column but not the underlying bedrock layers, whereas temperatures are calculated for both soil and bedrock layers. Also calculated are the temperature, mass, albedo, and density of the single-layer snowpack, the temperature and interception and storage of rain and snow on the vegetation canopy, and the temperature and depth of ponded water on the ground surface.

CLASSIC's biogeochemical component (CTEM) is a dynamic vegetation model coupled to CLASS. CLASS provides CTEM physical land surface information, including soil moisture, soil temperature and net radiation. CTEM uses this information to simulate photosynthesis in response to atmospheric CO2 concentration. CTEM considers three live vegetation components (leaves, stem, and roots) and two dead carbon pools (litter and soil). These five carbon pools are tracked, in the default CLASSIC configuration used here, for nine PFTs that map directly onto the four PFTs used by CLASS. Needleleaf trees are divided into their deciduous and evergreen types, broadleaf trees are divided into cold and drought deciduous and evergreen types, and crops and grasses are divided based on their photosynthetic pathways into C3 and C4 versions. In this study, the spatial distribution of PFTs has been prescribed to reduce the number of possible causes for model biases. The prognostic carbon masses of leaf, stem, and root simulated by CTEM are used to calculate the structural vegetation characteristics required by CLASS. The main processes simulated by CTEM include photosynthesis, canopy conductance, tissue turnover, allocation of carbon, and phenology ; dynamic root distribution ; maintenance, growth, and heterotrophic respiration ; wildfires ; competition for space between PFTs , and land use change . For the purpose of this study, CLASSIC's recently added nitrogen cycle has been turned off . A more detailed description of the model and its history is documented in . Details on the computation of radiation fluxes, surface albedo, heat fluxes, LAI, GPP, ecosystem respiration, and biomass are provided in the Appendix.

Meteorological inputs required by CLASSIC include surface downwelling shortwave (SW) and longwave (LW) radiation, near-surface air temperature, precipitation, near-surface specific humidity, surface pressure, and near-surface horizontal wind speed. Model outputs are potentially sensitive to uncertainties in the meteorological forcing. To account for this uncertainty, we drive CLASSIC with three different quasi-observed meteorological data sets, namely the Climate Research Unit – National Centers for Environmental Prediction version 8 (CRUNCEP; 1901–2016; ), the Climate Research Unit – Japanese 55-year Reanalysis version 2.0 (CRUJRAv2; 1901–2017; ; ), and the Global Soil Wetness Project Phase 3 (GSWP3) – WFDE5 over land merged with ERA5 over the ocean (W5E5) (GSWP3W5E5; 1901–2016; ). All meteorological forcings are disaggregated from 6-hourly to half-hourly time steps, following the methodology by .

Zonal mean values suggest that all three forcings are relatively consistent (Fig. ). The corresponding global mean values for CRUJRAv2, GSWP3W5E5, and CRUNCEP are 182, 178, and 183 W m-2 for downwelling SW radiation, 304, 309, and 300 W m-2 for downwelling LW radiation, 10 ∘C for near-surface air temperature (all three forcings), 59, 64, and 60 mm per month precipitation, 6.9, 7.4, and 6.9×10-3 kg kg-1 for specific humidity, 947, 948, and 950 hPa surface pressure, and 2.6, 3.3, and 2.6 m s-1 horizontal wind speed, respectively. With the exception of surface pressure, the data from CRUJRAv2 and GSWP3W5E5 tend to be more similar compared to CRUNCEP.

Our simulations are based on the trends in the land carbon cycle project (TRENDY) simulation protocol with time-varying CO2, climate, and land use (S3) . The protocol consists of a spinup for the year 1700 and two transient runs for the periods 1701–1900 and 1901–2016, respectively. The spinup uses a constant atmospheric CO2 concentration of 276.59 ppm, climate data from the early decades of the 20th century (i.e. 1901–1920), and a land use map with constant crops and pasture distribution. The first transient run uses the same climate as for the spinup but time-varying CO2 concentrations and land use for the 1701–1900 period. The second transient run uses time-varying CO2, climate, and land use for the 1901–2017 period.

Model outputs are evaluated against in situ observations and globally gridded observation-based reference data (Table ). The reference database covers 19 variables that form part of the energy, water, and carbon cycle. While most data sets consist of monthly mean values, some are assessed on an annual timescale (streamflow) or present a snapshot in time (vegetation biomass, organic soil carbon mass, and in situ LAI measurements provided by the Oak Ridge National Laboratory). All gridded reference data are spatially interpolated to the spatial resolution of a CLASSIC grid cell (2.8125∘×2.8125∘). In situ observations that fall into the same model grid cell are averaged prior to the comparison against model output. In most cases, the evaluation is based on the grid cell value, which reflects the average conditions across all land cover types located in a grid cell. For above-ground biomass, however, a comparison between the grid cell value and forest inventory data is not ideal, as the latter provides biomass estimates for trees only. We therefore only consider biomass simulated for trees when evaluating model results against forest inventory data. Details on each reference data set are provided next.

The Automated Model Benchmarking R package (AMBER; version 1.1.0) quantifies model performance using a skill score system originally developed by . The method employs five scores that assess the model's bias (Sbias), root-mean-square error (Srmse), seasonality (Sphase), interannual variability (Siav), and spatial distribution (Sdist). The main steps for computing a score usually include (i) computing a dimensionless statistical metric, (ii) scaling this metric onto a unit interval, and (iii) computing a spatial mean. All scores are dimensionless and range from 0 to 1, where increasing values imply better performance. These properties allow us to average skill scores across different statistical metrics in order to obtain an overall score for each variable (Soverall).

Figure  provides an overview of our experimental setup. As a first step, we create a small model ensemble by driving CLASSIC with three different meteorological data sets. The purpose of the ensemble is to determine the sensitivity of model performance to uncertainties in the meteorological forcing. Next, we evaluate each ensemble member using AMBER. The resulting model scores express how well model and reference data agree. A low score, however, does not necessarily imply poor model performance, as the lack of agreement may also be due to uncertainties in the forcing data and/or reference data. To assess to what extent model scores are affected by observational uncertainties, we also run AMBER with reference data only. The resulting benchmark scores quantify the level of agreement among different reference data sets. Comparing model scores against benchmark scores then shows how well CLASSIC performs relative to the performance of a reference data set. The remaining part of this section documents the details of the skill scores employed by AMBER.

RNS biases range from positive to negative values, depending on the choice of forcing (CRUJRAv2, GSWP3W5E5, CRUNCEP) and reference data (CERES, CLASSr, FluxCom, GEWEXSRB, and FLUXNET). For instance, global mean biases range from -15 % (-12.07 W m-2) when forcing CLASSIC with CRUNCEP and evaluating results against CERES (2000–2012), to +19 % (+12.24 W m-2) for CRUJRAv2 and CLASSr (2003–2009) (Fig. ). Evaluating model output against in situ measurements by FLUXNET yields mean biases that range from -8 % (-6.39 W m-2 for CRUNCEP) to +2 % (+1.99 W m-2 for CRUJRAv2) (Figs.  and a).

Zonal mean values in net surface radiation are in close agreement with the corresponding reference values, with biases that range from -20 to +20 W m-2 (Fig. ). The same figure shows that net surface radiation biases in the NH extratropics are negative when evaluating CLASSIC against CERES, FluxCom, and GEWEXSRB, and positive when choosing CLASSr as a reference data set. Considering all three model runs and four gridded reference data sets shows that for most grid cells annual mean biases can be both negative and positive, depending on which model run is compared against which reference data (Fig. a). The strongest annual mean bias is located in the Tibetan Plateau with values below -30 W m-2. Hovmöller diagrams show that CLASSIC captures the observed seasonality well across all latitudes (Fig. ). Monthly biases range from negative to positive values across most latitudes and months, depending on the choice of meteorological forcing and reference data. This suggests that model values are reasonably accurate across space and time when considering the uncertainties associated with the forcing and all four globally gridded reference data.

Other statistical metrics confirm that CLASSIC reproduces net surface radiation reasonably well. The relative measure of the centralized root-mean-square error (εcrmse) ranges from 0.30 to 0.49 (unitless), which is lower than for most other variables (Fig. ). Differences in the timing of modeled and observed seasonal peaks (θ) are less than 1 month. Modeled interannual variability is lower compared to CERES, CLASSr, GEWEXSRB, and FLUXNET, and larger than FluxCom (not shown). The relative interannual variability error (εiav) is reasonably small (0.30 to 0.76, unitless). CLASSIC reproduces the spatial variability of annual mean values well, where the ratio of model and reference standard deviation in space (σ) is close to unity (1.00 to 1.19). Also, the spatial correlation coefficients (R) range from 0.86 to 0.97. Averaging the resulting scores across all three ensemble members shows that Soverall ranges from 0.74 to 0.80 (unitless), depending on the reference data (Fig. a). The choice of meteorological forcing affects scores by less than 0.1 (Fig. b). Scores tend to be higher when forcing CLASSIC with GSWP3W5E5 (Fig. c).

Results for net SW radiation (RSS) and LW radiation (RLS) are mainly consistent with findings for net surface radiation described above. Net SW and LW radiation values produced by CLASSIC are reasonably accurate when considering uncertainties associated with meteorological forcing and reference data (CERES and GEWEXSRB). Global mean biases for net SW radiation range from -1 % (-1.65 W m-2) to -4 % (-5.99 W m-2) (Fig. ). For net LW radiation, global mean biases range from -17 % (-10.42 W m-2) to +6 % (+4.32 W m-2). CLASSIC captures the zonal mean values and seasonal patterns of both variables well (Figs. , , and ). However, CLASSIC also shows a systematic negative SW radiation bias in the NH extratropics from December to May (up to -40 W m-2) (Fig. ), which is consistent with a surface albedo bias as discussed further below. Results for other statistical metrics and corresponding scores are similar to the ones described for net surface radiation. For net SW radiation, εcrmse range between 0.22 and 0.38 (unitless), θ ranges from 0.38 to 0.56 months, εiav ranges from 0.11 to 0.53 (unitless), σ ranges from 0.94 to 1.13 (unitless), and R ranges from 0.92 to 0.98 (Fig. ). For net LW radiation, values in εcrmse and θ are considerably higher than for net SW radiation, where εcrmse ranges from 0.51 to 0.65 (unitless) and θ ranges from 0.63 to 0.76 months. The overall scores are higher for net SW radiation (Soverall≈0.80) than for net LW radiation (Soverall=0.72) (Fig. ). The choice of the forcing has a considerable impact on how well the model captures the interannual variability of both net SW and LW radiation. The difference between the maximum and minimum interannual variability scores (Siav) ranges from 0.10 to 0.21, depending on the reference data (Fig. b). For net LW radiation, the choice of the forcing has also a considerable impact on the bias score (ΔSbias: 0.17–0.2). CLASSIC performs best when forcing the model with GSWP3W5E5 (Fig. c).

Surface albedo (ALBS) shows a positive global mean bias across all three ensemble members and reference data sets (CERES, GEWEXSRB, and MODIS), ranging from +18 % to +42 % (+0.04 to +0.09, unitless) (Fig. ). This bias occurs mainly in the NH extratropics, with increasing values towards higher latitudes (+0.2; Figs.  and b). The strongest annual mean bias is located in the Tibetan Plateau with values above 0.3. In the NH extratropics, the bias is most evident during the cold season (+0.4 between 60 and 80∘ N from December to January) (Fig. ). Larger biases (+0.5) are present north of 80∘ N, but this involves only few grid cells. The timing of the bias suggests a possible relation with snow and/or the large solar zenith angle. The performance with respect to other statistical metrics tends to be lower than for net SW radiation. Values for εcrmse range between 0.98 and 1.12 (unitless), θ ranges from 1.67 to 2.18 months, εiav ranges from 0.38 to 0.48 (unitless), σ ranges from 1.12 to 2.04 (unitless), and R ranges from 0.77 to 0.91 (Fig. ). The corresponding overall score Soverall ranges between 0.48 and 0.59, depending on the reference data (Fig. ). The Soverall values are not very sensitive to the choice of forcing (ΔSoverall=0.01).

CLASSIC reproduces the partitioning of net surface radiation into sensible (HFSS), latent (HFLS), and soil heat flux (HFG) reasonably well. Global mean values in sensible heat flux simulated by CLASSIC are likely to be reasonable, with biases that range from negative to positive values, depending on the meteorological forcing (-23 % to +35 %, -8.33 to +10.82 W m-2) (Figs.  and ). Driving CLASSIC with CRUNCEP causes biases to be negative for all three reference data sets, while the opposite is true when running the model with CRUJRAv2. For latent heat flux, the sign of the bias is consistent across all ensemble members, with negative biases when compared against FluxCom (-12 % to -18 %, -5.35 to -8.32 W m-2) and FLUXNET (-7 % to -13 %, -3.16 to -5.83 W m-2) and reasonably small positive biases when evaluated against CLASSr (+5 % to +11 %, +1.55 to +3.67 W m-2). Biases in soil heat flux are negative across all ensemble members and both reference data sets (CLASSr and FLUXNET) (-32 % to -90 %). However, it must be noted that the corresponding absolute values are small (-0.14 to -1.31 W m-2), implying that the impact of annual mean soil heat flux biases on the surface energy balance is minor. The relative partitioning of net surface radiation into sensible and latent heat fluxes is well reproduced, with reasonably small positive and negative biases, depending on the meteorological forcing (Fig. b). A more detailed assessment of each heat flux is provided next.

CLASSIC reproduces the zonal mean patterns of sensible and latent heat flux well, with zonal mean biases of about ±20 W m-2 (Fig. ). For sensible heat flux, zonal mean biases range from positive to negative values across all latitudes, depending on the meteorological forcing (Figs.  and a). For latent heat flux, biases are primarily positive when compared against CLASSr and negative for FluxCom, regardless of the forcing (Figs.  and b). Zonal mean patterns of soil heat flux data from CLASSr are not well captured, with biases that become increasingly negative towards higher latitudes (-4 W m-2 in the Arctic) (Figs.  and a).

CLASSIC captures the seasonal cycle of all three heat fluxes reasonably well (Figs. , , and ). The negative latent heat flux biases are stronger during the warm season in both hemispheres. For soil heat fluxes, CLASSIC tends towards a more intense seasonal cycle in the extratropics, with positive biases in the warm and negative biases in the cold season.

Values of εrmse are similar among sensible and latent heat flux (0.45 to 0.77, unitless) and higher for soil heat flux (0.85 to 0.97) (Fig. ). For all three heat fluxes, time lags in seasonal peaks are close to 1 month. The interannual variability of sensible and latent heat fluxes tends to be smaller compared to the CLASSr reference data set and larger compared to FluxCom (not shown), implying that model values lie within the uncertainty of the reference data. The corresponding εiav values range from 0.25 to 2.42 (unitless), where the higher values are obtained when evaluating CLASSIC against FluxCom (Fig. ). For soil heat flux, modeled interannual variability is very similar to the corresponding reference values (not shown), leading to low values of εiav (0.28 to 0.36). Values of σ and R are reasonable for sensible heat flux (σ: 0.74 to 1.26, R: 0.78 to 0.89) and latent heat flux (σ: 0.94 to 1.08, R: 0.84 to 0.97), and poor for soil heat flux (σ: 0.15 to 0.17, R: 0.04 to 0.21). Consistently, the corresponding Soverall values are higher for latent and sensible heat flux (0.64 to 0.73) and lower for soil heat flux (0.53) (Fig. ). From all three heat fluxes, sensible heat flux is most sensitive to the choice of forcing, with ΔSoverall equal to 0.04 when driving the model with GSWP3W5E5.

Global mean values in soil moisture (MRSLL) of the top 10 cm are very close to the corresponding reference values from ESA (1979–2017), with biases ranging from -1 % (-0.28 kg m-2, CRUNCEP) to +5 % (about +1 kg m-2, CRUJRAv2 and GSWP3W5E5) (Fig. ). Zonal mean values are well reproduced with a tendency towards a positive bias north of 60∘ N (+10 kg m-2) (Fig. ). This bias is caused by clusters of grid cells located in different parts of the Arctic (Fig. b). The positive bias is present in all ensemble members and occurs from June to September (Fig. ). Other statistical metrics confirm that the performance does not differ largely among ensemble members (Fig. ). Values of εrmse and time lags of seasonal peaks (θ) are higher compared to most other variables (εrmse≤1.5, θ≈1.5 months). The spatial variability is larger than in the reference data (σ≈1.44, unitless) and correlation coefficients are relatively low (R≈0.62). The corresponding overall score is 0.52 and is not very sensitive to the choice of forcing (ΔSoverall=0.01) (Fig. ).

Global mean runoff (MRRO) biases range from +1 % (< 0.005 kg m-2 d-1, CRUNCEP) to +34 % (0.22 kg m-2 d-1, GSWP3W5E5) when evaluated against CLASSr (2003–2009) (Fig. ). This difference is consistent with the higher precipitation values found for the GSWP3W5E5 forcing. CLASSIC reproduces the zonal mean pattern of gridded reference runoff from CLASSr well, with mainly positive biases when forcing CLASSIC with GSWP3W5E5 (Fig. ). Smaller, including negative, zonal mean biases occur when forcing CLASSIC with CRUNCEP. Consistently, the sign of the bias varies for many grid cells depending on the forcing (Fig. a). Furthermore, the sign of the bias varies for most latitudes and months (Fig. ), suggesting that modeled runoff is within the uncertainty range of the meteorological forcing. CLASSIC reproduces the spatial patterns of runoff reasonably well, with σ values between 1.04 and 1.14 and R values between 0.87 and 0.88 (Fig. ). The timing of the seasonal peaks, however, differs by more than 2 months, and εrmse values exceed 1 when using CLASSr as reference data. The overall score averaged over the three ensemble members is 0.53, with a better score when driving CLASSIC with CRUNCEP and worse when using GSWP3W5E5 as a forcing data set (Fig. ).

Summing up the annual runoff from all grid cells located within each of the world's 50 largest river basins and comparing those values against annual mean streamflow at the corresponding river mouths yields biases that range between -14 % (-0.11 kg m-2 d-1, CRUNCEP, GRDC) to +20 % (+0.16 kg m-2 d-1, GSWP3W5E5, GRDC). For most basins, the sign of the bias remains constant regardless of the forcing (Fig. e for CRUJRAv2 as an example). CLASSIC reproduces the spatial patterns of streamflow reasonably well, with σ values close to 1 and R values ranging between 0.87 and 0.92 (Fig. ). The overall score averaged over the three ensemble members is 0.69. Performance is better when driving CLASSIC with CRUNCEP and worse when using GSWP3W5E5 as a forcing data set (Fig. ).

Global mean SNW biases range from -14 % (-0.49 cm, CRUNCEP-driven run evaluated against MERRA-2) to +70 % (+1.88 cm, GSWP3W5E5-driven run evaluated against Crocus) for the 1981–2017 period (Fig. ). Evaluating CLASSIC against in situ measurements compiled by shows that the mean bias can be negative (-10 %, -0.88 cm, CRUNCEP) or positive (+23 %, +2.07 cm, GSWP3W5E5), depending on the forcing (Figs.  and f). The higher snow water equivalent values in GSWP3W5E5-driven results are consistent with the higher precipitation values found for this forcing. For most evaluations, CLASSIC reproduces zonal mean values well when driven with CRUJRAv2 or CRUNCEP (Fig. ). The sign of the bias ranges from positive to negative for most grid cells (Fig. b). Positive biases tend to be larger in eastern Canada, northern Scandinavia, and northern Siberia, and during NH spring (Fig. b).

Other statistical metrics confirm that CLASSIC's ability to reproduce snow water equivalent tends to be weaker when driven with GSWP3W5E5 (εrmse: 0.55 to 0.91, θ: 0.38 to 0.77, εiav: 0.19 to 0.61, σ: 0.68 to 1.68, and R: 0.54 to 0.87) (Fig. ). The corresponding Soverall values range from 0.62 to 0.69, with lower scores when driving CLASSIC with GSWP3W5E5 and higher scores when using CRUJRAv2 (Fig. ). Site-level-based score values are similar to grid-level score values, with an overall score of 0.66.

Globally accumulated annual mean values of GPP computed from different reference data are of similar magnitude, with 115 Pg C yr-1 for FluxCom (1980–2013), 121 Pg C yr-1 for MODIS (2000–2016), and 130 Pg C yr-1 for GOSIF (2000–2017) (Table ). The corresponding model biases are reasonably small, with negative values when comparing CLASSIC against GOSIF (-2 % to -10 %), and positive values when evaluating against MODIS (<+1 % to +10 %) or FluxCom (+4 % to +14 %). An assessment with eddy covariance data shows that biases range from -6 % to -10 % (Fig. ). CLASSIC captures zonal mean GPP patterns well with no systematic bias pattern among evaluations (Fig. ). Annual mean biases range from negative to positive for most grid cells, depending on the forcing and reference data (Fig. a). Hovmöller diagrams show that CLASSIC captures the seasonal GPP cycle well across all latitudes, with no systematic biases (Fig. ). A more regional assessment, however, shows that the seasonal GPP cycle is out of phase in parts of the humid tropics of South America and Africa (not shown). Values in εrmse are less than 1 for all evaluations, and time lags of seasonal peaks are about 1 month on average (Fig. ). CLASSIC reproduces the interannual variability of GPP well when compared against FLUXNET, GOSIF, and MODIS (εiav<0.6). Higher εiav values are found when evaluating the model against FluxCom. However, it must be noted that interannual variability patterns from FluxCom are not reliable . Spatial patterns of GPP are well captured, with values of σ close to 1 and correlation coefficients around 0.91. The corresponding overall scores range from 0.58 (FluxCom) to 0.67 (MODIS), with higher scores when forcing CLASSIC with CRUJRAv2 (Fig. ).

Annual mean biases of RECO are reasonably small across FLUXNET sites, with mean biases ranging from +5 % to +12 %, depending on the forcing (Fig. ). Looking at the spatial distribution of FLUXNET sites suggests that the positive biases are more evident in the NH extratropics rather than in the tropics (Fig. b). Larger biases are present when comparing CLASSIC with FluxCom; however, those values are not reliable as FluxCom underestimates ecosystem respiration in the tropics . Other statistical metrics derived from FLUXNET data suggest that CLASSIC reproduces ecosystem respiration reasonably well (εrmse≈0.75, θ<1, εiav≈0.66, σ≈1.0, R≈0.69) (Fig. ). The corresponding overall score is 0.64 (Fig. ). This score value is insensitive to the choice of forcing.

Globally accumulated annual mean NEE computed from CT2019 is -4.2 Pg C yr-1 for the period 2000–2016 (Table ). The corresponding value from FluxCom is -24.9 Pg C yr-1 for the period 1980–2013, which is unrealistic and due to an underestimation of ecosystem respiration in the tropics . CLASSIC's estimate of annual mean NEE ranges from -4.2 to -4.6 Pg C yr-1 (2000–2016), which corresponds to a bias of about -1 % to -10 % when compared against CT2019, depending on the meteorological forcing. However, much larger biases apply when evaluating CLASSIC against FLUXNET (+88 %; Figs. c and ).

Zonal mean NEE patterns resemble values from CT2019 to a certain degree (Fig. ). The strength of the carbon sink is modeled to be stronger in the tropics (-0.15 g C m-2 d-1) and weaker in the extratropics (+0.10 g C m-2 d-1) compared to CT2019. FLUXNET data confirm the tendency towards a positive bias, i.e. weaker sink, in the NH extratropics (Fig. c). In the tropics, however, the sign of the bias is mixed, and the number of sites is too small to confirm whether or not CLASSIC overestimates the strength of the carbon sink at low latitudes.

Hovmöller diagrams suggest that CLASSIC reproduces the general pattern of the seasonal NEE cycle in the extratropics, where the strength of the carbon sink is strongest during the warm period (Fig. ). Other statistical metrics suggest a considerable mismatch between model and reference NEE (εrmse>1, θ>1, and R<0.2). The corresponding overall scores are 0.47 for FLUXNET and 0.53 for CT2019 (Fig. ).

Wildfires are estimated to burn an area of about 4.6× 106 km2 yr-1 (ESACCI from 2001–2017 and GFED4S from 2001–2015), causing emissions of about 2.9 Pg C yr-1 (CT2019, 2000–2017) (Table , where BURNT refers to area burned and FIRE refers to the associated emissions). CLASSIC reproduces the spatial extent of wildfires well considering the uncertainty of the meteorological forcing, with biases ranging from -28 % when forcing CLASSIC with CRUJRAv2, to +14 % for GSWP3W5E5. The higher fractional area burned values in GSWP3W5E5 are not related to precipitation, as GSWP3W5E5 is the wettest of the three forcing data sets across all latitudes and seasons. Instead, GSWP3W5E5 has larger values in near-surface wind speeds compared to the other forcing data sets, which increase the probability of wildfires. Model values for emissions from wildfires tend to be lower than observed, with biases ranging from -37 % (CRUJRAv2) to -7 % (GSWP3W5E5).

CLASSIC reproduces the zonal mean pattern of both variables well, with values that are highest at 10∘ S and 10∘ N (Fig. ). The sign of the zonal mean bias is consistent between both variables. Biases tend to be negative in the humid parts of the tropics and positive in the drier parts of the tropics (Figs.  and ). CLASSIC reproduces the seasonality of both variables well, with lower values when the solar zenith angle in the tropics is small (Figs.  and ), which is indicative of the position of the intertropical convergence zone. While biases are most evident in the tropics, the model also underestimates fractional area burned and associated emissions in the boreal forests. Other statistical metrics suggest that considerable differences between model and reference values remain (εrmse=1.0, θ>2 months, εiav≈1, and R≤0.51) (Fig. ). The resulting overall scores for fractional area burned and emissions from fires are 0.51 and 0.47, respectively (Fig. ). These score values are not very sensitive to the choice of forcing (ΔSoverall≤0.01).

Reference values from GEOCARBON estimate globally accumulated AGB with 214.5 Pg C. The corresponding values from CLASSIC range from 364.9 Pg C (+70 %) to 442.3 Pg C (+106 %) (Table ). The bias is largest between 20∘ N and 40∘ N, in particular in the eastern US and in East Asia (Figs.  and a). Comparing CLASSIC against vegetation carbon (i.e. above- and below-ground living biomass; CVEG) for the tropics provided by suggests smaller biases (+2 % for CRUNCEP to +33 % for CRUJRAv2) (Figs. , , and b). Spatial correlations are weak, with R≤0.62 for above-ground biomass and R≤0.71 for vegetation carbon (Fig. ). The corresponding overall scores are 0.55 (ΔSoverall=0.01) for above-ground biomass and 0.68 (ΔSoverall=0.05) for vegetation carbon in the tropics (Fig. ).

To investigate differences between CLASSIC and GEOCARBON in greater detail, we evaluate model and reference above-ground biomass values against forest inventory data (FOS). The comparison shows a larger negative bias for GEOCARBON (-50 %) and a smaller negative bias for CLASSIC (-3 % for CRUJRAv2 to -21 % for CRUNCEP) (Figs.  and d). This finding suggests that GEOCARBON may be subject to considerable uncertainties, which may contribute to the large bias when evaluating CLASSIC against GEOCARBON. More forest inventory data are required to reach a conclusive result, as the current spatial coverage of inventory plots provided by FOS is modest. Evaluating CLASSIC against FOS yields an overall score of 0.64 (Fig. ).

The global stock of soil carbon (CSOIL) is estimated with 1198 Pg C (HWSD). The corresponding values from CLASSIC range from about 956 Pg C (-20 %; CRUNCEP) to 1109 Pg C (-8 %; CRUJRAv2) (Table ). CLASSIC reproduces the zonal mean patterns of soil carbon well, with a tendency for negative biases in the Arctic (-10 kg C m-2) (Fig. ). Some of this bias is related to a small number of grid cells at high latitudes with particularly large biases (-40 kg C m-2) (Fig. a). The higher values in the reference data may be related to the presence of permafrost and peatland, which are currently not represented in CLASSIC. The spatial variability of soil carbon is well reproduced (σ≈0.92), but the spatial correlation is rather weak (R≈0.5) (Fig. ). The corresponding overall score is 0.68 (ΔSoverall=0.01) (Fig. ).

Evaluating CLASSIC with gridded LAI data from AVHRR (1982–2010) and MODIS (2000–2009) suggests large positive global mean biases that range from +31 % (+0.42 m-2 m-2) to +59 % (+0.88 m-2 m-2), depending on the forcing and reference data (Fig. ). While CLASSIC reproduces the general pattern of zonal mean LAI reasonably well, the model has a positive bias across most latitudes (Fig. ). However, biases have a local minimum at the Equator, especially when using MODIS as a reference. All six evaluations tend towards positive biases in forested regions of the NH extratropics and the drier parts of the tropics (Fig. b). In the more humid tropics of the Amazon and Congo basin, biases range from negative to positive values, depending on the forcing and reference data. Hovmöller diagrams show that the positive bias in the NH extratropics occurs mainly during the cold season, suggesting a possible relation with snow, leaf shedding, and/or the larger solar zenith angle (Fig. ). For most of the remaining months and latitudes, LAI biases range from positive to negative biases, suggesting that modeled values are reasonably accurate when considering the uncertainty of the meteorological forcing and reference data.

Other statistical metrics suggest further differences between model and reference LAI. Values of εrmse are higher than for most other variables (0.99 to 1.31), and time lags in seasonal peaks range between 1 and 2 months (Fig. ). The spatial variability of LAI is considerably larger for model data than gridded reference data (σ ranges from 1.27 to 1.66), and the correlation coefficients are reasonable but not impressive (0.7 to 0.8). The corresponding overall scores are 0.49 (ΔSoverall=0.01) and 0.53 (ΔSoverall=0.03) when evaluating CLASSIC against gridded LAI reference data from MODIS and AVHRR, respectively (Fig. ).

To investigate the apparent LAI bias further, we evaluate model and reference LAI values with in situ measurements of maximum LAI provided by ORNL. The comparisons show larger negative biases for MODIS (-37 %) and AVHRR (-33 %), and smaller negative biases for CLASSIC (-19 % to -24 %) (Figs.  and f for CLASSIC only). This suggests that (i) it is unlikely that CLASSIC overestimates maximum LAI, and (ii) the large positive LAI bias discussed above may in part be due to an underestimation of LAI by the gridded reference data set. Furthermore, evaluating CLASSIC against in situ LAI observations from CEOS shows mean biases that range from +1 % to +22 % (Figs.  and e). The corresponding values for MODIS and AVHRR are -6 % and -4 %, respectively (not shown). Given these findings, we conclude that the large positive LAI found for CLASSIC when evaluated against AVHRR and MODIS must be interpreted with caution. One possible reason for lower LAI in satellite-based reference data at high latitudes is the problems associated with LAI retrievals during winter months when the sun is low and the surface is snow covered.

Benchmark scores are produced by comparing two independent reference data sets (Fig. ). This exercise fulfills three purposes. First, benchmark scores quantify observational uncertainty, where lower values imply larger uncertainties. Second, we can evaluate the quality of gridded products using in situ measurements. Third, comparing model scores against benchmark scores shows how well CLASSIC performs relative to the performance of a reference data set. The comparison of two reference data sets is limited to the time period for which both data sets overlap.

Our results demonstrate that most benchmarks are considerably low, suggesting that observational uncertainties are large. For most variables, model skill scores are very similar to benchmark skill scores, implying that CLASSIC performs similarly to a reference data set. In some cases, CLASSIC even outperforms gridded reference data sets when evaluating gridded reference data and model output against in situ measurements. The remaining section discusses these findings in more detail.

Contrary to our expectations, most benchmark scores are far below unity (Fig. ). Only two variables show Soverall values equal to or greater than 0.8 (net surface radiation and net SW radiation). The largest uncertainties are found for NEE, with Soverall=0.49 when comparing CT2019 against FLUXNET. Even lower score values are found when evaluating FluxCom NEE against FLUXNET due to the known limitations of the FluxCom data set discussed in Sect. .

Comparing gridded reference data against in situ measurements shows subtle differences in how well gridded products perform for the same variable. For net surface radiation, Soverall values range from 0.79 for CLASSr to 0.83 for CERES when evaluated against FLUXNET. For latent heat flux, CLASSr (Soverall=0.71) performs slightly better than FluxCom (Soverall=0.68). For sensible heat flux, CLASSr and FluxCom perform equally well (Soverall=0.69). For snow water equivalent, the blended multi-reference mean data (ECCC; Soverall=0.66) perform very similarly to the individual reference data (Soverall: 0.64 to 0.66) when evaluated against in situ measurements provided by . For GPP, the Soverall values are highest for GOSIF (0.66), followed by MODIS (0.65) and FluxCom (0.63) when evaluated against FLUXNET. In the case of NEE, Soverall values are higher for FluxCom (0.56) than for CT2019 (0.49). However, it must be noted that the majority of FLUXNET sites are located in the NH extratropics, implying that corresponding scores reflect the performance in primarily non-tropical regions. For LAI, the Soverall values are 0.61 for MODIS and 0.60 for AVHRR when evaluated against in situ measurements from ORNL and 0.74 (MODIS) and 0.77 (AVHRR) when compared against data from CEOS.

In most cases, model skill score values are very similar to the corresponding benchmark score values. For instance, evaluating CERES net surface radiation against FLUXNET yields Soverall=0.83. The corresponding mean score value from all three CLASSIC ensemble members is 0.80. For a number of cases, the CLASSIC ensemble performs just as well as independent reference data sets. For instance, Soverall values for LW radiation are both 0.8 when evaluating either GEWEXSRB against CERES or the CLASSIC ensemble against CERES. In three cases (fractional area burned, LAI, and snow water equivalent), model scores and benchmark scores differ by more than 0.1, which is discussed next.

The Soverall benchmark score for fractional area burned is 0.62 (GFED4S versus ESACCI) and the corresponding model skill score is 0.51 (CLASSIC ensemble versus ESACCI). The score difference is driven by deficiencies in reproducing spatial patterns (ΔSdist=-0.27). For LAI, the Soverall benchmark score is 0.75 (AVHRR versus MODIS) and the Soverall model score is 0.49 (CLASSIC ensemble versus MODIS). This difference is primarily driven by lower model skill scores in Sbias and Srmse. However, in Sect. , we conclude that LAI biases derived from gridded reference data must be interpreted with caution. Comparing LAI benchmark scores derived from in situ measurements (AVHRR versus ORNL, Soverall=0.60) shows much greater similarity with the corresponding model skill scores (CLASSIC ensemble versus ORNL, Soverall=0.63). Similar results apply when conducting the same analysis with LAI from MODIS rather than AVHRR.

For snow water equivalent, the Soverall benchmark skill score exceeds the corresponding model skill score by 0.11 when evaluating Crocus and the CLASSIC ensemble against GS3. However, using in situ measurements yields benchmark and model skill scores that are more similar. For instance, evaluating the snow water equivalent reference ensemble ECCC and the CLASSIC ensemble against data from yields identical Soverall values (0.66).

For some variables, CLASSIC even outperforms gridded reference data when evaluated against in situ observations. This applies to net surface radiation from CLASSr, latent heat flux from FluxCom, snow water equivalent from Crocus, GS3, and MERRA-2, GPP from FluxCom and MODIS, and LAI from MODIS and AVHRR.

To rank model performance by variable, we compute the average model-benchmark score difference across all evaluations for each variable. Following this approach, model performance increases as follows, where the average model-benchmark score differences are provided in parenthesis: fractional area burned (-0.12), runoff (-0.1), soil heat flux (-0.09), LAI (-0.07), net SW radiation (-0.06), NEE (-0.05), above-ground biomass (-0.03), GPP (-0.03), surface albedo (-0.02), snow water equivalent (-0.01), net surface radiation (-0.01), sensible heat flux (-0.01), net LW radiation (0.01), latent heat flux (0.03), and ecosystem respiration (0.03). This ranking excludes the FluxCom NEE evaluation against CT2019 due to the unrealistic FluxCom NEE values in the tropics. To conclude, comparing model skill scores against benchmark scores derived from reference data suggests that model performance is reasonably good when considering the uncertainties associated with the reference data.