Articles | Volume 14, issue 1
Geosci. Model Dev., 14, 73–90, 2021
https://doi.org/10.5194/gmd-14-73-2021
Geosci. Model Dev., 14, 73–90, 2021
https://doi.org/10.5194/gmd-14-73-2021

Model experiment description paper 07 Jan 2021

Model experiment description paper | 07 Jan 2021

A multi-year short-range hindcast experiment with CESM1 for evaluating climate model moist processes from diurnal to interannual timescales

A multi-year short-range hindcast experiment with CESM1 for evaluating climate model moist processes from diurnal to interannual timescales
Hsi-Yen Ma1, Chen Zhou2, Yunyan Zhang1, Stephen A. Klein1, Mark D. Zelinka1, Xue Zheng1, Shaocheng Xie1, Wei-Ting Chen3, and Chien-Ming Wu3 Hsi-Yen Ma et al.
  • 1Lawrence Livermore National Laboratory, Livermore, CA, USA
  • 2School of Atmospheric Sciences, Nanjing University, Nanjing, China
  • 3Department of Atmospheric Sciences, National Taiwan University, Taipei, Taiwan

Correspondence: Hsi-Yen Ma (ma21@llnl.gov)

Abstract

We present a multi-year short-range hindcast experiment and its experimental design for better evaluation of both the mean state and variability of atmospheric moist processes in climate models from diurnal to interannual timescales and facilitate model development. We used the Community Earth System Model version 1 as the base model and performed a suite of 3 d hindcasts initialized every day starting at 00:00 Z from 1997 to 2012. Three processes – the diurnal cycle of clouds during different cloud regimes over the central US, precipitation and diabatic heating associated with the Madden–Julian Oscillation (MJO), and the response of precipitation, surface radiative and heat fluxes, as well as zonal wind stress to sea surface temperature anomalies associated with the El Niño–Southern Oscillation – are evaluated as examples to demonstrate how one can better utilize simulations from this experiment to gain insights into model errors and their connection to physical parameterizations or large-scale state. This is achieved by comparing the hindcasts with corresponding long-term observations for periods based on different phenomena. These analyses can only be done through this multi-year hindcast approach to establish robust statistics of the processes under well-controlled large-scale environment because these phenomena are either a result of interannual climate variability or only happen a few times in a given year (e.g., MJO, or cloud regime types). Furthermore, comparison of hindcasts to the typical simulations in climate mode with the same model allows one to infer what portion of a model's climate error directly comes from fast errors in the parameterizations of moist processes. As demonstrated here, model biases in the mean state and variability associated with parameterized moist processes usually develop within a few days and manifest within weeks to affect the simulations of large-scale circulation and ultimately the climate mean state and variability. Therefore, model developers can achieve additional useful understanding of the underlying problems in model physics by conducting a multi-year hindcast experiment.

1 Introduction

The representation of moist processes – clouds, convection, precipitation, and the associated radiative perturbations – and their interactions with the large-scale circulation in global climate models (GCMs) or Earth system models (ESMs) remains one of the grand challenges for the modeling community (Bony et al., 2015). Aside from using high-resolution cloud modeling (cloud-resolving models or large-eddy simulations) to study detailed cloud dynamics and physics and their interactions with large-scale environment, significant progress on process-level understanding of moist processes has also been achieved recently through the application of a climate model hindcast approach (Phillips et al., 2004; Williams et al., 2013) to gain insights relevant to the improvement of parameterizations in climate models. This progress has been made through either individual modeling studies (e.g., Xie et al., 2004, 2008; Klein et al., 2006; Barton et al., 2012, 2014; Medeiros et al., 2012; Hannah and Maloney 2014; Chandra et al., 2015; Van Weverberg et al., 2015; Zheng et al., 2016; 2017; Qin et al., 2018; Chen et al. 2019; Zhang et al., 2020) or coordinated model intercomparison projects (Lin et al., 2012; Williams et al., 2013; Ma et al., 2014; Klingaman et al., 2015; Xavier et al., 2015; Morcrette et al., 2018; Ma et al., 2018; Van Weverberg et al., 2018).

Earlier studies with climate model hindcast experiments usually focused on a relatively short period of time, such as those spanning Intensive Observation Periods or field campaigns. However, determining the robust aspects of certain cloud processes may not be achieved through these short simulations. Recent work in climate model studies has demonstrated the benefit of using multiple years of short-range hindcasts. For example, O'Brien et al. (2016) presented a direct comparison between observed and simulated weather events across multiple resolutions through the analysis of 5 d long hindcasts performed every day during a 5-year period. This hindcast modeling framework allows them to assess the degree to which increased resolution improves the fidelity of extreme events in the model. Further, Phillips et al. (2017) studied the land–atmosphere coupling over the US Southern Great Plains (SGP) using a suite of 16 years of short-range hindcasts as well as a free-running Atmospheric Model Intercomparison Project (AMIP; Gates, 1992) simulation for the same period in comparison with the US Department of Energy Atmospheric Radiation Measurement (ARM) observations. Although the surface climate state of the hindcasts deviates less from the observations in contrast to the AMIP simulation, they further identify that the model surface characteristics (e.g., vegetation cover) or physical parameterizations involving land–atmosphere coupling are more important factors than the performance of surface climate state in controlling the coupling behaviors. Chen et al. (2019) assessed precipitation biases in the Community Earth System Model version 1 (CESM1) during the abrupt onset of the South China Sea summer monsoon, a key precursor of the overall East Asian summer monsoon. A multi-year hindcast approach was utilized to obtain the well-constrained synoptic-scale horizontal circulation each year during the onset period. Their results highlighted the need for an appropriate representation of land–ocean convection interactions over coastal areas in order to improve the simulation of monsoon onset.

The above examples indicate the benefit of using multiple years of short-range hindcasts for robust process-level modeling studies in comparison with long-term observations. Also, evaluating ensemble short-range hindcasts with the same climate model can complement the traditional way of conducting AMIP-type model evaluation. In the present paper, we present a multi-year short-range hindcast experiment and its experimental design for better evaluating both the mean state and variability of atmospheric moist processes in climate models to facilitate model development using the CESM1 as the base model. This experiment provides a new opportunity to address several modeling issues associated with moist processes, which cannot be achieved from previous short Transpose-AMIP II hindcasts (Williams et al., 2013), or 1 or 2 years of short-range hindcasts that we conducted in the past (Xie et al., 2012; Ma et al., 2013, 2015). This is because these phenomena are either a result of interannual climate variability or only happen a few times in a given year, and thus we need multiple years to robustly quantify the errors associated with these phenomena. We will demonstrate the unique value of diagnosing systematic model errors from diurnal to interannual timescales with this suite of multi-year short-range hindcasts paired with long-term observations, such as from various satellites or from major field programs like the US Department of Energy ARM program. Process-level understanding can be achieved by comparing hindcasts with observations for periods based on the phenomena of interest rather than the climatological mean state. Three processes – the diurnal cycle of clouds during different cloud regimes at the ARM SGP site, precipitation and diabatic heating associated with the Madden–Julian Oscillation (MJO), and moist processes response to sea surface temperature (SST) anomalies associated with the El Niño–Southern Oscillation (ENSO) – are evaluated as examples to gain insights into model errors and their connection to physical parameterizations. We also demonstrate that systematic errors in the mean state of moist processes over the global scale are very robust and do not show significant interannual variations in error magnitudes or patterns over large spatial domains. Our focus of this paper is to document this multi-year experiment as a model description paper and provide examples on how to better utilize this suite of hindcasts. The remainder of this paper is organized into three sections. Section 2 describes the hindcast experiment design, experiments performed, and validation datasets. Section 3 presents three examples of how we can better utilize this suite of multi-year short-range hindcasts to evaluate the variability of moist processes over various timescales. Section 4 presents a summary.

2 Model experiments and validation data

2.1 Model and experiment design

All simulations were conducted with the CESM1 using the active atmospheric and land model components (version cesm1_0_5, FC5 compset; Neale et al., 2010). The atmospheric model component is the CAM5 with the finite volume dynamical core at a horizontal resolution of 0.9 latitude by 1.25 longitude and 30 vertical levels. The land model component is the Community Land Model version 4.0 (CLM4) with the same horizontal resolution. The ocean and sea ice components are prescribed with the National Oceanic and Atmospheric Administration (NOAA) optimum interpolation weekly SSTs and sea ice (Reynolds et al., 2002).

The hindcast procedure is based on Ma et al. (2015). In summary, we applied the horizontal velocities, temperature, specific humidity, and surface pressure from the European Centre for Medium-Range Weather Forecasts (ECMWF) ERA-Interim reanalysis (Dee et al., 2011) for the initial atmospheric states. We applied bilinear interpolation for the horizontal remapping for all the state variables to the model grid. For vertical remapping, we follow the procedure used at ECMWF when initializing model with foreign analysis: quadratic interpolation is used for temperature, linear interpolation is used for specific humidity, and a combination of linear and quadratic interpolation is used for zonal and meridional winds. To avoid spurious gravity waves associated with differences in topography between ERA-Interim and CAM5, we applied a spatial smoothing for the state variables (Gerrity and McPherson, 1970). We also adjusted the surface pressure associated with differences in topography between ERA-Interim and CAM5 using the hydrostatic approximation. A nudging simulation (horizontal velocities nudging only following Zhang et al., 2014) with CAM5/CLM4 was also performed to acquire other necessary variables (e.g., cloud and aerosol fields), which are not available from the ERA-Interim reanalysis for the atmospheric initial conditions. The nudging simulation was started from 1 January 1996 and stopped at 31 December 2012 with a 6 h relaxation timescale. During the nudging simulation, the reanalysis data are linearly interpolated between two time steps to match the model's current time.

We do not use land-surface conditions from the nudging simulation for the land-surface initial condition in the hindcasts. This is because in a nudging simulation, biased precipitation, winds, and surface fluxes are allowed to pass to the land model, which will cause larger biases in the simulated soil moisture and temperature (Ma et al., 2015). Instead, land initial conditions are taken from an offline land model simulation (I_2000 compset) forced by reanalysis and observations including precipitation, surface winds, and surface radiative fluxes (CRUNCEP, Nicolas Viovy, unpublished data) rather than coupled to an active atmospheric model. The default bilinear interpolation method is used to interpolate the forcing datasets to the CLM grid. The offline land model simulation started from 1990 to 2012, and we performed five cycles (1990 to 2012) for the offline simulation to allow proper spin-up of the land conditions. The carbon and nitrogen cycles in this particular CLM4 setup are not active. After that, we continued the offline land model simulation to the desired starting date and used the land model restart file (.r file) as the land initial condition. We have published documentation and relevant scripts in generating initial conditions on GitHub (https://github.com/PCMDI/CAPT, last access: 17 December 2020), which includes the initialization generation procedure, the nudging procedure, and land-surface spin-up procedure. The method to conduct a single hindcast is the same as performing an AMIP simulation, except we use the initial conditions from the procedure described above. The multi-year hindcast experiment is a suite of 3 d long hindcasts starting at 00:00 Z every day for the years of 1997 to 2012 (Fig. 1) using the initial conditions obtained from the procedure described above. We concatenated each hindcast from 24–48 h (48–72 h) lead time to form a pseudo day 2 (day 3) time series of 16-year duration from 1997 to 2012. Day 1 data are not analyzed to minimize the impact of model spin-up (Ma et al., 2013, 2014).

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f01

Figure 1Schematic diagram for the multi-year hindcast procedure (modified from Ma et al., 2015) applied to series of 3 d hindcasts from 1997 to 2012. Each hindcast is initialized with ERA-Interim reanalysis, and the starting time is 00:00 Z every day.

We also conducted a 16-year AMIP simulation with the same model for the same period. In this AMIP simulation, the state of the atmosphere evolves freely without constraints. Note that the nudging simulation mentioned above has the same model configuration as the AMIP simulation with the exception of the nudging terms. Also, the greenhouse gas and solar forcing is based on the setup of the CESM1 FC5 compset, which corresponds to the year 2000 level for all the simulation period. This is because the CMIP5 forcing data do not go beyond 2005. We also want to exclude the impact of the interannual variations in the solar and greenhouse gas forcings to our simulations so that we can better identify possible causes of model biases associated with parameterizations. To compare with high-temporal frequency observations collected at the ARM permanent sites as well as at various major field campaign locations within the simulation period, we have additionally generated output for every model time step (30 min interval) in addition to output for the entire global domain. Figure 2 and Table 1 identify their geographical locations and output grids. The location to obtain all the model output and necessary initial conditions to conduct the multi-year hindcasts are described in the code and data availability section.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f02

Figure 2Locations of model patch/site output associated with major field campaigns or US Department of Energy Atmospheric Radiation Measurement (ARM) sites. See Table 1 for detailed longitude and latitude information.

Table 1Locations of model patch/site output associated with major field campaigns or DOE ARM sites.

Download Print Version | Download XLSX

2.2 Strategy on performing the multi-year hindcasts on a high-performance computing system

Since each hindcast is independent and can be completed very quickly from less than half an hour to 2 h depending on the speed of the high-performance computing (HPC) system, one can easily bundle as many hindcasts as possible into one job submission on a HPC system. This is the concept of submitting multiple parallel jobs simultaneously in a single batch script. Most HPC systems nowadays have this capability and even encourage people to submit large jobs with a discount on the charge of computer hours. For example, a single hindcast takes 1 h to complete with five computer nodes. One can then request 1825 nodes for performing short-range hindcasts for a 1-year period in a single job submission. The queue time may require a longer wait but the entire hindcast will finish within an hour. One can submit multiple jobs in the queue for multiple years of hindcasts. One issue to keep in mind is the storage space for model output. There is usually a disk quota for scratch space on a HPC system. Therefore, a script or code to save model output to a long-term storage system, such as a high-performance storage system (HPSS), may be necessary to prevent reaching the scratch disk quota while the model is running.

2.3 Comparison datasets

Daily global observational precipitation is adopted from the Global Precipitation Climatology Project version 1.2 (GPCP; Adler et al., 2003). Absorbed shortwave flux at top of atmosphere (SWAbs) and outgoing longwave radiation (OLR), as well as net surface shortwave and longwave fluxes are obtained from Clouds and the Earth's Radiant Energy System (CERES) Energy Balanced And Filled (EBAF) observations (Loeb et al., 2009; Kato et al., 2013, Edition 2.8). Total cloud fraction is from the International Satellite Cloud Climatology Project (ISCCP) D2 dataset (Rossow and Schiffer, 1999). Global winds and surface turbulent heat fluxes are from the ERA-Interim reanalysis. Vertical profiles of cloud fraction at the ARM SGP site are from the ARM Best Estimate (ARMBE; Xie et al., 2010) Active Remote Sensing of Clouds data (ARSCL; Clothiaux et al., 2000, 2001). The available time period of the dataset is listed in Table 2. We interpolated all the datasets onto the model's grid for better comparison.

Table 2List of observation datasets.

Download Print Version | Download XLSX

3 Example analysis

Our goal here is to demonstrate the usefulness of the multi-year hindcasts in providing a different perspective on several long-standing moist process errors in GCMs through three examples. Note that identifying causes for individual model issue requires further investigation and is beyond the current scope of this model experiment description paper. For sensitivity tests to various parameter choices for a specific scheme or parameterization, it is not necessary to perform this suite of multi-year hindcast experiment once the issue has been identified. Instead, one could perform a “core experiment” (i.e., series of short-range hindcast over a 1-year period) as we proposed in Ma et al. (2015) or perform a set of hindcasts just for the set of key dates with the phenomena of interest (e.g., days with shallow cumulus at ARM SGP or phase 3 of various MJOs, which we will introduce in the following text).

3.1 Cloud regimes at the ARM SGP site

One common application of hindcasts for model evaluation is during major field campaigns where intensive observations are available at a very high temporal scale. However, field campaigns are usually confined to a certain short period and cannot determine the robust aspects of certain cloud processes, which are available only from long-term monitoring as provided by satellites or permanent ground-based sites. From over 10 years of cloud radar observations at the ARM SGP site, Zhang and Klein (2010) computed the diurnal cycle of cloud vertical structure for four distinct cloud regimes: daytime clear sky, daytime shallow convection, afternoon deep convection, and nighttime convection (Fig. 3a–d). They are defined as (1) daytime clear-sky day: precipitation rate of 0 mm d−1 at all hours of the day and cloud fraction  5 % at all levels between 08:00 and 16:00 local standard time (LST); (2) daytime shallow convection day: precipitation rate of 0 mm d−1 at all hours of the day, and shallow cumulus clouds are identified by Berg and Kassianov (2008), who first selected cumulus clouds based on fine-temporal-resolution ARSCL data at ARM SGP and then manually scrutinized cloud images taken by the Total Sky Imager (available online at http://www.arm.gov/instruments/tsi, last access: 17 December 2020) to eliminate low cloud types other than shallow cumulus; (3) afternoon deep convection day: the diurnal maximum hourly precipitation rate  1 mm d−1 and occurs between 15:00 and 20:00 LST and is at least twice the precipitation rate at any other hour of the day outside of 15:00–20:00 LST); and (4) nighttime deep convection day: the diurnal maximum hourly precipitation rate  1 mm d−1 and occurs between 00:00 and 07:00 LST. Each composite consists of somewhere between 79 and 229 d spanning the warm seasons for a 10-year period. We also created the same model composites (Fig. 3e–h) from the model grid box closest to the SGP site (Fig. 2) for the exact same days from the day 2 hindcasts. We use the model cloud fraction for this comparison because the variables for using a radar simulator (Zhang et al., 2018, 2019) were not saved at the time the hindcasts were done. This analysis, which cannot be achieved from the usual AMIP simulations, is a more precise method for model parameterization evaluation because it minimizes the impact of erroneous large-scale states on the clouds. This is because the atmospheric large-scale state is closer to observations during each diurnal cycle of the hindcasts than it is in the AMIP simulation. Furthermore, multi-year hindcasts provide a sufficient number of events to make a meaningful comparison with observations so that conclusions from such studies are more statistically robust.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f03

Figure 3Diurnal cycle of cloud fraction composites (%) from May to August in the years of 1997 to 2007 from (a, d) ARMBE ARSCL and (e–h) day 2 hindcasts for different convection regimes: (a, e) clear-sky regime, (b, f) fair-weather shallow cumulus regime, (c, g) late-afternoon deep convection regime, and (d, h) nighttime deep convection regime. Panels (a)(d) were modified from Fig. 3 in Zhang and Klein (2010).

In Fig. 3, the model overestimates high clouds regardless of cloud regimes, even for the clear-sky condition. For the clear-sky condition, the model also shows mid- and low-level clouds. One possible explanation is that the deep convection scheme in the model is triggered whenever the convective available potential energy (CAPE) is larger than 70 J kg−1. During the daytime in the warm seasons, CAPE is usually larger than the threshold and deep convection is easily triggered, resulting in the transport of water vapor and detrainment of cloud condensates. For the shallow convection regime, the model overestimates mid-level clouds by  4 %–6 % but underestimates shallow clouds by  10 %. For afternoon deep convective cloud regime, the model cannot simulate the transition from shallow to deep convective clouds. The deep convection clearly starts too early from around 11:00 LST rather than 15:00 LST. Also, the model underestimates both shallow and mid-level cloud fraction by  10 %. The model completely misses the nighttime convection regime and only shows some deep convection starting around noon. The too-early afternoon convection and the lack of nocturnal convection over land are common model problems as reported in previous studies (e.g., Dai 2006; Jiang et al., 2006; Covey et al., 2016). The missing nocturnal precipitation is likely related to the incapability of the model to capture elevated convection that often occurs at night at Southern Great Plains (Xie et al., 2019). We do realize that there are already small errors in the day 2 large-scale state and they can also contribute to the errors in the simulated cloud fields. Nevertheless, their impact is still much smaller compared to the errors due to parameterization deficiencies in convection.

With multi-year hindcasts and long-term cloud observations to build up robust statistics, these comparisons help identify specific cloud regime deficiencies under very similar large-scale meteorological conditions, and model developers can further focus on improving specific processes represented in the cloud and convection parameterizations.

3.2 Model biases associated with MJO

The MJO (Madden and Julian, 1971, 1972) is the dominant mode of intraseasonal variability in the tropics. The MJO has significant impacts on the global water cycle as it can interact with many weather and climate phenomena (Zhang, 2013). Nevertheless, contemporary GCMs still simulate a poor MJO including its weak amplitude and lack of eastward propagation (Jiang et al., 2015; Ahn et al., 2017). Recent studies suggest that the instability and propagation of the MJO are regulated by various feedback processes including cloud–radiation and wind–evaporation feedbacks (Sobel and Maloney, 2012, 2013; Adames and Kim, 2016; Ciesielski et al., 2017). These feedback processes may contribute to better MJO simulations if they are well represented in the GCMs. A particularly relevant process responsible for the eastward propagation of the MJO is the “pre-conditioning” process consisting of low-level moistening and shallow convective heating structure at the eastern edge of MJO deep convection (e.g., Jiang et al., 2011; Johnson and Ciesielski, 2013; Powell and Houze, 2013; Xu and Rutledge, 2014). This process destabilizes the environment encouraging subsequent development of deep convection.

As each MJO event is unique, one can take advantage of the multi-year hindcasts to composite precipitation, winds, and diabatic heating profiles based on observed MJO phases with the focus on identifying robust model biases associated with the MJO. The diabatic heating rate or apparent heating of large-scale motion system (Q1) consists of the heating due to radiation, the release of latent heat by net condensation, and vertical convergence of the vertical eddy transport of sensible heat (Yanai et al., 1973). In CESM1/CAM5, Q1 can be calculated through summing up all the tendency terms of all the diabatic processes. Figure 4 presents the observed composites of November to April 20–100 d bandpass-filtered NOAA interpolated outgoing longwave radiation (OLR) anomalies and horizontal wind anomalies at 850 hPa from ERA-Interim, as a function of the eight phases of the MJO (Wheeler and Hendon, 2004). The observed MJO shows a core of deep convection (center of negative OLR anomalies) over the Indian Ocean around 80 E associated with low-level convergence in winds during phase 2. The core of deep convection slowly propagates eastward, and the intensity of convection decreases (with OLR anomalies increasing) after the core of MJO crosses over the Maritime Continent and reaches the central Pacific (phases 6–8). Figure 5 shows composites of November to April precipitation and horizontal wind biases from day 3 hindcasts as a function of the eight phases of the MJO. We find that there is a dry bias in day 3 hindcasts over the core of deep convection (center of negative OLR anomalies in Fig. 4) associated with MJO and a wet bias to the east over the region of suppressed convection (center of positive OLR anomalies in Fig. 4) for all the phases as the MJO moves eastward. The dry bias is largest over Indian Ocean during phase 2, with a magnitude of 6 mm d−1, and the wet bias is largest over western Pacific during phase 8 with a magnitude of  5–6 mm d−1. The dry bias is usually attributed to the lack of organized convection in the model (Moncrieff et al., 2017), and the wet bias is consistent with the too frequently triggered deep convection scheme even under suppressed large-scale condition. Further, there is a persistent dry bias over Borneo and part of Sumatra, and a wet bias around the Maritime Continent for all the phases indicating a possible local effect of diurnal cycle of convection. The dry bias is more significant during phases 4 and 5 as the MJO crosses over the Maritime Continent. The 850 hPa winds show a biased low-level convergence near the Equator consistent with the excessive precipitation bias to the east over the region of suppressed convection.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f04

Figure 4Composites of November to April 20–100 d bandpass-filtered NOAA interpolated outgoing longwave radiation anomalies (color shades and contours, W m−2) and horizontal wind anomalies (vectors, m s−1) at 850 hPa from ERA-Interim, as a function of the eight phases of the MJO (Wheeler and Hendon 2004). The number of days for composites is indicated at the top of each panel. Years of analysis are from 1997 to 2012.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f05

Figure 5Composites of November to April precipitation bias (color shades, mm d−1), 850 hPa horizontal wind biases (vectors, m s−1), and anomalous 20–100 d bandpass-filtered Q1 vertical profiles (averaged over 10 S to 10 N, K d−1) from day 3 hindcasts, as a function of the eight phases of the MJO (Wheeler and Hendon 2004). The number of days comprising each composite is indicated at the top of each panel. The observed precipitation and winds are from GPCP and ERA-Interim, respectively. Only precipitation biases that are significant at the 95 % confidence level are shaded. The Q1 profiles are computed directly from the model's tendency terms with all the diabatic processes.

During phases 2 and 3 when the MJO is over the Indian Ocean, the anomalous Q1 profiles reveal that the magnitude of shallow heating is very weak (< 0.4 K d−1) to the east over the region of suppressed convection between 100 and 120 E in phase 2 and the heating is not restricted to low levels between 120 and 150 E in phase 3. Instead, there is an anomalous heating associated with deep convection in phase 3, which is not evident in the observations as indicated from many previous studies (e.g., Fig. 5a of Jiang et al., 2011). This suggests that the model fails to simulate the pre-conditioning moistening processes by shallow convection and the gradual transition from shallow to deep convection in phase 3 of day 3 hindcast composites. Figure 6 presents Hovmöller diagrams (longitude versus time in lag days) of rainfall anomalies along an equatorial band based on the lag correlation over an Indian Ocean box for both GPCP and day 3 hindcasts. The model shows an eastward propagation of precipitation anomalies associated with the MJO. However, the slightly lower correlation coefficients in the hindcasts ( 0.3 versus  0.4 in observation) east of 105 E is consistent with the weakening of MJO eastward propagation particularly over the Maritime Continent. Previous studies have identified Maritime Continent as a “barrier” for MJO propagation and most MJO episodes in the models fail to propagate across it due to several possible reasons, such as interactions of convection, clouds, surface fluxes, and local circulation within a diurnal cycle, land–sea contrast, terrain effect, or east–west low-level moisture gradient (Hagos et al., 2016; Jiang, 2017; Zhang and Ling, 2017). Although further diagnosis is required to fully understand the underlying problems for MJO propagation, our analysis with multi-year hindcasts indicates issues in the early development of biases in the shallow and deep convection associated with the MJO, as well as their interactions with the diurnal cycle of convection over the Maritime Continent.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f06

Figure 6Time–longitude anomalous rainfall correlation Hovmöller diagram along the Equator (10 S–10 N) based on GPCP and day 3 hindcasts. The rainfall anomalies associated with the MJO are derived based on lag correlation over an Indian Ocean box (5 S–5 N, 75–85 E) for northern winter (November–April) of 1997 to 2012.

Download

While we examined the composites of the MJO phases using a pseudo time series from the multi-year hindcasts, caution should be exercised when interpreting the results with the discontinuous time series. Specifically, one should avoid examining those processes mentioned above within a single short hindcast. One could, however, perform longer hindcasts, like those in Klingaman et al. (2015), to investigate how model physics interact with the large-scale environment and influence the propagation and evolution of the MJO with longer lead times.

3.3 Interannual variations of moist processes

3.3.1 Variations of moist processes associated with ENSO

Being the leading mode of interannual variability in the tropics and extratropics, ENSO has significant impact on both regional and global temperature, circulation, and moist processes through teleconnections. To gain insights into whether or not errors in the response of these fields to SST anomalies can be attributed to parameterization errors or whether errors in the circulation response to SST anomalies also contribute, one can further contrast the multi-year hindcasts with the behavior of a companion AMIP simulation with the same boundary conditions (SST and sea ice). This question cannot be addressed before with 1-year short-range hindcasts, as we proposed in Ma et al. (2015). To this end, we first selected several fields to compute their monthly anomalies and then regressed these anomalous fields onto the Niño3.4 index. Figure 7 shows the regression maps of precipitation, SWAbs, surface net flux (from atmosphere to the surface), and the surface zonal wind stress from observations, day 2 hindcasts, and the AMIP simulation (pattern statistics are shown in Table 3). The motivation for selecting these fields is because the tropical response of precipitation represents the atmospheric diabatic heating that forces circulation anomalies. On the other hand, surface radiation, turbulent heat fluxes, and wind stress provide critical heat and momentum forcings for SST anomalies and govern the ENSO behavior. The performance of these fields from an uncoupled atmospheric GCM is considered to be highly relevant for evaluation when it couples to an ocean model (Sun et al., 2006; Guilyardi et al., 2009).

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f07

Figure 7Regression maps of precipitation (mm d−1 K−1), absorbed shortwave radiation (W m−1 K−1), net surface heat flux (W m−1 K−1), surface zonal wind stress (N m−2 K−1) onto the Niño3.4 index from observations (left panels), day 2 hindcasts (middle panels), and AMIP simulation (right panels).

Download

Table 3Pattern statistics of regression maps of selected fields onto the Niño3.4 index between observations and model simulations (day 2 hindcasts or AMIP).

Download Print Version | Download XLSX

The responses of these fields from day 2 hindcasts show a better agreement both in the spatial patterns and magnitude with observations compared to the AMIP response (right column in Fig. 7). This is especially evident for precipitation, absorbed shortwave flux, and zonal wind stress over the western North Pacific, South Pacific Convergence Zone, and Indian Ocean. The remote teleconnections may be chaotic or poorly done by the model, causing a poor simulation in the AMIP mode. The large-scale state is well constrained in the hindcasts and the response of those fields to SST anomalies in Fig. 7 is far superior. This shows that remote errors are mostly the result of errors in circulation on long-term timescales, although the errors in circulation may be caused by model physics in the first place and deteriorate through feedback processes with time. This is evident as there are still biases in the hindcasts indicating problems from parameterizations in representing those responses to SST changes even over the local Niño3.4 region.

Surface net flux and zonal wind stress also show a greater change between hindcast and AMIP response compared to precipitation and SWAbs. It is reasonable for the latter two moist processes to show fewer changes as they are fast processes and the biases associated with model parameterizations usually develop within a few days of model integration (Xie et al., 2012; Ma et al., 2014). It is also reasonable for zonal wind stress to show greater change as the low-level winds are well constrained for the hindcasts. For surface net heat flux, the errors are contributed from various flux terms including radiation and turbulent heat fluxes, which are affected by both model physics and dynamics. Therefore, the net heat flux shows the lowest spatial correlation and larger root mean square errors in both hindcasts and the AMIP simulation compared to other fields.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f08

Figure 8Spatial pattern statistics of the bias errors (with respect to observations) of annual mean precipitation, ISCCP total cloud fraction (with τ> 1.3), absorbed shortwave radiation, and outgoing longwave radiation from the multi-year CAM5 hindcasts and an AMIP simulation. These statistics are illustrated with a Taylor diagram (Tayler, 2001) that shows the level of agreement of a given field to a common reference. The reference fields (REF) are the correspondent multi-year mean bias errors from the AMIP simulation. Each “x”, “+”, or “o” represents the day 2, day 3, or AMIP annual mean bias for individual years between 1997 and 2012 whenever the observations are available, and “2” and “3” represent the day 2 and day 3 hindcast mean biases averaged over all available years, respectively.

Download

3.3.2 Robustness of systematic errors

One question raised from earlier studies (Xie et al., 2012; Ma et al., 2014) of the correspondence between short- and long-timescale errors is whether systematic errors of moist processes show significant interannual variation in the mean state biases. Figure 8 shows the pattern statistics between errors from the individual annual means in the hindcasts or AMIP simulation, and errors in the 16-year mean of the AMIP simulation (the reference fields) for precipitation, total cloud fraction (from the ISCCP cloud simulator), SWAbs, and OLR. Compared to the long-term mean errors in the AMIP simulations, annual mean errors of the individual years for these fields show very similar magnitude in correlation and the normalized spatial standard deviation from the hindcasts at either time lag. This is also the case for individual AMIP years, although the correlations and standard deviations show a slightly larger spread. Compared to day 2 hindcasts, the correlations and standard deviations from day 3 hindcasts are closer to those from the AMIP simulations, indicating the bias growth toward the AMIP bias with hindcast lead time. We further find that the magnitudes of correlations for annual mean errors between individual hindcast years and the long-term AMIP simulations are not sensitive to the ENSO phase in a given year for these fields. This is also the case if seasonal means are compared (figures not shown here). These results suggest that mean errors in the moist processes are very robust and do not show significant interannual variations. Indeed, averaging the hindcast errors over many years (indicated by “2” or “3” in Fig. 8) only slightly improves the agreement with the AMIP reference field. Thus, one may identify robust model errors in the mean state from only 1 year of hindcasts with enough ensemble members (with the number of ensemble members greater than 15; Ma et al., 2014). A similar conclusion with multiple years of short AMIP-type simulations was also suggested by Wan et al. (2014). These results suggest that relatively short simulations will be effective at identifying the systematic moist process errors of a very high-resolution climate model which is too expensive to regularly perform multi-year simulations.

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f09

Figure 9Cloud metrics as defined using the ISCCP simulator in Klein et al. (2013) for day 2 and 3 hindcasts, as well as the AMIP simulation. These metrics are scalar measures of performance in simulating the space–time distribution of several cloud measures, with better performance indicated by smaller E values. ETCA measures total cloud amount, and ECTP−τ measures cloud-top pressure and optical depth in different categories of optically intermediate and thick clouds at high, middle, and low levels of the atmosphere. ESW and ELW measure the impacts on top-of-atmosphere shortwave and longwave radiation in the same categories used for ECTP−τ, respectively. The markers with error bars show the average and 1σ interannual variation in these error metrics. The square symbols are the error metrics computed by comparing model and observed monthly resolved climatological means. MODIS cloud amount for ETCA is plotted as a measure of observational uncertainty.

Download

https://gmd.copernicus.org/articles/14/73/2021/gmd-14-73-2021-f10

Figure 10Time series of monthly anomalies of global mean cloud radiative effect from CERES, day 2 and 3 hindcasts, as well as from the AMIP simulation. The correlation coefficient between the model and observation is shown in the upper right portion of each panel.

Download

It is also of interest to compare the absolute magnitude of errors in individual years to that of the long-term systematic error in the AMIP simulation. To do so, we calculated the annually averaged cloud error metrics proposed in Klein et al. (2013) in Fig. 9. These metrics are scalar measures of performance in simulating the space–time distribution of several cloud measures, with better performance indicated by smaller E values. ETCA measures the error in total cloud amount, and ECTP−τ measures the errors in the frequency of optically intermediate and thick clouds at high, middle, and low levels of the atmosphere. ESW and ELW measure the errors in the impacts on top-of-atmosphere shortwave and longwave radiation in the same cloud-top pressure and optical depth categories used for ECTP−τ, respectively. It is not surprising that the hindcasts show better performance in all the cloud metrics, as the large-scale circulation and state are not too far from the reanalysis. This is also true for the interannual variations in global mean cloud radiative effect at the top of the atmosphere (Fig. 10). We find that all the metrics and the cloud radiative effect show interannual variations, indicating that the circulation and state anomalies make a significant contribution to interannual variations, although these metrics or errors in the cloud radiative effect are not sensitive to ENSO phase. We further find that there is a larger difference between hindcasts and AMIP in the total cloud amount error metric (ETCA), implying that errors in the large-scale circulation and state make a larger contribution to errors in ETCA than cloud radiative properties (Fig. 10).

4 Summary

In this study, we present a multi-year short-range hindcast experiment and its experiment design for better evaluating both the mean state and variability of atmospheric moist processes in climate models from diurnal to interannual timescales to facilitate model development. We also demonstrate that one can obtain unique understanding on robust GCM systematic moist processes errors by diagnosing these processes with corresponding observations for periods based on different phenomena. The present experiment also demonstrates that it is now feasible to systematically evaluate climate model moist processes in deterministic weather prediction mode just as the moist processes in weather prediction models are often evaluated in analyses or reanalyses (Jakob, 1999; Yang et al., 2006). This experiment can also provide a very useful avenue to diagnose and understand critical processes regulating various climate and weather phenomena by taking advantage of detailed model output with a largely realistic representation of the large-scale state in hindcasts.

Three processes – the diurnal cycle of clouds during different cloud regimes at the ARM SGP site, precipitation and diabatic heating associated with the MJO, and the response of moist processes to ENSO SST anomalies – are evaluated as examples of using this multi-year hindcast experiment to gain insights into robust model errors and their connection to physical parameterizations and large-scale state. These analyses can only be done through this multi-year hindcast experiment to establish robust statistics of the processes under a well-controlled large-scale environment because these phenomena are either the result of interannual climate variability or only happen a few times in a given year (e.g., MJO, or cloud regime types). These comparisons identify specific model deficiencies that subsequent parameterization development should focus on. Results from the multi-year hindcasts also suggest that systematic errors in the mean state of moist processes are very robust and do not show significant interannual variation in error magnitude or patterns over a large spatial domain. Although we only showed examples relevant to moist processes, other processes related to planetary boundary layer or radiation schemes can also be examined through this suite of experiments. The proposed experiment and evaluation method also complement the existing ways of climate model evaluation, such as performing GCM simulations in the AMIP, or nudging mode. Comparison among the multi-year hindcasts, AMIP, and nudging simulations may provide more insights into the issues mentioned above.

In addition to processes indicated above, further studies on monsoon variability (e.g., South American and Asian monsoons; Chen et al., 2019), land–atmosphere interactions (Phillips et al., 2017), or detailed MJO studies with longer hindcast duration (Klingaman et al., 2015) are currently being explored with these hindcasts. As demonstrated in previous studies and here, model mean biases associated parameterized moist processes usually develop within a few days and manifest within weeks to affect the simulations of large-scale circulation and ultimately the climate mean state and variability. Therefore, model developers can achieve useful understanding of the underlying problems in model physics by conducting multiple years of hindcasts as demonstrated in the present work. Although a newer version of the CAM and CLM is now available (CAM6/CLM5), similar systematic errors associated with moist processes remain present in the latest model version. Therefore, it is still worthwhile to study these hindcasts and compare the results to hindcasts with the newer model version. In the meantime, we also plan to conduct another suite of multi-year hindcasts with the latest DOE Exascale Energy Earth System Model (E3SM; Golaz et al., 2019). We will also compare the results from E3SM to CESM1 to understand the impact of parameterization and model changes to the performance of moist processes since the atmospheric component of E3SM was originally branching from CAM5.3, which has very similar performance to CAM5 (Xie et al., 2018; Rasch et al., 2019). Note that E3SM version 1 has a new set of atmospheric physical parameterizations that are very similar to CAM6, the latest CAM. The hindcasts will also be available to the community.

Finally, the multi-year hindcast approach presented in this study is also intended to be one of the experiment protocols which will be used in the diurnal cycle of precipitation (DCP; https://portal.nersc.gov/cfs/capt/diurnal/, last access: 17 December 2020) model intercomparison project under the Global Energy and Water Exchanges (GEWEX) Global Atmospheric System Studies (GASS). This project is aimed at understanding the processes that control the diurnal and subdiurnal variations of precipitation over different climate regimes in observations and in models. The project will also identify the deficiencies and missing physics in current GCMs to gain insights for further improving the parameterization of convection.

Code and data availability

The model code is the CESM1 (cesm1_0_5, FC5 compset, F09_F09 resolution) and is available at http://www.cesm.ucar.edu/models/cesm1.0/ (last access: 17 December 2020, National Center for Atmospheric Research, 2020a). All model-necessary input files are available at https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/ (last access: 17 December 2020, National Center for Atmospheric Research, 2020b). The boundary conditions of SST and sea ice data are available at https://www.esrl.noaa.gov/psd/data/gridded/data.noaa.oisst.v2.html (last access: 17 December 2020, NOAA Physical Sciences Laboratory, 2020). The simulations are available online through NERSC Science Gateways (https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long, last access: 17 December 2020, Ma, 2020a). The initial conditions are located at https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long/IC/ (last access: 17 December 2020, Ma, 2020b). Detailed documentation for this experiment and variable list is available at https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long/CAPT_Long_output_cesm1_0_5_v5.pdf (last access: 17 December 2020, Ma et al., 2016).

Author contributions

HYM designed, performed, and analyzed the experiments, and wrote the first draft of the manuscript. SAK and SX contributed to the design of the experiments and to the interpretation of the results. CZ, YZ, MDZ, and XZ contributed to the analysis of experiments and the interpretation of the results. WTC and CMW contributed to the interpretation of the results. All coauthors contributed to the manuscript text.

Competing interests

The authors declare that they have no conflict of interest.

Acknowledgements

This study was funded by the US Department of Energy (DOE) Regional and Global Model Analysis program area, and the Atmospheric System Research and Atmospheric Radiation Measurement programs. This work was performed under the auspices of the US DOE by LLNL under contract no. DE-AC52-07NA27344.

Financial support

This research has been supported by the US Department of Energy, Office of Science (grant no. DE-AC52-07NA27344).

Review statement

This paper was edited by Christina McCluskey and reviewed by three anonymous referees.

References

Adames, Á, F. and Kim, D.: The MJO as a Dispersive, Convectively Coupled Moisture Wave: Theory and Observations, J. Atmos. Sci., 73, 913–941, https://doi.org/10.1175/JAS-D-15-0170.1, 2016. 

Adler, R. F., Huffman, G. J., Chang, A., Ferraro, R., Xie, P.-P., Janowiak, J., Rudolf, B., Schneider, U., Curtis, S., Bolvin, D., Gruber, A., Susskind, J., Arkin, P., and Nelkin, E.: The version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979–present), J. Hydrometeorol., 4, 1147–1167, 2003. 

Ahn, M.-S., Kim, D., Sperber, K. R., Kang, I.-S., Maloney, E., Waliser, D., and Hendon, H.: MJO simulation in CMIP5 climate models: MJO skill metrics and process-oriented diagnosis, Clim. Dynam., 49, 4023–4045, https://doi.org/10.1007/s00382-017-3558-4, 2017. 

Barton, N. P., Klein, S. A., Boyle, J. S., and Zhang, Y.: Arctic synoptic regimes: Comparing domain-wide Arctic cloud observations with CAM4 and CAM5 during similar dynamics, J. Geophys. Res., 117, D15205, https://doi.org/10.1029/2012JD017589, 2012. 

Barton, N. P., Klein, S. A., and Boyle, J. S.: On the contribution of longwave radiation to global climate model biases in Arctic lower tropospheric stability, J. Climate, 27, 7250–7269, https://doi.org/10.1175/JCLI-D-14-00126.1, 2014. 

Berg, L. K. and Kassianoy, E. I.: Temporal variability of fair- weather cumulus statistics at the ACRF SGP site, J. Climate, 21, 3344–3358, 2008. 

Bony, S., Stevens, B., Frierson, D. M. W., Jakob, C., Kageyama, M., Pincus, R., Shepherd, T. G., Sherwood, S. C., Siebesma, A. P., Sobel, A. H., Watanabe, M., and Webb, M. J.: Clouds, circulation and climate sensitivity, Nat. Geosci., 8, 261–268, https://doi.org/10.1038/NGEO2398, 2015. 

Chandra, A. S., Zhang, C., Klein, S. A., and Ma, H.-Y.: Low-cloud characteristics over the tropical western Pacific from ARM observations and CAM5 simulations, J. Geophys. Res.-Atmos., 120, 8953–8970, https://doi.org/10.1002/2015JD023369, 2015. 

Chen, W.-T., Wu, C.-M., and Ma, H.-Y.: Evaluating the bias of South China Sea summer monsoon precipitation associated with fast physical processes using climate model hindcast approach, J. Climate, 32, 4491–4507, https://doi.org/10.1175/JCLI-D-18-0660.1, 2019. 

Ciesielski, P. E., Johnson, R. H., Jiang, X., Zhang, Y., and Xie, S.: Relationships between radiation, clouds, and convection during DYNAMO, J. Geophys. Res.-Atmos., 122, 2529–2548, https://doi.org/10.1002/2016JD025965, 2017. 

Clothiaux, E. E., Ackerman, T. P., Mace, G. G., Moran, K. P., Marchand, R. T., Miller, M., and Martner, B. E.: Objective determination of cloud heights and radar reflectivities using a combination of active remote sensors at the ARM CART sites, J. Appl. Meteorol., 39, 645–665, 2000. 

Clothiaux, E. E., Miller, M. A., Perez, R. C., Turner, D. D., Moran, K. P., Martner, B. E., Ackerman, T. P., Mace, G. G., Marchand, R. T., Widener, K. B., Rodriguez, D. J., Uttal, T., Mather, J. H., Flynn, C. J., Gaustad, K. L., and Ermold, B.: The ARM millimeter wave cloud radars (MMCRs) and the active remote sensing of clouds (ARSCL) value added product (VAP), U.S. Department of Energy Tech. Memo. ARM VAP-002.1, 56 pp., 2001. 

Covey, C., Gleckler, P. J., Doutriaux, C., Williams, D. N., Dai, A., Fasullo, J., Trenberth, K., and Berg, A.: Metrics for diurnal cycle of precipitation: Toward routine benchmarks for climate models, J. Climate, 29, 4461–4471, 2016. 

Dai, A.: Precipitation characteristics in eighteen coupled climate models, J. Climate, 19, 4605–4630, 2006. 

Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P, Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hoìlm, E. V., Isaksen, L., Kallberg, P., Köhler, M., Matricardi, M., McNally, A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Theìpaut, J.-N., and Vitart, F.: The ERA-Interim reanalysis: configuration and performance of the data assimilation system, Q. J. Roy. Meteor. Soc., 137, 553–597, 2011. 

Gates, W. L.: AMIP: The Atmospheric Model Intercomparison Project, B. Am. Meteorol. Soc., 73, 1962–1970, 1992. 

Gerrity, J. P. and McPherson, R. D.: Noise analysis of a limited-area fine-mesh prediction model, ESSA Technical Memoranda, WBTM NMC 46, PB-191-188, 81 pp., 1970. 

Golaz, J., Caldwell, P. M., Van Roekel, L. P., Petersen, M. R., Tang, Q., Wolfe, J. D., Abeshu, G., Anantharaj, V., Asay-davis, X. S., Bader, D. C., Baldwin, S. A., Bisht, G., Bogenschutz, P. A., Branstetter, M., Brunke, M. A., Brus, S. R., Burrows, S. M., Cameron-smith, P. J., Donahue, A. S., Deakin, M., Easter, R. C., Evans, K. J., Feng, Y., Flanner, M., Foucar, J. G., Fyke, J. G., Hunke, E. C., Jacob, R. L., Jacobsen, D. W., Jeffery, N., Jones, P. W., Keen, N. D., Klein, S. A., Larson, V. E., Leung, L. R., Li, H., Lin, W., Lipscomb, W. H., Ma, P., Mccoy, R. B., Neale, R. B., Price, S. F., Qian, Y., Rasch, P. J., Eyre, J. E. J. R., Riley, W. J., Ringler, T. D., Roberts, A. F., Roesler, E. L., Salinger, A. G., Shaheen, Z., Shi, X., Singh, B., Veneziani, M., Wan, H., Wang, H., Wang, S., Williams, D. N., Wolfram, P. J., Worley, P. H., Xie, S., Yang, Y., Yoon, J.-H., Zelinka, M. D., Zender, C. S., Zeng, X., Zhang, C., Zhang, K., Zhang, Y., Zheng, X., Zhou, T., and Zhu, Q.: The DOE E3SM coupled model version 1: Overview and evaluation at standard resolution, J. Adv. Model. Earth Sy., 11, 2089–2129, https://doi.org/10.1029/2018MS001603, 2019. 

Guilyardi, E., Wittenberg, A., Fedorov, A., Collins, M., Wang, C., Capotondi, A., van Oldenborgh, G. J., and Stockdale, T.: Understanding El Niño in ocean-atmosphere general circulation models, B. Am. Meteorol. Soc., 90, 325–340, 2009. 

Hagos, S. M., Zhang, C., Feng, Z., Burleyson, C. D., DeMott, C., Kerns, B., Benedict, J., and Martini, M.: The impact of the diurnal cycle on the propagation of Madden-Julian Oscillation convection across the Maritime Continent, J. Adv. Model. Earth Sy., 8, 1552–1564, https://doi.org/10.1002/2016MS000725, 2016. 

Hannah, W. M. and Maloney, E. D.: The moist static energy budget in NCAR CAM5 hindcasts during DYNAMO, J. Adv. Model. Earth Sy., 6, 420–440, 2014. 

Jakob, C.: Cloud cover in the ECMWF reanalysis, J. Climate, 12, 947–959, 1999. 

Jiang, X.: Key processes for the eastward propagation of the Madden-Julian Oscillation based on multimodel simulations, J. Geophys. Res.-Atmos., 122, 755–770, https://doi.org/10.1002/2016JD025955, 2017. 

Jiang, X., Lau, N.-C., and Klein, S. A.: Role of eastward propagating convection systems in the diurnal cycle and seasonal mean of summertime rainfall over the U.S. Great Plains, Geophys. Res. Lett., 33, L19809, https://doi.org/10.1029/2006GL027022, 2006. 

Jiang, X., Waliser, D. E., Olson, W. S., Tao, W.-K., L'Ecuyer, T. S., Shige, S., Li, K.-F., Yung, Y. L., Lang, S., and Takayabu, Y. N.: Vertical diabatic heating structure of the MJO: Intercomparison between recent reanalyses and TRMM estimates, Mon. Weather Rev., 139, 3208–3223, 2011. 

Jiang, X., Waliser, D. E., Xavier, P. K., Petch, J., Klingaman, N. P., Woolnough, S. J., Guan, B., Bellon, G., Crueger, T., DeMott, C., Hannay, C., Lin, H., Hu, W., Kim, D., Lappen, C.-L., Lu, M.-M., Ma, H.-Y., Miyakawa, T., Ridout, J. A., Schubert, S. D., Scinocca, J., Seo, K.-H., Shindo, E., Song, X., Stan, C., Tseng, W.-L., Wang, W., Wu, T., Wu, X., Wyser, K., Zhang, G. J., Zhu, H.: Vertical structure and physical processes of the Madden-Julian oscillation: Exploring key model physics in climate simulations, J. Geophys. Res.-Atmos., 120, 4718–4748, https://doi.org/10.1002/2014JD022375, 2015. 

Johnson, R. H. and Ciesielski, P. E.: Structure and Properties of Madden–Julian Oscillations Deduced from DYNAMO Sounding Arrays, J. Atmos. Sci., 70, 3157–3179, https://doi.org/10.1175/JAS-D-13-065.1, 2013. 

Kato, S., Loeb, N. G., Fred, F. G., Doelling, D. R., Rutan, D. A., Caldwell, T. E., Yu, L., and Weller, R. A.: Surface irradiances consistent with CERES-derived top-of-atmosphere shortwave and longwave irradiances, J. Climate, 26, 2719–2740, https://doi.org/10.1175/JCLI-D-12-00436.1, 2013. 

Klein, S. A., Jiang, X., Boyle, J. S., Malyshev, S., and Xie, S.: Diagnosis of the summertime warm and dry bias over the U.S. Southern Great Plains in the GFDL climate model using a weather forecasting approach, Geophys. Res. Lett., 33, L18805, https://doi.org/10.1029/2006GL027567, 2006. 

Klein, S. A., Zhang, Y., Zelinka, M. D., Pincus, R., Boyle, J. S., and Gleckler, P. J: Are climate model simulations of clouds improving? An evaluation using the ISCCP simulator, J. Geophys. Res., 118, 1329–1342, https://doi.org/10.1002/jgrd.50141, 2013. 

Klingaman, N. P., Woolnough, S. J., Jiang, X., Waliser, D., Xavier, P. K., Petch, J., Caian, M., Hannay, C., Kim, D., Ma, H.-Y., Merryfield, W. J., Miyakawa, T., Pritchard, M., Ridout, J. A., Roehrig, R., Shindo, E., Vitart, F., Wang, H., Cavanaugh, N. R., Mapes, B. E., Shelly, A., and Zhang, G. J.: Vertical structure and physical processes of the Madden–Julian oscillation: Linking hindcast fidelity to simulated diabatic heating and moistening, J. Geophys. Res.-Atmos., 120, 4690–4717, https://doi.org/10.1002/2014JD022374, 2015. 

Lin, Y., Donner, L. J., Petch, J., Bechtold, P., Boyle J. S., Klein, S. A., Komori, T., Wapler, K., Willett, M., Xie, X., Zhao, M., Xie, S., McFarlane, S. A., and Schumacher, C.: TWP-ICE global atmospheric model intercomparison: convection responsiveness and resolution impact, J. Geophys. Res., 117, D09111, https://doi.org/10.1029/2011JD017018, 2012. 

Loeb, G. N., Wielicki, B. A., Doelling, D. R., Smith, G. L., Keyes, D. F., Kato, S., Manalo-Smith, N., and Wong, T.: Toward optimal closure of the Earth's top-of-atmosphere radiation budget, J. Climate, 22, 748–766, https://doi.org/10.1175/2008JCLI2637.1, 2009. 

Ma, H.-Y.: A multi-year short-range hindcast experiment with CESM1, Lawrence Livermore National Laboratory, HPSS archive, available at: https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long, last access: 17 December 2020a. 

Ma, H.-Y.: Initial conditions for the multi-year short-range hindcast experiment with CESM1, Lawrence Livermore National Laboratory, HPSS archive, available at: https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long/IC/, last access: 17 December 2020b. 

Ma, H.-Y., Xie, S., Boyle, J. S., Klein, S. A., and Zhang, Y.: Metrics and diagnostics for precipitation-related processes in climate model short-range hindcasts, J. Climate, 26, 1516–1534, 2013. 

Ma, H. Y., Xie, S., Klein, S. A., Williams, K. D., Boyle, J. S., Bony, S., Douville, H., Fermepin, S., Medeiros, B., Tyteca, S., and Watanabe, M.: On the correspondence between mean forecast errors and climate errors in CMIP5 models, J. Climate, 27, 1781–1798, 2014. 

Ma, H.-Y., Chuang, C. C., Klein, S. A., Lo, M.-H., Zhang, Y., Xie, S., Zheng, X., Ma, P.-L., Zhang, Y., and Phillips, T. J.: An improved hindcast approach for evaluation and diagnosis of physical processes in global climate models, J. Adv. Model. Earth Sy., 7, 1810–1827, https://doi.org/10.1002/2015MS000490, 2015. 

Ma, H.-Y., Klein, S. A., and Xie, S.: Documentation for Multi-year (1997–2012) CAPT Hindcast Output, Lawrence Livermore National Laboratory, available at: https://portal.nersc.gov/archive/home/h/hyma/www/CAPT/CAPT_Long/CAPT_Long_output_cesm1_0_5_v5.pdf (last access: 17 December 2020), 2016. 

Ma, H. Y., Klein, S. A., Xie, S., Zhang, C., Tang, S., Tang, Q., Morcrette, C. J., Van Weverberg, K., Petch, J., Ahlgrimm, M., Berg, L. K., Cheruy, F., Cole, J., Forbes, R., Gustafson Jr., W. I., Huang, M., Liu, Y., Merryfield, W., Qian, Y., Roehrig, R., and Wang, Y.-C.: CAUSES: On the role of surface energy budget errors to the warm surface air temperature error over the Cen- tral United States, J. Geophys. Res.-Atmos., 123, 2888–2909, https://doi.org/10.1002/2017JD027194, 2018. 

Madden, R. A. and Julian, P. R.: Detection of a 40–50 day oscillation in the zonal wind in the tropical Pacific, J. Atmos. Sci., 28, 702–708, 1971. 

Madden, R. A. and Julian, P. R.: Description of global-scale circulation cells in the tropics with a 40–50 day period, J. Atmos. Sci., 29, 1109–1123, 1972. 

Medeiros, B., Williamson, D. L., Hannay, C., and Olson, J. G.: Southeast Pacific stratocumulus in the Community Atmosphere Model, J. Climate, 25, 6175–6192, 2012. 

Moncrieff, M. W., Liu, C., and Bogenschutz, P.: Simulation, modeling, and dynamically based parameterization of organized tropical convection for global climate models, J. Atmos. Sci., 74, 1363–1380, 2017. 

Morcrette, C. J., Van Weverberg, K., Ma, H. Y., Ahlgrimm, M., Bazile, E., Berg, L. K., Cheng, A., Cheruy, F., Cole, J., Forbes, R., and Gustafson Jr., W. I.: Introduction to CAUSES: Description of weather and climate models and their near-surface temperature errors in 5 day hindcasts near the Southern Great Plains, J. Geophys. Res.-Atmos., 123, 2655–2683, https://doi.org/10.1002/2017JD027199, 2018. 

National Center for Atmospheric Research (NCAR): CESM Models, UCAR/NCAR, Boulder, CO, available at: http://www.cesm.ucar.edu/models/cesm1.0/, last access: 17 December 2020a. 

National Center for Atmospheric Research (NCAR): CESM Subversion input data repository, UCAR/NCAR, Boulder, CO, available at: https://svn-ccsm-inputdata.cgd.ucar.edu/trunk/inputdata/, last access: 17 December 2020b. 

Neale, R. B., Chen, C. C., Gettelman, A., Lauritzen, P. H., Park, S., Williamson, D. L., Conley, A. J., Garcia, R., Kinnison, D., Lamarque, J. F., and Marsh, D.: Description of the NCAR community atmosphere model (CAM 5.0), NCAR Tech. Note NCAR/TN-486+ STR, 2010. 

NOAA Physical Sciences Laboratory: NOAA Optimum Interpolation (OI) Sea Surface Temperature (SST) V2, NOAA/OAR/ESRL PSL, Boulder, Colorado, USA, available at: https://www.esrl.noaa.gov/psd/data/gridded/data.noaa.oisst.v2.html, last access: 17 December 2020. 

O'Brien, T. A., Collins, W. D., Kashinath, K., Rübel, O., Byna, S., Gu, J., Krishnan, H., and Ullrich, P. A.: Resolution dependence of precipitation statistical fidelity in hindcast simulations, J. Adv. Model. Earth Sy., 8, 976–990, https://doi.org/10.1002/2016MS000671, 2016. 

Phillips, T. J., Potter, G. L., Williamson, D. L., Cederwall, R. T., Boyle, J. S., Fiorino, M., Hnilo, J. J., Olson, J. G., Xie, S., and Yio, J. J.: Evaluating parameterizations in general circulation models: Climate simulation meets weather prediction, B. Am. Meteorol. Soc., 85, 1903–1915, 2004. 

Phillips, T. J., Klein, S. A., Ma, H.-Y., Tang, Q., Xie, S., Williams, I. N., Santanello, J. A., Cook, D. R., and Torn, M. S.: Using ARM Observations to Evaluate Climate Model Simulations of Land-Atmosphere Coupling on the U.S. Southern Great Plains, J. Geophys. Res.-Atmos., 122, 11524–11548. https://doi.org/10.1002/2017JD027141, 2017. 

Powell, S. W. and Houze, R. A.: The cloud population and onset of the Madden-Julian Oscillation over the Indian Ocean during DYNAMO-AMIE, J. Geophys. Res.-Atmos., 118, 11979–11995, https://doi.org/10.1002/2013JD020421, 2013. 

Qin, Y., Lin, Y., Xu, S., Ma, H.-Y., and Xie, S.: A diagnostic PDF cloud scheme to improve subtropical low clouds in NCAR Community Atmosphere Model (CAM5), J. Adv. Model. Earth Sys., 10, 320–341, 2018. 

Rasch, P., Xie, S., Ma, P.-L., Lin, W., Wang, H., Tang, Q., Burrows, S., Caldwell, P., Zhang, K., Easter, R., Cameron-Smith, P., Singh, B., Wan, H., Golaz, J.-C., Harrop, B., Roesler, E., Bacmeister, J., Larson, V., Evans, K., Qian, Y., Taylor, M., Leung, R., Zhang, Y., Brent, L., Branstettor, M., Hannay, C., Mahajan, S., Mametjanov, A., Neale, R., Richter, J., Yoon, J., Zender, C., Bader, D., Flanner, M., Foucar, J., Jacob, R., Keen, N., Klein, S., Liu, X., Salinger, A., Shrivastava, M., and Yang, Y.: An Overview of the Atmospheric Component of the Energy Exascale Earth System Model, J. Adv. Model. Earth Syst., 11, 2377–2411, https://doi.org/10.1029/2019MS001629, 2019. 

Reynolds, R. W., Rayner, N. A., Smith, T. M., Stokes, D. C., and Wang, W.: An improved in situ and satellite SST analysis for climate, J. Climate, 15, 1609–1625, 2002. 

Rossow, W. B. and Schiffer, R. A.: Advances in Understanding Clouds from ISCCP, B. Am. Meteorol. Soc., 80, 2261–2288, 1999. 

Sobel, A. and Maloney, E.: An Idealized Semi-Empirical Framework for Modeling the Madden–Julian Oscillation, J. Atmos. Sci., 69, 1691–1705, https://doi.org/10.1175/jas-d-11-0118.1, 2012. 

Sobel, A. and Maloney, E.: Moisture Modes and the Eastward Propagation of the MJO, J. Atmos. Sci., 70, 187–192, https://doi.org/10.1175/Jas-D-12-0189.1, 2013. 

Sun, D.-Z., Thang, T., Covey, C., Klein, S. A., Collins, W. D., Hack, J. J., Kiehl, J. T., Meehl, G. A., Held, I. M., and Suarez, M.: Radiative and dynamical feedbacks over the equatorial cold tongue: Results from nine atmospheric GCMs, J. Climate, 19, 4059–4074, 2006. 

Taylor, K. E.: Summarizing multiple aspects of model performance in a single diagram, J. Geophys. Res., 106, 7183–7192, 2001. 

Van Weverberg, K., Morcrette, C. J., Ma, H.-Y., Klein, S. A., and Petch, J. C.: Using regime analysis to identify the contribution of clouds to surface temperature errors in weather and climate models, Q. J. Roy. Meteor. Soc., 141, 3190–3206, https://doi.org/10.1002/qj.2603, 2015. 

Van Weverberg, K., Morcrette, C. J., Petch, J., Klein, S. A., Ma, H. Y., Zhang, C., Xie, S., Tang, Q., Gustafson Jr., W. I., Qian, Y., and Berg, L. K.: CAUSES: Attribution of surface radia- tion biases in NWP and climate models near the U.S. South- ern Great Plains, J. Geophys. Res.-Atmos., 123, 3612–3644, https://doi.org/10.1002/2017JD027188, 2018. 

Wan, H., Rasch, P. J., Zhang, K., Qian, Y., Yan, H., and Zhao, C.: Short ensembles: an efficient method for discerning climate-relevant sensitivities in atmospheric general circulation models, Geosci. Model Dev., 7, 1961–1977, https://doi.org/10.5194/gmd-7-1961-2014, 2014. 

Wheeler, M. C. and Hendon, H. H.: An all-season real-time multivariate MJO index: Development of an index for monitoring and prediction, Mon. Weather Rev., 132, 1917–1932, 2004. 

Williams, K. D., Bodas-Salcedo, A., Déqué, M., Fermepin, S., Medeiros, B., Watanabe, M., Jakob, C., Klein, S. A., Senior, C. A., and Williamson, D. L.: The Transpose-AMIP II Experiment and Its Application to the Understanding of Southern Ocean Cloud Biases in Climate Models, J. Climate, 26, 3258–3274, 2013. 

Xavier, P. K., Petch, J. C., Klingaman, N. P., Woolnough, S. J., Jiang, X., Waliser, D. E., Caian, M., Cole, J., Hagos, S. M., Hannay, C., and Kim, D.: Vertical structure and physical processes of the Madden-Julian Oscillation: Biases and uncertainties at short range, J. Geophys. Res.-Atmos., 120, 4749–4763, https://doi.org/10.1002/2014JD022718, 2015. 

Xie, S., Zhang, M. H., Boyle, J. S., Cederwall, R. T., Potter, G. L., and Lin, W. Y.: Impact of a revised convective triggering mechanism on CAM2 model simulations: results from short-range weather forecasts, J. Geophys. Res., 109, D14102, https://doi.org/10.1029/2004JD004692, 2004. 

Xie, S., Boyle, J. S., Klein, S. A., Liu, X., and Ghan, S.: Simulations of Arctic mixed-phase clouds in forecasts with CAM3 and AM2 for M-PACE, J. Geophys. Res., 113, D04211, https://doi.org/10.1029/2007JD009225, 2008. 

Xie, S., Ma, H.-Y., Boyle, J. S., Klein, S. A., and Zhang, Y.: On the correspondence between short- and long- timescale systematic errors in CAM4/CAM5 for the Years of Tropical Convection, J. Climate, 25, 7937–7955, 2012. 

Xie, S., Lin, W., Rasch, P. J., Ma, P.-L., Neale, R., Larson, V. E., Qian, Y., Bogenschutz, P. A., Caldwell, P., Cameron-Smith, P., Golaz, J.-C., Mahajan, S., Singh, B., Tang, Q., Wang, H., Yoon, J.-H., Zhang, K., and Zhang, Y.: Understanding cloud and convective characteristics in version 1 of the E3SM atmosphere model, J. Adv. Model. Earth Sy., 10, 2618–2644, https://doi.org/10.1029/2018MS001350, 2018. 

Xie, S., Wang, Y.-C., Lin, W., Ma, H.-Y., Tang, Q., Tang, S., Zheng, X., Golaz, J.-C., Zhang, G., and Zhang, M.: Improved Diurnal Cycle of Precipitation in E3SM with a Revised Convective Triggering Function, J. Adv. Model. Earth Sy., 11, 2290–2310, https://doi.org/10.1029/2019MS001702, 2019. 

Xie, S. C., McCoy, R. B., Klein, S. A., Cederwall, R. T., Wiscombe, W. J., Clothiaux, E. E., Gaustad, K. L., Golaz, J. C., Hall, S. D., Jensen, M. P., Johnson, K. L., Lin, Y. L., Long, C. N., Mather, J. H., McCord, R. A., McFarlane, S. A., Palanisamy, G., Shi, Y., and Turner, D. D. D.: Arm Climate Modeling Best Estimate Data a New Data Product for Climate Studies, B. Am. Meteorol. Soc., 91, 13–20, https://doi.org/10.1175/2009bams2891.1, 2010. 

Xu, W. and Rutledge, S. A.: Convective characteristics of the Madden–Julian Oscillation over the Central Indian Ocean observed by shipborne radar during DYNAMO, J. Atmos. Sci., 71, 2859–2877, 2014. 

Yanai, M., Esbensen, S., and Chu, J.-H.: Determination of bulk properties of tropical cloud clusters from large-scale heat and moisture budgets, J. Atmos. Sci., 30, 611–627, 1973. 

Yang, F., Pan, H., Krueger, S. K., Moorthi, S., and Lord, S. J.: Evaluation of the NCEP Global Forecast System at the ARM SGP site, Mon. Weather Rev., 134, 3668–3690, 2006. 

Zhang, C.: Madden–Julian Oscillation: Bridging weather and climate, B. Am. Meteorol. Soc., 94, 1849–1870, https://doi.org/10.1175/bams-d-12-00026.1, 2013. 

Zhang, C. and Ling, J.: Barrier effect of the Indo-Pacific Maritime Continent on the MJO: Perspectives from tracking MJO precipitation, J. Climate, 30, 3439–3459, 2017. 

Zhang, K., Wan, H., Liu, X., Ghan, S. J., Kooperman, G. J., Ma, P.-L., Rasch, P. J., Neubauer, D., and Lohmann, U.: Technical Note: On the use of nudging for aerosol–climate model intercomparison studies, Atmos. Chem. Phys., 14, 8631–8645, https://doi.org/10.5194/acp-14-8631-2014, 2014. 

Zhang, M., Xie, S., Liu, X., Lin, W., Zhang K., Ma, H.-Y., Zheng, X., and Zhang, Y.: Toward Understanding the Simulated Phase Partitioning of Arctic Single-Layer Mixed-Phase Clouds in E3SM, Earth Space Sci., 7, e2020EA001125, https://doi.org/10.1002/essoar.10502164.1, 2020. 

Zhang, Y. and Klein, S. A.: Mechanisms affecting the transition from shallow to deep convection over land: Inferences from observations of the diurnal cycle collected at the ARM Southern Great Plains Site, J. Atmos. Sci., 67, 2943–2959, 2010. 

Zhang, Y., Xie, S., Klein, S. A., Marchand, R., Kollias, P., Clothiaux, E. E., Lin, W., Johnson, K., Swales, D., Bodas-Salcedo, A., Tang, S., Haynes, J. M., Collis, S., Jensen, M., Bharadwaj, N., Hardin, J., and Isom, B.: The ARM cloud radar simulator for global climate models: Bridging field data and climate models, B. Am. Meteorol. Soc., 99, 21–26, https://doi.org/10.1175/BAMS-D-16-0258.1, 2018. 

Zhang, Y., Xie, S., Lin, W., Klein, S. A., Zelinka, M. D., Ma, P.-L., Rasch, P. J., Qian, Y., Tang, Q., and Ma, H.-Y.: Evaluation of clouds in version 1 of E3SM Atmosphere Model with satellite simulators, J. Adv. Model. Earth Sy., 11, 1253–1268, https://doi.org/10.1029/2018MS001562, 2019.  

Zheng, X., Klein, S. A., Ma, H.-Y., Bogenschutz, P., Gettelman, A., and Larson, V. E.: Assessment of marine boundary layer cloud simulations in the Community Atmosphere Model with Cloud Layers Unified By Binormals and updated microphysics scheme based on ARM observations from the Azores, J. Geophys. Res.-Atmos., 121, 8472–8492, https://doi.org/10.1002/2016JD025274, 2016. 

Zheng, X., Klein, S. A., Ma, H.-Y., Caldwell, P. M., Larson, V. E., Gettelman, A., and Bogenschutz, P.: A cloudy planetary boundary layer oscillation arising from the coupling of turbulence with precipitation in climate simulations, J. Adv. Model. Earth Sy., 9, 1973–1993, https://doi.org/10.1002/2017MS000993, 2017. 

Download
Short summary
We propose an experimental design of a suite of multi-year, short-term hindcasts and compare them with corresponding observations or measurements for periods based on different weather and climate phenomena. This atypical way of evaluating model performance is particularly useful and beneficial, as these hindcasts can give scientists a robust picture of modeled precipitation, and cloud and radiation processes from their diurnal variation to year-to-year variability.