Advances in supercomputing power continue to enable refinements in the resolutions of general circulation models (GCMs) used to simulate the effects of climate variability and anthropogenic climate change. As GCMs have become better able to resolve regional-scale boundary features (e.g., orography, coastlines), the scientific community has paid increasing attention to these models' representations of local and regional hydrological extremes e.g.,, including the sensitivity of those extremes to climate change e.g.,. Robust projections of local and regional changes in extremes with anthropogenic warming are essential to underpin decisions on adaptation strategies; accurate predictions of these extremes in response to natural climate variability are critical for preserving lives and livelihoods, for example, through emergency response and anticipatory aid efforts, particularly on sub-seasonal–seasonal scales.

Despite refinements in resolution and efforts to revise the treatment of sub-gridscale processes, such as deep convection, climate models are criticized routinely for their inability to represent the observed frequency, intensity and persistence of precipitation. compared daily precipitation in 18 GCMs from the Third Coupled Model Intercomparison Project (CMIP3) against satellite-derived analyses from the Tropical Rainfall Measuring Mission (TRMM) dataset across 50∘ S–50∘ N. The CMIP3 models produced precipitation too frequently, particularly light precipitation (< 10 mm day-1), but did not produce heavy precipitation (> 20 mm day-1) frequently enough. Models performed similarly poorly when compared against gridded gauge data over land . obtained similar results at the sub-daily scale, demonstrating that 30 min averaged rainfall (sampled every 3 h) from the Geophysical Fluid Dynamics Laboratory model was biased towards low intensities relative to TRMM. Revisions to the convective parameterization, particularly the closure and the triggering function, increased heavy precipitation frequency and reduced light precipitation frequency. employed observations from the CloudSat spaceborne cloud-profiling radar to show that although contemporary GCMs produced reasonable seasonal and annual precipitation accumulations, these accumulations arose from highly biased daily precipitation distributions; models produced precipitation far too frequently and far too lightly. The strong preference for persistent, light daily accumulations led the authors to call the GCMs' simulated world “dreary”. Such biases lead to erroneously large moisture recycling over land, with consequences for the simulation of the global hydrological cycle e.g.,.

More recently, found that GCMs from the Fifth Coupled Model Intercomparison Project (CMIP5) had improved somewhat in their daily precipitation distributions relative to their CMIP3 counterparts, particularly through an increase in the frequency of intense precipitation and a reduction in the overall frequency of precipitation. showed improved skill for 1–5 day precipitation extremes in CMIP5 relative to CMIP3. However, concluded that CMIP5 GCMs still produced 3 h rain rates of 1–10 mm day-1 too frequently over the southeastern United States, compared to gridded gauge data. When the models did produce heavier events, those events were too persistent.

Although the studies above highlight a heightened focus on the GCM representations of hydrological extremes – which are inherently small-scale, short-lived features – most evaluation of GCM precipitation focuses on gross spatial (e.g., regional averages) and temporal (e.g., monthly and seasonal means) characteristics e.g.,. Where attention is paid to shorter-term variability, studies have adopted a phenomenological approach, analyzing precipitation associated with synoptic features, such as mesoscale fronts and convective systems e.g.,, or sub-seasonal modes, such as the Madden–Julian Oscillation MJO; e.g.,. Yet the processes that produce precipitation in GCMs – the interactions between the sub-gridscale parameterizations and the resolved dynamics – function on the native gridscale and time step of the models, not on a 3 h or daily mean basis or on a regional average. Although it is often hypothesized that biases in the distributions of spatially and/or temporally averaged precipitation are the result of errors at the gridpoint, time step level, few studies have examined the spatial and temporal characteristics of precipitation at these most fundamental scales. In isolated single-column model experiments, convective parameterizations have been shown to produce highly intermittent time step precipitation e.g.,, but it is not clear how, or even if, this behavior influences the distributions of precipitation at larger and longer scales. Information about the spatial and temporal characteristics of gridscale precipitation are far more useful for informing parameterization development than information about regional biases in seasonal, or even daily, accumulations.

The dearth of studies focused on the gridscale and time step may be due to a lack of data, since large GCM intercomparison efforts such as CMIP5 do not collect time step output to limit the volume of data produced. However, a recent model-evaluation project focused on the MJO collected time step data from 10 GCMs for a limited number of short hindcast simulations . found that models differed considerably in the degree of time-step-to-time-step precipitation variability over a 5∘×5∘ region of the equatorial Indian Ocean, as computed by the root mean squared difference of area-averaged time step precipitation, but did not connect this variability to other scales or examine the spectra of rainfall intensities. There was no relationship between time step precipitation variability and MJO fidelity .

Another reason for the lack of attention to time step precipitation may be a scarcity of suitable diagnostics to compare the characteristics of precipitation variability among models, and between models and observations across spatial and temporal scales. Previous studies have focused mainly on frequency distributions of precipitation intensities, computed mainly at the model gridscale but often on time-averaged or selectively sampled data (e.g., one 30 min model time step per 3 h). While these results are useful, they do not consider the coherence of precipitation features in space and time. Diagnostics of precipitation coherence require sampling many gridpoints and time steps, which can be computationally cumbersome when working with high-frequency, fine-resolution data.

In this manuscript, we introduce diagnostics designed to describe precipitation variability and scale interactions in observations and models across a range of spatial and temporal scales. These diagnostics were developed with a view to condensing large data volumes from sub-daily output of O(10) km scale GCMs into a set of measures of precipitation frequency, intensity and spatial and temporal coherence, to improve understanding of observed rainfall variability and compare simulated and observed precipitation characteristics across a range of scales. The diagnostics form a small software package entitled “Analyzing Scales of Precipitation”, version 1.0 (ASoP1). In Sect. , we describe the ASoP1 diagnostics, then introduce the MJO hindcast dataset mentioned above. In Sect.  we demonstrate how the diagnostics can be used to discern and evaluate model behavior by applying them to the MJO hindcast dataset as well as to satellite-derived precipitation analyses. We discuss our results in Sect.  and summarize our findings in Sect. .

As this manuscript focuses on novel diagnostics of precipitation, we devote Sect.  to a thorough explanation of our methods, including examples using satellite-derived precipitation analyses. To demonstrate the ability of these diagnostics to compare precipitation characteristics among a range of model configurations, we apply the ASoP1 diagnostics to sub-daily tropical precipitation from ten models from the “Vertical structure and physical processes of the Madden–Julian Oscillation” model-evaluation project, described in Sect.  and shown in Table , as well as 3 h data from two satellite-derived analyses: TRMM 3B42 product, version 7A ; hereafter “TRMM” and the National Oceanic and Atmospheric Administration Climate Prediction Center Morphing Technique, version 1.0 hereafter “CMORPH”. We employ two observation-based datasets to provide a measure of observational uncertainty in our diagnostics. Both products are derived from a combination of infrared and microwave sounders. The TRMM 3B42 algorithm combines available precipitation data from microwave sounders, then fills the gaps in this dataset by merging precipitation data from infrared sounders, which have been first calibrated against the microwave sounders over a longer time period. TRMM 3B42 is calibrated on a monthly basis against gauge data from the Global Precipitation Climatology Centre. The 3B42 algorithm uses many microwave sounders, not only the TRMM Microwave Imager; “TRMM” refers to the source of the data, not the instrument. CMORPH combines precipitation data from microwave sensors only, but fills the gaps between microwave satellite overpasses by advecting the precipitation field using vectors derived from infrared-based cloud observations. Both products have been shown to under-detect light rainfall rates e.g.,. We use TRMM and CMORPH in the domain 60–160∘ E, 10∘ S–10∘ N for two periods in boreal winter 2009–2010, the choice of which is described in Sect. .

TRMM and CMORPH have a native horizontal resolution of 0.25∘×0.25∘, which is finer than any of the models analyzed. Because the diagnosed spatial and temporal scales of precipitation will vary with horizontal resolution, we use an area-weighted averaging method to interpolate TRMM and CMORPH to a 1.25∘×1.25 grid, which is approximately the median resolution of the models (147 km). A robust validation of any one model would require averaging TRMM and CMORPH to the model's native resolution, or preferably to a common resolution coarser than the model's native grid, as our results suggest. However, model validation is not the purpose of our study; therefore, for clarity of presentation we compare the models to the 1.25∘ TRMM and CMORPH data to indicate observed scales of precipitation at a resolution comparable to, but not exactly equal to, the models' resolution. The example diagnostics below demonstrate the effects of horizontal resolution on the scales of precipitation, using 0.25 and 1.25∘ CMORPH data.

We discuss the diagnostics first, as they are designed to be applied to any model or observed dataset at scales ranging from the model time step to a sub-seasonal average, and from the gridscale to O(1000 km) regions, depending on the phenomena and scales of interest. The results we show in Sect.  for time step and 3 h precipitation are only one example use of these diagnostics. In all our diagnostics, we scale precipitation rates to mm day-1, since this unit is commonly used in other studies. However, it should be remembered that a fixed value in mm day-1 equates to various rainfall intensities depending on the temporal scale considered (e.g., a 20 min time step or a 3 h average).

In all figures, we order the GASS/YoTC models alphabetically by abbreviation (Table ) except that we place MetUM-GA3 first. MetUM-GA3 often displays behavior distinct from the other models. Because of the attention paid to MetUM-GA3 in our discussion, and because MetUM is the subject of our future work, we choose to separate this model to emphasize its unique behavior.

Our diagnostics reveal that analyzing temporally or spatially averaged precipitation can hide a wealth of information about model behavior on the native gridscale and time step. This is true even for relatively small averaging scales, such as 3 h means or 2×2 gridboxes (our 5.6∘×5.6∘ regions were 4 times the gridscale of the coarsest resolution models in our dataset). Analysis of gridpoint, time step precipitation is critical for developing sub-gridscale parameterizations, since these are the scales at which the parameterizations interact with the resolved dynamics. Such analysis can identify potentially undesirable characteristics, such as the strong spatial and temporal intermittency in convection in MetUM-GA3. Nearly all of the convection in MetUM-GA3 is very strong, producing precipitation rates > 100 mm day-1 on a time step (Fig. a); furthermore, convection is often isolated to a single gridpoint and time step (Fig. a). Although there are no verifying observations for the model time step data that cover comparable spatial and temporal domains, it is difficult to believe that this behavior is representative of oceanic tropical convection. These intense, isolated precipitation features must be associated with intense, isolated column heating. Over a sequence of time steps, this behavior produces a “checkerboard”-style spatial pattern of heating that shifts from one time step to the next as gridpoint convection triggers quasi-randomly. It is not clear whether the model dynamics respond to this strong gridscale heating, or only to the average heating over several gridpoints and time steps, but gravity waves triggered by the intermittent heating in one column may influence the likelihood of convection at neighboring gridpoints on subsequent time steps, disrupting convective organization and the propagation of waves with longer periods and larger horizontal scales (e.g., Kelvin waves, the MJO). Understanding the controls on spatial and temporal intermittency in MetUM convection, as well as the influences of that intermittency on the model dynamics, tropical convective variability and the mean state, are all active areas of research inspired by our diagnostics.

Future research should also seek to validate time step model convection against high-resolution observations, through comparison of model data against ground-based or spaceborne precipitation radar measurements (e.g., from the Global Precipitation Measurement mission). These comparisons must take care to analyze observed and simulated data at comparable spatial and temporal scales, given our results on the effects of spatial and temporal averaging on the distributions and coherence of precipitation. Model development efforts to reduce or remove undesirable intermittency may involve single-column model experiments, in which the effects of changes in sub-gridscale physics can be isolated from feedbacks through the resolved dynamics e.g.,, although we stress that physics–dynamics coupling may have a substantial effect on the model behaviors and diagnostics presented here.

Although MetUM-GA3 is the model with the most intermittent precipitation in our study, CNRM-AM, CanCM4, GISS-E2, ECEarth3 and SPCAM3 display varying degrees of intermittency (Fig. ). It is likely that all of those models have a self-limiting character to their convective parameterizations, such that the effect of convection on one time step reduces the probability of convection for one or several subsequent time steps. Preliminary analysis of MetUM-GA3 (not shown) suggests that convection on one time step produces downdraft cooling that stabilizes the vertical temperature profile near the lifting condensation level (LCL), the stability across which is used in the diagnosis of deep convection (i.e., to diagnose deep convection, the parcel must be able to ascend through the LCL). Although instability may remain aloft, the model is unable to convect on subsequent time steps until the profile again becomes unstable at the LCL. There are a variety of mechanisms by which a parameterization can be self-limiting, which will depend on the precise design of the parameterization; a detailed examination of the convective parameterizations of 10 GCMs is outside the scope of this study, but our analysis of this behavior may be of interest to individual modeling centers to understand and improve their parameterizations.

On the gridpoint and time step scale, the worlds simulated by these models are definitely not “dreary” e.g.,, at least over the Warm Pool domain considered here. In most models, the total precipitation consists of a variety of time step rates that span 1–100 mm day-1, with most precipitation falling in time steps with precipitation rates > 10 mm day-1 (Fig. a). Only when the time step data are averaged to 3 h means do the precipitation spectra begin to collapse to be lighter (Fig. b) and more persistent (Fig. ) than in the satellite-derived analyses. The narrower spectra arise from the tendency for one time step with heavy precipitation to be followed by several time steps with no precipitation; the persistence of 3 h rain rates suggests that the time step intermittency occurs consistently in each 3 h window. These results imply that the self-limiting character of a model's convection, displayed through temporal intermittency in time step precipitation, prevents the model from producing enough consecutive time steps of heavy precipitation, or enough consecutive time steps of no precipitation, to generate a broader distribution of 3 h mean rates. An observer stationed on an island in the Warm Pool in many of these models would be constantly dodging intense, short-lived downpours, not standing in the persistent light rain implied by past studies' analyses of 3 h or daily mean data.

Much of our analysis has focused on time step and gridpoint data from GCMs, the formulations of which include spatial and temporal smoothing (either implicitly or explicitly), as well as truncation errors, both of which lead to an underestimation of energy on the smallest resolved scales. Previous studies have found that the “effective resolution” of a GCM – the scales at which the truncation and smoothing have no effect, or zero power – is several times the native resolution e.g.,, such that the time step, gridpoint data are unreliable and should be discarded. While we do not argue with the conclusions of those studies, we believe that it remains important to examine the characteristics of native resolution data for several reasons: (a) to inform parameterization development, as discussed above; (b) to understand the effects of intermittency on these scales, however under-resolved, because that intermittency may influence the larger and longer scales in a GCM; and (c) because despite previous conclusions on effective resolution, the scientific community is increasingly using gridscale, instantaneous output from models with ever-finer horizontal resolution to study extreme events and their responses to natural variability and anthropogenic climate change e.g.,.

We used 2-day hindcasts from the “Vertical structure and physical processes of the MJO” model-evaluation project, which is the only known source of time step, gridpoint precipitation data from many contemporary models. However, this dataset has limitations. First, only two sets of 22 2-day forecasts were performed, each for a case study of an MJO event in boreal winter 2009–2010. Although each set of forecasts samples the MJO active and suppressed phase, limiting the possibility of sensitivity to MJO phase, there is an active MJO in the analysis domain throughout the dataset, which may bias the simulated precipitation characteristics. We plan to address this issue in a future study by computing our diagnostics for across an entire season of MetUM time step data. Second, the spatial domain of the data is limited to the deep tropical Warm Pool; the dataset may not represent the full spectrum of tropical convection in the models or satellite-derived analyses. Third, all forecasts were initialized from ECMWF analyses. found this led to an initialization shock, the strength of which varied among the models. To reduce the effect of the shock, we removed the first 12 h of each forecast, as in , but it is possible that our findings are influenced by the shock and may not represent the model's intrinsic behavior. Removing the first 24 h of each forecast made only a very small difference to our results and did not affect our conclusions.

The analysis in Sect.  is only one potential use of these diagnostics. Understanding the spatial and temporal characteristics of precipitation is important for a variety of applications. Computing precipitation spectra (Fig. ) and 2-D histograms (Fig. ) for daily-mean or pentad-mean precipitation from models and observations could give insight into the simulated levels of synoptic and intraseasonal variance in a particular region, for instance the active and break periods of the major monsoons. Spatial maps of contributions from sections of the precipitation spectra (Fig. ) could aid understanding of whether biases in simulated mean precipitation are due primarily to biases in frequency or in intensity. Spatial and temporal coherence diagnostics (Fig. ) may provide information on convective aggregation, which is important for tropical cyclones and the MJO. All of these diagnostics could be used to compare precipitation characteristics from simulations of the same model at various horizontal resolutions, or with perturbations to one or several parameters, to assist model development and assessment. We believe that these diagnostics will be useful primarily on sub-monthly and sub-2000 km scales, as larger and longer scales are likely dominated by the seasonal cycle rather than the individual synoptic or mesoscale systems that produce precipitation.

When comparing datasets with different spatial and temporal resolutions, it is commonplace to average all data to the resolution of the coarsest dataset. However, our results show that any spatial or temporal averaging can alter precipitation characteristics, such that for the purposes of these diagnostics it is unfair to compare a lower-resolution dataset at its native resolution to a higher-resolution dataset that has been averaged to the lower resolution. Instead, we recommend comparing the datasets at their native resolutions – to understand the behavior of each dataset – as well as at a common resolution at least 2 times (in each direction) that of the coarsest dataset in space and time. This is still not a clean comparison because the effects of averaging increase with the number of points combined (up to some asymptotic limit), but at least it allows both datasets to “experience” some averaging in space and time.

We have developed a range of diagnostics to identify the spatial and temporal characteristics of precipitation in observations and GCMs; these diagnostics form a small software package, “Analyzing Scales of Precipitation” version 1.0 (ASOP1). The ASoP1 diagnostics are designed be applied to sub-monthly data at horizontal resolutions O(1000 km) or finer, to assess precipitation variability associated with phenomena ranging from individual cloud systems to mesoscale weather systems and synoptic fronts. The diagnostics are motivated by the increasing attention paid to the simulation of local and regional hydrological extremes in fine-resolution GCMs – which often requires gridscale, instantaneous precipitation data – while model evaluation has remained focused primarily on monthly and seasonal accumulations. Sub-gridscale parameterization development requires information about the spatial and temporal variability of precipitation at the native gridscale and time step, since these are the scales at which the parameterizations operate. The ASoP1 diagnostics include 1-D histograms and spatial maps of the contributions of intensity ranges to the total precipitation (e.g., Fig.  and Fig. ); 2-D histograms of precipitation rates at the same gridpoint on consecutive time intervals (e.g., Fig. a), which show the temporal persistence of precipitation; the average correlation of precipitation at a range of distances and temporal lags, correlated against precipitation at a central gridpoint (Fig. c), computed by dividing the analysis domain into a series of non-overlapping sub-regions (e.g., Fig. b); average correlations as a function of either physical distance (in km) or time, with which one can compare datasets with different spatial and temporal resolutions (e.g., Fig. ); and summary metrics that can be used to easily track the effects of changes to model resolution, physics or dynamics on the spatial and temporal coherence of precipitation (Table ).

To demonstrate the value of these diagnostics, we apply them to 10 models from the “Vertical structure and physical processes of the MJO” model-evaluation project (Table ), which collected time step data at the native model horizontal resolution over an extended warm pool domain (10∘ S–10∘ N, 60–160∘ E) from 44 2-day hindcasts during two strong MJO events in boreal winter 2009–2010. At the time step and gridscale, some models produce precipitation features that are highly coherent in space and time, while others produce intermittent precipitation that resembles uncorrelated noise (Fig. ). MetUM-GA3 is the model with the most intermittent precipitation, with a weakly negative lag-1 auto-correlation of time step precipitation and no statistically significant correlations between precipitation at neighboring gridpoints (Fig. ). We found no relationship between the level of intermittency and either horizontal resolution or the length of the model time step. Models with intermittent precipitation tend to produce more of their total precipitation from very heavy events – often exceeding 100 mm day-1 in the case of MetUM-GA3 – while models with persistent time step precipitation, such as ECEarth3, generate more frequent precipitation with moderate intensities of 10–50 mm day-1 (Figs.  and ). Strong and highly intermittent convection, such as that in MetUM-GA3, will be associated with strong and intermittent column heating, which may interact with the resolved dynamics, affecting the spectrum of tropical wave activity and even the mean state. The effects of this intermittency remain unclear, but are an active area of research. The fact that 5 of the 10 GCMs in this study produce heavy time step precipitation rates, interspersed by time steps of little or no precipitation, contradicts the common criticism that GCMs simulate a “dreary state” in the tropics of continual light precipitation, which arose from studies that analyzed 3 h or daily-averaged precipitation e.g.,. In fact, many models continually produce short-lived, intense downpours throughout the warm pool.

Averaging time step, gridscale data in either time (to 3 h means) or space (to 5.6∘×5.6∘) considerably reduces inter-model variations in the spatial and temporal scales of precipitation (Figs.  and , Table ), as well as in the spectra of precipitation intensities (Fig. ) and the temporal persistence of precipitation rates (Figs.  and ). This is because spatial or temporal averaging has a greater effect on intermittent precipitation than on persistent precipitation. When compared to TRMM and CMORPH satellite-derived precipitation analyses over the same period and domain, all models produce 3 h precipitation features that are too broad and too persistent, despite the fact that many of those same models produce time step precipitation features that are isolated in both space and time (Fig. ). This emphasizes that averaging in either space or time can hide a wealth of information about the intrinsic behavior of GCMs.

Averaging 3 h data from the models, TRMM and CMORPH, to a common 5.6∘×5.6∘ grid improves the agreement among the models, as well as between the models and the satellite-derived analyses (Figs.  and , Table ). We hypothesize that the strong agreement among the models indicates that these are the scales at which the convection in these models is in balance with the synoptic-scale, dynamical systems that produce precipitation. This convergence of model behavior may be enhanced by the fact that these data are from short (2-day) forecasts initialized from the same ECMWF analyses, which means that the representation of these dynamical systems are much more similar among models than if the data came from free-running climate simulations.

These results represent only one possible use of the ASoP1 diagnostics, which we believe will be useful for model development and evaluation at longer (e.g., daily, synoptic) and larger (e.g., regional averages) scales, as well as at the native gridpoint and time step. In particular, these diagnostics would be ideal for understanding the effects of horizontal resolution and changes to physical parameters on the simulated spatial and temporal scales of precipitation, and for comparing the characteristics of precipitation and their representation in models in different tropical regions.

The ASoP1 diagnostics package is coded in Python 2. The code is available under the terms of the Apache 2.0 license from the lead author's GitHub repository at https://github.com/nick-klingaman/ASoP (Klingaman and Martin, 2016). There are two software packages: ASoP-Spectral, which computes the 1-D histograms and maps of the contributions of specific intensity bins to the total precipitation; and ASoP-Coherence, which computes the 2-D histograms, the correlations with distance and time and the spatial and temporal coherence summary metrics. The user must install several Python packages prior to running the code; a list of these is given at the top of each python code file in the package. These packages also have software dependencies. The hardware requirements for running the code will vary based on the size of the dataset the user wishes to analyze, particularly for the amount of system memory (RAM) required. The analysis shown in this manuscript was performed on a Quad-core desktop workstation with 32GB RAM.

Data from the “Vertical structure and physical processes of the Madden–Julian oscillation” project can be obtained from the Earth System Grid Federation: https://www.earthsystemcog.org/projects/gass-yotc-mip (Waliser et al., 2016).

TRMM 3B42 version 7A data can be obtained from http://disc.sci.gsfc.nasa.gov/TRMM (National Aeronautics and Space Administration, 2016).

CMORPH version 1.0 data can be obtained from ftp://ftp.cpc.ncep.noaa.gov/precip/global_CMORPH/3-hourly_025deg (National Centers for Environmental Prediction Climate Prediction Center, 2016).