The evaluation of models in general is a nontrivial task and can, due to epistemological and practical reasons, never be considered complete. Due to this incompleteness, a model may yield correct results for the wrong reasons, i.e., via a different chain of processes than found in observations. While guidelines and strategies exist in the atmospheric sciences to maximize the chances that models are correct for the right reasons, these are mostly applicable to full physics models, such as numerical weather prediction models. The Intermediate Complexity Atmospheric Research (ICAR) model is an atmospheric model employing linear mountain wave theory to represent the wind field. In this wind field, atmospheric quantities such as temperature and moisture are advected and a microphysics scheme is applied to represent the formation of clouds and precipitation. This study conducts an in-depth process-based evaluation of ICAR, employing idealized simulations to increase the understanding of the model and develop recommendations to maximize the probability that its results are correct for the right reasons. To contrast the obtained results from the linear-theory-based ICAR model to a full physics model, idealized simulations with the Weather Research and Forecasting (WRF) model are conducted. The impact of the developed recommendations is then demonstrated with a case study for the South Island of New Zealand. The results of this investigation suggest three modifications to improve different aspects of ICAR simulations. The representation of the wind field within the domain improves when the dry and the moist Brunt–Väisälä frequencies are calculated in accordance with linear mountain wave theory from the unperturbed base state rather than from the time-dependent perturbed atmosphere. Imposing boundary conditions at the upper boundary that are different to the standard zero-gradient boundary condition is shown to reduce errors in the potential temperature and water vapor fields. Furthermore, the results show that there is a lowest possible model top elevation that should not be undercut to avoid influences of the model top on cloud and precipitation processes within the domain. The method to determine the lowest model top elevation is applied to both the idealized simulations and the real terrain case study. Notable differences between the ICAR and WRF simulations are observed across all investigated quantities such as the wind field, water vapor and hydrometeor distributions, and the distribution of precipitation. The case study indicates that the precipitation maximum calculated by the ICAR simulation employing the developed recommendations is spatially shifted upwind in comparison to an unmodified version of ICAR. The cause for the shift is found in influences of the model top on cloud formation and precipitation processes in the ICAR simulations. Furthermore, the results show that when model skill is evaluated from statistical metrics based on comparisons to surface observations only, such an analysis may not reflect the skill of the model in capturing atmospheric processes like gravity waves and cloud formation.

All numerical models of natural systems are approximations to reality. They generate predictions that may further the understanding of natural processes and allow the model to be tested against measurements. However, the complete verification or demonstration of the truth of such a model is impossible for epistemological and practical reasons

These propositions include models employed in the Earth sciences, such as coupled atmosphere–ocean general circulation models, numerical weather prediction models and regional climate models. Those models approximate and simplify the world and processes in it by discretizing the governing equations in time and space and by modeling subgrid-scale processes with adequate parameterizations

The Intermediate Complexity Atmospheric Research model

However, in the literature the evaluation efforts for ICAR have so far focused mainly on comparisons to precipitation measurements or WRF output.

This study aims to improve the understanding of the ICAR model and develop recommendations that maximize the probability that the results of ICAR simulations, such as the spatial distribution of precipitation, are correct and caused by the physical processes modeled by ICAR (correct for the right reasons) and not by numerical artifacts or any influence of the model top. For a given initial state, a correct representation of the fields of wind, temperature and moisture, as well as of the microphysical processes, are a necessity to obtain the correct distribution of precipitation. Therefore, simulations of an idealized mountain ridge are employed to investigate and evaluate the respective fields and processes in ICAR. This study first quantitatively and qualitatively analyzes how closely the ICAR wind and potential temperature fields match the analytical solution for the ideal ridge and contrasts them with a WRF simulation to infer the aspects not captured by linear theory (Sect.

ICAR is an atmospheric model based on linear mountain wave theory

ICAR stores all dependent variables on a 3-D staggered Arakawa C-grid

In contrast to dynamical downscaling models, ICAR avoids solving the Navier–Stokes equations of motion explicitly. Instead, ICAR calculates the perturbations to the horizontal background winds analytically for a given time step by employing linearized Boussinesq-approximated governing equations that are solved in frequency space with the Fourier transformation

The vertical wind speed perturbation

ICAR allows for the selection of different microphysics (MP) schemes. In this study an updated version of the Thompson MP scheme is employed

The microphysics species,

In linear mountain wave theory, the wind field is entirely determined by the topography and the background state of the atmosphere

Note that since ICAR is based on the equations derived by

The investigations described in this study were conducted with a modified version of ICAR 1.0.1. All modifications are publicly available for download (

From the initial state of

However, in linear mountain wave theory

ICAR imposes a zero-gradient boundary condition (ZG BC) at the upper boundary on all quantities subject to numerical advection; see Eq. (

In the following, the mass levels are indexed from

To arrive at the discrete equations of the upwind advection, the flux divergences

While the effect described above is related to downdrafts at the model top, note that updrafts, on the other hand, may cause moisture to be transported out of the domain, leading to a mass loss. However, for

A solution to address both issues would potentially be to include a relaxation layer directly beneath the model top

To investigate ICAR with respect to the influence of the elevation of the upper boundary and the boundary conditions applied to it, idealized numerical simulations and a real case study are conducted. Simulations are run with ICAR-O, ICAR-N and WRF in order to assess to what degree ICAR simulations approximate the results of the analytical solution and a full physics model. In addition, WRF is employed to infer differences due to nonlinearities.

Simulations in this study are conducted with version 1.0.1 of ICAR (ICAR-O) and version 4.1.1 of WRF. Additionally, a modification of ICAR-O, referred to as ICAR-N, where the Brunt–Väisälä frequency

The ideal case consists of an infinite ridge extending along the south–north direction in the domain and westerly flow. The horizontal grid spacings of ICAR and WRF are chosen as

Idealized ICAR simulations are run for different model top elevations. The elevation of the upper boundary of the domain, referred to as model top elevation

The topography is given by a witch of Agnesi ridge defined by

The vertical potential temperature profile of the base state

Overview of the combinations of topographies and soundings (scenarios) used to initialize the idealized ICAR simulations. Here

For the default scenario with the

ICAR calculates the perturbations to the horizontal background wind with Eq. (

In this study the effect of the boundary conditions (BCs) imposed by ICAR at the upper boundary of the simulation domain is investigated. To this end several alternative BCs to the existing zero-gradient boundary condition are added to the ICAR code, their abbreviations, mathematical formulation and their numerical implementation are summarized in Table

For this study, options are added to the ICAR code that allow the application of different BCs to water vapor, potential temperature and the hydrometeors (cloud water, ice, rain, snow and graupel), hereafter referred to as a set of boundary conditions. To indicate which BCs were applied to what group in a specific model run, the runs are labeled with a three-digit code; see Table

The 10 combinations of BCs tested in the sensitivity study are listed in Table

Overview of all types of boundary conditions that were imposed at the model top of ICAR in the sensitivity study. The table lists the ID number, the abbreviation used in this study, the full name and equation of the BC evaluated at

Combinations of BCs tested in the sensitivity study with idealized simulations. Each column represents a combination of three BCs used in a specific simulation. Each digit of the three-digit code refers to the ID number of a specific BC listed in Table

All evaluations conducted in this study focus on cross sections along the west–east axis of the domain, oriented parallel to the background flow. Since ICAR does not currently support periodic boundary conditions, the ICAR domain is extended along the south–north axis to minimize influences from the boundaries (see Sect.

The effect of the Brunt–Väisälä frequency calculation method is investigated with a comparison of the

For the evaluation in this study the mixing ratios of the microphysics species are assigned to three groups: water vapor

The sensitivity of the physical processes simulated by ICAR-N to the elevation of the upper boundary and the imposed boundary conditions (BCs) is inferred from the total mass of the MP species in the cross section and the spatial distribution of potential temperature, the MP species and the 12 h accumulated precipitation

Differences in the spatial distribution of time-averaged quantities

To quantify the improvement of one simulation, with a set of boundary conditions BCs and model top

To characterize the effect of increasing the model top elevation on the SSE while keeping the set of boundary conditions unchanged, RE is evaluated for increasing values of

The quantity

To investigate the effects of the suggested modifications to ICAR on the distribution of precipitation for a real-world application, a case study is conducted for the Southern Alps on the South Island of New Zealand located in the southwestern Pacific Ocean. Furthermore, the procedure to identify the lowest possible model top elevation

To maintain comparability to

The case study focuses on 6 May 2015 LT (local time), a day with stably stratified large-scale northwesterly flow throughout the troposphere impinging on the Southern Alps over a 24 h period. Upstream of the South Island, ERAI exhibits a 24 h averaged relative humidity of more than

Figure

Generally, the horizontal west–east and vertical perturbations to the background state calculated by ICAR-N reproduce those obtained from the analytical expressions well (cf. Fig.

The wind and potential temperature fields simulated by ICAR-O (Fig.

Vertical cross sections of the horizontal perturbation wind component

WRF is not expected to perfectly reproduce the analytical solution due to the occurrence of nonlinearities for the chosen non-dimensional mountain height of

Figure

The reduction of error (RE), dependent on the chosen combination of boundary conditions (

The choice of an alternative BC over the standard ZG BC has the largest potential for a reduction of error when (i) the grid cells of the uppermost vertical level coincide with regions of vertical convergence where

The normalized vertical flux gradient of

For the investigated scenarios, altering the boundary condition applied to

The mean absolute error (MAE) of potential temperature in ICAR-N simulations employing ZG BCs (000) with different model top settings

Figure

The panels show the minimum model top elevation

As shown in Fig.

The reduction of error (RE), dependent on

Panel

Note that the spread of RE dependent on

The results show that the total masses of the microphysics species alone are not sufficient to determine whether the processes within the domain are influenced by the model top. In other words, the spatial distribution of these quantities needs to be taken into account as well. Conversely, even though the error in the distribution of

This section investigates how the lowest possible model top elevation

For a witch of Agnesi ridge with

The dependence of the lowest possible model top elevation

This section compares the spatial distribution of water vapor

With respect to water vapor ICAR-N is drier upwind of the topographical ridge and wetter downwind in comparison to WRF (see Fig.

Mixing ratios (color contours) of water vapor

Clear differences between the ICAR-N and WRF simulations are observed for suspended hydrometeors. While the approximate shape of the windward cap cloud (Fig.

The majority of precipitating hydrometeors in ICAR-N are observed windward of the topographical ridge, extending over most of the upwind slope (Fig.

Perturbations of the horizontal wind component

Figure

The distribution of precipitation in ICAR-N is asymmetric with a gradual increase until the maximum is reached and a steeper decrease after that. While in WRF

The maximum of accumulated snow in WRF is

The absence of graupel in ICAR-N compared to WRF can be traced to the MP scheme and is a result of the atmospheric conditions it encounters. The Thompson MP predicts graupel formation if riming growth exceeds the depositional growth of snow

The previous sections have demonstrated that (i) the Brunt–Väisälä frequency needs to be diagnosed from the background stratification in order to model a realistic perturbation flow field with ICAR, that (ii) it further requires a minimum model top elevation (which is dependent on the orography and the atmospheric background state), and that (iii) a combination of ZG and CG BCs (BC codes 011 and 111) are optimal to be used at the top of the ICAR model domain. The effects of these suggested modifications to ICAR on a real-world application are investigated with a case study conducted for the Southern Alps on the South Island of New Zealand located in the southwestern Pacific Ocean (Fig.

The Southern Alps are a mountain range approximately

For this region two ICAR-O and one ICAR-N simulations are conducted. ICAR-O calculates the Brunt–Väisälä frequency

The reduction of error (RE) of the simulations for the South Island of New Zealand for

The resulting patterns of

Clear differences can be observed in the distributions of

Furthermore, this artificial cloud in ICAR-O

Cross sections along the South Island of New Zealand (line A–B in Fig.

These results strongly indicate that the low model top setting of

The results highlight that a more accurate representation of the wind fields is obtained only when the Brunt–Väisälä frequency, in accordance with linear mountain wave theory, is calculated from the unperturbed background state of the atmosphere (ICAR-N) rather than from the perturbed state (ICAR-O). The remaining differences of the wind fields in ICAR-N to the analytical solution may be attributable to two causes: firstly, to solve the governing equations ICAR numerically calculates the Fourier transform of the topography

ICAR is intended as a computationally frugal alternative to full physics models, in principle allowing for very low model top elevations. While employing a low model top to take advantage of the associated computational cheapness is tempting, increased efficiency should not come at the cost of the physical fidelity of the model. The results in this study clearly show that there is a lowest possible model top elevation

This study strongly suggests that no general value for

The determination of

A comparison between ICAR-N and WRF simulations conducted for the same topography and sounding reveals substantial differences in the spatial distributions of

For strongly stratified atmospheric conditions, a constant-gradient BC was found to cause numerical stability issues in the idealized and real case simulations alike. Future studies could investigate further BC options that might allow a better approximation of the potential temperature profile: such approaches might, for instance, (i) analytically diagnose

The case study investigates the effect of the proposed modifications to ICAR on a real-world application for the South Island of New Zealand. It reveals that these modifications shift the distribution of precipitation upwind, leading to drier conditions in the alpine range but wetter coastal regions. The method for the determination of

With regard to the case study, the unmodified version of ICAR (ICAR-O) is found to produce enhanced precipitation in the alpine range due to artifacts (heightened mixing ratios of hydrometeors) in the topmost vertical levels in the horizontal vicinity of topographical peaks. This additionally caused the very low model top elevation found with the method employed in

The key findings and recommendations based on the extensive process-based evaluation of ICAR are summarized in the following.

There is a minimum possible model top elevation

Results show that in order to avoid spurious influences of the upper boundary to the microphysical processes within the domain,

Determining an exact value for

The method described in this study to determine

While most of the tested boundary conditions (in comparison to the default zero-gradient boundary condition) are suitable to reduce the errors in the water vapor and potential temperature fields, no tested combination of these boundary conditions results in a lower value for

Model skill, when inferred only from comparisons to surface observations, does not necessarily reflect the model skill in representing atmospheric processes.

The representation of the wind field in ICAR is improved by ensuring that the Brunt–Väisälä frequency is calculated from the background state of the atmosphere provided by the forcing data. Note that the current version of ICAR employs the perturbed state of the domain.

The modified version ICAR v1.0.1 employed for the simulations (

The investigation and its design, the simulations and their analysis, and the visualization of the results and writing (original draft and editing) were carried out by JH. The conceptualization of the paper was a joint effort from all authors, as were the discussion and refinement of the methods presented. Additionally, funding acquisition and project administration were carried out by MH.

The authors declare that they have no conflict of interest.

The computational results have been achieved with the high-performance computing support from Cheyenne (

This research has been supported by the Austrian Science Fund (FWF) (grant no. 28006-N32).

This paper was edited by Rohitash Chandra and reviewed by three anonymous referees.