Articles | Volume 11, issue 4
Geosci. Model Dev., 11, 1577–1590, 2018
Geosci. Model Dev., 11, 1577–1590, 2018

Methods for assessment of models 19 Apr 2018

Methods for assessment of models | 19 Apr 2018

On the effect of model parameters on forecast objects

Caren Marzban1,2, Corinne Jones2, Ning Li2, and Scott Sandgathe1 Caren Marzban et al.
  • 1Applied Physics Laboratory, Univ. of Washington, Seattle, WA 98195 USA
  • 2Department of Statistics, Univ. of Washington, Seattle, WA 98195 USA

Abstract. Many physics-based numerical models produce a gridded, spatial field of forecasts, e.g., a temperature map. The field for some quantities generally consists of spatially coherent and disconnected objects. Such objects arise in many problems, including precipitation forecasts in atmospheric models, eddy currents in ocean models, and models of forest fires. Certain features of these objects (e.g., location, size, intensity, and shape) are generally of interest. Here, a methodology is developed for assessing the impact of model parameters on the features of forecast objects. The main ingredients of the methodology include the use of (1) Latin hypercube sampling for varying the values of the model parameters, (2) statistical clustering algorithms for identifying objects, (3) multivariate multiple regression for assessing the impact of multiple model parameters on the distribution (across the forecast domain) of object features, and (4) methods for reducing the number of hypothesis tests and controlling the resulting errors. The final output of the methodology is a series of box plots and confidence intervals that visually display the sensitivities. The methodology is demonstrated on precipitation forecasts from a mesoscale numerical weather prediction model.

Short summary
Some numerical models generate "maps", for example temperature or precipitation maps produced by numerical weather prediction models. These maps often contain "objects", for example a storm. Features of these objects are generally affected by the parameters of the numerical model. This paper puts forth a methodology for exposing both the strength and the statistical significance of the effect of the model parameters on object features.