Preprints
https://doi.org/10.5194/gmd-2024-56
https://doi.org/10.5194/gmd-2024-56
Submitted as: methods for assessment of models
 | 
18 Apr 2024
Submitted as: methods for assessment of models |  | 18 Apr 2024
Status: this preprint is currently under review for the journal GMD.

ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool

Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini

Abstract. The accurate calibration of parameters in atmospheric and Earth system models is crucial for improving their performance, but remains a challenge due to their inherent complexity, which is reflected in input-output relationships often characterized by multiple interactions between the parameters and thus hindering the use of simple sensitivity analysis methods. This paper introduces the Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool (ML-AMPSIT), a new tool designed with the aim of providing a simple and flexible framework to estimate the sensitivity and importance of parameters in complex numerical weather prediction models. This tool leverages the strengths of multiple regression-based and probabilistic machine learning methods including LASSO, Support Vector Machine, Classification and Decision Trees, Random Forest, Extreme Gradient Boosting, Gaussian Process Regression, and Bayesian Ridge Regression. These regression algorithms are used to construct computationally inexpensive surrogate models to effectively predict model outputs from input parameters, thereby significantly reducing the computational burden of running high-fidelity models for sensitivity analysis. Moreover, the multi-method approach allows for a comparative analysis of the results. Through a detailed case study with the Weather Research and Forecasting (WRF) model coupled with the Noah-MP land surface model, ML-AMPSIT is demonstrated to efficiently predict the behavior of Noah-MP model parameters with a relatively small number of model runs, by simulating a sea breeze circulation over an idealized flat domain. This paper points out how ML-AMPSIT can be an efficient tool for performing sensitivity and importance analysis also for complex models, guiding the user through the different steps and allowing for a simplification and automatization of the process.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this preprint. The responsibility to include appropriate place names lies with the authors.
Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini

Status: open (until 18 Jun 2024)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse
  • CEC1: 'Comment on gmd-2024-56', Juan Antonio Añel, 12 May 2024 reply
    • AC1: 'Reply on CEC1', Dario Di Santo, 13 May 2024 reply
      • CEC2: 'Reply on AC1', Juan Antonio Añel, 18 May 2024 reply
Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini

Model code and software

ML-AMPSIT Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini https://doi.org/10.5281/zenodo.10789930

Dario Di Santo, Cenlin He, Fei Chen, and Lorenzo Giovannini

Viewed

Total article views: 298 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
210 67 21 298 12 10
  • HTML: 210
  • PDF: 67
  • XML: 21
  • Total: 298
  • BibTeX: 12
  • EndNote: 10
Views and downloads (calculated since 18 Apr 2024)
Cumulative views and downloads (calculated since 18 Apr 2024)

Viewed (geographical distribution)

Total article views: 297 (including HTML, PDF, and XML) Thereof 297 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 20 May 2024
Download
Short summary
This paper presents ML-AMPSIT, a new tool that exploits different machine learning algorithms to perform sensitivity analysis for atmospheric models, providing a computationally efficient way to identify key parameters that affect model output. The tool is tested by taking as a case study the simulation of a sea breeze circulation over flat terrain with the WRF/Noah-MP model, investigating the sensitivity of model results to different vegetation-related parameters.