Submitted as: methods for assessment of models 03 Aug 2020

Submitted as: methods for assessment of models | 03 Aug 2020

Review status: a revised version of this preprint was accepted for the journal GMD.

Testing the Reliability of Interpretable Neural Networks in Geoscience Using the Madden-Julian Oscillation

Benjamin A. Toms1, Karthik Kashinath2, Prabhat2, and Da Yang2,3 Benjamin A. Toms et al.
  • 1Department of Atmospheric Science, Colorado State University, Fort Collins, CO
  • 2Lawrence Berkeley National Laboratory, Berkeley, California
  • 3University of California, Davis, Davis, California

Abstract. We test the reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications by applying them to a commonly studied geophysical phenomenon, the Madden-Julian Oscillation. The Madden-Julian Oscillation is a multi-scale pattern within the tropical atmosphere that has been extensively studied over the past decades, which makes it an ideal test case to ensure the interpretability methods can recover the current state of knowledge regarding its spatial structure. The neural networks can, indeed, reproduce the current state of knowledge and can also provide new insights into the seasonality of the Madden-Julian Oscillation and its relationships with atmospheric state variables.

The neural network identifies the phase of the Madden-Julian Oscillation twice as accurately as linear regression, which means that nonlinearities used by the neural network are important to the structure of the Madden-Julian Oscillation. Interpretations of the neural network show that it accurately captures the spatial structures of the Madden-Julian Oscillation, suggest that the nonlinearities of the Madden-Julian Oscillation are manifested through the uniqueness of each event, and offer physically meaningful insights into its relationship with atmospheric state variables. We also use the interpretations to identify the seasonality of the MJO, and find that the conventionally defined extended seasons should be shifted later by one month. More generally, this study suggests that neural networks can be reliably interpreted for geoscientific applications and may thereby serve as a dependable method for testing geoscientific hypotheses.

Benjamin A. Toms et al.

Status: final response (author comments only)
Status: final response (author comments only)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Login for authors/topical editors] [Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement

Benjamin A. Toms et al.

Data sets

Data for GMD 2020-152 Benjamin Toms

Model code and software

Data for GMD 2020-152 Benjamin Toms

Benjamin A. Toms et al.


Total article views: 536 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
357 173 6 536 17 13
  • HTML: 357
  • PDF: 173
  • XML: 6
  • Total: 536
  • BibTeX: 17
  • EndNote: 13
Views and downloads (calculated since 03 Aug 2020)
Cumulative views and downloads (calculated since 03 Aug 2020)

Viewed (geographical distribution)

Total article views: 382 (including HTML, PDF, and XML) Thereof 377 with geography defined and 5 with unknown origin.
Country # Views %
  • 1


Latest update: 04 Mar 2021
Short summary
We test whether a type of machine learning called neural networks can be used trustfully within the geosciences. We do so by challenging the networks to understand the spatial patterns of a commonly studied geoscientific phenomenon. The neural networks can correctly identify the spatial patterns, which lends confidence that similar networks can be used for more uncertain problems. The results of this study may give geoscientists confidence when using neural networks in their research.