We test the reliability of two neural network interpretation techniques, backward optimization and layerwise relevance propagation, within geoscientific applications by applying them to a commonly studied geophysical phenomenon, the Madden–Julian oscillation. The Madden–Julian oscillation is a multi-scale pattern within the tropical atmosphere that has been extensively studied over the past decades, which makes it an ideal test case to ensure the interpretability methods can recover the current state of knowledge regarding its spatial structure. The neural networks can, indeed, reproduce the current state of knowledge and can also provide new insights into the seasonality of the Madden–Julian oscillation and its relationships with atmospheric state variables.

The neural network identifies the phase of the Madden–Julian oscillation twice as accurately as a linear regression approach, which means that nonlinearities used by the neural network are important to the structure of the Madden–Julian oscillation. Interpretations of the neural network show that it accurately captures the spatial structures of the Madden–Julian oscillation, suggest that the nonlinearities of the Madden–Julian oscillation are manifested through the uniqueness of each event, and offer physically meaningful insights into its relationship with atmospheric state variables. We also use the interpretations to identify the seasonality of the Madden–Julian oscillation and find that the conventionally defined extended seasons should be shifted later by 1 month. More generally, this study suggests that neural networks can be reliably interpreted for geoscientific applications and may thereby serve as a dependable method for testing geoscientific hypotheses.

Neural networks have the potential to improve our understanding of the earth system in ways that are unique from other statistical and machine learning methods. Recent research within the geosciences has shown that neural networks can be used to accelerate climate model parameterizations

Neural networks may be particularly useful within the geosciences if the relationships contained within their learned parameters can be understood and interpreted. Numerous methods have been proposed for such interpretation within the computer science community and have even been shown to be applicable to improving the understanding of geoscientific phenomena such as El Niño–Southern Oscillation (ENSO), sources of seasonal predictability, and severe convective storms

The Madden–Julian oscillation (MJO;

We use the MJO as an opportunity to test whether interpretable neural networks can capture known patterns of variability within complex geoscientific data, and we then extend our analysis into inferring new information about the MJO itself. We also provide a new definition of MJO seasonality, for both the conventional outgoing longwave radiation definition and across atmospheric state variables. The aim of this paper is threefold: (1) to highlight the ability of neural networks to capture complex relationships within geoscientific data; (2) to test neural network interpretation methods to ensure they can reliably infer the relationships captured by neural networks; and (3) to use the interpretations to gain new insights into the MJO. This paper thereby offers a conceptual guideline for how a geoscientist might go about using a neural network to discover new patterns within geoscientific data. Those interested in the MJO itself will also find new insights into its spatial structures and seasonality.

We first discuss the data we use to define the MJO and then detail how we design a neural network to infer information about its spatial structure and seasonality.

We define the MJO according to the Outgoing Longwave Radiation MJO Index

While the process of calculating OMI is complicated, the resultant phase-space and spatial perspectives of the MJO are relatively simple, as shown in Fig.

Spatial and phase-space perspectives of the Madden–Julian oscillation.

We test whether a neural network can identify the phase of the MJO given inputs of cloud characteristics and atmospheric state variables. The inputs to the neural network are tropical (30

We design a neural network to be as simple as possible while still ensuring it can capture any relationships between the input atmospheric state variables and the phase of the MJO. We use fully connected networks, which can be thought of as a chain of nonlinear regression functions that map the relationships between input and outputs datasets. The neural network has one input layer, two subsequent “hidden” layers with 64 and 128 nodes each, and one output layer with eight nodes, each of which represents a phase of the MJO (Fig.

Schematic for the neural network used in this study. The first layer ingests vectorized input images, with two subsequent hidden layers (the first with 64 nodes and the second with 128 nodes) and an output layer of 8 nodes that correspond to the eight phases of the MJO. A separate neural network is trained for each calendar week of the year.

We train separate neural networks on data from 121 d bins centered on each calendar week of the year in order to study the seasonality of the MJO. Each neural network is therefore tasked with identifying the phase of the MJO according to the outgoing longwave radiation and state variable patterns during the period of the year to which it is assigned. Comparisons between interpretations of each neural network offer insights into the seasonality of the MJO, as discussed in subsequent sections.

Neural network interpretability generally becomes more challenging with increasing network complexity

The novelty of this paper is the demonstrated ability to interpret what the neural networks have learned and to then gather scientific value from the interpretations. We use two interpretation methods which we briefly discuss here and are explained in more extensive detail in the context of geoscience within

Backward optimization uses the same method that is used to train a neural network (i.e., backpropagation) to instead interpret what a trained network has learned

Layerwise relevance propagation (LRP) interprets the neural network's decision-making process for each individual input sample

We first ensure the neural networks are accurate enough to offer scientifically valuable interpretations. As a reminder, we train separate neural networks on data from 121 d windows centered on each calendar week of the year. The accuracy of the neural network for the window centered on 10 January is presented from both a deterministic and probabilistic perspective in Fig.

Example visualizations of the accuracy of the neural networks, in this case for the neural network centered on 10 January.

An important question regarding the usage of neural networks is whether they out-perform conventional methods, such as regression. If regression performs similarly to a neural network, then the increased complexity and nonlinearity of a neural network is not required. We therefore similarly use a form of linear regression to identify the phase of the MJO using the input state variables and outgoing longwave radiation across 121 d windows centered on each calendar week. The multi-output linear regression models have no hidden nodes and no nonlinearities but are otherwise identical to the neural networks (i.e., as in Fig.

The accuracy of the neural network and multi-output linear regression approaches for each calendar week throughout the year. The neural network accuracy is plotted in blue, and the regression accuracy is plotted in red. The solid lines show the accuracy for all input samples, and the dashed lines show the accuracy if a one-phase error is permitted.

We use backward optimization and layerwise relevance propagation (LRP) to infer the spatial structure of the MJO and its seasonality according to the neural networks. Examples of LRP applied to inputs for the neural network trained on the 121 d window centered on 10 January are shown in Fig.

Example relevance heatmaps from the layerwise relevance propagation interpretation technique. The outgoing longwave radiation fields from four example inputs into the neural network are shown, each corresponding to a separate phase-7 MJO day. The corresponding relevance heatmaps are shown below each example outgoing longwave radiation field and show where the neural network focuses its attention to determine that the examples are associated with a phase-7 MJO day.

We next test the neural networks more rigorously and challenge them to identify the most common spatial structures of the MJO across its eight phases. To do so, we use backward optimization and optimize inputs such that the spatial patterns within the inputs make the neural networks most confident that the inputs are associated with a particular phase of the MJO. Numerically, this means that the outputs associated with the optimized inputs have a likelihood of approximately 1 in the phase for which they are optimized, and likelihoods of 0 for all other phases. We again only show the optimized outgoing longwave radiation fields for simplicity, although the optimization also identifies the characteristic patterns in the 15 other state variables.

The spatial pattern of the MJO during boreal winter (10 January) and boreal summer (1 August) according to both OMI and the neural networks is shown in Fig.

Because the neural network so accurately captures the seasonal evolution of the MJO within the outgoing longwave radiation composites, we now extend the interpretations to study the seasonality of the MJO. We first test how the spatial structure of the MJO changes across seasons using LRP. To do so, we calculate the composite relevance for each variable for each calendar week of the year and present the annual evolution of the relevance in Fig.

Composite normalized LRP relevance across all variables for each calendar week throughout the year. The relevance is normalized to sum to 1 across all variables for each calendar week (i.e., along the vertical axis).

The fact that upper-tropospheric anomalies are most important for identifying the MJO during boreal winter may explain the seasonality in coupling between the MJO and the stratosphere

Optimized patterns for phase 6 of the MJO for different periods of the year. The central date on which the neural network is trained for each optimization is shown in the title of each subfigure. Each subfigure shows outgoing longwave radiation, 850 mbar zonal wind, 200 mbar zonal wind, 200 mbar meridional wind, 200 mbar temperature, and 850 mbar specific humidity.

We now examine the optimal spatial patterns of the MJO throughout the year to provide some spatial context to the seasonality of the relevances shown in Fig.

Mechanistic studies of the MJO commonly depend on accurate definitions of when each MJO seasonal mode occurs, since the spatial structures of the winter and summer modes differ so substantially (Fig.

Seasonality of the Madden–Julian oscillation according to interpretations of the neural networks. The extended boreal summer and winter modes are shown in red and blue, respectively, and periods of transition are denoted by the lighter red and blue colors. The winter (summer) mode is defined as periods during which the correlation between the optimized MJO pattern on 10 January (1 August) and the optimized pattern for each respective calendar week is greater than 0.75. and the transition periods extend between these two modes. The extended boreal winter mode is defined as periods during which the optimized pattern for each respective calendar week is more highly correlated with the 10 January optimized pattern than the 1 August optimized pattern and vice versa for the extended boreal summer mode.

Finally, we define extended boreal winter as the period during which the correlation between each weekly optimal pattern and the 10 January optimal pattern is greater than that between the weekly optimal patterns and the 1 August optimal pattern. Extended boreal summer spans the rest of the year. Using this definition, extended boreal winter MJO extends from early November through late April across most state variables and from mid-November through late April for outgoing longwave radiation in particular (dark and light colors in Fig.

We have tested the ability of interpretable neural networks to identify complex, multi-scale geophysical phenomena via their application to the Madden–Julian oscillation (MJO). We first evaluated whether neural networks can identify the MJO and then used neural network interpretability methods to study the seasonality and spatial structure of the MJO and its relationship to atmospheric state variables. Our study therefore contributes both to the general usage of neural networks within geoscience and to knowledge of the MJO itself, so we separate our discussion of the implications for both communities below.

We have shown that neural networks are highly interpretable, even for complex, multi-scale geophysical phenomena. Two methods proposed by the computer science community – backward optimization and layerwise relevance propagation – provide particularly useful interpretations of neural networks

The capability of neural networks to include nonlinearities and simultaneously model different input patterns that lead to similar outputs proved useful for studying the seasonality of the MJO. The neural networks identified the phase of the MJO twice as accurately as the multi-output linear regression, which implies that interpretations of the neural network characterize the MJO more accurately than the linear regression approach. We hypothesized that the increase in accuracy was caused by the neural networks' ability to model the uniqueness of each MJO event, which is not feasible using conventional linear approaches such as regression. The amount of neural network complexity required for tasks across the geosciences will vary greatly, so the benefits of interpretable neural networks are also likely to vary across sub-disciplines. We have found that a baseline approach of comparing the accuracy of neural networks to more simple methods such as linear regression is useful in determining the necessity of a neural network.

Based on this study and other supporting work

We also used neural networks as an approach to better understand the spatial structure and seasonality of the MJO. Our results are generally consistent with the thorough body of literature on the MJO, which supports the reliability and robustness of interpretable neural networks within geoscience.

Consistent with previous studies, we find that the spatial structure of the MJO generally exhibits two dominant modes of variability distinguished between the boreal summer and winter. We find that the extended boreal winter mode of the MJO occurs between early November and late April, with the boreal summer mode occurring throughout the remainder of the year. This definition of the extended seasons is delayed 1 month compared to conventional definitions, which use an extended boreal winter of October through March. Furthermore, the seasonality of the relationship between the MJO and atmospheric state variables is more complex, with each variable exhibiting a unique seasonality. Some state variables such as lower-tropospheric zonal winds exhibit a uni-modal seasonality, whereas others such as upper-tropospheric zonal winds exhibit a bi-modal seasonality. We also find that upper-tropospheric thermodynamic anomalies are particularly useful in identifying the MJO during boreal winter, which may relate to the enhanced coupling between the MJO and stratospheric processes during this season.

Consistent with previous studies, we find that the spatial structure of the MJO generally exhibits two dominant modes of variability distinguished between the boreal summer and winter. We also extend our analysis to test numerous aspects of the MJO, from its nonlinearities to its relationships with atmospheric state variables. The key points of this analysis are as follows:

The neural networks identify the phase of the MJO twice as accurately as the multi-output linear regression approach, which suggests that nonlinearities are important to the structure of the MJO. These nonlinearities are reflected in the spatial uniqueness of each MJO event, given that the composite structure of the MJO identified by the neural networks and linear methods are remarkably similar (Figs.

Each state variable exhibits a unique seasonality in its relationship with the MJO. For example, some state variables such as lower-tropospheric zonal winds exhibit a uni-modal seasonality, whereas others such as upper-tropospheric zonal winds exhibit a bi-modal seasonality (Figs.

Upper-tropospheric thermodynamic anomalies are particularly important for identifying the MJO during boreal winter, which may relate to the enhanced coupling between the MJO and stratospheric processes during this season (Fig.

We find that the extended boreal winter mode occurs between early November and late April, while the boreal summer mode occurs throughout the remainder of the year. This definition of the extended seasons is delayed 1 month compared to the conventional definition, which uses an extended boreal winter of October through March (Fig.

Our results show that neural networks are highly interpretable, even for spatially complex geoscientific applications. Because of the high reliability of the interpretations, neural networks are viable tools for testing hypotheses related to the MJO and other spatially complex geophysical phenomena. More complex hypotheses can now be tested: for example, does horizontal advection of the lower-tropospheric mean moisture by the MJO circulation govern the propagation of the MJO

All data used in this study and an example script for training a neural network and generating the LRP heatmaps and optimal input fields are available at the following DOI:

BAT, KK, P, and DY conceived the idea and designed the experiment. KK and P provided expertise on neural networks throughout the project. BAT performed the analysis and wrote the paper.

The authors declare that they have no conflict of interest.

This research has been supported by the U.S. Department of Energy (grant no. DE-FG02-97ER25308), the U.S. Department of Energy, Office of Science (grant no. DE-AC02-05CH11231), the U.S. Department of Energy, Office of Science (grant no. DE-AC02-05CH11231), the National Institute of Food and Agriculture (grant no. CA-D-LAW-2462-RR), and the Packard Foundation for Science and Engineering.

This paper was edited by Richard Neale and reviewed by two anonymous referees.