Articles | Volume 16, issue 11
https://doi.org/10.5194/gmd-16-3123-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-16-3123-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Review and perspective paper
| 
02 Jun 2023
Review and perspective paper |
| 02 Jun 2023
| 02 Jun 2023
Differentiable programming for Earth system modeling
Maximilian Gelbrecht
CORRESPONDING AUTHOR
Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany
Potsdam Institute for Climate Impact Research, Potsdam, Germany
Alistair White
Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany
Potsdam Institute for Climate Impact Research, Potsdam, Germany
Sebastian Bathiany
Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany
Potsdam Institute for Climate Impact Research, Potsdam, Germany
Niklas Boers
Earth System Modelling, School of Engineering and Design, Technical University of Munich, Munich, Germany
Potsdam Institute for Climate Impact Research, Potsdam, Germany
Department of Mathematics and Global Systems Institute, University of Exeter, Exeter, UK
Related authors
No articles found.
Takahito Mitsui, Peter Ditlevsen, Niklas Boers, and Michel Crucifix
Earth Syst. Dynam., 16, 1569–1584, https://doi.org/10.5194/esd-16-1569-2025, https://doi.org/10.5194/esd-16-1569-2025, 2025
Short summary
Short summary
The late Pleistocene glacial cycles are dominated by a 100-kyr periodicity, rather than other major astronomical periods like 19, 23, 41, or 400 kyr. Various models propose distinct mechanisms to explain this, but their diversity may obscure the key factor behind the 100-kyr periodicity. We propose a timescale-matching hypothesis, suggesting that the ice-sheet climate system responds to astronomical forcing at ∼100 kyr because its intrinsic timescale is closer to 100 kyr than to other periods.
This article is included in the Encyclopedia of Geosciences
Colin Jones, Isaline Bossert, Donovan P. Dennis, Hazel Jeffery, Chris D. Jones, Torben Koenigk, Sina Loriani, Benjamin Sanderson, Roland Séférian, Klaus Wyser, Shuting Yang, Manabu Abe, Sebastian Bathiany, Pascale Braconnot, Victor Brovkin, Friedrich A. Burger, Patrica Cadule, Frederic S. Castruccio, Gokhan Danabasoglu, Andrea Dittus, Jonathan F. Donges, Friederike Fröb, Thomas Frölicher, Goran Georgievski, Chuncheng Guo, Aixue Hu, Peter Lawrence, Paul Lerner, José Licón-Saláiz, Bette Otto-Bliesner, Anastasia Romanou, Elena Shevliakova, Yona Silvy, Didier Swingedouw, Jerry Tjiputra, Jeremy Walton, Andy Wiltshire, Ricarda Winkelmann, Richard Wood, Tokuta Yokohata, and Tilo Ziehn
EGUsphere, https://doi.org/10.5194/egusphere-2025-3604, https://doi.org/10.5194/egusphere-2025-3604, 2025
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
We introduce a new Earth system model experiment protocol to help researchers understand how Earth might respond to positive, zero, and negative carbon emissions. This protocol enables different models to be compared following similar warming and cooling rates. Researchers use the models to explore how the Earth reacts to different climate futures, including the risk of tipping points being exceeded and whether changes can be reversed. The results will support improved long-term climate policy.
This article is included in the Encyclopedia of Geosciences
Michael Aich, Philipp Hess, Baoxiang Pan, Sebastian Bathiany, Yu Huang, and Niklas Boers
EGUsphere, https://doi.org/10.5194/egusphere-2025-2646, https://doi.org/10.5194/egusphere-2025-2646, 2025
Short summary
Short summary
Accurately simulating rainfall is essential to understand the impacts of climate change, especially extreme events such as floods and droughts. Climate models simulate the atmosphere at a coarse resolution and often misrepresent precipitation, leading to biased and overly smooth fields. We improve the precipitation using a machine learning model that is data-efficient, preserves key climate signals such as trends and variability, and significantly improves the representation of extreme events.
This article is included in the Encyclopedia of Geosciences
Ricarda Winkelmann, Donovan P. Dennis, Jonathan F. Donges, Sina Loriani, Ann Kristin Klose, Jesse F. Abrams, Jorge Alvarez-Solas, Torsten Albrecht, David Armstrong McKay, Sebastian Bathiany, Javier Blasco Navarro, Victor Brovkin, Eleanor Burke, Gokhan Danabasoglu, Reik V. Donner, Markus Drüke, Goran Georgievski, Heiko Goelzer, Anna B. Harper, Gabriele Hegerl, Marina Hirota, Aixue Hu, Laura C. Jackson, Colin Jones, Hyungjun Kim, Torben Koenigk, Peter Lawrence, Timothy M. Lenton, Hannah Liddy, José Licón-Saláiz, Maxence Menthon, Marisa Montoya, Jan Nitzbon, Sophie Nowicki, Bette Otto-Bliesner, Francesco Pausata, Stefan Rahmstorf, Karoline Ramin, Alexander Robinson, Johan Rockström, Anastasia Romanou, Boris Sakschewski, Christina Schädel, Steven Sherwood, Robin S. Smith, Norman J. Steinert, Didier Swingedouw, Matteo Willeit, Wilbert Weijer, Richard Wood, Klaus Wyser, and Shuting Yang
EGUsphere, https://doi.org/10.5194/egusphere-2025-1899, https://doi.org/10.5194/egusphere-2025-1899, 2025
Short summary
Short summary
The Tipping Points Modelling Intercomparison Project (TIPMIP) is an international collaborative effort to systematically assess tipping point risks in the Earth system using state-of-the-art coupled and stand-alone domain models. TIPMIP will provide a first global atlas of potential tipping dynamics, respective critical thresholds and key uncertainties, generating an important building block towards a comprehensive scientific basis for policy- and decision-making.
This article is included in the Encyclopedia of Geosciences
Nils Bochow, Anna Poltronieri, and Niklas Boers
The Cryosphere, 18, 5825–5863, https://doi.org/10.5194/tc-18-5825-2024, https://doi.org/10.5194/tc-18-5825-2024, 2024
Short summary
Short summary
Using the latest climate models, we update the understanding of how the Greenland ice sheet responds to climate changes. We found that precipitation and temperature changes in Greenland vary across different regions. Our findings suggest that using uniform estimates for temperature and precipitation for modelling the response of the ice sheet can overestimate ice loss in Greenland. Therefore, this study highlights the need for spatially resolved data in predicting the ice sheet's future.
This article is included in the Encyclopedia of Geosciences
Clara Hummel, Niklas Boers, and Martin Rypdal
EGUsphere, https://doi.org/10.5194/egusphere-2024-3567, https://doi.org/10.5194/egusphere-2024-3567, 2024
Short summary
Short summary
We revisit early warning signals (EWS) for past abrupt climate changes known as Dansgaard-Oeschger (DO) events. Using advanced statistical methods, we find fewer significant EWS than previously reported. While some signals appear consistent across Greenland ice core records, they are not enough to identify the still unknown physical mechanisms behind DO events. This study highlights the complexity of predicting climate changes and urges caution in interpreting (paleo-)climate data.
This article is included in the Encyclopedia of Geosciences
Vasilis Dakos, Chris A. Boulton, Joshua E. Buxton, Jesse F. Abrams, Beatriz Arellano-Nava, David I. Armstrong McKay, Sebastian Bathiany, Lana Blaschke, Niklas Boers, Daniel Dylewsky, Carlos López-Martínez, Isobel Parry, Paul Ritchie, Bregje van der Bolt, Larissa van der Laan, Els Weinans, and Sonia Kéfi
Earth Syst. Dynam., 15, 1117–1135, https://doi.org/10.5194/esd-15-1117-2024, https://doi.org/10.5194/esd-15-1117-2024, 2024
Short summary
Short summary
Tipping points are abrupt, rapid, and sometimes irreversible changes, and numerous approaches have been proposed to detect them in advance. Such approaches have been termed early warning signals and represent a set of methods for identifying changes in the underlying behaviour of a system across time or space that might indicate an approaching tipping point. Here, we review the literature to explore where, how, and which early warnings have been used in real-world case studies so far.
This article is included in the Encyclopedia of Geosciences
Maya Ben-Yami, Lana Blaschke, Sebastian Bathiany, and Niklas Boers
EGUsphere, https://doi.org/10.5194/egusphere-2024-1106, https://doi.org/10.5194/egusphere-2024-1106, 2024
Preprint archived
Short summary
Short summary
Recent work has used observations to find statistical signs that the Atlantic Meridional Overturning Circulation (AMOC) may be approaching a collapse. We find that in complex climate models in which the AMOC does not collapse before 2100, the statistical signs that are present in the observations are not found in the 1850–2014 equivalent model time series. This indicates that the observed statistical signs are not prone to false positives.
This article is included in the Encyclopedia of Geosciences
Takahito Mitsui and Niklas Boers
Clim. Past, 20, 683–699, https://doi.org/10.5194/cp-20-683-2024, https://doi.org/10.5194/cp-20-683-2024, 2024
Short summary
Short summary
In general, the variance and short-lag autocorrelations of the fluctuations increase in a system approaching a critical transition. Using these indicators, we identify statistical precursor signals for the Dansgaard–Oeschger cooling events recorded in two climatic proxies of three Greenland ice core records. We then provide a dynamical systems theory that bridges the gap between observing statistical precursor signals and the physical precursor signs empirically known in paleoclimate research.
This article is included in the Encyclopedia of Geosciences
Nico Wunderling, Anna S. von der Heydt, Yevgeny Aksenov, Stephen Barker, Robbin Bastiaansen, Victor Brovkin, Maura Brunetti, Victor Couplet, Thomas Kleinen, Caroline H. Lear, Johannes Lohmann, Rosa Maria Roman-Cuesta, Sacha Sinet, Didier Swingedouw, Ricarda Winkelmann, Pallavi Anand, Jonathan Barichivich, Sebastian Bathiany, Mara Baudena, John T. Bruun, Cristiano M. Chiessi, Helen K. Coxall, David Docquier, Jonathan F. Donges, Swinda K. J. Falkena, Ann Kristin Klose, David Obura, Juan Rocha, Stefanie Rynders, Norman Julius Steinert, and Matteo Willeit
Earth Syst. Dynam., 15, 41–74, https://doi.org/10.5194/esd-15-41-2024, https://doi.org/10.5194/esd-15-41-2024, 2024
Short summary
Short summary
This paper maps out the state-of-the-art literature on interactions between tipping elements relevant for current global warming pathways. We find indications that many of the interactions between tipping elements are destabilizing. This means that tipping cascades cannot be ruled out on centennial to millennial timescales at global warming levels between 1.5 and 2.0 °C or on shorter timescales if global warming surpasses 2.0 °C.
This article is included in the Encyclopedia of Geosciences
Takahito Mitsui, Matteo Willeit, and Niklas Boers
Earth Syst. Dynam., 14, 1277–1294, https://doi.org/10.5194/esd-14-1277-2023, https://doi.org/10.5194/esd-14-1277-2023, 2023
Short summary
Short summary
The glacial–interglacial cycles of the Quaternary exhibit 41 kyr periodicity before the Mid-Pleistocene Transition (MPT) around 1.2–0.8 Myr ago and ~100 kyr periodicity after that. The mechanism generating these periodicities remains elusive. Through an analysis of an Earth system model of intermediate complexity, CLIMBER-2, we show that the dominant periodicities of glacial cycles can be explained from the viewpoint of synchronization theory.
This article is included in the Encyclopedia of Geosciences
Sara M. Vallejo-Bernal, Frederik Wolf, Niklas Boers, Dominik Traxl, Norbert Marwan, and Jürgen Kurths
Hydrol. Earth Syst. Sci., 27, 2645–2660, https://doi.org/10.5194/hess-27-2645-2023, https://doi.org/10.5194/hess-27-2645-2023, 2023
Short summary
Short summary
Employing event synchronization and complex networks analysis, we reveal a cascade of heavy rainfall events, related to intense atmospheric rivers (ARs): heavy precipitation events (HPEs) in western North America (NA) that occur in the aftermath of land-falling ARs are synchronized with HPEs in central and eastern Canada with a delay of up to 12 d. Understanding the effects of ARs in the rainfall over NA will lead to better anticipating the evolution of the climate dynamics in the region.
This article is included in the Encyclopedia of Geosciences
Keno Riechers, Leonardo Rydin Gorjão, Forough Hassanibesheli, Pedro G. Lind, Dirk Witthaut, and Niklas Boers
Earth Syst. Dynam., 14, 593–607, https://doi.org/10.5194/esd-14-593-2023, https://doi.org/10.5194/esd-14-593-2023, 2023
Short summary
Short summary
Paleoclimate proxy records show that the North Atlantic climate repeatedly transitioned between two regimes during the last glacial interval. This study investigates a bivariate proxy record from a Greenland ice core which reflects past Greenland temperatures and large-scale atmospheric conditions. We reconstruct the underlying deterministic drift by estimating first-order Kramers–Moyal coefficients and identify two separate stable states in agreement with the aforementioned climatic regimes.
This article is included in the Encyclopedia of Geosciences
Taylor Smith, Ruxandra-Maria Zotta, Chris A. Boulton, Timothy M. Lenton, Wouter Dorigo, and Niklas Boers
Earth Syst. Dynam., 14, 173–183, https://doi.org/10.5194/esd-14-173-2023, https://doi.org/10.5194/esd-14-173-2023, 2023
Short summary
Short summary
Multi-instrument records with varying signal-to-noise ratios are becoming increasingly common as legacy sensors are upgraded, and data sets are modernized. Induced changes in higher-order statistics such as the autocorrelation and variance are not always well captured by cross-calibration schemes. Here we investigate using synthetic examples how strong resulting biases can be and how they can be avoided in order to make reliable statements about changes in the resilience of a system.
This article is included in the Encyclopedia of Geosciences
Eirik Myrvoll-Nilsen, Keno Riechers, Martin Wibe Rypdal, and Niklas Boers
Clim. Past, 18, 1275–1294, https://doi.org/10.5194/cp-18-1275-2022, https://doi.org/10.5194/cp-18-1275-2022, 2022
Short summary
Short summary
In layer counted proxy records each measurement is accompanied by a timestamp typically measured by counting periodic layers. Knowledge of the uncertainty of this timestamp is important for a rigorous propagation to further analyses. By assuming a Bayesian regression model to the layer increments we express the dating uncertainty by the posterior distribution, from which chronologies can be sampled efficiently. We apply our framework to dating abrupt warming transitions during the last glacial.
This article is included in the Encyclopedia of Geosciences
Keno Riechers, Takahito Mitsui, Niklas Boers, and Michael Ghil
Clim. Past, 18, 863–893, https://doi.org/10.5194/cp-18-863-2022, https://doi.org/10.5194/cp-18-863-2022, 2022
Short summary
Short summary
Building upon Milancovic's theory of orbital forcing, this paper reviews the interplay between intrinsic variability and external forcing in the emergence of glacial interglacial cycles. It provides the reader with historical background information and with basic theoretical concepts used in recent paleoclimate research. Moreover, it presents new results which confirm the reduced stability of glacial-cycle dynamics after the mid-Pleistocene transition.
This article is included in the Encyclopedia of Geosciences
Keno Riechers and Niklas Boers
Clim. Past, 17, 1751–1775, https://doi.org/10.5194/cp-17-1751-2021, https://doi.org/10.5194/cp-17-1751-2021, 2021
Short summary
Short summary
Greenland ice core data show that the last glacial cycle was punctuated by a series of abrupt climate shifts comprising significant warming over Greenland, retreat of North Atlantic sea ice, and atmospheric reorganization. Statistical analysis of multi-proxy records reveals no systematic lead or lag between the transitions of proxies that represent different climatic subsystems, and hence no evidence for a potential trigger of these so-called Dansgaard–Oeschger events can be found.
This article is included in the Encyclopedia of Geosciences
Cited articles
Arias, P., Bellouin, N., Coppola, E., Jones, R., Krinner, G., Marotzke, J.,
Naik, V., Palmer, M., Plattner, G.-K., Rogelj, J., Rojas, M., Sillmann, J.,
Storelvmo, T., Thorne, P., Trewin, B., Achuta Rao, K., Adhikary, B., Allan,
R., Armour, K., Bala, G., Barimalala, R., Berger, S., Canadell, J., Cassou,
C., Cherchi, A., Collins, W., Collins, W., Connors, S., Corti, S., Cruz, F.,
Dentener, F., Dereczynski, C., Di Luca, A., Diongue Niang, A., Doblas-Reyes,
F., Dosio, A., Douville, H., Engelbrecht, F., Eyring, V., Fischer, E.,
Forster, P., Fox-Kemper, B., Fuglestvedt, J., Fyfe, J., Gillett, N.,
Goldfarb, L., Gorodetskaya, I., Gutierrez, J., Hamdi, R., Hawkins, E.,
Hewitt, H., Hope, P., Islam, A., Jones, C., Kaufman, D., Kopp, R., Kosaka,
Y., Kossin, J., Krakovska, S., Lee, J.-Y., Li, J., Mauritsen, T., Maycock,
T., Meinshausen, M., Min, S.-K., Monteiro, P., Ngo-Duc, T., Otto, F., Pinto,
I., Pirani, A., Raghavan, K., Ranasinghe, R., Ruane, A., Ruiz, L., Sallée,
J.-B., Samset, B., Sathyendranath, S., Seneviratne, S., Sörensson, A.,
Szopa, S., Takayabu, I., Tréguier, A.-M., van den Hurk, B., Vautard, R., von
Schuckmann, K., Zaehle, S., Zhang, X., and Zickfeld, K.: Technical Summary, in: Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, Cambridge, United Kingdom and New
York, NY, USA,
33–144, https://doi.org/10.1017/9781009157896.002, 2021. a
Berger, M., Aftosmis, M., and Muman, S.: Analysis of Slope Limiters on
Irregular Grids, 43rd AIAA Aerospace Sciences Meeting and Exhibit
10–13 January 2005, https://doi.org/10.2514/6.2005-490, 2005. a
Beucler, T., Rasp, S., Pritchard, M., and Gentine, P.: Achieving Conservation
of Energy in Neural Network Emulators for Climate Modeling, ArXiv,
https://doi.org/10.48550/ARXIV.1906.06622, 2019. a
Beucler, T., Pritchard, M., Rasp, S., Ott, J., Baldi, P., and Gentine, P.:
Enforcing Analytic Constraints in Neural Networks Emulating Physical Systems,
Phys. Rev. Lett., 126, 098302, https://doi.org/10.1103/PhysRevLett.126.098302, 2021. a
Bezgin, D. A., Buhendwa, A. B., and Adams, N. A.: JAX-Fluids: A
fully-differentiable high-order computational fluid dynamics solver for
compressible two-phase flows, Comput. Phys. Commun., 282, 108527,
https://doi.org/10.1016/j.cpc.2022.108527, 2023. a
Blondel, M., Berthet, Q., Cuturi, M., Frostig, R., Hoyer, S., Llinares-López,
F., Pedregosa, F., and Vert, J.-P.: Efficient and Modular Implicit
Differentiation, ArXiv, https://doi.org/10.48550/ARXIV.2105.15183, 2021. a
Blonigan, P. J., Fernandez, P., Murman, S. M., Wang, Q., Rigas, G., and Magri,
L.: Toward a chaotic adjoint for LES, ArXiv, https://doi.org/10.48550/ARXIV.1702.06809, 2017. a
Bolton, T. and Zanna, L.: Applications of Deep Learning to Ocean Data Inference
and Subgrid Parameterization, J. Adv. Model. Earth Sy.,
11, 376–399, https://doi.org/10.1029/2018MS001472, 2019. a
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin,
D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., and
Zhang, Q.: JAX: composable transformations of Python+NumPy programs, GitHub [code],
http://github.com/google/jax (last access: 30 May 2023), 2018. a, b, c, d
Campagne, J.-E., Lanusse, F., Zuntz, J., Boucaud, A., Casas, S., Karamanis, M.,
Kirkby, D., Lanzieri, D., Li, Y., and Peel, A.: JAX-COSMO: An End-to-End
Differentiable and GPU Accelerated Cosmology Library, 6, Cosmology and Nongalactic Astrophysics, https://doi.org/10.21105/astro.2302.05163, 2023. a
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D.: Neural
Ordinary Differential Equations, ArXiv, https://doi.org/10.48550/ARXIV.1806.07366, 2018. a
Chizat, L., Oyallon, E., and Bach, F.: On Lazy Training in Differentiable
Programming, in: Advances in Neural Information Processing Systems, edited by:
Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R., Curran Associates, vol. 32,
Inc.,
https://proceedings.neurips.cc/paper/2019/file/ae614c557843b1df326cb29c57225459-Paper.pdf (last access: 30 May 2023),
2019. a
Dauvergne, B. and Hascoët, L.: The Data-Flow Equations of Checkpointing in
Reverse Automatic Differentiation, in: Computational Science – ICCS 2006,
edited by: Alexandrov, V. N., van Albada, G. D., Sloot, P. M. A., and
Dongarra, J., 566–573, Springer Berlin Heidelberg, Berlin, Heidelberg,
2006. a
de Bézenac, E., Pajot, A., and Gallinari, P.: Deep learning for physical
processes: incorporating prior scientific knowledge, J. Statist.
Mech. Theory and Experiment, 2019, 124009,
https://doi.org/10.1088/1742-5468/ab3195, 2019. a
Duane, S., Kennedy, A., Pendleton, B. J., and Roweth, D.: Hybrid Monte Carlo,
Phys. Lett. B, 195, 216–222,
https://doi.org/10.1016/0370-2693(87)91197-X, 1987. a
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., and Taylor, K. E.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization, Geosci. Model Dev., 9, 1937–1958, https://doi.org/10.5194/gmd-9-1937-2016, 2016. a
Farrell, P. E., Ham, D. A., Funke, S. W., and Rognes, M. E.: Automated
Derivation of the Adjoint of High-Level Transient Finite Element Programs,
SIAM J. Sci. Comput., 35, C369–C393,
https://doi.org/10.1137/120873558, 2013. a, b, c, d
Ge, H., Xu, K., and Ghahramani, Z.: Turing: A Language for Flexible
Probabilistic Inference, in: Proceedings of the Twenty-First International
Conference on Artificial Intelligence and Statistics, edited by: Storkey, A.
and Perez-Cruz, F., Proc. Mach. Learn.
Res., 84, 1682–1690,
https://proceedings.mlr.press/v84/ge18b.html (last access: 30 May 2023), 2018. a
Gelbrecht, M., Boers, N., and Kurths, J.: Neural partial differential equations
for chaotic systems, New J. Phys., 23, 043005,
https://doi.org/10.1088/1367-2630/abeb90, 2021. a
Giering, R. and Kaminski, T.: Recipes for Adjoint Code Construction, ACM Trans.
Math. Softw., 24, 437–474, https://doi.org/10.1145/293686.293695, 1998. a, b
Griewank, A. and Walther, A.: Algorithm 799: Revolve: An Implementation of
Checkpointing for the Reverse or Adjoint Mode of Computational
Differentiation, ACM Trans. Math. Softw., 26, 19–45,
https://doi.org/10.1145/347837.347846, 2000. a
Guillaumin, A. P. and Zanna, L.: Stochastic-Deep Learning Parameterization of
Ocean Momentum Forcing, J. Adv. Model. Earth Sy., 13,
e2021MS002534, https://doi.org/10.1029/2021MS002534, 2021. a
Gutiérrez, M. S. and Lucarini, V.: Response and Sensitivity Using Markov
Chains, J. Stat. Phys., 179, 1572–1593,
https://doi.org/10.1007/s10955-020-02504-4, 2020. a
Häfner, D., Jacobsen, R. L., Eden, C., Kristensen, M. R. B., Jochum, M., Nuterman, R., and Vinter, B.: Veros v0.1 – a fast and versatile ocean simulator in pure Python, Geosci. Model Dev., 11, 3299–3312, https://doi.org/10.5194/gmd-11-3299-2018, 2018. a
Häfner, D., Nuterman, R., and Jochum, M.: Fast, Cheap, and
Turbulent—Global Ocean Modeling With GPU Acceleration in Python, J.
Adv. Model. Earth Sy., 13, e2021MS002717,
https://doi.org/10.1029/2021MS002717, 2021. a, b
Hascoët, L. and Pascual, V.: The Tapenade Automatic Differentiation tool:
Principles, Model, and Specification, ACM T. Math.
Softw., 39, 20:1–20:43,
https://doi.org/10.1145/2450153.2450158, 2013. a, b
Hatfield, S., Chantry, M., Dueben, P., Lopez, P., Geer, A., and Palmer, T.:
Building Tangent-Linear and Adjoint Models for Data Assimilation With Neural
Networks, J. Adv. Model. Earth Sy., 13, e2021MS002521,
https://doi.org/10.1029/2021MS002521, 2021. a
Holl, P., Thuerey, N., and Koltun, V.: Learning to Control PDEs with Differentiable Physics, International Conference on Learning Representations, https://openreview.net/forum?id=HyeSin4FPB (last access: 31 May 2023), 2020. a
Hopcroft, P. O. and Valdes, P. J.: Paleoclimate-conditioning reveals a North
Africa land–atmosphere tipping point, P. Natl.
Acad. Sci. USA, 118, e2108783118, https://doi.org/10.1073/pnas.2108783118, 2021. a
Hourdin, F., Mauritsen, T., Gettelman, A., Golaz, J.-C., Balaji, V., Duan, Q.,
Folini, D., Ji, D., Klocke, D., Qian, Y., Rauser, F., Rio, C., Tomassini, L.,
Watanabe, M., and Williamson, D.: The Art and Science of Climate Model
Tuning, B. Am. Meteorol. Soc., 98, 589–602,
https://doi.org/10.1175/BAMS-D-15-00135.1, 2017. a, b, c, d
Irrgang, C., Boers, N., Sonnewald, M., Barnes, E. A., Kadow, C., Staneva, J.,
and Saynisch-Wagner, J.: Towards neural Earth system modelling by integrating
artificial intelligence in Earth system science, Nat. Mach. Int.,
3, 667–674, https://doi.org/10.1038/s42256-021-00374-3, 2021. a, b
Jouvet, G., Cordonnier, G., Kim, B., Lüthi, M., Vieli, A., and Aschwanden, A.:
Deep learning speeds up ice flow modelling by several orders of magnitude,
J. Glaciol., 68, 651–664, https://doi.org/10.1017/jog.2021.120, 2022. a
Kalmikov, A. G. and Heimbach, P.: A Hessian-Based Method for Uncertainty
Quantification in Global Ocean State Estimation, SIAM J. Sci.
Comput., 36, S267–S295, https://doi.org/10.1137/130925311, 2014. a
Kaminski, T., Knorr, W., Schürmann, G., Scholze, M., Rayner, P. J., Zaehle,
S., Blessing, S., Dorigo, W., Gayler, V., Giering, R., Gobron, N., Grant,
J. P., Heimann, M., Hooker-Stroud, A., Houweling, S., Kato, T., Kattge, J.,
Kelley, D., Kemp, S., Koffi, E. N., Köstler, C., Mathieu, P.-P., Pinty,
B., Reick, C. H., Rödenbeck, C., Schnur, R., Scipal, K., Sebald, C.,
Stacke, T., van Scheltinga, A. T., Vossbeck, M., Widmann, H., and Ziehn, T.:
The BETHY/JSBACH Carbon Cycle Data Assimilation System: experiences and
challenges, J. Geophys. Res.-Biogeo., 118, 1414–1426,
https://doi.org/10.1002/jgrg.20118, 2013. a
Kennedy, M. C. and O'Hagan, A.: Bayesian calibration of computer models,
J. Ro. Stat. Soc. B,
63, 425–464, https://doi.org/10.1111/1467-9868.00294, 2001. a
Kim, S., Ji, W., Deng, S., Ma, Y., and Rackauckas, C.: Stiff neural ordinary
differential equations, Chaos, 31, 093122, https://doi.org/10.1063/5.0060697, 2021. a
Klöwer, M., Hatfield, S., Croci, M., Düben, P. D., and Palmer, T. N.:
Fluid simulations accelerated with 16 bits: Approaching 4x speedup on A64FX
by squeezing ShallowWaters.jl into Float16, J. Adv. Model.
Earth Sy., 14, e2021MS002684, https://doi.org/10.1029/2021MS002684, 2022. a
Logg, A., Mardal, K.-A., and Wells, G. (Eds.): Automated Solution of Differential Equations by the Finite Element Method, vol. 84, Springer
Science & Business Media, https://doi.org/10.1007/978-3-642-23099-8, 2012. a, b
Loose, N. and Heimbach, P.: Leveraging Uncertainty Quantification to Design
Ocean Climate Observing Systems, J. Adv. Model. Earth Sy., 13, e2020MS002386, https://doi.org/10.1029/2020MS002386, 2021. a, b
Lucarini, V., Ragone, F., and Lunkeit, F.: Predicting Climate Change Using
Response Theory: Global Averages and Spatial Patterns, J. Stat.
Phys., 166, 1036–1064, https://doi.org/10.1007/s10955-016-1506-z, 2017. a
Lyu, G., Köhl, A., Matei, I., and Stammer, D.: Adjoint-Based Climate Model
Tuning: Application to the Planet Simulator, J. Adv. Model.
Earth Sy., 10, 207–222, https://doi.org/10.1002/2017MS001194,
2018. a, b, c
Marotzke, J., Giering, R., Zhang, K. Q., Stammer, D., Hill, C., and Lee, T.:
Construction of the adjoint MIT ocean general circulation model and
application to Atlantic heat transport sensitivity, J. Geophys. Res.-Oceans, 104, 29529–29547,
https://doi.org/10.1029/1999JC900236, 1999. a
Mauritsen, T., Stevens, B., Roeckner, E., Crueger, T., Esch, M., Giorgetta, M.,
Haak, H., Jungclaus, J., Klocke, D., Matei, D., Mikolajewicz, U., Notz, D.,
Pincus, R., Schmidt, H., and Tomassini, L.: Tuning the climate of a global
model, J. Adv. Model. Earth Sy., 4,
https://doi.org/10.1029/2012MS000154, 2012. a, b, c, d, e
Metz, L., Freeman, C. D., Schoenholz, S. S., and Kachman, T.: Gradients are Not
All You Need, ArXiv, https://doi.org/10.48550/ARXIV.2111.05803, 2021. a, b
Michalak, K. and Ollivier-Gooch, C.: Differentiability of slope limiters on
unstructured grids, in: Proceedings of fourteenth annual conference of the
computational fluid dynamics society of Canada, https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=Differentiability+of+Slope+Limiters+on+Unstructured+Grids&btnG= (last access: 31 May 2023), 2006. a
Mitusch, S. K., Funke, S. W., and Dokken, J. S.: dolfin-adjoint 2018.1:
automated adjoints for FEniCS and Firedrake, J. Open Source Softw.,
4, 1292, https://doi.org/10.21105/joss.01292, 2019. a, b, c, d
Moses, W. and Churavy, V.: Instead of Rewriting Foreign Code for Machine
Learning, Automatically Synthesize Fast Gradients, in: Advances in Neural
Information Processing Systems, edited by: Larochelle, H., Ranzato, M.,
Hadsell, R., Balcan, M. F., and Lin, H., 33, 12472–12485,
Curran Associates, Inc.,
https://proceedings.neurips.cc/paper/2020/file/9332c513ef44b682e9347822c2e457ac-Paper.pdf (last access: 30 May 2023),
2020. a, b, c, d, e
Ni, A. and Wang, Q.: Sensitivity analysis on chaotic dynamical systems by
Non-Intrusive Least Squares Shadowing (NILSS), J. Comput.
Phys., 347, 56–77, https://doi.org/10.1016/j.jcp.2017.06.033, 2017. a
OpenAI: ChatGPT: Optimizing Language Models for Dialogue,
https://openai.com/blog/chatgpt/ (last access: 30 May 2023), 2022. a
Palmer, T. and Stevens, B.: The scientific challenge of understanding and
estimating climate change, P. Natl. Acad. Sci. USA,
116, 24390–24395, https://doi.org/10.1073/pnas.1906691116, 2019. a
Petra, N., Martin, J., Stadler, G., and Ghattas, O.: A Computational Framework
for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic
Newton MCMC with Application to Ice Sheet Flow Inverse Problems, SIAM J. Sci. Comput., 36, A1525–A1555, https://doi.org/10.1137/130934805, 2014. a
Rabier, F., Thépaut, J.-N., and Courtier, P.: Extended assimilation and
forecast experiments with a four-dimensional variational assimilation system,
Q. J. Roy. Meteor. Soc., 124, 1861–1887,
https://doi.org/10.1002/qj.49712455005, 1998. a
Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K., Supekar, R.,
Skinner, D., Ramadhan, A., and Edelman, A.: Universal Differential Equations
for Scientific Machine Learning, ArXiv, https://doi.org/10.48550/ARXIV.2001.04385, 2020. a, b, c
Raissi, M., Perdikaris, P., and Karniadakis, G.: Physics-informed neural
networks: A deep learning framework for solving forward and inverse problems
involving nonlinear partial differential equations, J. Comput.
Phys., 378, 686–707, https://doi.org/10.1016/j.jcp.2018.10.045,
2019. a
Rasp, S.: Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0), Geosci. Model Dev., 13, 2185–2196, https://doi.org/10.5194/gmd-13-2185-2020, 2020. a, b
Rasp, S., Pritchard, M. S., and Gentine, P.: Deep learning to represent subgrid
processes in climate models, P. Natl. Acad. Sci. USA,
115, 9684–9689, https://doi.org/10.1073/pnas.1810286115, 2018. a, b
Rathgeber, F., Ham, D. A., Mitchell, L., Lange, M., Luporini, F., Mcrae, A.
T. T., Bercea, G.-T., Markall, G. R., and Kelly, P. H. J.: Firedrake, ACM
T. Math. Softw., 43, 1–27, https://doi.org/10.1145/2998441,
2016. a, b
Rayner, P. J., Scholze, M., Knorr, W., Kaminski, T., Giering, R., and Widmann,
H.: Two decades of terrestrial carbon fluxes from a carbon cycle data
assimilation system (CCDAS), Global Biogeochem. Cycles, 19, GB2026,
https://doi.org/10.1029/2004GB002254, 2005. a
Ruelle, D.: General linear response formula in statistical mechanics, and the
fluctuation-dissipation theorem far from equilibrium, Phys. Lett. A, 245,
220–224, https://doi.org/10.1016/S0375-9601(98)00419-8, 1998. a
Schneider, T., Lan, S., Stuart, A., and Teixeira, J.: Earth System Modeling
2.0: A Blueprint for Models That Learn From Observations and Targeted
High-Resolution Simulations, Geophys. Res. Lett., 44,
12396–12417, https://doi.org/10.1002/2017GL076101, 2017. a
Schoenholz, S. and Cubuk, E. D.: JAX MD: A Framework for Differentiable
Physics, in: Advances in Neural Information Processing Systems, edited by:
Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H., 33,
11428–11441, Curran Associates, Inc.,
https://proceedings.neurips.cc/paper_files/paper/2020/file/83d3d4b6c9579515e1679aca8cbc8033-Paper.pdf (last access: 30 May 2023),
2020. a
Souhar, O., Faure, J. B., and Paquier, A.: Automatic sensitivity analysis of a
finite volume model for two-dimensional shallow water flows, Environ.
Fluid Mech., 7, 303–315, https://doi.org/10.1007/s10652-007-9028-5, 2007. a
Stammer, D., Wunsch, C., Giering, R., Eckert, C., Heimbach, P., Marotzke, J.,
Adcroft, A., Hill, C. N., and Marshall, J.: Global ocean circulation during
1992–1997, estimated from ocean observations and a general circulation
model, J. Geophys. Res.-Oceans, 107, 1-1–1-27,
https://doi.org/10.1029/2001JC000888, 2002. a
Thacker, W. C.: The role of the Hessian matrix in fitting models to
measurements, J. Geophys. Res.-Oceans, 94, 6177–6196,
https://doi.org/10.1029/JC094iC05p06177, 1989.
a
Tsai, W.-P., Feng, D., Pan, M., Beck, H., Lawson, K., Yang, Y., Liu, J., and
Shen, C.: From calibration to parameter learning: Harnessing the scaling
effects of big data in geoscientific modeling, Nat. Commun., 12,
5988, https://doi.org/10.1038/s41467-021-26107-z, 2021. a
Valdes, P.: Built for stability, Nat. Geosci., 4, 414–416,
https://doi.org/10.1038/ngeo1200, 2011. a
Vettoretti, G., Ditlevsen, P., Jochum, M., and Rasmussen, S. O.: Atmospheric
CO2 control of spontaneous millennial-scale ice age climate oscillations,
Nat. Geosci., 15, 300–306, https://doi.org/10.1038/s41561-022-00920-7, 2022. a
Villa, U., Petra, N., and Ghattas, O.: HIPPYlib: An Extensible Software
Framework for Large-Scale Inverse Problems Governed by PDEs: Part I:
Deterministic Inversion and Linearized Bayesian Inference, ACM Trans. Math.
Softw., 47, 1–34, https://doi.org/10.1145/3428447, 2021. a
Volodina, V. and Challenor, P.: The importance of uncertainty quantification in
model reproducibility, Philosophical Transactions of the Royal Society A:
Mathematical, Phys. Eng. Sci., 379, 20200071,
https://doi.org/10.1098/rsta.2020.0071, 2021. a
Wang, P., Jiang, J., Lin, P., Ding, M., Wei, J., Zhang, F., Zhao, L., Li, Y., Yu, Z., Zheng, W., Yu, Y., Chi, X., and Liu, H.: The GPU version of LASG/IAP Climate System Ocean Model version 3 (LICOM3) under the heterogeneous-compute interface for portability (HIP) framework and its large-scale application , Geosci. Model Dev., 14, 2781–2799, https://doi.org/10.5194/gmd-14-2781-2021, 2021. a
Wang, Q., Hu, R., and Blonigan, P.: Least Squares Shadowing sensitivity
analysis of chaotic limit cycle oscillations, J. Comput.
Phys., 267, 210–224, https://doi.org/10.1016/j.jcp.2014.03.002, 2014. a, b
Williamson, D. B., Blaker, A. T., and Sinha, B.: Tuning without over-tuning: parametric uncertainty quantification for the NEMO ocean model, Geosci. Model Dev., 10, 1789–1816, https://doi.org/10.5194/gmd-10-1789-2017, 2017. a
Yuval, J., O'Gorman, P. A., and Hill, C. N.: Use of Neural Networks for Stable,
Accurate and Physically Consistent Parameterization of Subgrid Atmospheric
Processes With Good Performance at Reduced Precision, Geophys. Res. Lett., 48, e2020GL091363, https://doi.org/10.1029/2020GL091363, 2021. a
Zanna, L. and Bolton, T.: Deep Learning of Unresolved Turbulent Ocean Processes
in Climate Models, John Wiley & Sons, Ltd, chap. 20, 298–306,
https://doi.org/10.1002/9781119646181.ch20, 2021. a
Executive editor
This paper reviews the technique of differentiable programming in Earth System Modeling.
Short summary
Differential programming is a technique that enables the automatic computation of derivatives of the output of models with respect to model parameters. Applying these techniques to Earth system modeling leverages the increasing availability of high-quality data to improve the models themselves. This can be done by either using calibration techniques that use gradient-based optimization or incorporating machine learning methods that can learn previously unresolved influences directly from data.
Differential programming is a technique that enables the automatic computation of derivatives of...
