Articles | Volume 16, issue 15
https://doi.org/10.5194/gmd-16-4501-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-16-4501-2023
© Author(s) 2023. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Using probabilistic machine learning to better model temporal patterns in parameterizations: a case study with the Lorenz 96 model
Department of Computer Science, University of Cambridge, Cambridge, UK
British Antarctic Survey, Cambridge, UK
Hannah M. Christensen
Department of Physics, University of Oxford, Oxford, UK
J. Scott Hosking
British Antarctic Survey, Cambridge, UK
The Alan Turing Institute, London, UK
Damon J. Wischik
Department of Computer Science, University of Cambridge, Cambridge, UK
Related authors
No articles found.
Marc Girona-Mata, Andrew Orr, Martin Widmann, Daniel Bannister, Ghulam Hussain Dars, Scott Hosking, Jesse Norris, David Ocio, Tony Phillips, Jakob Steiner, and Richard E. Turner
Hydrol. Earth Syst. Sci., 29, 3073–3100, https://doi.org/10.5194/hess-29-3073-2025, https://doi.org/10.5194/hess-29-3073-2025, 2025
Short summary
Short summary
We introduce a novel method for improving daily precipitation maps in mountain regions and pilot it across three basins in the Hindu Kush Himalaya (HKH). The approach leverages climate model and weather station data, along with statistical or machine learning techniques. Our results show that this approach outperforms traditional methods, especially in remote ungauged areas, suggesting that it could be used to improve precipitation maps across much of the HKH, as well as other mountain regions.
Emma Ford, Manuela I. Brunner, Hannah Christensen, and Louise Slater
EGUsphere, https://doi.org/10.5194/egusphere-2025-1493, https://doi.org/10.5194/egusphere-2025-1493, 2025
Short summary
Short summary
This study aims to improve prediction and understanding of extreme flood events in UK near-natural catchments. We develop a machine learning framework to assess the contribution of different features to flood magnitude estimation. We find weather patterns are weak predictors and stress the importance of evaluating model performance across and within catchments.
Malcolm J. Roberts, Kevin A. Reed, Qing Bao, Joseph J. Barsugli, Suzana J. Camargo, Louis-Philippe Caron, Ping Chang, Cheng-Ta Chen, Hannah M. Christensen, Gokhan Danabasoglu, Ivy Frenger, Neven S. Fučkar, Shabeh ul Hasson, Helene T. Hewitt, Huanping Huang, Daehyun Kim, Chihiro Kodama, Michael Lai, Lai-Yung Ruby Leung, Ryo Mizuta, Paulo Nobre, Pablo Ortega, Dominique Paquin, Christopher D. Roberts, Enrico Scoccimarro, Jon Seddon, Anne Marie Treguier, Chia-Ying Tu, Paul A. Ullrich, Pier Luigi Vidale, Michael F. Wehner, Colin M. Zarzycki, Bosong Zhang, Wei Zhang, and Ming Zhao
Geosci. Model Dev., 18, 1307–1332, https://doi.org/10.5194/gmd-18-1307-2025, https://doi.org/10.5194/gmd-18-1307-2025, 2025
Short summary
Short summary
HighResMIP2 is a model intercomparison project focusing on high-resolution global climate models, that is, those with grid spacings of 25 km or less in the atmosphere and ocean, using simulations of decades to a century in length. We are proposing an update of our simulation protocol to make the models more applicable to key questions for climate variability and hazard in present-day and future projections and to build links with other communities to provide more robust climate information.
Kenza Tazi, Andrew Orr, Javier Hernandez-González, Scott Hosking, and Richard E. Turner
Hydrol. Earth Syst. Sci., 28, 4903–4925, https://doi.org/10.5194/hess-28-4903-2024, https://doi.org/10.5194/hess-28-4903-2024, 2024
Short summary
Short summary
This work aims to improve the understanding of precipitation patterns in High-mountain Asia, a crucial water source for around 1.9 billion people. Through a novel machine learning method, we generate high-resolution precipitation predictions, including the likelihoods of floods and droughts. Compared to state-of-the-art methods, our method is simpler to implement and more suitable for small datasets. The method also shows accuracy comparable to or better than existing benchmark datasets.
Anna Vaughan, Will Tebbutt, J. Scott Hosking, and Richard E. Turner
Geosci. Model Dev., 15, 251–268, https://doi.org/10.5194/gmd-15-251-2022, https://doi.org/10.5194/gmd-15-251-2022, 2022
Short summary
Short summary
We develop a new method for climate downscaling, i.e. transforming low-resolution climate model output to high-resolution projections, using a deep-learning model known as a convolutional conditional neural process. This model is shown to outperform an ensemble of baseline methods for downscaling daily maximum temperature and precipitation and provides a powerful new downscaling framework for climate impact studies.
Andrew Orr, J. Scott Hosking, Aymeric Delon, Lars Hoffmann, Reinhold Spang, Tracy Moffat-Griffin, James Keeble, Nathan Luke Abraham, and Peter Braesicke
Atmos. Chem. Phys., 20, 12483–12497, https://doi.org/10.5194/acp-20-12483-2020, https://doi.org/10.5194/acp-20-12483-2020, 2020
Short summary
Short summary
Polar stratospheric clouds (PSCs) are clouds found in the Antarctic winter stratosphere and are implicated in the formation of the ozone hole. These clouds can sometimes be formed or enhanced by mountain waves, formed as air passes over hills or mountains. However, this important mechanism is missing in coarse-resolution climate models, limiting our ability to simulate ozone. This study examines an attempt to include the effects of mountain waves and their impact on PSCs and ozone.
Cited articles
Agarwal, N., Kondrashov, D., Dueben, P., Ryzhov, E., and Berloff, P.: A
Comparison of Data-Driven Approaches to Build Low-Dimensional Ocean Models,
J. Adv. Model. Earth Sy., 13, e2021MS002537, https://doi.org/10.1029/2021MS002537, 2021. a
Arcomano, T., Szunyogh, I., Wikner, A., Pathak, J., Hunt, B. R., and Ott, E.: A
Hybrid Approach to Atmospheric Modeling That Combines Machine Learning With a
Physics-Based Numerical Model, J. Adv. Model. Earth Sy.,
14, e2021MS002712, https://doi.org/10.1029/2021MS002712, 2022. a, b
Arjovsky, M., Chintala, S., and Bottou, L.: Wasserstein generative adversarial
networks, in: International conference on machine learning,
PMLR, 214–223, https://doi.org/10.48550/arXiv.1701.07875, 2017. a
Arnold, H. M., Moroz, I. M., and Palmer, T. N.: Stochastic parametrizations and
model uncertainty in the Lorenz’96 system, Philosophical Transactions of
the Royal Society A: Mathematical, Phys. Eng. Sci., 371,
20110479, https://doi.org/10.1098/rsta.2011.0479, 2013. a, b, c, d
Bahdanau, D., Cho, K., and Bengio, Y.: Neural machine translation by jointly
learning to align and translate, arXiv [preprint],
https://doi.org/10.48550/arXiv.1409.0473, 2014. a
Berner, J., Jung, T., and Palmer, T.: Systematic model error: The impact of
increased horizontal resolution versus improved stochastic and deterministic
parameterizations, J. Climate, 25, 4946–4962, 2012. a
Beucler, T., Pritchard, M., Gentine, P., and Rasp, S.: Towards
physically-consistent, data-driven models of convection, in: IGARSS 2020-2020
IEEE International Geoscience and Remote Sensing Symposium,
IEEE, 3987–3990, https://doi.org/10.48550/arXiv.2002.08525, 2020. a, b
Beucler, T., Pritchard, M., Rasp, S., Ott, J., Baldi, P., and Gentine, P.:
Enforcing analytic constraints in neural networks emulating physical systems,
Phys. Rev. Lett., 126, 098302, https://doi.org/10.1103/PhysRevLett.126.098302, 2021. a, b
Casella, G. and Berger, R. L.: Chap. 6.3, The Likelihood
Principle, in: Statistical inference, Duxbury Press, p. 290, ISBN 0-534-24312-6, 2002. a
Chattopadhyay, A., Hassanzadeh, P., and Subramanian, D.: Data-driven predictions of a multiscale Lorenz 96 chaotic system using machine-learning methods: reservoir computing, artificial neural network, and long short-term memory network, Nonlin. Processes Geophys., 27, 373–389, https://doi.org/10.5194/npg-27-373-2020, 2020a. a, b
Chattopadhyay, A., Subel, A., and Hassanzadeh, P.: Data-driven
super-parameterization using deep learning: Experimentation with multiscale
Lorenz 96 systems and transfer learning, Journal of Advances in Modeling
Earth Systems, 12, e2020MS002084, https://doi.org/10.1029/2020MS002084, 2020b. a
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F.,
Schwenk, H., and Bengio, Y.: Learning phrase representations using RNN
encoder-decoder for statistical machine translation, arXiv [preprint],
https://doi.org/10.48550/arXiv.1406.1078,
2014. a, b
Christensen, H. M., Berner, J., Coleman, D. R., and Palmer, T.: Stochastic
parameterization and El Niño–Southern Oscillation, J. Climate,
30, 17–38, 2017. a
Chung, J., Ahn, S., and Bengio, Y.: Hierarchical multiscale recurrent neural
networks, arXiv [preprint],
https://doi.org/10.48550/arXiv.1609.01704, 2016. a
Crommelin, D. and Vanden-Eijnden, E.: Subgrid-scale parameterization with
conditional Markov chains, J. Atmos. Sci., 65,
2661–2675, 2008. a
Eck, D. and Schmidhuber, J.: A first look at music composition using lstm
recurrent neural networks, Istituto Dalle Molle Di Studi Sull Intelligenza
Artificiale, 103, 2002. a
Esteban, C., Hyland, S. L., and Rätsch, G.: Real-valued (medical) time
series generation with recurrent conditional gans, arXiv [preprint]
https://doi.org/10.48550/arXiv.1706.02633, 2017. a
Freedman, D. and Diaconis, P.: On the histogram as a density estimator: L 2
theory, Z. Wahrscheinlichkeit.,
57, 453–476, 1981. a
Goodfellow, I.: Nips 2016 tutorial: Generative adversarial networks, arXiv
[preprint],
https://doi.org/10.48550/arXiv.1701.00160, 2016. a
Goodfellow, I., Bengio, Y., and Courville, A.: Deep Learning, chap. Sequence
Modeling: Recurrent and Recursive Nets, MIT Press, 372–384,
http://www.deeplearningbook.org (last access: 1 March 2023), 2016. a
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D.,
Ozair, S., Courville, A., and Bengio, Y.: Generative adversarial networks,
arXiv [preprint],
https://doi.org/10.48550/arXiv.1406.2661, 2014. a, b
Graves, A.: Generating sequences with recurrent neural networks, arXiv [preprint]
https://doi.org/10.48550/arXiv:1308.0850, 2013. a
Grover, A., Dhar, M., and Ermon, S.: Flow-gan: Combining maximum likelihood and
adversarial learning in generative models, in: Proceedings of the AAAI
Conference on Artificial Intelligence, vol. 32, https://doi.org/10.48550/arXiv.1705.08868, 2018. a
Guillaumin, A. P. and Zanna, L.: Stochastic-deep learning parameterization of
ocean momentum forcing, J. Adv. Model. Earth Sy., 13,
e2021MS002534, https://doi.org/10.1029/2021MS002534, 2021. a
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.:
Improved training of wasserstein gans, arXiv [preprint], https://doi.org/10.48550/arXiv:1704.00028, 2017. a
Hasselmann, K.: Stochastic climate models part I. Theory, Tellus, 28, 473–485,
1976. a
Hochreiter, S. and Schmidhuber, J.: Long short-term memory, Neural Comput.,
9, 1735–1780, 1997. a
Johnson, S. J., Stockdale, T. N., Ferranti, L., Balmaseda, M. A., Molteni, F., Magnusson, L., Tietsche, S., Decremer, D., Weisheimer, A., Balsamo, G., Keeley, S. P. E., Mogensen, K., Zuo, H., and Monge-Sanz, B. M.: SEAS5: the new ECMWF seasonal forecast system, Geosci. Model Dev., 12, 1087–1117, https://doi.org/10.5194/gmd-12-1087-2019, 2019. a
Kingma, D. P. and Ba, J.: Adam: A method for stochastic optimization, arXiv
[preprint], https://doi.org/10.48550/arXiv:1412.6980, 2014. a
Kingma, D. P. and Welling, M.: Auto-encoding variational bayes, arXiv [preprint], https://doi.org/10.48550/arXiv:1312.6114, 2013. a
Krasnopolsky, V. M., Fox-Rabinovitz, M. S., and Belochitski, A. A.: Using
ensemble of neural networks to learn stochastic convection parameterizations
for climate and numerical weather prediction models from data simulated by a
cloud resolving model, Advances in Artificial Neural Systems, 2013, 5-5, https://doi.org/10.1155/2013/485913, 2013. a, b
Kwasniok, F.: Data-based stochastic subgrid-scale parametrization: an approach
using cluster-weighted modelling, Philos. T. Roy.
Soc. A, 370, 1061–1086,
2012. a
Leutbecher, M. and Palmer, T. N.: Ensemble forecasting, J.
Comput. Phys., 227, 3515–3539, 2008. a
Leutbecher, M., Lock, S.-J., Ollinaho, P., Lang, S. T., Balsamo, G., Bechtold,
P., Bonavita, M., Christensen, H. M., Diamantakis, M., Dutra, E., English, S., Fisher, M., Forbes, R. M., Goddard, J., Haiden, T., Hogan, R. J., Juricke, S., Lawrence, H., MacLeod, D., Magnusson, L., Malardel, S., Massart, S., Sandu, I., Smolarkiewicz, P. K., Subramanian, A., Vitart, F., Wedi, N., and Weisheimer, A.:
Stochastic representations of model uncertainties at ECMWF: State of the art
and future vision, Q. J. Roy. Meteorol. Soc.,
143, 2315–2339, 2017. a, b
Liu, Y., Kutz, J. N., and Brunton, S. L.: Hierarchical deep learning of
multiscale differential equation time-steppers, arXiv [preprint], https://doi.org/10.48550/arXiv:2008.09768, 2020. a
Lorenz, E. N.: Regimes in simple systems, J. Atmos. Sci,
63, 2056–2073, 2006. a
Mogren, O.: C-RNN-GAN: Continuous recurrent neural networks with adversarial
training, arXiv [preprint], https://doi.org/10.48550/arXiv:1611.09904, 2016. a, b
Molteni, F., Stockdale, T., Balmaseda, M., Balsamo, G., Buizza, R., Ferranti,
L., Magnusson, L., Mogensen, K., Palmer, T., and Vitart, F.: The new ECMWF
seasonal forecast system (System 4), European Centre for
medium-range weather forecasts Reading, vol. 49, https://doi.org/10.21957/4nery093i, 2011. a, b
Palmer, T. N., Buizza, R., Doblas-Reyes, F., Jung, T., Leutbecher, M., Shutts,
G. J., Steinheimer, M., and Weisheimer, A.: Stochastic parametrization and
model uncertainty, https://doi.org/10.21957/ps8gbwbdv, 2009. a, b, c, d
Parthipan, R.: raghul-parthipan/l96_rnn: v1.1.0 (v1.1.0), Zenodo [code], https://doi.org/10.5281/zenodo.7837814, 2023. a
Parthipan, R. and Wischik, D. J.: Don't Waste Data: Transfer Learning to
Leverage All Data for Machine-Learnt Climate Model Emulation, arXiv [preprint], https://doi.org/10.48550/arXiv:2210.04001, 2022. a
Rasp, S.: Coupled online learning as a way to tackle instabilities and biases in neural network parameterizations: general algorithms and Lorenz 96 case study (v1.0), Geosci. Model Dev., 13, 2185–2196, https://doi.org/10.5194/gmd-13-2185-2020, 2020. a, b
Sanchez, C., Williams, K. D., and Collins, M.: Improved stochastic physics
schemes for global weather and climate models, Q. J. Roy.
Meteorol. Soc., 142, 147–159, 2016. a
Schneider, T., Teixeira, J., Bretherton, C. S., Brient, F., Pressel, K. G.,
Schär, C., and Siebesma, A. P.: Climate goals and computing the future of
clouds, Nat. Clim. Change, 7, 3–5, 2017. a
Selten, F. M. and Branstator, G.: Preferred regime transition routes and
evidence for an unstable periodic orbit in a baroclinic model, J.
Atmos. Sci., 61, 2267–2282, 2004. a
Silverman, B. W.: Density estimation for statistics and data analysis,
CRC press, vol. 26, ISBN 978-0412246203, 1986. a
Skamarock, W. C., Klemp, J. B., Dudhia, J., Gill, D. O., Liu, Z., Berner, J.,
Wang, W., Powers, J. G., Duda, M. G., Barker, D. M., and Huang, X.-Y.: A description of
the advanced research WRF model version 4, National Center for Atmospheric
Research: Boulder, CO, USA, 145, https://doi.org/10.5065/1dfh-6p97, 2019. a, b
Stephenson, D., Hannachi, A., and O'Neill, A.: On the existence of multiple
climate regimes, Q. J. Roy. Meteor. Soc., 130, 583–605, 2004. a
Stockdale, T. N., Anderson, D. L., Balmaseda, M. A., Doblas-Reyes, F.,
Ferranti, L., Mogensen, K., Palmer, T. N., Molteni, F., and Vitart, F.: ECMWF
seasonal forecast system 3 and its prediction of sea surface temperature,
Clim. Dynam., 37, 455–471, 2011. a
Sutskever, I., Vinyals, O., and Le, Q. V.: Sequence to sequence learning with
neural networks, in: Advances in neural information processing systems,
3104–3112, https://doi.org/10.48550/arXiv.1409.3215, 2014. a, b
Theis, L., van den Oord, A., and Bethge, M.: A note on the evaluation of
generative models, arXiv [preprint],
https://doi.org/10.48550/arXiv:1511.01844, 2015. a
Vlachas, P. R., Byeon, W., Wan, Z. Y., Sapsis, T. P., and Koumoutsakos, P.:
Data-driven forecasting of high-dimensional chaotic systems with long
short-term memory networks, Proc. Roy. Soc. A:, 474, 20170844, https://doi.org/10.1098/rspa.2017.0844, 2018. a
Vlachas, P. R., Pathak, J., Hunt, B. R., Sapsis, T. P., Girvan, M., Ott, E.,
and Koumoutsakos, P.: Backpropagation algorithms and reservoir computing in
recurrent neural networks for the forecasting of complex spatiotemporal
dynamics, Neural Networks, 126, 191–217, 2020. a
Walters, D., Baran, A. J., Boutle, I., Brooks, M., Earnshaw, P., Edwards, J., Furtado, K., Hill, P., Lock, A., Manners, J., Morcrette, C., Mulcahy, J., Sanchez, C., Smith, C., Stratton, R., Tennant, W., Tomassini, L., Van Weverberg, K., Vosper, S., Willett, M., Browse, J., Bushell, A., Carslaw, K., Dalvi, M., Essery, R., Gedney, N., Hardiman, S., Johnson, B., Johnson, C., Jones, A., Jones, C., Mann, G., Milton, S., Rumbold, H., Sellar, A., Ujiie, M., Whitall, M., Williams, K., and Zerroukat, M.: The Met Office Unified Model Global Atmosphere 7.0/7.1 and JULES Global Land 7.0 configurations, Geosci. Model Dev., 12, 1909–1963, https://doi.org/10.5194/gmd-12-1909-2019, 2019. a
Wilks, D. S.: Empirical Distributions and Exploratory Data Analysis, in: Statistical methods in the atmospheric sciences, Academic
press, 100, 33–35, 2011. a
Yuval, J., O'Gorman, P. A., and Hill, C. N.: Use of neural networks for stable,
accurate and physically consistent parameterization of subgrid atmospheric
processes with good performance at reduced precision, Geophys. Res.
Lett., 48, e2020GL091363, https://doi.org/10.1029/2020GL091363, 2021. a, b
Zelinka, M. D., Myers, T. A., McCoy, D. T., Po-Chedley, S., Caldwell, P. M.,
Ceppi, P., Klein, S. A., and Taylor, K. E.: Causes of higher climate
sensitivity in CMIP6 models, Geophys. Res. Lett., 47,
e2019GL085782, https://doi.org/10.1029/2019GL085782, 2020. a
Zhao, M., Cong, Y., Dai, S., and Carin, L.: Bridging Maximum Likelihood and
Adversarial Learning via α-Divergence, in: Proceedings of the AAAI
Conference on Artificial Intelligence, 34, 6901–6908, https://doi.org/10.48550/arXiv.2007.06178, 2020. a
Short summary
How can we create better climate models? We tackle this by proposing a data-driven successor to the existing approach for capturing key temporal trends in climate models. We combine probability, allowing us to represent uncertainty, with machine learning, a technique to learn relationships from data which are undiscoverable to humans. Our model is often superior to existing baselines when tested in a simple atmospheric simulation.
How can we create better climate models? We tackle this by proposing a data-driven successor to...