Articles | Volume 17, issue 24
https://doi.org/10.5194/gmd-17-8909-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-17-8909-2024
© Author(s) 2024. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Development and technical paper
| 
19 Dec 2024
Development and technical paper |
| 19 Dec 2024
| 19 Dec 2024
The effect of lossy compression of numerical weather prediction data on data analysis: a case study using enstools-compression 2023.11
Meteorological Institute, Ludwig Maximilian University of Munich, 80333 Munich, Germany
Robert Redl
Meteorological Institute, Ludwig Maximilian University of Munich, 80333 Munich, Germany
Marc Rautenhaus
Hub of Computing and Data Science, Visual Data Analysis Group, Universität Hamburg, 20146 Hamburg, Germany
Tobias Selz
Meteorological Institute, Ludwig Maximilian University of Munich, 80333 Munich, Germany
Takumi Matsunobu
Meteorological Institute, Ludwig Maximilian University of Munich, 80333 Munich, Germany
Kameswar Rao Modali
Hub of Computing and Data Science, Visual Data Analysis Group, Universität Hamburg, 20146 Hamburg, Germany
George Craig
Meteorological Institute, Ludwig Maximilian University of Munich, 80333 Munich, Germany
Related authors
Francisco J. Doblas-Reyes, Jenni Kontkanen, Irina Sandu, Mario Acosta, Mohammed Hussam Al Turjmam, Ivan Alsina-Ferrer, Miguel Andrés-Martínez, Leo Arriola, Marvin Axness, Marc Batlle Martín, Peter Bauer, Tobias Becker, Daniel Beltrán, Sebastian Beyer, Hendryk Bockelmann, Pierre-Antoine Bretonnière, Sebastien Cabaniols, Silvia Caprioli, Miguel Castrillo, Aparna Chandrasekar, Suvarchal Cheedela, Victor Correal, Emanuele Danovaro, Paolo Davini, Jussi Enkovaara, Claudia Frauen, Barbara Früh, Aina Gaya Àvila, Paolo Ghinassi, Rohit Ghosh, Supriyo Ghosh, Iker González, Katherine Grayson, Matthew Griffith, Ioan Hadade, Christopher Haine, Carl Hartick, Utz-Uwe Haus, Shane Hearne, Heikki Järvinen, Bernat Jiménez, Amal John, Marlin Juchem, Thomas Jung, Jessica Kegel, Matthias Kelbling, Kai Keller, Bruno Kinoshita, Theresa Kiszler, Daniel Klocke, Lukas Kluft, Nikolay Koldunov, Tobias Kölling, Joonas Kolstela, Luis Kornblueh, Sergey Kosukhin, Aleksander Lacima-Nadolnik, Jeisson Javier Leal Rojas, Jonni Lehtiranta, Tuomas Lunttila, Anna Luoma, Pekka Manninen, Alexey Medvedev, Sebastian Milinski, Ali Omar Abdelazim Mohammed, Sebastian Müller, Devaraju Naryanappa, Natalia Nazarova, Sami Niemelä, Bimochan Niraula, Henrik Nortamo, Aleksi Nummelin, Matteo Nurisso, Pablo Ortega, Stella Paronuzzi, Xabier Pedruzo Bagazgoitia, Charles Pelletier, Carlos Peña, Suraj Polade, Himansu Pradhan, Rommel Quintanilla, Tiago Quintino, Thomas Rackow, Jouni Räisänen, Maqsood Mubarak Rajput, René Redler, Balthasar Reuter, Nuno Rocha Monteiro, Francesc Roura-Adserias, Silva Ruppert, Susan Sayed, Reiner Schnur, Tanvi Sharma, Dmitry Sidorenko, Outi Sievi-Korte, Albert Soret, Christian Steger, Bjorn Stevens, Jan Streffing, Jaleena Sunny, Luiggi Tenorio, Stephan Thober, Ulf Tigerstedt, Oriol Tinto, Juha Tonttila, Heikki Tuomenvirta, Lauri Tuppi, Ginka Van Thielen, Emanuele Vitali, Jost von Hardenberg, Ingo Wagner, Nils Wedi, Jan Wehner, Sven Willner, Xavier Yepes-Arbós, Florian Ziemen, and Janos Zimmermann
EGUsphere, https://doi.org/10.5194/egusphere-2025-2198, https://doi.org/10.5194/egusphere-2025-2198, 2025
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
The Climate Change Adaptation Digital Twin (Climate DT) pioneers the operationalisation of climate projections. The system produces global simulations with local granularity for adaptation decision-making. Applications are embedded to generate tailored indicators. A unified workflow orchestrates all components in several supercomputers. Data management ensures consistency and streaming enables real-time use. It is a complementary innovation to initiatives like CMIP, CORDEX, and climate services.
Ralf Döscher, Mario Acosta, Andrea Alessandri, Peter Anthoni, Thomas Arsouze, Tommi Bergman, Raffaele Bernardello, Souhail Boussetta, Louis-Philippe Caron, Glenn Carver, Miguel Castrillo, Franco Catalano, Ivana Cvijanovic, Paolo Davini, Evelien Dekker, Francisco J. Doblas-Reyes, David Docquier, Pablo Echevarria, Uwe Fladrich, Ramon Fuentes-Franco, Matthias Gröger, Jost v. Hardenberg, Jenny Hieronymus, M. Pasha Karami, Jukka-Pekka Keskinen, Torben Koenigk, Risto Makkonen, François Massonnet, Martin Ménégoz, Paul A. Miller, Eduardo Moreno-Chamarro, Lars Nieradzik, Twan van Noije, Paul Nolan, Declan O'Donnell, Pirkka Ollinaho, Gijs van den Oord, Pablo Ortega, Oriol Tintó Prims, Arthur Ramos, Thomas Reerink, Clement Rousset, Yohan Ruprich-Robert, Philippe Le Sager, Torben Schmith, Roland Schrödner, Federico Serva, Valentina Sicardi, Marianne Sloth Madsen, Benjamin Smith, Tian Tian, Etienne Tourigny, Petteri Uotila, Martin Vancoppenolle, Shiyu Wang, David Wårlind, Ulrika Willén, Klaus Wyser, Shuting Yang, Xavier Yepes-Arbós, and Qiong Zhang
Geosci. Model Dev., 15, 2973–3020, https://doi.org/10.5194/gmd-15-2973-2022, https://doi.org/10.5194/gmd-15-2973-2022, 2022
Short summary
Short summary
The Earth system model EC-Earth3 is documented here. Key performance metrics show physical behavior and biases well within the frame known from recent models. With improved physical and dynamic features, new ESM components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond.
Oriol Tintó Prims, Mario C. Acosta, Andrew M. Moore, Miguel Castrillo, Kim Serradell, Ana Cortés, and Francisco J. Doblas-Reyes
Geosci. Model Dev., 12, 3135–3148, https://doi.org/10.5194/gmd-12-3135-2019, https://doi.org/10.5194/gmd-12-3135-2019, 2019
Short summary
Short summary
Mixed-precision approaches can provide substantial speed-ups for both computing- and memory-bound codes, requiring little effort. A novel method to enable modern and legacy codes to benefit from a reduction of precision without sacrificing accuracy is presented. Using a precision emulator and a divide-and-conquer algorithm it identifies the parts that cannot handle reduced precision and the ones that can. The method has been proved using two ocean models, NEMO and ROMS, with promising results.
Francisco J. Doblas-Reyes, Jenni Kontkanen, Irina Sandu, Mario Acosta, Mohammed Hussam Al Turjmam, Ivan Alsina-Ferrer, Miguel Andrés-Martínez, Leo Arriola, Marvin Axness, Marc Batlle Martín, Peter Bauer, Tobias Becker, Daniel Beltrán, Sebastian Beyer, Hendryk Bockelmann, Pierre-Antoine Bretonnière, Sebastien Cabaniols, Silvia Caprioli, Miguel Castrillo, Aparna Chandrasekar, Suvarchal Cheedela, Victor Correal, Emanuele Danovaro, Paolo Davini, Jussi Enkovaara, Claudia Frauen, Barbara Früh, Aina Gaya Àvila, Paolo Ghinassi, Rohit Ghosh, Supriyo Ghosh, Iker González, Katherine Grayson, Matthew Griffith, Ioan Hadade, Christopher Haine, Carl Hartick, Utz-Uwe Haus, Shane Hearne, Heikki Järvinen, Bernat Jiménez, Amal John, Marlin Juchem, Thomas Jung, Jessica Kegel, Matthias Kelbling, Kai Keller, Bruno Kinoshita, Theresa Kiszler, Daniel Klocke, Lukas Kluft, Nikolay Koldunov, Tobias Kölling, Joonas Kolstela, Luis Kornblueh, Sergey Kosukhin, Aleksander Lacima-Nadolnik, Jeisson Javier Leal Rojas, Jonni Lehtiranta, Tuomas Lunttila, Anna Luoma, Pekka Manninen, Alexey Medvedev, Sebastian Milinski, Ali Omar Abdelazim Mohammed, Sebastian Müller, Devaraju Naryanappa, Natalia Nazarova, Sami Niemelä, Bimochan Niraula, Henrik Nortamo, Aleksi Nummelin, Matteo Nurisso, Pablo Ortega, Stella Paronuzzi, Xabier Pedruzo Bagazgoitia, Charles Pelletier, Carlos Peña, Suraj Polade, Himansu Pradhan, Rommel Quintanilla, Tiago Quintino, Thomas Rackow, Jouni Räisänen, Maqsood Mubarak Rajput, René Redler, Balthasar Reuter, Nuno Rocha Monteiro, Francesc Roura-Adserias, Silva Ruppert, Susan Sayed, Reiner Schnur, Tanvi Sharma, Dmitry Sidorenko, Outi Sievi-Korte, Albert Soret, Christian Steger, Bjorn Stevens, Jan Streffing, Jaleena Sunny, Luiggi Tenorio, Stephan Thober, Ulf Tigerstedt, Oriol Tinto, Juha Tonttila, Heikki Tuomenvirta, Lauri Tuppi, Ginka Van Thielen, Emanuele Vitali, Jost von Hardenberg, Ingo Wagner, Nils Wedi, Jan Wehner, Sven Willner, Xavier Yepes-Arbós, Florian Ziemen, and Janos Zimmermann
EGUsphere, https://doi.org/10.5194/egusphere-2025-2198, https://doi.org/10.5194/egusphere-2025-2198, 2025
This preprint is open for discussion and under review for Geoscientific Model Development (GMD).
Short summary
Short summary
The Climate Change Adaptation Digital Twin (Climate DT) pioneers the operationalisation of climate projections. The system produces global simulations with local granularity for adaptation decision-making. Applications are embedded to generate tailored indicators. A unified workflow orchestrates all components in several supercomputers. Data management ensures consistency and streaming enables real-time use. It is a complementary innovation to initiatives like CMIP, CORDEX, and climate services.
Tim Radke, Susanne Fuchs, Christian Wilms, Iuliia Polkova, and Marc Rautenhaus
Geosci. Model Dev., 18, 1017–1039, https://doi.org/10.5194/gmd-18-1017-2025, https://doi.org/10.5194/gmd-18-1017-2025, 2025
Short summary
Short summary
In our study, we built upon previous work to investigate the patterns artificial intelligence (AI) learns to detect atmospheric features like tropical cyclones (TCs) and atmospheric rivers (ARs). As primary objective, we adopt a method to explain the AI used and investigate the plausibility of learned patterns. We find that plausible patterns are learned for both TCs and ARs. Hence, the chosen method is very useful for gaining confidence in the AI-based detection of atmospheric features.
Christoph Fischer, Andreas H. Fink, Elmar Schömer, Marc Rautenhaus, and Michael Riemer
Geosci. Model Dev., 17, 4213–4228, https://doi.org/10.5194/gmd-17-4213-2024, https://doi.org/10.5194/gmd-17-4213-2024, 2024
Short summary
Short summary
This study presents a method for identifying and tracking 3-D potential vorticity structures within African easterly waves (AEWs). Each identified structure is characterized by descriptors, including its 3-D position and orientation, which have been validated through composite comparisons. A trough-centric perspective on the descriptors reveals the evolution and distinct characteristics of AEWs. These descriptors serve as valuable statistical inputs for the study of AEW-related phenomena.
Konstantin Krüger, Andreas Schäfler, Martin Weissmann, and George C. Craig
Weather Clim. Dynam., 5, 491–509, https://doi.org/10.5194/wcd-5-491-2024, https://doi.org/10.5194/wcd-5-491-2024, 2024
Short summary
Short summary
Initial conditions of current numerical weather prediction models insufficiently represent the sharp vertical gradients across the midlatitude tropopause. Observation-space data assimilation output is used to study the influence of assimilated radiosondes on the tropopause. The radiosondes reduce systematic biases of the model background and sharpen temperature and wind gradients in the analysis. Tropopause sharpness is still underestimated in the analysis, which may impact weather forecasts.
Hyunju Jung, Peter Knippertz, Yvonne Ruckstuhl, Robert Redl, Tijana Janjic, and Corinna Hoose
Weather Clim. Dynam., 4, 1111–1134, https://doi.org/10.5194/wcd-4-1111-2023, https://doi.org/10.5194/wcd-4-1111-2023, 2023
Short summary
Short summary
A narrow rainfall belt in the tropics is an important feature for large-scale circulations and the global water cycle. The accurate simulation of this rainfall feature has been a long-standing problem, with the reasons behind that unclear. We present a novel diagnostic tool that allows us to disentangle processes important for rainfall, which changes due to modifications in model. Using our diagnostic tool, one can potentially identify sources of uncertainty in weather and climate models.
Christoph Neuhauser, Maicon Hieronymus, Michael Kern, Marc Rautenhaus, Annika Oertel, and Rüdiger Westermann
Geosci. Model Dev., 16, 4617–4638, https://doi.org/10.5194/gmd-16-4617-2023, https://doi.org/10.5194/gmd-16-4617-2023, 2023
Short summary
Short summary
Numerical weather prediction models rely on parameterizations for sub-grid-scale processes, which are a source of uncertainty. We present novel visual analytics solutions to analyze interactively the sensitivities of a selected prognostic variable to multiple model parameters along trajectories regarding similarities in temporal development and spatiotemporal relationships. The proposed workflow is applied to cloud microphysical sensitivities along coherent strongly ascending trajectories.
Andreas A. Beckert, Lea Eisenstein, Annika Oertel, Tim Hewson, George C. Craig, and Marc Rautenhaus
Geosci. Model Dev., 16, 4427–4450, https://doi.org/10.5194/gmd-16-4427-2023, https://doi.org/10.5194/gmd-16-4427-2023, 2023
Short summary
Short summary
We investigate the benefit of objective 3-D front detection with modern interactive visual analysis techniques for case studies of extra-tropical cyclones and comparisons of frontal structures between different numerical weather prediction models. The 3-D frontal structures show agreement with 2-D fronts from surface analysis charts and augment them in the vertical dimension. We see great potential for more complex studies of atmospheric dynamics and for operational weather forecasting.
Konstantin Krüger, Andreas Schäfler, Martin Wirth, Martin Weissmann, and George C. Craig
Atmos. Chem. Phys., 22, 15559–15577, https://doi.org/10.5194/acp-22-15559-2022, https://doi.org/10.5194/acp-22-15559-2022, 2022
Short summary
Short summary
A comprehensive data set of airborne lidar water vapour profiles is compared with ERA5 reanalyses for a robust characterization of the vertical structure of the mid-latitude lower-stratospheric moist bias. We confirm a moist bias of up to 55 % at 1.3 km altitude above the tropopause and uncover a decreasing bias beyond. Collocated O3 and H2O observations reveal a particularly strong bias in the mixing layer, indicating insufficiently modelled transport processes fostering the bias.
Takumi Matsunobu, Christian Keil, and Christian Barthlott
Weather Clim. Dynam., 3, 1273–1289, https://doi.org/10.5194/wcd-3-1273-2022, https://doi.org/10.5194/wcd-3-1273-2022, 2022
Short summary
Short summary
This study quantifies the impact of poorly constrained parameters used to represent aerosol–cloud–precipitation interactions on precipitation and cloud forecasts associated with uncertainties in input atmospheric states. Uncertainties in these parameters have a non-negligible impact on daily precipitation amount and largely change the amount of cloud. The comparison between different weather situations reveals that the impact becomes more important when convection is triggered by local effects.
Andreas Alexander Beckert, Lea Eisenstein, Annika Oertel, Timothy Hewson, George C. Craig, and Marc Rautenhaus
Weather Clim. Dynam. Discuss., https://doi.org/10.5194/wcd-2022-36, https://doi.org/10.5194/wcd-2022-36, 2022
Preprint withdrawn
Short summary
Short summary
This study revises and extends a previously presented 3-D objective front detection method and demonstrates its benefits to analyse weather dynamics in numerical simulation data. Based on two case studies of extratropical cyclones, we demonstrate the evaluation of conceptual models from dynamic meteorology, illustrate the benefits of our interactive analysis approach by comparing fronts in data with different model resolutions, and study the impact of convection on fronts.
Christoph Fischer, Andreas H. Fink, Elmar Schömer, Roderick van der Linden, Michael Maier-Gerber, Marc Rautenhaus, and Michael Riemer
Geosci. Model Dev., 15, 4447–4468, https://doi.org/10.5194/gmd-15-4447-2022, https://doi.org/10.5194/gmd-15-4447-2022, 2022
Short summary
Short summary
Potential vorticity (PV) analysis plays a central role in studying atmospheric dynamics. For example, anomalies in the PV field near the tropopause are linked to extreme weather events. In this study, an objective strategy to identify these anomalies is presented and evaluated. As a novel concept, it can be applied to three-dimensional (3-D) data sets. Supported by 3-D visualizations, we illustrate advantages of this new analysis over existing studies along a case study.
Raphael Kriegmair, Yvonne Ruckstuhl, Stephan Rasp, and George Craig
Nonlin. Processes Geophys., 29, 171–181, https://doi.org/10.5194/npg-29-171-2022, https://doi.org/10.5194/npg-29-171-2022, 2022
Short summary
Short summary
Our regional numerical weather prediction models run at kilometer-scale resolutions. Processes that occur at smaller scales not yet resolved contribute significantly to the atmospheric flow. We use a neural network (NN) to represent the unresolved part of physical process such as cumulus clouds. We test this approach on a simplified, yet representative, 1D model and find that the NN corrections vastly improve the model forecast up to a couple of days.
Ralf Döscher, Mario Acosta, Andrea Alessandri, Peter Anthoni, Thomas Arsouze, Tommi Bergman, Raffaele Bernardello, Souhail Boussetta, Louis-Philippe Caron, Glenn Carver, Miguel Castrillo, Franco Catalano, Ivana Cvijanovic, Paolo Davini, Evelien Dekker, Francisco J. Doblas-Reyes, David Docquier, Pablo Echevarria, Uwe Fladrich, Ramon Fuentes-Franco, Matthias Gröger, Jost v. Hardenberg, Jenny Hieronymus, M. Pasha Karami, Jukka-Pekka Keskinen, Torben Koenigk, Risto Makkonen, François Massonnet, Martin Ménégoz, Paul A. Miller, Eduardo Moreno-Chamarro, Lars Nieradzik, Twan van Noije, Paul Nolan, Declan O'Donnell, Pirkka Ollinaho, Gijs van den Oord, Pablo Ortega, Oriol Tintó Prims, Arthur Ramos, Thomas Reerink, Clement Rousset, Yohan Ruprich-Robert, Philippe Le Sager, Torben Schmith, Roland Schrödner, Federico Serva, Valentina Sicardi, Marianne Sloth Madsen, Benjamin Smith, Tian Tian, Etienne Tourigny, Petteri Uotila, Martin Vancoppenolle, Shiyu Wang, David Wårlind, Ulrika Willén, Klaus Wyser, Shuting Yang, Xavier Yepes-Arbós, and Qiong Zhang
Geosci. Model Dev., 15, 2973–3020, https://doi.org/10.5194/gmd-15-2973-2022, https://doi.org/10.5194/gmd-15-2973-2022, 2022
Short summary
Short summary
The Earth system model EC-Earth3 is documented here. Key performance metrics show physical behavior and biases well within the frame known from recent models. With improved physical and dynamic features, new ESM components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond.
Marcel Meyer, Iuliia Polkova, Kameswar Rao Modali, Laura Schaffer, Johanna Baehr, Stephan Olbrich, and Marc Rautenhaus
Weather Clim. Dynam., 2, 867–891, https://doi.org/10.5194/wcd-2-867-2021, https://doi.org/10.5194/wcd-2-867-2021, 2021
Short summary
Short summary
Novel techniques from computer science are used to study extreme weather events. Inspired by the interactive 3-D visual analysis of the recently released ERA5 reanalysis data, we improve commonly used metrics for measuring polar winter storms and outbreaks of cold air. The software (Met.3D) that we have extended and applied as part of this study is freely available and can be used generically for 3-D visualization of a broad variety of atmospheric processes in weather and climate data.
Yong Wang, Guang J. Zhang, Shaocheng Xie, Wuyin Lin, George C. Craig, Qi Tang, and Hsi-Yen Ma
Geosci. Model Dev., 14, 1575–1593, https://doi.org/10.5194/gmd-14-1575-2021, https://doi.org/10.5194/gmd-14-1575-2021, 2021
Short summary
Short summary
A stochastic deep convection parameterization is implemented into the US Department of Energy Energy Exascale Earth System Model Atmosphere Model version 1 (EAMv1). Compared to the default model, the well-known problem of
too much light rain and too little heavy rainis largely alleviated over the tropics with the stochastic scheme. Results from this study provide important insights into the model performance of EAMv1 when stochasticity is included in the deep convective parameterization.
Xavier Fettweis, Stefan Hofer, Uta Krebs-Kanzow, Charles Amory, Teruo Aoki, Constantijn J. Berends, Andreas Born, Jason E. Box, Alison Delhasse, Koji Fujita, Paul Gierz, Heiko Goelzer, Edward Hanna, Akihiro Hashimoto, Philippe Huybrechts, Marie-Luise Kapsch, Michalea D. King, Christoph Kittel, Charlotte Lang, Peter L. Langen, Jan T. M. Lenaerts, Glen E. Liston, Gerrit Lohmann, Sebastian H. Mernild, Uwe Mikolajewicz, Kameswarrao Modali, Ruth H. Mottram, Masashi Niwano, Brice Noël, Jonathan C. Ryan, Amy Smith, Jan Streffing, Marco Tedesco, Willem Jan van de Berg, Michiel van den Broeke, Roderik S. W. van de Wal, Leo van Kampenhout, David Wilton, Bert Wouters, Florian Ziemen, and Tobias Zolles
The Cryosphere, 14, 3935–3958, https://doi.org/10.5194/tc-14-3935-2020, https://doi.org/10.5194/tc-14-3935-2020, 2020
Short summary
Short summary
We evaluated simulated Greenland Ice Sheet surface mass balance from 5 kinds of models. While the most complex (but expensive to compute) models remain the best, the faster/simpler models also compare reliably with observations and have biases of the same order as the regional models. Discrepancies in the trend over 2000–2012, however, suggest that large uncertainties remain in the modelled future SMB changes as they are highly impacted by the meltwater runoff biases over the current climate.
Oriol Tintó Prims, Mario C. Acosta, Andrew M. Moore, Miguel Castrillo, Kim Serradell, Ana Cortés, and Francisco J. Doblas-Reyes
Geosci. Model Dev., 12, 3135–3148, https://doi.org/10.5194/gmd-12-3135-2019, https://doi.org/10.5194/gmd-12-3135-2019, 2019
Short summary
Short summary
Mixed-precision approaches can provide substantial speed-ups for both computing- and memory-bound codes, requiring little effort. A novel method to enable modern and legacy codes to benefit from a reduction of precision without sacrificing accuracy is presented. Using a precision emulator and a divide-and-conquer algorithm it identifies the parts that cannot handle reduced precision and the ones that can. The method has been proved using two ocean models, NEMO and ROMS, with promising results.
M. Rautenhaus, M. Kern, A. Schäfler, and R. Westermann
Geosci. Model Dev., 8, 2329–2353, https://doi.org/10.5194/gmd-8-2329-2015, https://doi.org/10.5194/gmd-8-2329-2015, 2015
Short summary
Short summary
This article presents "Met.3D", a new open-source tool for the interactive 3D visualization of numerical ensemble weather predictions. Met.3D builds a bridge from proven 2D visualization methods commonly used in meteorology to 3D visualization and implements approaches to using the ensemble to allow the user to assess forecast uncertainty. The article is the first part of a two-paper study discussing how 3D and ensemble visualization can be used in a meaningful way suited to weather forecasting.
M. Rautenhaus, C. M. Grams, A. Schäfler, and R. Westermann
Geosci. Model Dev., 8, 2355–2377, https://doi.org/10.5194/gmd-8-2355-2015, https://doi.org/10.5194/gmd-8-2355-2015, 2015
Short summary
Short summary
This article presents the application of interactive 3D visualization of ensemble
weather predictions to forecasting warm conveyor belt situations during aircraft-based atmospheric research campaigns. A method to predict 3D probabilities of the spatial occurrence of WCBs is developed and integrated into the 3D visualization tool "Met.3D", introduced in the first part of this two-paper study. A case study demonstrates the use of 3D and uncertainty visualization for weather forecasting.
Related subject area
Earth and space science informatics
DNS (v1.0): an open-source ray-tracing tool for space geodetic techniques
The OpenMindat v1.0.0 R package: a machine interface to Mindat open data to facilitate data-intensive geoscience discoveries
A time-dependent three-dimensional dayside magnetopause model based on quasi-elastodynamic theory
DustNet (v1): skilful neural network predictions of dust aerosols over the Saharan desert
RiverBedDynamics v1.0: a Landlab component for computing two-dimensional sediment transport and river bed evolution
A GPU parallelization of the neXtSIM-DG dynamical core (v0.3.1)
The Earth System Grid Federation (ESGF) Virtual Aggregation (CMIP6 v20240125)
Can AI be enabled to perform dynamical downscaling? A latent diffusion model to mimic kilometer-scale COSMO5.0_CLM9 simulations
Moving beyond post hoc explainable artificial intelligence: a perspective paper on lessons learned from dynamical climate modeling
Remote-sensing-based forest canopy height mapping: some models are useful, but might they provide us with even more insights when combined?
FLAML version 2.3.3 model-based assessment of gross primary productivity at forest, grassland, and cropland ecosystem sites
Checking the consistency of 3D geological models
GNNWR: an open-source package of spatiotemporal intelligent regression methods for modeling spatial and temporal nonstationarity
A New Reduction Model for Enhancing the Interpolation Accuracy of VMF1/VMF3 Tropospheric Products in GNSS Applications
FZStats v1.0: a raster statistics toolbox for simultaneous management of spatial stratified heterogeneity and positional dependence in Python
Random forests with spatial proxies for environmental modelling: opportunities and pitfalls
An improved global pressure and zenith wet delay model with optimized vertical correction considering the spatiotemporal variability in multiple height-scale factors
kNNDM CV: k-fold nearest-neighbour distance matching cross-validation for map accuracy estimation
Accelerating Lagrangian transport simulations on graphics processing units: performance optimizations of Massive-Parallel Trajectory Calculations (MPTRAC) v2.6
Focal-TSMP: deep learning for vegetation health prediction and agricultural drought assessment from a regional climate simulation
Tomofast-x 2.0: an open-source parallel code for inversion of potential field data with topography using wavelet compression
Functional analysis of variance (ANOVA) for carbon flux estimates from remote sensing data
The 4D reconstruction of dynamic geological evolution processes for renowned geological features
Machine learning for numerical weather and climate modelling: a review
Overcoming barriers to enable convergence research by integrating ecological and climate sciences: the NCAR–NEON system Version 1
A close look at using national ground stations for the statistical modeling of NO2
Ensemble of optimised machine learning algorithms for predicting surface soil moisture content at a global scale
Hazard assessment modeling and software development of earthquake-triggered landslides in the Sichuan–Yunnan area, China
A generalized spatial autoregressive neural network method for three-dimensional spatial interpolation
The Common Community Physics Package (CCPP) Framework v6
Causal deep learning models for studying the Earth system
A methodological framework for improving the performance of data-driven models: a case study for daily runoff prediction in the Maumee domain, USA
SHAFTS (v2022.3): a deep-learning-based Python package for simultaneous extraction of building height and footprint from sentinel imagery
Bayesian atmospheric correction over land: Sentinel-2/MSI and Landsat 8/OLI
Twenty-five years of the IPCC Data Distribution Centre at the DKRZ and the Reference Data Archive for CMIP data
Effectiveness and computational efficiency of absorbing boundary conditions for full-waveform inversion
LAND-SUITE V1.0: a suite of tools for statistically based landslide susceptibility zonation
Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5
Fast infrared radiative transfer calculations using graphics processing units: JURASSIC-GPU v2.0
CSDMS: a community platform for numerical modeling of Earth surface processes
A new methodological framework for geophysical sensor combinations associated with machine learning algorithms to understand soil attributes
Model calibration using ESEm v1.1.0 – an open, scalable Earth system emulator
Turbidity maximum zone index: a novel model for remote extraction of the turbidity maximum zone in different estuaries
dh2loop 1.0: an open-source Python library for automated processing and classification of geological logs
Copula-based synthetic data augmentation for machine-learning emulators
Automated geological map deconstruction for 3D model construction using map2loop 1.0 and map2model 1.0
A spatially explicit approach to simulate urban heat mitigation with InVEST (v3.8.0)
S-SOM v1.0: a structural self-organizing map algorithm for weather typing
Using Shapley additive explanations to interpret extreme gradient boosting predictions of grassland degradation in Xilingol, China
Current status on the need for improved accessibility to climate models code
Florian Zus, Kyriakos Balidakis, Ali Hasan Dogan, Rohith Thundathil, Galina Dick, and Jens Wickert
Geosci. Model Dev., 18, 4951–4964, https://doi.org/10.5194/gmd-18-4951-2025, https://doi.org/10.5194/gmd-18-4951-2025, 2025
Short summary
Short summary
Atmospheric signal propagation effects are one of the largest error sources in the analysis of space geodetic techniques. Inaccuracies in the modelling map into errors in positioning, navigation and timing. We describe the open-source ray-tracing tool DNS and show the two outstanding features of this tool compared to previous model developments: it can handle both the troposphere and the ionosphere, and it does so efficiently. This makes the tool perfectly suited for geoscientific applications.
Xiang Que, Jiyin Zhang, Weilin Chen, Jolyon Ralph, and Xiaogang Ma
Geosci. Model Dev., 18, 4455–4467, https://doi.org/10.5194/gmd-18-4455-2025, https://doi.org/10.5194/gmd-18-4455-2025, 2025
Short summary
Short summary
This paper describes an R package as the machine interface to the open data of Mindat.org, one of the world's most widely used databases of mineral species and their distribution. In the past decades, many geoscientists have been using Mindat data, but an open data service has never been fully established. The machine interface described in this paper will be an efficient way to meet the overwhelming data needs.
Yaxin Gu, Yi Wang, Fengsi Wei, Xueshang Feng, Andrey Samsonov, Xiaojian Song, Boyi Wang, Pingbing Zuo, Chaowei Jiang, Yalan Chen, Xiaojun Xu, and Zilu Zhou
Geosci. Model Dev., 18, 4215–4229, https://doi.org/10.5194/gmd-18-4215-2025, https://doi.org/10.5194/gmd-18-4215-2025, 2025
Short summary
Short summary
Understanding how solar wind interacts with Earth’s protective magnetic shield is a frontier issue in space physics, critical for predicting space weather risks. However, most existing magnetopause models cannot capture this time-dependent process, and numerical simulations demand significant computational resources. We have developed the first time-dependent 3D model capable of showing these shield changes. It is significantly more precise and performs nearly instant computations.
Trish E. Nowak, Andy T. Augousti, Benno I. Simmons, and Stefan Siegert
Geosci. Model Dev., 18, 3509–3532, https://doi.org/10.5194/gmd-18-3509-2025, https://doi.org/10.5194/gmd-18-3509-2025, 2025
Short summary
Short summary
The DustNet model uses deep neural networks to accurately predict Saharan mineral dust transport in the atmosphere. It offers fast and precise forecasts with predictions achieved in just 2.1 s on a standard computer. This innovative approach outperforms traditional models, which take hours to produce a forecast and use high-energy supercomputers. By making high-quality dust monitoring accessible and efficient, DustNet can improve weather, climate, and air quality forecasts.
Angel D. Monsalve, Samuel R. Anderson, Nicole M. Gasparini, and Elowyn M. Yager
Geosci. Model Dev., 18, 3427–3451, https://doi.org/10.5194/gmd-18-3427-2025, https://doi.org/10.5194/gmd-18-3427-2025, 2025
Short summary
Short summary
Rivers shape landscapes by moving sediments and changing their beds, but existing computer models oversimplify these processes by only considering erosion. We developed new software that simulates how rivers transport sediments and change over time through both erosion and deposition. This helps scientists and engineers better predict river behavior for water management, flood prevention, and ecosystem protection.
Robert Jendersie, Christian Lessig, and Thomas Richter
Geosci. Model Dev., 18, 3017–3040, https://doi.org/10.5194/gmd-18-3017-2025, https://doi.org/10.5194/gmd-18-3017-2025, 2025
Short summary
Short summary
Accurate computer simulations are critical to understanding how climate change will affect local communities. An important part of such simulations is sea ice, which affects even distant areas in the long term. In our work, we explore how GPUs (graphics processing units), computer chips originally designed for gaming allow for faster simulation of sea ice with a new software, the neXtSIM-DG dynamical core. We discuss multiple options and demonstrate that using GPUs makes more accurate simulations feasible.
Ezequiel Cimadevilla, Bryan N. Lawrence, and Antonio S. Cofiño
Geosci. Model Dev., 18, 2461–2478, https://doi.org/10.5194/gmd-18-2461-2025, https://doi.org/10.5194/gmd-18-2461-2025, 2025
Short summary
Short summary
The Earth System Grid Federation (ESGF) stores an enormous amount of climate data spread across millions of files in data centres all over the world. Accessing and working with this scientific information is quite complex. This work presents ESGF Virtual Aggregation, an approach that combines data from different sources into a format that is ready for analysis straightaway.
Elena Tomasi, Gabriele Franch, and Marco Cristoforetti
Geosci. Model Dev., 18, 2051–2078, https://doi.org/10.5194/gmd-18-2051-2025, https://doi.org/10.5194/gmd-18-2051-2025, 2025
Short summary
Short summary
High-resolution weather data are crucial for many applications, typically generated via resource-intensive numerical models through dynamical downscaling. We developed an AI model using latent diffusion models (LDMs) to mimic this process, increasing weather data resolution over Italy from 25 to 2 km. LDM outperforms other methods, accurately capturing local patterns and extreme events. This approach offers a cost-effective alternative, with potential disruptive application in climate sciences.
Ryan J. O'Loughlin, Dan Li, Richard Neale, and Travis A. O'Brien
Geosci. Model Dev., 18, 787–802, https://doi.org/10.5194/gmd-18-787-2025, https://doi.org/10.5194/gmd-18-787-2025, 2025
Short summary
Short summary
We draw from traditional climate modeling practices to make recommendations for machine-learning (ML)-driven climate science. Our intended audience is climate modelers who are relatively new to ML. We show how component-level understanding – obtained when scientists can link model behavior to parts within the overall model – should guide the development and evaluation of ML models. Better understanding yields a stronger basis for trust in the models. We highlight several examples to demonstrate.
Nikola Besic, Nicolas Picard, Cédric Vega, Jean-Daniel Bontemps, Lionel Hertzog, Jean-Pierre Renaud, Fajwel Fogel, Martin Schwartz, Agnès Pellissier-Tanon, Gabriel Destouet, Frédéric Mortier, Milena Planells-Rodriguez, and Philippe Ciais
Geosci. Model Dev., 18, 337–359, https://doi.org/10.5194/gmd-18-337-2025, https://doi.org/10.5194/gmd-18-337-2025, 2025
Short summary
Short summary
The creation of advanced mapping models for forest attributes, utilizing remote sensing data and incorporating machine or deep learning methods, has become a key area of interest in the domain of forest observation and monitoring. This paper introduces a method where we blend and collectively interpret five models dedicated to estimating forest canopy height. We achieve this through Bayesian model averaging, offering a comprehensive analysis of these remote-sensing-based products.
Jie Lai, Yuan Zhang, Anzhi Wang, Wenli Fei, Yiwei Diao, Rongping Li, and Jiabin Wu
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2024-169, https://doi.org/10.5194/gmd-2024-169, 2025
Revised manuscript accepted for GMD
Short summary
Short summary
In this study, a new model called FLAML-LUE was created by combining the FLAML model with LUE-based models, the latter provides the key variables of vegetation growth for modeling. These models utilize the Fast Lightweight Automated Machine Learning (FLAML) framework, using variables of LUE models, to investigate the potential of estimating site-scale GPP.
Marion N. Parquer, Eric A. de Kemp, Boyan Brodaric, and Michael J. Hillier
Geosci. Model Dev., 18, 71–100, https://doi.org/10.5194/gmd-18-71-2025, https://doi.org/10.5194/gmd-18-71-2025, 2025
Short summary
Short summary
This is a proof-of-concept paper outlining a general approach to how 3D geological models would be checked to be geologically
reasonable. We do this with a consistency-checking tool that looks at geological feature pairs and their spatial, temporal, and internal polarity characteristics. The idea is to assess if geological relationships from a specific 3D geological model match what is allowed in the real world from the perspective of geological principles.
Ziyu Yin, Jiale Ding, Yi Liu, Ruoxu Wang, Yige Wang, Yijun Chen, Jin Qi, Sensen Wu, and Zhenhong Du
Geosci. Model Dev., 17, 8455–8468, https://doi.org/10.5194/gmd-17-8455-2024, https://doi.org/10.5194/gmd-17-8455-2024, 2024
Short summary
Short summary
In geography, understanding how relationships between different factors change over time and space is crucial. This study implements two neural-network-based spatiotemporal regression models and an open-source Python package named Geographically Neural Network Weighted Regression to capture relationships between factors. This makes it a valuable tool for researchers in fields such as environmental science, urban planning, and public health.
Peng Sun, Kefei Zhang, Dantong Zhu, Dongsheng Zhao, Shuangshuang Shi, Xuexi Liu, Minghao Zhang, and Suqin Wu
Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2024-123, https://doi.org/10.5194/gmd-2024-123, 2024
Revised manuscript accepted for GMD
Short summary
Short summary
GNSS signal is delayed when it transmits through the neutral gas. In this contribution, a new model was developed for reducing the VMF1/VMF3 grid-wise ground-surface ZHD and ZWD values to the target height to improve the ZHD and ZWD interpolation performance. Test results showed that the accuracy of the ZHD, ZWD interpolated from the VMF1/VMF3 products deduced by the new model was significantly improved compared to traditional methods.
Na Ren, Daojun Zhang, and Qiuming Cheng
EGUsphere, https://doi.org/10.5194/egusphere-2024-2461, https://doi.org/10.5194/egusphere-2024-2461, 2024
Short summary
Short summary
While Focal Statistics and Zonal Statistics deal with Spatial Position Dependence (SPD) and Spatial Stratified Heterogeneity (SSH) separately, the developed Focal-Zonal Mixed Statistics can handle both simultaneously. This new tool has the potential to become a general statistics tool. The integrated FZStats v1.0 toolbox in this study includes all three models mentioned above, providing new methodological support for understanding and addressing spatial statistical issues.
Carles Milà, Marvin Ludwig, Edzer Pebesma, Cathryn Tonne, and Hanna Meyer
Geosci. Model Dev., 17, 6007–6033, https://doi.org/10.5194/gmd-17-6007-2024, https://doi.org/10.5194/gmd-17-6007-2024, 2024
Short summary
Short summary
Spatial proxies, such as coordinates and distances, are often used as predictors in random forest models for predictive mapping. In a simulation and two case studies, we investigated the conditions under which their use is appropriate. We found that spatial proxies are not always beneficial and should not be used as a default approach without careful consideration. We also provide insights into the reasons behind their suitability, how to detect them, and potential alternatives.
Chunhua Jiang, Xiang Gao, Huizhong Zhu, Shuaimin Wang, Sixuan Liu, Shaoni Chen, and Guangsheng Liu
Geosci. Model Dev., 17, 5939–5959, https://doi.org/10.5194/gmd-17-5939-2024, https://doi.org/10.5194/gmd-17-5939-2024, 2024
Short summary
Short summary
With ERA5 hourly data, we show spatiotemporal characteristics of pressure and zenith wet delay (ZWD) and propose an empirical global pressure and ZWD grid model with a broader operating space which can provide accurate pressure, ZWD, zenith hydrostatic delay, and zenith tropospheric delay estimates for any selected time and location over globe. IGPZWD will be of great significance for the tropospheric augmentation in real-time GNSS positioning and atmospheric water vapor remote sensing.
Jan Linnenbrink, Carles Milà, Marvin Ludwig, and Hanna Meyer
Geosci. Model Dev., 17, 5897–5912, https://doi.org/10.5194/gmd-17-5897-2024, https://doi.org/10.5194/gmd-17-5897-2024, 2024
Short summary
Short summary
Estimation of map accuracy based on cross-validation (CV) in spatial modelling is pervasive but controversial. Here, we build upon our previous work and propose a novel, prediction-oriented k-fold CV strategy for map accuracy estimation in which the distribution of geographical distances between prediction and training points is taken into account when constructing the CV folds. Our method produces more reliable estimates than other CV methods and can be used for large datasets.
Lars Hoffmann, Kaveh Haghighi Mood, Andreas Herten, Markus Hrywniak, Jiri Kraus, Jan Clemens, and Mingzhao Liu
Geosci. Model Dev., 17, 4077–4094, https://doi.org/10.5194/gmd-17-4077-2024, https://doi.org/10.5194/gmd-17-4077-2024, 2024
Short summary
Short summary
Lagrangian particle dispersion models are key for studying atmospheric transport but can be computationally intensive. To speed up simulations, the MPTRAC model was ported to graphics processing units (GPUs). Performance optimization of data structures and memory alignment resulted in runtime improvements of up to 75 % on NVIDIA A100 GPUs for ERA5-based simulations with 100 million particles. These optimizations make the MPTRAC model well suited for future high-performance computing systems.
Mohamad Hakam Shams Eddin and Juergen Gall
Geosci. Model Dev., 17, 2987–3023, https://doi.org/10.5194/gmd-17-2987-2024, https://doi.org/10.5194/gmd-17-2987-2024, 2024
Short summary
Short summary
In this study, we use deep learning and a climate simulation to predict the vegetation health as it would be observed from satellites. We found that the developed model can help to identify regions with a high risk of agricultural drought. The main applications of this study are to estimate vegetation products for periods where no satellite data are available and to forecast the future vegetation response to climate change based on climate scenarios.
Vitaliy Ogarko, Kim Frankcombe, Taige Liu, Jeremie Giraud, Roland Martin, and Mark Jessell
Geosci. Model Dev., 17, 2325–2345, https://doi.org/10.5194/gmd-17-2325-2024, https://doi.org/10.5194/gmd-17-2325-2024, 2024
Short summary
Short summary
We present a major release of the Tomofast-x open-source gravity and magnetic inversion code that is enhancing its performance and applicability for both industrial and academic studies. We focus on real-world mineral exploration scenarios, while offering flexibility for applications at regional scale or for crustal studies. The optimisation work described in this paper is fundamental to allowing more complete descriptions of the controls on magnetisation, including remanence.
Jonathan Hobbs, Matthias Katzfuss, Hai Nguyen, Vineet Yadav, and Junjie Liu
Geosci. Model Dev., 17, 1133–1151, https://doi.org/10.5194/gmd-17-1133-2024, https://doi.org/10.5194/gmd-17-1133-2024, 2024
Short summary
Short summary
The cycling of carbon among the land, oceans, and atmosphere is a closely monitored process in the global climate system. These exchanges between the atmosphere and the surface can be quantified using a combination of atmospheric carbon dioxide observations and computer models. This study presents a statistical method for investigating the similarities and differences in the estimated surface–atmosphere carbon exchange when different computer model assumptions are invoked.
Jiateng Guo, Zhibin Liu, Xulei Wang, Lixin Wu, Shanjun Liu, and Yunqiang Li
Geosci. Model Dev., 17, 847–864, https://doi.org/10.5194/gmd-17-847-2024, https://doi.org/10.5194/gmd-17-847-2024, 2024
Short summary
Short summary
This study proposes a 3D and temporally dynamic (4D) geological modeling method. Several simulation and actual cases show that the 4D spatial and temporal evolution of regional geological formations can be modeled easily using this method with smooth boundaries. The 4D modeling system can dynamically present the regional geological evolution process under the timeline, which will be helpful to the research and teaching on the formation of typical and complex geological features.
Catherine O. de Burgh-Day and Tennessee Leeuwenburg
Geosci. Model Dev., 16, 6433–6477, https://doi.org/10.5194/gmd-16-6433-2023, https://doi.org/10.5194/gmd-16-6433-2023, 2023
Short summary
Short summary
Machine learning (ML) is an increasingly popular tool in the field of weather and climate modelling. While ML has been used in this space for a long time, it is only recently that ML approaches have become competitive with more traditional methods. In this review, we have summarized the use of ML in weather and climate modelling over time; provided an overview of key ML concepts, methodologies, and terms; and suggested promising avenues for further research.
Danica L. Lombardozzi, William R. Wieder, Negin Sobhani, Gordon B. Bonan, David Durden, Dawn Lenz, Michael SanClements, Samantha Weintraub-Leff, Edward Ayres, Christopher R. Florian, Kyla Dahlin, Sanjiv Kumar, Abigail L. S. Swann, Claire M. Zarakas, Charles Vardeman, and Valerio Pascucci
Geosci. Model Dev., 16, 5979–6000, https://doi.org/10.5194/gmd-16-5979-2023, https://doi.org/10.5194/gmd-16-5979-2023, 2023
Short summary
Short summary
We present a novel cyberinfrastructure system that uses National Ecological Observatory Network measurements to run Community Terrestrial System Model point simulations in a containerized system. The simple interface and tutorials expand access to data and models used in Earth system research by removing technical barriers and facilitating research, educational opportunities, and community engagement. The NCAR–NEON system enables convergence of climate and ecological sciences.
Foeke Boersma and Meng Lu
EGUsphere, https://doi.org/10.5194/egusphere-2023-1260, https://doi.org/10.5194/egusphere-2023-1260, 2023
Short summary
Short summary
Air pollution harms health and society. Understanding and predicting it is crucial. Various models are developed to model air pollution. However, the consistency exhibited by a model in different areas is commonly neglected. Our study accounts for this and shows lower accuracy near busy roads, but higher in less populated areas. Considering location characteristics in air pollution predictions is important in comparing statistical models and understanding the health-society-space relationship.
Qianqian Han, Yijian Zeng, Lijie Zhang, Calimanut-Ionut Cira, Egor Prikaziuk, Ting Duan, Chao Wang, Brigitta Szabó, Salvatore Manfreda, Ruodan Zhuang, and Bob Su
Geosci. Model Dev., 16, 5825–5845, https://doi.org/10.5194/gmd-16-5825-2023, https://doi.org/10.5194/gmd-16-5825-2023, 2023
Short summary
Short summary
Using machine learning, we estimated global surface soil moisture (SSM) to aid in understanding water, energy, and carbon exchange. Ensemble models outperformed individual algorithms in predicting SSM under different climates. The best-performing ensemble included K-neighbours Regressor, Random Forest Regressor, and Extreme Gradient Boosting. This is important for hydrological and climatological applications such as water cycle monitoring, irrigation management, and crop yield prediction.
Xiaoyi Shao, Siyuan Ma, and Chong Xu
Geosci. Model Dev., 16, 5113–5129, https://doi.org/10.5194/gmd-16-5113-2023, https://doi.org/10.5194/gmd-16-5113-2023, 2023
Short summary
Short summary
Scientific understandings of the distribution of coseismic landslides, followed by emergency and medium- and long-term risk assessment, can reduce landslide risk. The aim of this study is to propose an improved three-stage spatial prediction strategy and develop corresponding hazard assessment software called Mat.LShazard V1.0, which provides a new application tool for coseismic landslide disaster prevention and mitigation in different stages.
Junda Zhan, Sensen Wu, Jin Qi, Jindi Zeng, Mengjiao Qin, Yuanyuan Wang, and Zhenhong Du
Geosci. Model Dev., 16, 2777–2794, https://doi.org/10.5194/gmd-16-2777-2023, https://doi.org/10.5194/gmd-16-2777-2023, 2023
Short summary
Short summary
We develop a generalized spatial autoregressive neural network model used for three-dimensional spatial interpolation. Taking the different changing trend of geographic elements along various directions into consideration, the model defines spatial distance in a generalized way and integrates it into the process of spatial interpolation with the theories of spatial autoregression and neural network. Compared with traditional methods, the model achieves better performance and is more adaptable.
Dominikus Heinzeller, Ligia Bernardet, Grant Firl, Man Zhang, Xia Sun, and Michael Ek
Geosci. Model Dev., 16, 2235–2259, https://doi.org/10.5194/gmd-16-2235-2023, https://doi.org/10.5194/gmd-16-2235-2023, 2023
Short summary
Short summary
The Common Community Physics Package is a collection of physical atmospheric parameterizations for use in Earth system models and a framework that couples the physics to a host model’s dynamical core. A primary goal for this effort is to facilitate research and development of physical parameterizations and physics–dynamics coupling methods while offering capabilities for numerical weather prediction operations, for example in the upcoming implementation of the Global Forecast System (GFS) v17.
Tobias Tesch, Stefan Kollet, and Jochen Garcke
Geosci. Model Dev., 16, 2149–2166, https://doi.org/10.5194/gmd-16-2149-2023, https://doi.org/10.5194/gmd-16-2149-2023, 2023
Short summary
Short summary
A recent statistical approach for studying relations in the Earth system is to train deep learning (DL) models to predict Earth system variables given one or several others and use interpretable DL to analyze the relations learned by the models. Here, we propose to combine the approach with a theorem from causality research to ensure that the deep learning model learns causal rather than spurious relations. As an example, we apply the method to study soil-moisture–precipitation coupling.
Yao Hu, Chirantan Ghosh, and Siamak Malakpour-Estalaki
Geosci. Model Dev., 16, 1925–1936, https://doi.org/10.5194/gmd-16-1925-2023, https://doi.org/10.5194/gmd-16-1925-2023, 2023
Short summary
Short summary
Data-driven models (DDMs) gain popularity in earth and environmental systems, thanks in large part to advancements in data collection techniques and artificial intelligence (AI). The performance of these models is determined by the underlying machine learning (ML) algorithms. In this study, we develop a framework to improve the model performance by optimizing ML algorithms and demonstrate the effectiveness of the framework using a DDM to predict edge-of-field runoff in the Maumee domain, USA.
Ruidong Li, Ting Sun, Fuqiang Tian, and Guang-Heng Ni
Geosci. Model Dev., 16, 751–778, https://doi.org/10.5194/gmd-16-751-2023, https://doi.org/10.5194/gmd-16-751-2023, 2023
Short summary
Short summary
We developed SHAFTS (Simultaneous building Height And FootprinT extraction from Sentinel imagery), a multi-task deep-learning-based Python package, to estimate average building height and footprint from Sentinel imagery. Evaluation in 46 cities worldwide shows that SHAFTS achieves significant improvement over existing machine-learning-based methods.
Feng Yin, Philip E. Lewis, and Jose L. Gómez-Dans
Geosci. Model Dev., 15, 7933–7976, https://doi.org/10.5194/gmd-15-7933-2022, https://doi.org/10.5194/gmd-15-7933-2022, 2022
Short summary
Short summary
The proposed SIAC atmospheric correction method provides consistent surface reflectance estimations from medium spatial-resolution satellites (Sentinel 2 and Landsat 8) with per-pixel uncertainty information. The outputs from SIAC have been validated against a wide range of ground measurements, and it shows that SIAC can provide accurate estimations of both surface reflectance and atmospheric parameters, with meaningful uncertainty information.
Martina Stockhause and Michael Lautenschlager
Geosci. Model Dev., 15, 6047–6058, https://doi.org/10.5194/gmd-15-6047-2022, https://doi.org/10.5194/gmd-15-6047-2022, 2022
Short summary
Short summary
The Data Distribution Centre (DDC) of the Intergovernmental Panel on Climate Change (IPCC) celebrates its 25th anniversary in 2022. DDC Partner DKRZ has supported the IPCC Assessments and preserved the quality-assured, citable climate model data underpinning the Assessment Reports over these years over the long term. With the introduction of the IPCC FAIR Guidelines into the current AR6, the value of DDC services has been recognized. However, DDC sustainability remains unresolved.
Daiane Iglesia Dolci, Felipe A. G. Silva, Pedro S. Peixoto, and Ernani V. Volpe
Geosci. Model Dev., 15, 5857–5881, https://doi.org/10.5194/gmd-15-5857-2022, https://doi.org/10.5194/gmd-15-5857-2022, 2022
Short summary
Short summary
We investigate and compare the theoretical and computational characteristics of several absorbing boundary conditions (ABCs) for the full-waveform inversion (FWI) problem. The different ABCs are implemented in an optimized computational framework called Devito. The computational efficiency and memory requirements of the ABC methods are evaluated in the forward and adjoint wave propagators, from simple to realistic velocity models.
Mauro Rossi, Txomin Bornaetxea, and Paola Reichenbach
Geosci. Model Dev., 15, 5651–5666, https://doi.org/10.5194/gmd-15-5651-2022, https://doi.org/10.5194/gmd-15-5651-2022, 2022
Short summary
Short summary
LAND-SUITE is a software package designed to support landslide susceptibility zonation. The software integrates, extends, and completes LAND-SE (Rossi et al., 2010; Rossi and Reichenbach, 2016). The software is implemented in R, a free software environment for statistical computing and graphics, and gives expert users the possibility to perform easier, more flexible, and more informed statistically based landslide susceptibility applications and zonations.
Ashesh Chattopadhyay, Mustafa Mustafa, Pedram Hassanzadeh, Eviatar Bach, and Karthik Kashinath
Geosci. Model Dev., 15, 2221–2237, https://doi.org/10.5194/gmd-15-2221-2022, https://doi.org/10.5194/gmd-15-2221-2022, 2022
Short summary
Short summary
There is growing interest in data-driven weather forecasting, i.e., to predict the weather by using a deep neural network that learns from the evolution of past atmospheric patterns. Here, we propose three components to add to the current data-driven weather forecast models to improve their performance. These components involve a feature that incorporates physics into the neural network, a method to add data assimilation, and an algorithm to use several different time intervals in the forecast.
Paul F. Baumeister and Lars Hoffmann
Geosci. Model Dev., 15, 1855–1874, https://doi.org/10.5194/gmd-15-1855-2022, https://doi.org/10.5194/gmd-15-1855-2022, 2022
Short summary
Short summary
The efficiency of the numerical simulation of radiative transport is shown on modern server-class graphics cards (GPUs). The low-cost prefactor on GPUs compared to general-purpose processors (CPUs) enables future large retrieval campaigns for multi-channel data from infrared sounders aboard low-orbit satellites. The validated research software JURASSIC is available in the public domain.
Gregory E. Tucker, Eric W. H. Hutton, Mark D. Piper, Benjamin Campforts, Tian Gan, Katherine R. Barnhart, Albert J. Kettner, Irina Overeem, Scott D. Peckham, Lynn McCready, and Jaia Syvitski
Geosci. Model Dev., 15, 1413–1439, https://doi.org/10.5194/gmd-15-1413-2022, https://doi.org/10.5194/gmd-15-1413-2022, 2022
Short summary
Short summary
Scientists use computer simulation models to understand how Earth surface processes work, including floods, landslides, soil erosion, river channel migration, ocean sedimentation, and coastal change. Research benefits when the software for simulation modeling is open, shared, and coordinated. The Community Surface Dynamics Modeling System (CSDMS) is a US-based facility that supports research by providing community support, computing tools and guidelines, and educational resources.
Danilo César de Mello, Gustavo Vieira Veloso, Marcos Guedes de Lana, Fellipe Alcantara de Oliveira Mello, Raul Roberto Poppiel, Diego Ribeiro Oquendo Cabrero, Luis Augusto Di Loreto Di Raimo, Carlos Ernesto Gonçalves Reynaud Schaefer, Elpídio Inácio Fernandes Filho, Emilson Pereira Leite, and José Alexandre Melo Demattê
Geosci. Model Dev., 15, 1219–1246, https://doi.org/10.5194/gmd-15-1219-2022, https://doi.org/10.5194/gmd-15-1219-2022, 2022
Short summary
Short summary
We used soil parent material, terrain attributes, and geophysical data from the soil surface to test and compare different and unprecedented geophysical sensor combination, as well as different machine learning algorithms to model and predict several soil attributes. Also, we analyzed the importance of pedoenvironmental variables. The soil attributes were modeled throughout different machine learning algorithms and related to different geophysical sensor combinations.
Duncan Watson-Parris, Andrew Williams, Lucia Deaconu, and Philip Stier
Geosci. Model Dev., 14, 7659–7672, https://doi.org/10.5194/gmd-14-7659-2021, https://doi.org/10.5194/gmd-14-7659-2021, 2021
Short summary
Short summary
The Earth System Emulator (ESEm) provides a fast and flexible framework for emulating a wide variety of Earth science datasets and tools for constraining (or tuning) models of any complexity. Three distinct use cases are presented that demonstrate the utility of ESEm and provide some insight into the use of machine learning for emulation in these different settings. The open-source Python package is freely available so that it might become a valuable tool for the community.
Chongyang Wang, Li Wang, Danni Wang, Dan Li, Chenghu Zhou, Hao Jiang, Qiong Zheng, Shuisen Chen, Kai Jia, Yangxiaoyue Liu, Ji Yang, Xia Zhou, and Yong Li
Geosci. Model Dev., 14, 6833–6846, https://doi.org/10.5194/gmd-14-6833-2021, https://doi.org/10.5194/gmd-14-6833-2021, 2021
Short summary
Short summary
The turbidity maximum zone (TMZ) is a special phenomenon in estuaries worldwide. However, the extraction methods and criteria used to describe the TMZ vary significantly both spatially and temporally. This study proposes an new index, the turbidity maximum zone index, based on the corresponding relationship of total suspended solid concentration and Chl a concentration, which could better extract TMZs in different estuaries and on different dates.
Ranee Joshi, Kavitha Madaiah, Mark Jessell, Mark Lindsay, and Guillaume Pirot
Geosci. Model Dev., 14, 6711–6740, https://doi.org/10.5194/gmd-14-6711-2021, https://doi.org/10.5194/gmd-14-6711-2021, 2021
Short summary
Short summary
We have developed a software that allows the user to extract and standardize drill hole information from legacy datasets and/or different drilling campaigns. It also provides functionality to upscale the lithological information. These functionalities were possible by developing thesauri to identify and group geological terminologies together.
David Meyer, Thomas Nagler, and Robin J. Hogan
Geosci. Model Dev., 14, 5205–5215, https://doi.org/10.5194/gmd-14-5205-2021, https://doi.org/10.5194/gmd-14-5205-2021, 2021
Short summary
Short summary
A major limitation in training machine-learning emulators is often caused by the lack of data. This paper presents a cheap way to increase the size of training datasets using statistical techniques and thereby improve the performance of machine-learning emulators.
Mark Jessell, Vitaliy Ogarko, Yohan de Rose, Mark Lindsay, Ranee Joshi, Agnieszka Piechocka, Lachlan Grose, Miguel de la Varga, Laurent Ailleres, and Guillaume Pirot
Geosci. Model Dev., 14, 5063–5092, https://doi.org/10.5194/gmd-14-5063-2021, https://doi.org/10.5194/gmd-14-5063-2021, 2021
Short summary
Short summary
We have developed software that allows the user to extract sufficient information from unmodified digital maps and associated datasets that we are able to use to automatically build 3D geological models. By automating the process we are able to remove human bias from the procedure, which makes the workflow reproducible.
Martí Bosch, Maxence Locatelli, Perrine Hamel, Roy P. Remme, Jérôme Chenal, and Stéphane Joost
Geosci. Model Dev., 14, 3521–3537, https://doi.org/10.5194/gmd-14-3521-2021, https://doi.org/10.5194/gmd-14-3521-2021, 2021
Short summary
Short summary
The article presents a novel approach to simulate urban heat mitigation from land use/land cover data based on three biophysical mechanisms: tree shade, evapotranspiration and albedo. An automated procedure is proposed to calibrate the model parameters to best fit temperature observations from monitoring stations. A case study in Lausanne, Switzerland, shows that the approach outperforms regressions based on satellite data and provides valuable insights into design heat mitigation policies.
Quang-Van Doan, Hiroyuki Kusaka, Takuto Sato, and Fei Chen
Geosci. Model Dev., 14, 2097–2111, https://doi.org/10.5194/gmd-14-2097-2021, https://doi.org/10.5194/gmd-14-2097-2021, 2021
Short summary
Short summary
This study proposes a novel structural self-organizing map (S-SOM) algorithm. The superiority of S-SOM is that it can better recognize the difference (or similarity) among spatial (or temporal) data used for training and thus improve the clustering quality compared to traditional SOM algorithms.
Batunacun, Ralf Wieland, Tobia Lakes, and Claas Nendel
Geosci. Model Dev., 14, 1493–1510, https://doi.org/10.5194/gmd-14-1493-2021, https://doi.org/10.5194/gmd-14-1493-2021, 2021
Short summary
Short summary
Extreme gradient boosting (XGBoost) can provide alternative insights that conventional land-use models are unable to generate. Shapley additive explanations (SHAP) can interpret the results of the purely data-driven approach. XGBoost achieved similar and robust simulation results. SHAP values were useful for analysing the complex relationship between the different drivers of grassland degradation.
Juan A. Añel, Michael García-Rodríguez, and Javier Rodeiro
Geosci. Model Dev., 14, 923–934, https://doi.org/10.5194/gmd-14-923-2021, https://doi.org/10.5194/gmd-14-923-2021, 2021
Short summary
Short summary
This work shows that it continues to be hard, if not impossible, to obtain some of the most used climate models worldwide. We reach this conclusion through a systematic study and encourage all development teams and research centres to make public the models they use to produce scientific results.
Cited articles
Alted, F.: Why modern CPUs are starving and what can be done about it, CiSE, 12, 68–71, https://doi.org/10.1109/MCSE.2010.51, 2010. a
Ayachit, U., Geveci, B., Moreland, K., Patchett, J., and Ahrens, J.: The ParaView Visualization Application, in: High Performance Visualization, edited by Bethel, E. W., Childs, H., and Hansen, C., chap. 18, CRC Press, 383–400, https://doi.org/10.1201/b12985-31, 2012. a
Bachmann, K., Keil, C., and Weissmann, M.: Impact of radar data assimilation and orography on predictability of deep convection, Q. J. Roy. Meteor. Soc., 145, 117–130, https://doi.org/10.1002/qj.3412, 2018. a
Bachmann, K., Keil, C., Craig, G. C., Weissmann, M., and Welzbacher, C. A.: Predictability of Deep Convection in Idealized and Operational Forecasts: Effects of Radar Data Assimilation, Orography, and Synoptic Weather Regime, Mon. Weather Rev., 148, 63–81, https://doi.org/10.1175/mwr-d-19-0045.1, 2019. a
Baker, A. H., Xu, H., Dennis, J. M., Levy, M. N., Nychka, D., Mickelson, S. A., Edwards, J., Vertenstein, M., and Wegener, A.: A methodology for evaluating the impact of data compression on climate simulation data, in: Proceedings of the 23rd international symposium on High-performance parallel and distributed computing, HPDC'14: The 23rd International Symposium on High-Performance Parallel and Distributed Computing, 23–27 June 2014, Vancouver BC Canada, ACM Press, https://doi.org/10.1145/2600212.2600217, 2014. a, b, c
Baker, A. H., Hammerling, D. M., Mickelson, S. A., Xu, H., Stolpe, M. B., Naveau, P., Sanderson, B., Ebert-Uphoff, I., Samarasinghe, S., De Simone, F., Carbone, F., Gencarelli, C. N., Dennis, J. M., Kay, J. E., and Lindstrom, P.: Evaluating lossy data compression on climate simulation data within a large ensemble, Geosci. Model Dev., 9, 4381–4403, https://doi.org/10.5194/gmd-9-4381-2016, 2016. a, b, c
Baker, A. H., Xu, H., Hammerling, D. M., Li, S., and Clyne, J. P.: Toward a Multi-method Approach: Lossy Data Compression for Climate Simulation Data, in: Lecture Notes in Computer Science, Springer International Publishing, 30–42, https://doi.org/10.1007/978-3-319-67630-2_3, 2017. a, b
Baker, A. H., Hammerling, D. M., and Turton, T. L.: Evaluating image quality measures to assess the impact of lossy data compression applied to climate simulation data, Comput. Graph. Forum, 38, 517–528, https://doi.org/10.1111/cgf.13707, 2019. a
Baker, A. H., Pinard, A., and Hammerling, D. M.: On a Structural Similarity Index Approach for Floating-Point Data, IEEE T. Vis. Comput. Gr., 30, 6261–6274, https://doi.org/10.1109/TVCG.2023.3332843, 2024. a
Ballester-Ripoll, R. and Pajarola, R.: Lossy volume compression using Tucker truncation and thresholding, Vis. Comput., 32, 1433–1446, https://doi.org/10.1007/s00371-015-1130-y, 2015. a, b
Ballester-Ripoll, R., Lindstrom, P., and Pajarola, R.: TTHRESH: Tensor Compression for Multidimensional Visual Data, IEEE T. Vis. Comput. Gr., 26, 2891–2903, https://doi.org/10.1109/tvcg.2019.2904063, 2020. a, b
Balsa Rodríguez, M., Gobbetti, E., Iglesias Guitián, J., Makhinya, M., Marton, F., Pajarola, R., and Suter, S.: State-of-the-Art in Compressed GPU-Based Direct Volume Rendering: State-of-the-Art in Compressed GPU-Based DVR, Comput. Graph. Forum, 33, 77–100, https://doi.org/10.1111/cgf.12280, 2014. a
Ben-Kiki, O., Evans, C., and Ingerson, B.: YAML Ain't Markup Language (YAML™) Version 1.2, https://yaml.org/spec/1.2/spec.html (last access: 22 November 2023), 2009. a
Beyer, J., Hadwiger, M., and Pfister, H.: State-of-the-Art in GPU-Based Large-Scale Volume Visualization: GPU-Based Large-Scale Volume Visualization, Comput. Graph. Forum, 34, 13–37, https://doi.org/10.1111/cgf.12605, 2015. a
Blosc Development Team: Blosc: A blocking, shuffling and loss-less compression library, https://www.blosc.org, last access: 19 August 2022. a
Boutell, T.: PNG (Portable Network Graphics) Specification Version 1.0, Tech. Rep., https://doi.org/10.17487/rfc2083, 1997. a
Bray, T.: The JavaScript Object Notation (JSON) Data Interchange Format, RFC 8259, Internet Engineering Task Force, https://doi.org/10.17487/RFC8259, 2017. a
Burden, A., Burden, R., and Faires, J. D.: Numerical Analysis, CENGAGE Learning Custom Publishing, 10 edn., ISBN 9780357705315, 2015. a
Caron, J.: Compression by Bit-Shaving, https://web.archive.org/web/20141024232623/https://www.unidata.ucar.edu/blogs/developer/entry/compression_by_bit_shaving (last access: 22 November 2023), 2014. a
Collet, Y.: LZ4 Frame Format Description, GitHub Repository [code], https://github.com/lz4/lz4 (last access: 14 October 2024), 2020. a
Collet, Y. and Kucherawy, M.: Zstandard Compression and the 'application/zstd' Media Type, RFC 8878, https://doi.org/10.17487/RFC8878, 2021. a
Collette, A.: Python and HDF5, O'Reilly Media, Inc., ISBN 9781449367831, 2013. a
Craig, G. C., Fink, A. H., Hoose, C., Janjić, T., Knippertz, P., Laurian, A., Lerch, S., Mayer, B., Miltenberger, A., Redl, R., Riemer, M., Tempest, K. I., and Wirth, V.: Waves to Weather: Exploring the Limits of Predictability of Weather, B. Am. Meteorol. Soc., 102, E2151–E2164, https://doi.org/10.1175/BAMS-D-20-0035.1, 2021. a, b
Delaunay, X., Courtois, A., and Gouillon, F.: Evaluation of lossless and lossy algorithms for the compression of scientific datasets in netCDF-4 or HDF5 files, Geosci. Model Dev., 12, 4099–4113, https://doi.org/10.5194/gmd-12-4099-2019, 2019. a, b
Deutsch, P. and Gailly, J.: RFC 1950: ZLIB Compressed Data Format Specification, Internet Engineering Task Force (IETF), https://doi.org/10.17487/RFC1950, 1996. a
Dey, S. R. A., Leoncini, G., Roberts, N. M., Plant, R. S., and Migliorini, S.: A Spatial View of Ensemble Spread in Convection Permitting Ensembles, Mon. Weather Rev., 142, 4091–4107, https://doi.org/10.1175/mwr-d-14-00172.1, 2014. a
Donayre Holtz, S.: Lossy Compression of Climate Data Using Convolutional Autoencoders, Master Thesis, Karlsruher Institut für Technologie (KIT), https://doi.org/10.5445/IR/1000144742, 2022. a
Düben, P. D., Leutbecher, M., and Bauer, P.: New Methods for Data Storage of Model Output from Ensemble Simulations, Mon. Weather Rev., 147, 677–689, https://doi.org/10.1175/mwr-d-18-0170.1, 2019. a, b
enstools-compression Contributors: enstools-compression Documentation, https://enstools-compression.readthedocs.io/en/latest/, last access: 22 November 2023. a
Gneiting, T., Westveld, A. H., and Goldman, T.: Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation, Mon. Weather Rev., 133, 1098–1118, https://doi.org/10.1175/MWR2904.1, 2005. a
Gong, Q., Chen, J., Whitney, B., Liang, X., Reshniak, V., Banerjee, T., Lee, J., Rangarajan, A., Wan, L., Vidal, N., Liu, Q., Gainaru, A., Podhorszki, N., Archibald, R., Ranka, S., and Klasky, S.: MGARD: A multigrid framework for high-performance, error-controlled data compression and refactoring, SoftwareX, 24, 101590, https://doi.org/10.1016/j.softx.2023.101590, 2023. a
h5netcdf Contributors: h5netcdf: A Python interface for the netCDF4 file-format based on h5py, https://h5netcdf.org, last access: 22 November 2023. a
Hersbach, H., Bell, B., Berrisford, P., Blavati, G., Horányi, A., Muñoz, J. S., Nicolas, J., Peubey, C., Radu, R., Rozum, I., Schepers, D., Simmons, A., Soci, C., Dee, D., and Thépaut, J.-N.: ERA5 hourly data on pressure levels from 1959 to present, Copernicus Climate Change Service (C3S) Climate Data Store (CDS) [data set], https://doi.org/10.24381/cds.bd0915c6, 2018. a, b
Hoang, D., Klacansky, P., Bhatia, H., Bremer, P.-T., Lindstrom, P., and Pascucci, V.: A Study of the Trade-off Between Reducing Precision and Reducing Resolution for Data Analysis and Visualization, IEEE T. Vis. Comput. Gr., 25, 1193–1203, https://doi.org/10.1109/TVCG.2018.2864853, 2019. a, b, c
Hoang, D., Summa, B., Bhatia, H., Lindstrom, P., Klacansky, P., Usher, W., Bremer, P.-T., and Pascucci, V.: Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution, IEEE T. Vis. Comput. Gr., 27, 603–613, https://doi.org/10.1109/TVCG.2020.3030381, 2021. a
Hoyer, S. and Hamman, J.: xarray: N-D labeled Arrays and Datasets in Python, Journal of Open Research Software, 5, 10, https://doi.org/10.5334/jors.148, 2017. a
Kern, M., Hewson, T., Sadlo, F., Westermann, R., and Rautenhaus, M.: Robust Detection and Visualization of Jet-Stream Core Lines in Atmospheric Flow, IEEE T. Vis. Comput. Gr., 24, 893–902, https://doi.org/10.1109/tvcg.2017.2743989, 2018. a, b
Kochenderfer, M. J. and Wheeler, T. A.: Algorithms for Optimization, The MIT Press, ISBN 9780262039420, 2019. a
Koranne, S.: Hierarchical Data Format 5: HDF5, in: Handbook of Open Source Tools, Springer US, 191–200, https://doi.org/10.1007/978-1-4419-7719-9_10, 2010. a
Kunkel, J., Novikova, A., Betke, E., and Schaare, A.: Toward Decoupling the Selection of Compression Algorithms from Quality Constraints, in: Lecture Notes in Computer Science, Springer International Publishing, 3–14, https://doi.org/10.1007/978-3-319-67630-2_1, 2017. a, b
Lawrence, B. N., Kunkel, J. M., Churchill, J., Massey, N., Kershaw, P., and Pritchard, M.: Beating data bottlenecks in weather and climate science, in: Extreme Data Workshop 2018 Forschungszentrum Jülich, 18–19 September 2018 Proceedings, edited by: Schultz, M., Pleiter, D., and Bauer, P., vol. 40 of Schriften des Forschungszentrums Jülich IAS Series, Forschungszentrum Jülich GmbH Zentralbibliothek, Verlag Jülich, 18–19, ISBN 978-3-95806-392-1, https://pdfs.semanticscholar.org/9881/ed9d9e16cb70fba9456fb0905bf28c450ce0.pdf#page=38 (last access: 14 October 2024), 2019. a
Li, S., Jaroszynski, S., Pearse, S., Orf, L., and Clyne, J.: VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System Science, Atmosphere, 10, 488, https://doi.org/10.3390/atmos10090488, 2019. a
Li, S., Lindstrom, P., and Clyne, J.: Lossy Scientific Data Compression With SPERR, in: 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 37th IEEE International Parallel & Distributed Processing Symposium, 15–19 May 2023, Hilton St. Petersburg Bayfront Hotel St. Petersburg, Florida USA, 1007–1017, https://doi.org/10.1109/IPDPS54959.2023.00104, 2023. a
Liang, X., Di, S., Tao, D., Li, S., Li, S., Guo, H., Chen, Z., and Cappello, F.: Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets, in: 2018 IEEE International Conference on Big Data (Big Data), 2–7 July 2018, San Francisco, CA, USA, https://doi.org/10.1109/bigdata.2018.8622520, 2018. a, b, c, d
Liang, X., Guo, H., Di, S., Cappello, F., Raj, M., Liu, C., Ono, K., Chen, Z., and Peterka, T.: Toward Feature-Preserving 2D and 3D Vector Field Compression, in: 2020 IEEE Pacific Visualization Symposium (PacificVis), 14–17 April 2020, Tianjin, China, 81–90, ISBN 978-1-72815-697-2, https://doi.org/10.1109/PacificVis48177.2020.6431, 2020. a
Lindstrom, P. and Isenburg, M.: Fast and Efficient Compression of Floating-Point Data, IEEE T. Vis. Comput. Gr., 12, 1245–1250, https://doi.org/10.1109/tvcg.2006.143, 2006. a
Liu, J., Di, S., Zhao, K., Jin, S., Tao, D., Liang, X., Chen, Z., and Cappello, F.: Exploring Autoencoder-based Error-bounded Compression for Scientific Data, 2021 IEEE International Conference on Cluster Computing (CLUSTER), 7–10 September 2021, Portland, OR, USA, https://doi.org/10.1109/Cluster48925.2021.00034, 2021. a, b, c, d
Lu, Y., Jiang, K., Levine, J. A., and Berger, M.: Compressive Neural Representations of Volumetric Scalar Fields, Comput. Graph. Forum, 40, 135–146, https://doi.org/10.1111/cgf.14295, 2021. a
Matsunobu, T., Keil, C., and Barthlott, C.: The impact of microphysical uncertainty conditional on initial and boundary condition uncertainty under varying synoptic control, Weather Clim. Dynam., 3, 1273–1289, https://doi.org/10.5194/wcd-3-1273-2022, 2022. a, b
Matsunobu, T., Puh, M., and Keil, C.: Flow- and scale-dependent spatial predictability of convective precipitation combining different model uncertainty representations, Q. J. Roy. Meteor. Soc., 150, 2364–2381, https://doi.org/10.1002/qj.4713, 2024. a
Norton, A. and Clyne, J.: The VAPOR Visualization Application, in: High Performance Visualization, edited by Bethel, E. W., Childs, H., and Hansen, C., chap. 20, CRC Press, 415–428, https://doi.org/10.1201/b12985-33, 2012. a
Poppick, A., Nardi, J., Feldman, N., Baker, A. H., Pinard, A., and Hammerling, D. M.: A statistical analysis of lossily compressed climate model data, Comput. Geosci., 145, 104599, https://doi.org/10.1016/j.cageo.2020.104599, 2020. a, b
Rautenhaus, M., Grams, C. M., Schäfler, A., and Westermann, R.: Three-dimensional visualization of ensemble weather forecasts – Part 2: Forecasting warm conveyor belt situations for aircraft-based field campaigns, Geosci. Model Dev., 8, 2355–2377, https://doi.org/10.5194/gmd-8-2355-2015, 2015a. a
Rautenhaus, M., Kern, M., Schäfler, A., and Westermann, R.: Three-dimensional visualization of ensemble weather forecasts – Part 1: The visualization tool Met.3D (version 1.0), Geosci. Model Dev., 8, 2329–2353, https://doi.org/10.5194/gmd-8-2329-2015, 2015b. a, b
Rautenhaus, M., Bottinger, M., Siemen, S., Hoffman, R., Kirby, R. M., Mirzargar, M., Rober, N., and Westermann, R.: Visualization in Meteorology – A Survey of Techniques and Tools for Data Analysis Tasks, IEEE T. Vis. Comput. Gr., 24, 3268–3296, https://doi.org/10.1109/tvcg.2017.2779501, 2018. a, b, c
Redl, R., Tintó-Prims, O., baurflorian, and cagau: wavestoweather/enstools: Release v2022.11.1, Zenodo [code], https://doi.org/10.5281/zenodo.7331674, 2022. a, b, c
Rew, R., Davis, G., Emmerson, S., Cormack, C., Caron, J., Pincus, R., Hartnett, E., Heimbigner, D., Appel, L., and Fisher, W.: Unidata NetCDF, NSF [software], https://doi.org/10.5065/D6H70CW6, 1989. a
Roberts, N. M. and Lean, H. W.: Scale-Selective Verification of Rainfall Accumulations from High-Resolution Forecasts of Convective Events, Mon. Weather Rev., 136, 78–97, https://doi.org/10.1175/2007mwr2123.1, 2008. a
Schäfler, A., Craig, G., Wernli, H., Arbogast, P., Doyle, J. D., McTaggart-Cowan, R., Methven, J., Rivière, G., Ament, F., Boettcher, M., Bramberger, M., Cazenave, Q., Cotton, R., Crewell, S., Delanoë, J., Dörnbrack, A., Ehrlich, A., Ewald, F., Fix, A., Grams, C. M., Gray, S. L., Grob, H., Groß, S., Hagen, M., Harvey, B., Hirsch, L., Jacob, M., Kölling, T., Konow, H., Lemmerz, C., Lux, O., Magnusson, L., Mayer, B., Mech, M., Moore, R., Pelon, J., Quinting, J., Rahm, S., Rapp, M., Rautenhaus, M., Reitebuch, O., Reynolds, C. A., Sodemann, H., Spengler, T., Vaughan, G., Wendisch, M., Wirth, M., Witschas, B., Wolf, K., and Zinner, T.: The North Atlantic Waveguide and Downstream Impact Experiment, B. Am. Meteorol. Soc., 99, 1607–1637, https://doi.org/10.1175/bams-d-17-0003.1, 2018. a
Selz, T., Riemer, M., and Craig, G. C.: The Transition from Practical to Intrinsic Predictability of Midlatitude Weather, J. Atmos. Sci., 79, 2013–2030, https://doi.org/10.1175/jas-d-21-0271.1, 2022. a, b, c, d
Sprenger, M. and Wernli, H.: The LAGRANTO Lagrangian analysis tool – version 2.0, Geosci. Model Dev., 8, 2569–2586, https://doi.org/10.5194/gmd-8-2569-2015, 2015. a
Tao, D., Di, S., Guo, H., Chen, Z., and Cappello, F.: Z-checker: A framework for assessing lossy compression of scientific data, Int. J. High Perform. Comput. Appl., 33, 285–303, https://doi.org/10.1177/1094342017737147, 2019a. a
Tao, D., Di, S., Liang, X., Chen, Z., and Cappello, F.: Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP, IEEE Trans. Parallel Distrib. Syst., 30, 1857–1871, https://doi.org/10.1109/TPDS.2019.2894404, 2019b. a, b
The HDF Group: HDF5 Filters, https://support.hdfgroup.org/documentation/hdf5/latest/group___h5_z.html (last access: 14 October 2024), 2023. a
Tintó Prims, O.: wavestoweather/enstools-compression: Release v2023.11.1, Zenodo [code], https://doi.org/10.5281/zenodo.7327682, 2023. a
Tintó Prims, O.: The effect of lossy compression of numerical weather prediction data on data analysis: software to reproduce figures using enstools-compression, Zenodo [software], https://doi.org/10.5281/zenodo.10998604, 2024. a
Underwood, R., Malvoso, V., Calhoun, J. C., Di, S., and Cappello, F.: Productive and Performant Generic Lossy Data Compression with LibPressio, in: 2021 7th International Workshop on Data Analysis and Reduction for Big Scientific Data (DRBSD-7), 14 November 2021, St. Louis, Missouri, USA, 1–10, https://doi.org/10.1109/DRBSD754563.2021.00005, 2021. a
Vincent, T., V. Armando Solé, Kieffer, J., Kittisopikul, M., Florian-G, Plaswig, F., Valls, V., Klein, J., Gerstel, M., Junyuewang, and Payno: silx-kit/hdf5plugin: 3.3.1: 2022/06/03, Zenodo [code], https://doi.org/10.5281/zenodo.7257761, 2022. a, b
Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E.: Image Quality Assessment: From Error Visibility to Structural Similarity, IEEE Trans. Image Process., 13, 600–612, https://doi.org/10.1109/tip.2003.819861, 2004. a, b
Weiss, S., Hermüller, P., and Westermann, R.: Fast Neural Representations for Direct Volume Rendering, Comput. Graph. Forum, 41, 196–211, https://doi.org/10.1111/cgf.14578, 2022. a
Zender, C. S.: Bit Grooming: statistically accurate precision-preserving quantization with compression, evaluated in the netCDF Operators (NCO, v4.4.8+), Geosci. Model Dev., 9, 3199–3211, https://doi.org/10.5194/gmd-9-3199-2016, 2016. a, b, c, d
Zhao, K., Di, S., Lian, X., Li, S., Tao, D., Bessac, J., Chen, Z., and Cappello, F.: SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors, in: 2020 IEEE International Conference on Big Data (Big Data), IEEE, 10–13 December 2020, online, https://doi.org/10.1109/bigdata50022.2020.9378449, 2020. a, b, c, d, e, f, g, h
Short summary
Advanced compression techniques can drastically reduce the size of meteorological datasets (by 5 to 150 times) without compromising the data's scientific value. We developed a user-friendly tool called
enstools-compressionthat makes this compression simple for Earth scientists. This tool works seamlessly with common weather and climate data formats. Our work shows that lossy compression can significantly improve how researchers store and analyze large meteorological datasets.
Advanced compression techniques can drastically reduce the size of meteorological datasets (by 5...
