Articles | Volume 19, issue 17
https://doi.org/10.5194/gmd-19-8501-2026
© Author(s) 2026. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-19-8501-2026
© Author(s) 2026. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Development and technical paper
| 
11 Sep 2026
Development and technical paper |
| 11 Sep 2026
| 11 Sep 2026
Task aggregation as a strategy to optimize Earth System Model workflows in HPC: assessing real scenarios with EC-Earth
Barcelona Supercomputing Center (BSC), Barcelona, Spain
University of Cantabria (UC), Santander, Spain
Manuel G. Marciani
Barcelona Supercomputing Center (BSC), Barcelona, Spain
Miguel Castrillo
Barcelona Supercomputing Center (BSC), Barcelona, Spain
Mario C. Acosta
Barcelona Supercomputing Center (BSC), Barcelona, Spain
Related authors
No articles found.
M. Andrea Orihuela-García, Yohan Ruprich-Robert, Vladimir Lapin, Saskia Loosveldt Tomas, Raffaele Bernardello, Margarida Samsó-Cabré, Pierre-Antoine Bretonnière, Miguel Castrillo, and Martí Galí
Biogeosciences, 23, 4083–4112, https://doi.org/10.5194/bg-23-4083-2026, https://doi.org/10.5194/bg-23-4083-2026, 2026
Short summary
Short summary
We use a biogeochemical model to examine how particulate organic carbon (POC) is produced, transformed and transported to the deep ocean. Vertical flux attenuation arises from several interacting processes: export pathways, particle lability, temperature-dependent degradation, and zooplankton transformations, leading to strongest attenuation in the most productive region. Together with extensive comparison to observations, such a budgeting approach can help constrain model projections.
Oriol Tintó Prims, Sergi Palomas, Simone Vacondio, and Mario C. Acosta
EGUsphere, https://doi.org/10.5194/egusphere-2026-1661, https://doi.org/10.5194/egusphere-2026-1661, 2026
Short summary
Short summary
Earth System models are our main tools to study the climate. They consist of different components, like the atmosphere and ocean. We investigated the hidden inefficiencies of running these components one after the other rather than simultaneously. By defining a new metric, we enable developers to quantify the exact cost of this sequential approach. This helps identify wasted computing resources, ultimately guiding the design of faster and more efficient climate models for the future.
Francisco J. Doblas-Reyes, Jenni Kontkanen, Irina Sandu, Mario Acosta, Mohammed Hussam Al Turjmam, Ivan Alsina-Ferrer, Miguel Andrés-Martínez, Costanza Anerdi, Leo Arriola, Marvin Axness, Marc Batlle Martín, Peter Bauer, Tobias Becker, Daniel Beltrán, Sebastian Beyer, Hendryk Bockelmann, Pierre-Antoine Bretonnière, Sebastien Cabaniols, Silvia Caprioli, Miguel Castrillo, Aparna Chandrasekar, Suvarchal Cheedela, Victor Correal, Emanuele Danovaro, Paolo Davini, Jussi Enkovaara, Claudia Frauen, Barbara Früh, Aina Gaya Àvila, Paolo Ghinassi, Rohit Ghosh, Supriyo Ghosh, Iker González, Katherine Grayson, Matthew Griffith, Ioan Hadade, Christopher Haine, Carl Hartick, Utz-Uwe Haus, Shane Hearne, Heikki Järvinen, Bernat Jiménez, Amal John, Marlin Juchem, Thomas Jung, Jessica Kegel, Matthias Kelbling, Kai Keller, Bruno Kinoshita, Theresa Kiszler, Daniel Klocke, Lukas Kluft, Nikolay Koldunov, Tobias Kölling, Joonas Kolstela, Luis Kornblueh, Sergey Kosukhin, Aleksander Lacima-Nadolnik, Jeisson Javier Leal Rojas, Jonni Lehtiranta, Tuomas Lunttila, Anna Luoma, Pekka Manninen, Alexey Medvedev, Sebastian Milinski, Ali Mohammed, Sebastian Müller, Devaraju Naryanappa, Natalia Nazarova, Sami Niemelä, Bimochan Niraula, Henrik Nortamo, Aleksi Nummelin, Matteo Nurisso, Pablo Ortega, Stella Paronuzzi, Xabier Pedruzo-Bagazgoitia, Charles Pelletier, Carlos Peña, Suraj Polade, Himansu Kesari Pradhan, Rommel Quintanilla, Tiago Quintino, Thomas Rackow, Jouni Räisänen, Maqsood Mubarak Rajput, René Redler, Balthasar Reuter, Nuno Rocha Monteiro, Francesc Roura-Adserias, Silva Ruppert, Susan Sayed, Reiner Schnur, Tanvi Sharma, Dmitry Sidorenko, Outi Sievi-Korte, Albert Soret, Christian Steger, Bjorn Stevens, Jan Streffing, Jaleena Sunny, Luiggi Tenorio, Stephan Thober, Ulf Tigerstedt, Oriol Tinto, Juha Tonttila, Heikki Tuomenvirta, Lauri Tuppi, Ginka Van Thielen, Emanuele Vitali, Jost von Hardenberg, Ingo Wagner, Nils Wedi, Jan Wehner, Sven Willner, Xavier Yepes-Arbós, Florian Ziemen, and Janos Zimmermann
Geosci. Model Dev., 19, 2821–2848, https://doi.org/10.5194/gmd-19-2821-2026, https://doi.org/10.5194/gmd-19-2821-2026, 2026
Short summary
Short summary
The Climate Change Adaptation Digital Twin (Climate DT) pioneers the operationalisation of global climate projections. It produces global simulations with local granularity for adaptation decision-making. Applications are embedded to generate tailored indicators. A unified workflow orchestrates all components in several supercomputers. Data management ensures consistency and streaming enables real-time use. It is a complementary innovation to initiatives like CMIP, CORDEX, and climate services.
Kai R. Keller, Marta Alerany Solé, and Mario Acosta
Geosci. Model Dev., 18, 10221–10243, https://doi.org/10.5194/gmd-18-10221-2025, https://doi.org/10.5194/gmd-18-10221-2025, 2025
Short summary
Short summary
Can we be sure that different computing environments, that should not change the model climate, indeed leave the climate unaltered? In this article, we present a novel methodology that answers whether two model climates are statistically the same. Besides a new methodology, able to detect significant differences between two model climates 60 % more accurately than a similar recent state-of-the-art method, we also provide an analysis on what actually constitutes a different climate.
Manuel G. Marciani, Miguel Castrillo, Gladys Utrera, Mario C. Acosta, Bruno P. Kinoshita, and Francisco Doblas-Reyes
Geosci. Model Dev., 18, 9709–9721, https://doi.org/10.5194/gmd-18-9709-2025, https://doi.org/10.5194/gmd-18-9709-2025, 2025
Short summary
Short summary
Earth System Model simulations are typically run on large, highly congested flagship computers using workflows. These workflows can consist of thousands of tasks. If these tasks are queued individually, the wait time can add up, resulting in a long response time. In this paper, we explore a technique for aggregating tasks into a single submission. We found that this simple technique reduced the time spent in the queue by up to 7 %.
Sergi Palomas, Mario C. Acosta, Gladys Utrera, and Etienne Tourigny
Geosci. Model Dev., 18, 3661–3679, https://doi.org/10.5194/gmd-18-3661-2025, https://doi.org/10.5194/gmd-18-3661-2025, 2025
Short summary
Short summary
We present an automatic tool that optimizes resource distribution in coupled climate models, enhancing speed and reducing computational costs without requiring expert knowledge. Users can set energy/time criteria or limit resource usage. Tested on various European Community Earth System Model (EC-Earth) configurations and high-performance computing (HPC) platforms, it achieved up to 34 % faster simulations with fewer resources.
Eduardo Moreno-Chamarro, Thomas Arsouze, Mario Acosta, Pierre-Antoine Bretonnière, Miguel Castrillo, Eric Ferrer, Amanda Frigola, Daria Kuznetsova, Eneko Martin-Martinez, Pablo Ortega, and Sergi Palomas
Geosci. Model Dev., 18, 461–482, https://doi.org/10.5194/gmd-18-461-2025, https://doi.org/10.5194/gmd-18-461-2025, 2025
Short summary
Short summary
We present the high-resolution model version of the EC-Earth global climate model to contribute to HighResMIP. The combined model resolution is about 10–15 km in both the ocean and atmosphere, which makes it one of the finest ever used to complete historical and scenario simulations. This model is compared with two lower-resolution versions, with a 100 km and a 25 km grid. The three models are compared with observations to study the improvements thanks to the increased resolution.
Colin G. Jones, Fanny Adloff, Ben B. B. Booth, Peter M. Cox, Veronika Eyring, Pierre Friedlingstein, Katja Frieler, Helene T. Hewitt, Hazel A. Jeffery, Sylvie Joussaume, Torben Koenigk, Bryan N. Lawrence, Eleanor O'Rourke, Malcolm J. Roberts, Benjamin M. Sanderson, Roland Séférian, Samuel Somot, Pier Luigi Vidale, Detlef van Vuuren, Mario Acosta, Mats Bentsen, Raffaele Bernardello, Richard Betts, Ed Blockley, Julien Boé, Tom Bracegirdle, Pascale Braconnot, Victor Brovkin, Carlo Buontempo, Francisco Doblas-Reyes, Markus Donat, Italo Epicoco, Pete Falloon, Sandro Fiore, Thomas Frölicher, Neven S. Fučkar, Matthew J. Gidden, Helge F. Goessling, Rune Grand Graversen, Silvio Gualdi, José M. Gutiérrez, Tatiana Ilyina, Daniela Jacob, Chris D. Jones, Martin Juckes, Elizabeth Kendon, Erik Kjellström, Reto Knutti, Jason Lowe, Matthew Mizielinski, Paola Nassisi, Michael Obersteiner, Pierre Regnier, Romain Roehrig, David Salas y Mélia, Carl-Friedrich Schleussner, Michael Schulz, Enrico Scoccimarro, Laurent Terray, Hannes Thiemann, Richard A. Wood, Shuting Yang, and Sönke Zaehle
Earth Syst. Dynam., 15, 1319–1351, https://doi.org/10.5194/esd-15-1319-2024, https://doi.org/10.5194/esd-15-1319-2024, 2024
Short summary
Short summary
We propose a number of priority areas for the international climate research community to address over the coming decade. Advances in these areas will both increase our understanding of past and future Earth system change, including the societal and environmental impacts of this change, and deliver significantly improved scientific support to international climate policy, such as future IPCC assessments and the UNFCCC Global Stocktake.
Mario C. Acosta, Sergi Palomas, Stella V. Paronuzzi Ticco, Gladys Utrera, Joachim Biercamp, Pierre-Antoine Bretonniere, Reinhard Budich, Miguel Castrillo, Arnaud Caubel, Francisco Doblas-Reyes, Italo Epicoco, Uwe Fladrich, Sylvie Joussaume, Alok Kumar Gupta, Bryan Lawrence, Philippe Le Sager, Grenville Lister, Marie-Pierre Moine, Jean-Christophe Rioual, Sophie Valcke, Niki Zadeh, and Venkatramani Balaji
Geosci. Model Dev., 17, 3081–3098, https://doi.org/10.5194/gmd-17-3081-2024, https://doi.org/10.5194/gmd-17-3081-2024, 2024
Short summary
Short summary
We present a collection of performance metrics gathered during the Coupled Model Intercomparison Project Phase 6 (CMIP6), a worldwide initiative to study climate change. We analyse the metrics that resulted from collaboration efforts among many partners and models and describe our findings to demonstrate the utility of our study for the scientific community. The research contributes to understanding climate modelling performance on the current high-performance computing (HPC) architectures.
Marcus Falls, Raffaele Bernardello, Miguel Castrillo, Mario Acosta, Joan Llort, and Martí Galí
Geosci. Model Dev., 15, 5713–5737, https://doi.org/10.5194/gmd-15-5713-2022, https://doi.org/10.5194/gmd-15-5713-2022, 2022
Short summary
Short summary
This paper describes and tests a method which uses a genetic algorithm (GA), a type of optimisation algorithm, on an ocean biogeochemical model. The aim is to produce a set of numerical parameters that best reflect the observed data of particulate organic carbon in a specific region of the ocean. We show that the GA can provide optimised model parameters in a robust and efficient manner and can also help detect model limitations, ultimately leading to a reduction in the model uncertainties.
Enza Di Tomaso, Jerónimo Escribano, Sara Basart, Paul Ginoux, Francesca Macchia, Francesca Barnaba, Francesco Benincasa, Pierre-Antoine Bretonnière, Arnau Buñuel, Miguel Castrillo, Emilio Cuevas, Paola Formenti, María Gonçalves, Oriol Jorba, Martina Klose, Lucia Mona, Gilbert Montané Pinto, Michail Mytilinaios, Vincenzo Obiso, Miriam Olid, Nick Schutgens, Athanasios Votsis, Ernest Werner, and Carlos Pérez García-Pando
Earth Syst. Sci. Data, 14, 2785–2816, https://doi.org/10.5194/essd-14-2785-2022, https://doi.org/10.5194/essd-14-2785-2022, 2022
Short summary
Short summary
MONARCH reanalysis of desert dust aerosols extends the existing observation-based information for mineral dust monitoring by providing 3-hourly upper-air, surface and total column key geophysical variables of the dust cycle over Northern Africa, the Middle East and Europe, at a 0.1° horizontal resolution in a rotated grid, from 2007 to 2016. This work provides evidence of the high accuracy of this data set and its suitability for air quality and health and climate service applications.
Matthew L. Dawson, Christian Guzman, Jeffrey H. Curtis, Mario Acosta, Shupeng Zhu, Donald Dabdub, Andrew Conley, Matthew West, Nicole Riemer, and Oriol Jorba
Geosci. Model Dev., 15, 3663–3689, https://doi.org/10.5194/gmd-15-3663-2022, https://doi.org/10.5194/gmd-15-3663-2022, 2022
Short summary
Short summary
Progress in identifying complex, mixed-phase physicochemical processes has resulted in an advanced understanding of the evolution of atmospheric systems but has also introduced a level of complexity that few atmospheric models were designed to handle. We present a flexible treatment for multiphase chemical processes for models of diverse scale, from box up to global models. This enables users to build a customized multiphase mechanism that is accessible to a much wider community.
Ralf Döscher, Mario Acosta, Andrea Alessandri, Peter Anthoni, Thomas Arsouze, Tommi Bergman, Raffaele Bernardello, Souhail Boussetta, Louis-Philippe Caron, Glenn Carver, Miguel Castrillo, Franco Catalano, Ivana Cvijanovic, Paolo Davini, Evelien Dekker, Francisco J. Doblas-Reyes, David Docquier, Pablo Echevarria, Uwe Fladrich, Ramon Fuentes-Franco, Matthias Gröger, Jost v. Hardenberg, Jenny Hieronymus, M. Pasha Karami, Jukka-Pekka Keskinen, Torben Koenigk, Risto Makkonen, François Massonnet, Martin Ménégoz, Paul A. Miller, Eduardo Moreno-Chamarro, Lars Nieradzik, Twan van Noije, Paul Nolan, Declan O'Donnell, Pirkka Ollinaho, Gijs van den Oord, Pablo Ortega, Oriol Tintó Prims, Arthur Ramos, Thomas Reerink, Clement Rousset, Yohan Ruprich-Robert, Philippe Le Sager, Torben Schmith, Roland Schrödner, Federico Serva, Valentina Sicardi, Marianne Sloth Madsen, Benjamin Smith, Tian Tian, Etienne Tourigny, Petteri Uotila, Martin Vancoppenolle, Shiyu Wang, David Wårlind, Ulrika Willén, Klaus Wyser, Shuting Yang, Xavier Yepes-Arbós, and Qiong Zhang
Geosci. Model Dev., 15, 2973–3020, https://doi.org/10.5194/gmd-15-2973-2022, https://doi.org/10.5194/gmd-15-2973-2022, 2022
Short summary
Short summary
The Earth system model EC-Earth3 is documented here. Key performance metrics show physical behavior and biases well within the frame known from recent models. With improved physical and dynamic features, new ESM components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond.
Xavier Yepes-Arbós, Gijs van den Oord, Mario C. Acosta, and Glenn D. Carver
Geosci. Model Dev., 15, 379–394, https://doi.org/10.5194/gmd-15-379-2022, https://doi.org/10.5194/gmd-15-379-2022, 2022
Short summary
Short summary
Climate prediction models produce a large volume of simulated data that sometimes might not be efficiently managed. In this paper we present an approach to address this issue by reducing the computing time and storage space. As a case study, we analyse the output writing process of the ECMWF atmospheric model called IFS, and we integrate into it a data writing tool called XIOS. The results suggest that the integration between the two components achieves an adequate computational performance.
Cited articles
Acosta, M. C., Palomas, S., Paronuzzi Ticco, S. V., Utrera, G., Biercamp, J., Bretonniere, P.-A., Budich, R., Castrillo, M., Caubel, A., Doblas-Reyes, F., Epicoco, I., Fladrich, U., Joussaume, S., Kumar Gupta, A., Lawrence, B., Le Sager, P., Lister, G., Moine, M.-P., Rioual, J.-C., Valcke, S., Zadeh, N., and Balaji, V.: The computational and energy cost of simulation and storage for climate science: lessons from CMIP6, Geoscientific Model Development, 17, 3081–3098, https://doi.org/10.5194/gmd-17-3081-2024, 2024. a, b, c
Ahn, D. H., Bass, N., Chu, A., Garlick, J., Grondona, M., Herbein, S., Ingólfsson, H. I., Koning, J., Patki, T., Scogland, T. R., Springmeyer, B., and Taufer, M.: Flux: Overcoming scheduling challenges for exascale workflows, Future Generat. Comput. Syst., 110, 202–213, https://doi.org/10.1016/j.future.2020.04.006, 2020. a
Aumont, O., Ethé, C., Tagliabue, A., Bopp, L., and Gehlen, M.: PISCES-v2: an ocean biogeochemical model for carbon and ecosystem studies, Geosci. Model Dev., 8, 2465–2513, https://doi.org/10.5194/gmd-8-2465-2015, 2015. a
Autosubmit Team: Autosubmit v3.15 Documentation, https://autosubmit.readthedocs.io/en/3.15.0-branch/ (last access: 24 August 2026), 2023. a
Babuji, Y., Woodard, A., Li, Z., Katz, D. S., Clifford, B., Kumar, R., Lacinski, L., Chard, R., Wozniak, J. M., Foster, I., Wilde, M., and Chard, K.: Parsl: Pervasive Parallel Programming in Python, in: HPDC '19, Association for Computing Machinery, New York, NY, USA, 25–36, ISBN 9781450366700, https://doi.org/10.1145/3307681.3325400, 2019. a
Balaji, V., Maisonnave, E., Zadeh, N., Lawrence, B. N., Biercamp, J., Fladrich, U., Aloisio, G., Benson, R., Caubel, A., Durachta, J., Foujols, M.-A., Lister, G., Mocavero, S., Underwood, S., and Wright, G.: CPMIP: measurements of real computational performance of Earth system models in CMIP6, Geosci. Model Dev., 10, 19–34, https://doi.org/10.5194/gmd-10-19-2017, 2017. a, b
Banchelli, F., Garcia-Gasulla, M., Mantovani, F., Vinyals, J., Pocurull, J., Vicente, D., Eguzkitza, B., Galeazzo, F. C. C., Acosta, M. C., and Girona, S.: Introducing MareNostrum5: A European pre-exascale energy-efficient system designed to serve a broad spectrum of scientific workloads, arXiv [preprint], https://doi.org/10.48550/arXiv.2503.09917, 2025. a
Beránek, J., Böhm, A., Palermo, G., Martinovič, J., and Jansík, B.: HyperQueue: Efficient and ergonomic task graphs on HPC clusters, SoftwareX, 27, 101814, https://doi.org/10.1016/j.softx.2024.101814, 2024. a, b
Buizza, R., Alonso-Balmaseda, M., Brown, A., English, S., Forbes, R., Geer, A., Haiden, T., Leutbecher, M., Magnusson, L., Rodwell, M., Sleigh, M., Stockdale, T., Vitart, F., and Wedi, N.: The development and evaluation process followed at ECMWF to upgrade the Integrated Forecasting System (IFS), European Centre for Medium Range Weather Forecasts, https://doi.org/10.21957/xzopnhty9, 2018. a
Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P. J., Mayani, R., Chen, W., Ferreira da Silva, R., Livny, M., and Wenger, K.: Pegasus, a workflow management system for science automation, Future Generat. Comput. Syst., 46, 17–35, https://doi.org/10.1016/j.future.2014.10.008, 2015. a
Döscher, R., Acosta, M., Alessandri, A., Anthoni, P., Arsouze, T., Bergman, T., Bernardello, R., Boussetta, S., Caron, L. P., Carver, G., Castrillo, M., Catalano, F., Cvijanovic, I., Davini, P., Dekker, E., Doblas-Reyes, F. J., Docquier, D., Echevarria, P., Fladrich, U., Fuentes-Franco, R., Gröger, M., Hardenberg, J. V., Hieronymus, J., Karami, M. P., Keskinen, J. P., Koenigk, T., Makkonen, R., Massonnet, F., Ménégoz, M., Miller, P. A., Moreno-Chamarro, E., Nieradzik, L., Van Noije, T., Nolan, P., O'donnell, D., Ollinaho, P., Van Den Oord, G., Ortega, P., Prims, O. T., Ramos, A., Reerink, T., Rousset, C., Ruprich-Robert, Y., Le Sager, P., Schmith, T., Schrödner, R., Serva, F., Sicardi, V., Sloth Madsen, M., Smith, B., Tian, T., Tourigny, E., Uotila, P., Vancoppenolle, M., Wang, S., Wårlind, D., Willén, U., Wyser, K., Yang, S., Yepes-Arbós, X., and Zhang, Q.: The EC-Earth3 Earth system model for the Coupled Model Intercomparison Project 6, Geosci. Model Dev., 15, 2973–3020, https://doi.org/10.5194/gmd-15-2973-2022, 2022. a
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., and Taylor, K. E.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization, Geosci. Model Dev., 9, 1937–1958, https://doi.org/10.5194/gmd-9-1937-2016, 2016. a
Ferrer, E., Montane, G., Castrillo, M., and Garcia, A.: Auto-EC-Earth: An automatic workflow to manage climate modelling experiments using Autosubmit, EGU General Assembly 2025, Vienna, Austria, 27 Apr–2 May 2025, EGU25-8621, https://doi.org/10.5194/egusphere-egu25-8621, 2025. a
Fujitsu: FUJITSU Software Technical Computing Suite V4.0L20, Fujitsu, https://software.fujitsu.com/jp/manual/manualindex/p21000155e.html (last access: 5 December 2025), 2021. a
Garcia Lopez, A., Arriola Meikle, L., Montane Pinto, G., Castrillo, M., de Paula Kinoshita, B., Ferrer Escuin, E., and Gaya Avila, A.: Enabling reliable workflow development with an advanced Testing Suite, in: EGU General Assembly 2025, Vienna, Austria, EGU25-8305, 2025. a
Goitia, P.: Machine utilization and statistics of EC-Earth3 workflows running on MareNostrum 4, MareNostrum 5, and MeluXina, Zenodo [data set], https://doi.org/10.5281/zenodo.15292084, 2025. a
Goitia, P. and Giménez de Castro, M.: Header scripts to gather statistics and usage records of EC-Earth3 workflows from MareNostrum 4, MareNostrum 5, and Meluxina, Zenodo [code], https://doi.org/10.5281/zenodo.15673462, 2025. a, b
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow with wrappers in MeluXina, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2063.1, 2025a. a
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow without wrappers in MeluXina, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2062.1, 2025b. a
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow with wrappers in MareNostrum 4, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2067.1, 2025c. a
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow without wrappers in MareNostrum 4, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2066.1, 2025d. a
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow with wrappers in MareNostrum 5, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2065.1, 2025e. a
Goitia, P., Ferrer, E., Garcia, A., Bonet, G., Montane, G., and Castrillo, M.: EC-Earth3 workflow without wrappers in MareNostrum 5, WorkflowHub [code], https://doi.org/10.48546/workflowhub.workflow.2064.1, 2025f. a
Gustafsson, O. J. R., Wilkinson, S. R., Bacall, F., Soiland-Reyes, S., Leo, S., Pireddu, L., Owen, S., Juty, N., Fernández, J. M., Brown, T., Ménager, H., Grüning, B., Capella-Gutierrez, S., Coppens, F., and Goble, C.: WorkflowHub: a registry for computational workflows, Sci. Data, 12, 837, https://doi.org/10.1038/s41597-025-04786-3, 2025. a
Hoffmann, J., Bauer, P., Sandu, I., Wedi, N., Geenen, T., and Thiemert, D.: Destination Earth – A digital twin in support of climate services, Clim. Serv., 30, 100394, https://doi.org/10.1016/j.cliser.2023.100394, 2023. a
Huber, S. P., Zoupanos, S., Uhrin, M., Talirz, L., Kahle, L., Häuselmann, R., Gresch, D., Müller, T., Yakutovich, A. V., Andersen, C. W., Ramirez, F. F., Adorf, C. S., Gargiulo, F., Kumbhar, S., Passaro, E., Johnston, C., Merkys, A., Cepellotti, A., Mounet, N., Marzari, N., Kozinsky, B., and Pizzi, G.: AiiDA 1.0, a scalable computational infrastructure for automated reproducible workflows and data provenance, Sci. Data, 7, https://doi.org/10.1038/s41597-020-00638-4, 2020. a, b
Irrmann, G., Masson, S., Maisonnave, E., Guibert, D., and Raffin, E.: Improving ocean modeling software NEMO 4.0 benchmarking and communication efficiency, Geosci. Model Dev., 15, 1567–1582, https://doi.org/10.5194/gmd-15-1567-2022, 2022. a
Jette, M. A. and Wickberg, T.: Architecture of the Slurm Workload Manager, in: Job Scheduling Strategies for Parallel Processing. JSSPP 2023. Lecture Notes in Computer Science, vol. 14283, edited by: Klusáček, D., Corbalán, J., and Rodrigo Gonzalo, P., Springer Nature, Cham, Switzerland, 3–23, https://doi.org/10.1007/978-3-031-43943-8_1, 2023. a
Jones, M. D., White, J. P., Innus, M., DeLeon, R. L., Simakov, N., Palmer, J. T., Gallo, S. M., Furlani, T. R., Showerman, M., Brunner, R., Kot, A., Bauer, G., Bode, B., Enos, J., and Kramer, W.: Workload Analysis of Blue Waters, arXiv [preprint], https://doi.org/10.48550/arXiv.1703.00924, 2017. a
Madec, G. and the NEMO System Team: NEMO ocean engine, Scientific Notes of IPSL Climate Modelling Center, 27, Zenodo, https://doi.org/10.5281/zenodo.6334656, 2022. a
Manubens-Gil, D., Vegas-Regidor, J., Prodhomme, C., Mula-Valls, O., and Doblas-Reyes, F. J.: Seamless management of ensemble climate prediction experiments on HPC platforms, in: 2016 International Conference on High Performance Computing & Simulation (HPCS), 895–900, https://doi.org/10.1109/HPCSim.2016.7568429, 2016. a, b
Marciani, M. G., Castrillo, M., Utrera, G., Acosta, M. C., Kinoshita, B. P., and Doblas-Reyes, F.: Evaluating the impact of task aggregation in workflows with shared resource environments: use case for the MONARCH application, Geosci. Model Dev., 18, 9709–9721, https://doi.org/10.5194/gmd-18-9709-2025, 2025. a, b
Merzky, A., Turilli, M., Maldonado, M., Santcroos, M., and Jha, S.: Using Pilot Systems to Execute Many Task Workloads on Supercomputers, in: Job Scheduling Strategies for Parallel Processing, edited by: Klusáček, D., Cirne, W., and Desai, N., Springer International Publishing, Cham, 61–82, https://doi.org/10.1007/978-3-030-10632-4_4, 2019. a
Merzky, A., Turilli, M., Titov, M., Al-Saadi, A., and Jha, S.: Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms, IEEE T. Parallel Distrib. Syst., 33, 818–829, 2022. a
Mickelson, S., Bertini, A., Strand, G., Paul, K., Nienhouse, E., Dennis, J., and Vertenstein, M.: A new end-to-end workflow for the Community Earth System Model (version 2.0) for the Coupled Model Intercomparison Project Phase 6 (CMIP6), Geosci. Model Dev., 13, 5567–5581, https://doi.org/10.5194/gmd-13-5567-2020, 2020. a
Mölder, F., Jablonski, K., Letcher, B., Hall, M., Tomkins-Tinch, C., Sochat, V., Forster, J., Lee, S., Twardziok, S., Kanitz, A., Wilm, A., Holtgrewe, M., Rahmann, S., Nahnsen, S., and Köster, J.: Sustainable data analysis with Snakemake, F1000Research, 10, https://doi.org/10.12688/f1000research.29032.2, 2021. a, b
Oliver, H., Shin, M., Matthews, D., Sanders, O., Bartholomew, S., Clark, A., Fitzpatrick, B., van Haren, R., Hut, R., and Drost, N.: Workflow Automation for Cycling Systems, Comput. Sci. Eng., 21, 7–21, https://doi.org/10.1109/MCSE.2019.2906593, 2019. a
Patel, T., Liu, Z., Kettimuthu, R., Rich, P., Allcock, W., and Tiwari, D.: Job Characteristics on Large-Scale Systems: Long-Term Analysis, Quantification, and Implications, in: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 1–17, https://doi.org/10.1109/SC41405.2020.00088, 2020. a
Puiggros, A., Castrillo, M., de Paula Kinoshita, B., Bretonniere, P.-A., and Agudetse, V.: Enhancing Data Provenance in Workflow Management: Integrating FAIR Principles into Autosubmit and SUNSET, EGU General Assembly 2025, Vienna, Austria, 27 April–2 May 2025, EGU25-2142, https://doi.org/10.5194/egusphere-egu25-2142, 2025. a
Rodrigo, G. P., Östberg, P.-O., Elmroth, E., Antypas, K., Gerber, R., and Ramakrishnan, L.: Towards understanding HPC users and systems: A NERSC case study, J. Parallel Distrib. Comput., 111, 206–221, https://doi.org/10.1016/j.jpdc.2017.09.002, 2018. a
Rousset, C., Vancoppenolle, M., Madec, G., Fichefet, T., Flavoni, S., Barthélemy, A., Benshila, R., Chanut, J., Levy, C., Masson, S., and Vivier, F.: The Louvain-La-Neuve sea ice model LIM3.6: global and regional capabilities, Geosci. Model Dev., 8, 2991–3005, https://doi.org/10.5194/gmd-8-2991-2015, 2015. a
SchedMD: Fair Tree Fairshare Algorithm, https://slurm.schedmd.com/fair_tree.html (last access: 5 December 2025), 2025a. a
SchedMD: Multifactor Priority Plugin, https://slurm.schedmd.com/priority_multifactor.html (last access: 5 December 2025), 2025b. a
Schlagkamp, S., Ferreira da Silva, R., Allcock, W., Deelman, E., and Schwiegelshohn, U.: Consecutive Job Submission Behavior at Mira Supercomputer, in: HPDC '16, Association for Computing Machinery, New York, NY, USA, 93–96, ISBN 9781450343145, https://doi.org/10.1145/2907294.2907314, 2016. a
Smith, B., Wårlind, D., Arneth, A., Hickler, T., Leadley, P., Siltberg, J., and Zaehle, S.: Implications of incorporating N cycling and N limitations on primary production in an individual-based dynamic vegetation model, Biogeosciences, 11, 2027–2054, https://doi.org/10.5194/bg-11-2027-2014, 2014. a
Suarez, E., Bockelmann, H., Eicker, N., Eitzinger, J., El Sayed, S., Fieseler, T., Frank, M., Frech, P., Giesselmann, P., Hackenberg, D., Hager, G., Herten, A., Ilsche, T., Koller, B., Laure, E., Manzano, C., Oeste, S., Ott, M., Reuter, K., Schneider, R., Thust, K., and von St. Vieth, B.: Energy-aware operation of HPC systems in Germany, Front. High Perform. Comput., 3, https://doi.org/10.3389/fhpcp.2025.1520207, 2025. a
TOP500: TOP500 List, https://top500.org/ (last access: 1 June 2025), 2025. a
Turilli, M., Santcroos, M., and Jha, S.: A Comprehensive Perspective on Pilot-Job Systems, ACM Comput. Surv., 51, https://doi.org/10.1145/3177851, 2018. a
van Noije, T. P. C., Le Sager, P., Segers, A. J., van Velthoven, P. F. J., Krol, M. C., Hazeleger, W., Williams, A. G., and Chambers, S. D.: Simulation of tropospheric chemistry and aerosols with the climate model EC-Earth, Geosci. Model Dev., 7, 2435–2475, https://doi.org/10.5194/gmd-7-2435-2014, 2014. a
Short summary
Earth System Model workflows commonly run on highly congested high-performance computing platforms, meaning that each individual workflow task potentially faces lengthy waiting times in the queues of the schedulers. In this work, we evaluate the task aggregation approach in EC-Earth3 workflows to reduce the queue times and, consequently, the total execution time. The results show up to 23% more simulated years over the same time span, with queuing times reduced by up to a factor of 12.33.
Earth System Model workflows commonly run on highly congested high-performance computing...
