Task aggregation as a strategy to optimize Earth System Model workflows in HPC: assessing real scenarios with EC-Earth

Goitia, Pablo; Marciani, Manuel G.; Castrillo, Miguel; Acosta, Mario C.

doi:10.5194/gmd-19-8501-2026

Articles | Volume 19, issue 17

https://doi.org/10.5194/gmd-19-8501-2026

© Author(s) 2026. This work is distributed under
the Creative Commons Attribution 4.0 License.

https://doi.org/10.5194/gmd-19-8501-2026

© Author(s) 2026. This work is distributed under
the Creative Commons Attribution 4.0 License.

Articles | Volume 19, issue 17

Development and technical paper

|

11 Sep 2026

Development and technical paper |

| 11 Sep 2026

Task aggregation as a strategy to optimize Earth System Model workflows in HPC: assessing real scenarios with EC-Earth

Pablo Goitia, Manuel G. Marciani, Miguel Castrillo, and Mario C. Acosta

Data sets

Machine utilization and statistics of EC-Earth3 workflows running on MareNostrum 4, MareNostrum 5, and MeluXina Pablo Goitia https://doi.org/10.5281/zenodo.15292084

Model code and software

EC-Earth3 workflow with wrappers in MareNostrum 4 Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2067.1

EC-Earth3 workflow without wrappers in MareNostrum 4 Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2066.1

EC-Earth3 workflow with wrappers in MareNostrum 5 Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2065.1

EC-Earth3 workflow without wrappers in MareNostrum 5 Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2064.1

EC-Earth3 workflow with wrappers in MeluXina Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2063.1

EC-Earth3 workflow without wrappers in MeluXina Pablo Goitia et al. https://doi.org/10.48546/workflowhub.workflow.2062.1

Header scripts to gather statistics and usage records of EC-Earth3 workflows from MareNostrum 4, MareNostrum 5, and Meluxina Pablo Goitia and Manuel Giménez de Castro https://doi.org/10.5281/zenodo.15673462

Short summary

Earth System Model workflows commonly run on highly congested high-performance computing platforms, meaning that each individual workflow task potentially faces lengthy waiting times in the queues of the schedulers. In this work, we evaluate the task aggregation approach in EC-Earth3 workflows to reduce the queue times and, consequently, the total execution time. The results show up to 23% more simulated years over the same time span, with queuing times reduced by up to a factor of 12.33.