Articles | Volume 18, issue 8
https://doi.org/10.5194/gmd-18-2461-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-18-2461-2025
© Author(s) 2025. This work is distributed under
the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
The Earth System Grid Federation (ESGF) Virtual Aggregation (CMIP6 v20240125)
Ezequiel Cimadevilla
CORRESPONDING AUTHOR
Instituto de Física de Cantabria (IFCA), CSIC-Universidad de Cantabria, Santander, Spain
Bryan N. Lawrence
National Centre for Atmospheric Science, Department of Meteorology, University of Reading, Reading, UK
Antonio S. Cofiño
Instituto de Física de Cantabria (IFCA), CSIC-Universidad de Cantabria, Santander, Spain
Related authors
No articles found.
Paul J. Durack, Karl E. Taylor, Peter J. Gleckler, Gerald A. Meehl, Bryan N. Lawrence, Curt Covey, Ronald J. Stouffer, Guillaume Levavasseur, Atef Ben-Nasser, Sebastien Denvil, Martina Stockhause, Jonathan M. Gregory, Martin Juckes, Sasha K. Ames, Fabrizio Antonio, David C. Bader, John P. Dunne, Daniel Ellis, Veronika Eyring, Sandro L. Fiore, Sylvie Joussaume, Philip Kershaw, Jean-Francois Lamarque, Michael Lautenschlager, Jiwoo Lee, Chris F. Mauzey, Matthew Mizielinski, Paola Nassisi, Alessandra Nuzzo, Eleanor O’Rourke, Jeffrey Painter, Gerald L. Potter, Sven Rodriguez, and Dean N. Williams
EGUsphere, https://doi.org/10.5194/egusphere-2024-3729, https://doi.org/10.5194/egusphere-2024-3729, 2025
Short summary
Short summary
CMIP6 was the most expansive and ambitious Model Intercomparison Project (MIP), the latest in a history, extending four decades. CMIP engaged a growing community focused on improving climate understanding, and quantifying and attributing observed climate change being experienced today. The project's profound impact is due to the combining the latest climate science and technology, enabling the latest-generation climate simulations and increasing community attention in every successive phase.
Colin G. Jones, Fanny Adloff, Ben B. B. Booth, Peter M. Cox, Veronika Eyring, Pierre Friedlingstein, Katja Frieler, Helene T. Hewitt, Hazel A. Jeffery, Sylvie Joussaume, Torben Koenigk, Bryan N. Lawrence, Eleanor O'Rourke, Malcolm J. Roberts, Benjamin M. Sanderson, Roland Séférian, Samuel Somot, Pier Luigi Vidale, Detlef van Vuuren, Mario Acosta, Mats Bentsen, Raffaele Bernardello, Richard Betts, Ed Blockley, Julien Boé, Tom Bracegirdle, Pascale Braconnot, Victor Brovkin, Carlo Buontempo, Francisco Doblas-Reyes, Markus Donat, Italo Epicoco, Pete Falloon, Sandro Fiore, Thomas Frölicher, Neven S. Fučkar, Matthew J. Gidden, Helge F. Goessling, Rune Grand Graversen, Silvio Gualdi, José M. Gutiérrez, Tatiana Ilyina, Daniela Jacob, Chris D. Jones, Martin Juckes, Elizabeth Kendon, Erik Kjellström, Reto Knutti, Jason Lowe, Matthew Mizielinski, Paola Nassisi, Michael Obersteiner, Pierre Regnier, Romain Roehrig, David Salas y Mélia, Carl-Friedrich Schleussner, Michael Schulz, Enrico Scoccimarro, Laurent Terray, Hannes Thiemann, Richard A. Wood, Shuting Yang, and Sönke Zaehle
Earth Syst. Dynam., 15, 1319–1351, https://doi.org/10.5194/esd-15-1319-2024, https://doi.org/10.5194/esd-15-1319-2024, 2024
Short summary
Short summary
We propose a number of priority areas for the international climate research community to address over the coming decade. Advances in these areas will both increase our understanding of past and future Earth system change, including the societal and environmental impacts of this change, and deliver significantly improved scientific support to international climate policy, such as future IPCC assessments and the UNFCCC Global Stocktake.
Bjorn Stevens, Stefan Adami, Tariq Ali, Hartwig Anzt, Zafer Aslan, Sabine Attinger, Jaana Bäck, Johanna Baehr, Peter Bauer, Natacha Bernier, Bob Bishop, Hendryk Bockelmann, Sandrine Bony, Guy Brasseur, David N. Bresch, Sean Breyer, Gilbert Brunet, Pier Luigi Buttigieg, Junji Cao, Christelle Castet, Yafang Cheng, Ayantika Dey Choudhury, Deborah Coen, Susanne Crewell, Atish Dabholkar, Qing Dai, Francisco Doblas-Reyes, Dale Durran, Ayoub El Gaidi, Charlie Ewen, Eleftheria Exarchou, Veronika Eyring, Florencia Falkinhoff, David Farrell, Piers M. Forster, Ariane Frassoni, Claudia Frauen, Oliver Fuhrer, Shahzad Gani, Edwin Gerber, Debra Goldfarb, Jens Grieger, Nicolas Gruber, Wilco Hazeleger, Rolf Herken, Chris Hewitt, Torsten Hoefler, Huang-Hsiung Hsu, Daniela Jacob, Alexandra Jahn, Christian Jakob, Thomas Jung, Christopher Kadow, In-Sik Kang, Sarah Kang, Karthik Kashinath, Katharina Kleinen-von Königslöw, Daniel Klocke, Uta Kloenne, Milan Klöwer, Chihiro Kodama, Stefan Kollet, Tobias Kölling, Jenni Kontkanen, Steve Kopp, Michal Koran, Markku Kulmala, Hanna Lappalainen, Fakhria Latifi, Bryan Lawrence, June Yi Lee, Quentin Lejeun, Christian Lessig, Chao Li, Thomas Lippert, Jürg Luterbacher, Pekka Manninen, Jochem Marotzke, Satoshi Matsouoka, Charlotte Merchant, Peter Messmer, Gero Michel, Kristel Michielsen, Tomoki Miyakawa, Jens Müller, Ramsha Munir, Sandeep Narayanasetti, Ousmane Ndiaye, Carlos Nobre, Achim Oberg, Riko Oki, Tuba Özkan-Haller, Tim Palmer, Stan Posey, Andreas Prein, Odessa Primus, Mike Pritchard, Julie Pullen, Dian Putrasahan, Johannes Quaas, Krishnan Raghavan, Venkatachalam Ramaswamy, Markus Rapp, Florian Rauser, Markus Reichstein, Aromar Revi, Sonakshi Saluja, Masaki Satoh, Vera Schemann, Sebastian Schemm, Christina Schnadt Poberaj, Thomas Schulthess, Cath Senior, Jagadish Shukla, Manmeet Singh, Julia Slingo, Adam Sobel, Silvina Solman, Jenna Spitzer, Philip Stier, Thomas Stocker, Sarah Strock, Hang Su, Petteri Taalas, John Taylor, Susann Tegtmeier, Georg Teutsch, Adrian Tompkins, Uwe Ulbrich, Pier-Luigi Vidale, Chien-Ming Wu, Hao Xu, Najibullah Zaki, Laure Zanna, Tianjun Zhou, and Florian Ziemen
Earth Syst. Sci. Data, 16, 2113–2122, https://doi.org/10.5194/essd-16-2113-2024, https://doi.org/10.5194/essd-16-2113-2024, 2024
Short summary
Short summary
To manage Earth in the Anthropocene, new tools, new institutions, and new forms of international cooperation will be required. Earth Virtualization Engines is proposed as an international federation of centers of excellence to empower all people to respond to the immense and urgent challenges posed by climate change.
Mario C. Acosta, Sergi Palomas, Stella V. Paronuzzi Ticco, Gladys Utrera, Joachim Biercamp, Pierre-Antoine Bretonniere, Reinhard Budich, Miguel Castrillo, Arnaud Caubel, Francisco Doblas-Reyes, Italo Epicoco, Uwe Fladrich, Sylvie Joussaume, Alok Kumar Gupta, Bryan Lawrence, Philippe Le Sager, Grenville Lister, Marie-Pierre Moine, Jean-Christophe Rioual, Sophie Valcke, Niki Zadeh, and Venkatramani Balaji
Geosci. Model Dev., 17, 3081–3098, https://doi.org/10.5194/gmd-17-3081-2024, https://doi.org/10.5194/gmd-17-3081-2024, 2024
Short summary
Short summary
We present a collection of performance metrics gathered during the Coupled Model Intercomparison Project Phase 6 (CMIP6), a worldwide initiative to study climate change. We analyse the metrics that resulted from collaboration efforts among many partners and models and describe our findings to demonstrate the utility of our study for the scientific community. The research contributes to understanding climate modelling performance on the current high-performance computing (HPC) architectures.
Thibault Hallouin, Richard J. Ellis, Douglas B. Clark, Simon J. Dadson, Andrew G. Hughes, Bryan N. Lawrence, Grenville M. S. Lister, and Jan Polcher
Geosci. Model Dev., 15, 9177–9196, https://doi.org/10.5194/gmd-15-9177-2022, https://doi.org/10.5194/gmd-15-9177-2022, 2022
Short summary
Short summary
A new framework for modelling the water cycle in the land system has been implemented. It considers the hydrological cycle as three interconnected components, bringing flexibility in the choice of the physical processes and their spatio-temporal resolutions. It is designed to foster collaborations between land surface, hydrological, and groundwater modelling communities to develop the next-generation of land system models for integration in Earth system models.
Jorge Baño-Medina, Rodrigo Manzanas, Ezequiel Cimadevilla, Jesús Fernández, Jose González-Abad, Antonio S. Cofiño, and José Manuel Gutiérrez
Geosci. Model Dev., 15, 6747–6758, https://doi.org/10.5194/gmd-15-6747-2022, https://doi.org/10.5194/gmd-15-6747-2022, 2022
Short summary
Short summary
Deep neural networks are used to produce downscaled regional climate change projections over Europe for temperature and precipitation for the first time. The resulting dataset, DeepESD, is analyzed against state-of-the-art downscaling methodologies, reproducing more accurately the observed climate in the historical period and showing plausible future climate change signals with low computational requirements.
Cited articles
Abernathey, R. P., Augspurger, T., Banihirwe, A., Blackmon-Luca, C. C., Crone, T. J., Gentemann, C. L., Hamman, J. J., Henderson, N., Lepore, C., McCaie, T. A., Robinson, N. H., and Signell, R. P.: Cloud-Native Repositories for Big Scientific Data, Comput. Sci. Eng., 23, 26–35, https://doi.org/10.1109/MCSE.2021.3059437, 2021. a, b, c
Asadnabizadeh, M.: Critical findings of the sixth assessment report (AR6) of working Group I of the intergovernmental panel on climate change (IPCC) for global climate change policymaking a summary for policymakers (SPM) analysis, Int. J. Clim. Chang. Str., 15, 652–670, https://doi.org/10.1108/IJCCSM-04-2022-0049, 2023. a
Balaji, V., Taylor, K. E., Juckes, M., Lawrence, B. N., Durack, P. J., Lautenschlager, M., Blanton, C., Cinquini, L., Denvil, S., Elkington, M., Guglielmo, F., Guilyardi, E., Hassell, D., Kharin, S., Kindermann, S., Nikonov, S., Radhakrishnan, A., Stockhause, M., Weigel, T., and Williams, D.: Requirements for a global data infrastructure in support of CMIP6, Geosci. Model Dev., 11, 3659–3680, https://doi.org/10.5194/gmd-11-3659-2018, 2018. a, b, c
Banihirwe, A., Long, M., Grover, M., bonnland, Kent, J., Bourgault, P., Squire, D., Busecke, J., Spring, A., Schulz, H., Paul, K., RondeauG, and Kölling, T.: intake/intake-esm: intake-esm v2023.11.10, Zenodo [code], https://doi.org/10.5281/zenodo.3491062, 2023. a
Busecke, J. and Stern, C.: How to transform thousands of CMIP6 datasets to Zarr with Pangeo Forge and why we should never do this again!, Zenodo, https://doi.org/10.5281/zenodo.10229275, 2023. a
Busecke, J., Ritschel, M., Maroon, E., Nicholas, T., and Readthedocs-Assistant: jbusecke/xMIP: v0.7.1, Zenodo [code], https://doi.org/10.5281/zenodo.3678662, 2023. a
Caron, J., Davis, E., Hermida, M., Heimbigner, D., Arms, S., Ward-Garrison, C., May, R., Madry, L., Kambic, R., and Johnson, H.: Unidata THREDDS Data Server, UniData [data set], https://doi.org/10.5065/D6N014KG, 1997. a, b
Caron, J., Davis, E., Hermida, M., Heimbigner, D., Arms, S., Ward-Garrison, C., May, R., Madry, L., Kambic, R., Van Dam II, H., and Johnson, H.: Unidata NetCDF-Java Library, UniData [data set], https://doi.org/10.5065/DA15-J131, 2009. a, b
Cimadevilla, E.: ESGF-VA-CMIP6, Zenodo [data set], https://doi.org/10.5281/zenodo.14987358, 2025. a
Cimadevilla, E., Lawrence, B. N., and Cofiño, A. S.: The ESGF Virtual Aggregation (CMIP6 v20240125), Zenodo [code], https://doi.org/10.5281/zenodo.14203625, 2024. a
Cinquini, L., Crichton, D., Mattmann, C., Harney, J., Shipman, G., Wang, F., Ananthakrishnan, R., Miller, N., Denvil, S., Morgan, M., Pobre, Z., Bell, G. M., Drach, B., Williams, D., Kershaw, P., Pascoe, S., Gonzalez, E., Fiore, S., and Schweitzer, R.: The Earth System Grid Federation: An open infrastructure for access to distributed geospatial data, in: 2012 IEEE 8th International Conference on E-Science, IEEE, Chicago, IL, USA, 8–12 October 2012, 10 pp., ISBN 978-1-4673-4466-1 978-1-4673-4467-8 978-1-4673-4465-4, https://doi.org/10.1109/eScience.2012.6404471, 2012. a, b
Collier, N., Grover, M., and Stachelek, J.: esgf2-us/intake-esgf, GitHub [code], https://github.com/esgf2-us/intake-esgf (last access: 12 April 2024), 2024. a
Durant, M.: fsspec/kerchunk, GitHub [code], https://github.com/fsspec/kerchunk (last access: 9 May 2024), 2024. a
Dwyer, J. L., Roy, D. P., Sauer, B., Jenkerson, C. B., Zhang, H. K., and Lymburner, L.: Analysis Ready Data: Enabling Analysis of the Landsat Archive, Remote Sensing, 10, 1363, https://doi.org/10.3390/rs10091363, 2018. a, b
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., and Taylor, K. E.: Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization, Geosci. Model Dev., 9, 1937–1958, https://doi.org/10.5194/gmd-9-1937-2016, 2016. a, b
Fiore, S., Nassisi, P., Nuzzo, A., Mirto, M., Cinquini, L., Williams, D., and Aloisio, G.: A climate change community gateway for data usage & data archive metrics across the earth system grid federation, in: CEUR Workshop Proceedings, vol. 2975, https://www.scopus.com/inward/record.uri?eid=2-s2.0-85117857366&partnerID=40&md5=4882870b6cda97c5595337cb15c624b2 (last access: 6 May 2024), 2021. a
Garcia, J., Fox, P., West, P., and Zednik, S.: Developing service-oriented applications in a grid environment: Experiences using the OPeNDAP back-end-server, Earth Sci. Inform., 2, 133–139, https://doi.org/10.1007/s12145-008-0017-0, 2009. a
Gutiérrez, J. M., Jones, R. G., Narisma, G. T., Alves, L. M., Amjad, M., Gorodetskaya, I. V., Grose, M., Klutse, N. A. B., Krakovska, S., Li, J., Martínez-Castro, D., Mearns, L. O., Mernild, S. H., Ngo-Duc, T., van den Hurk, B., and Yoon, J.-H.: Atlas, in: Climate Change 2021: The Physical Science Basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, edited by: Masson-Delmotte, V., Zhai, P., Pirani, A., Connors, S. L., Péan, C., Berger, S., Caud, N., Chen, Y., Goldfarb, L., Gomis, M. I., Huang, M., Leitzell, K., Lonnoy, E., Matthews, J. B. R., Maycock, T. K., Waterfield, T., Yelekçi, O., Yu, R., and Zhou, B., 1927–2058, Cambridge University Press, https://doi.org/10.1017/9781009157896.021, 2021. a
Gutowski Jr., W. J., Giorgi, F., Timbal, B., Frigon, A., Jacob, D., Kang, H.-S., Raghavan, K., Lee, B., Lennard, C., Nikulin, G., O'Rourke, E., Rixen, M., Solman, S., Stephenson, T., and Tangang, F.: WCRP COordinated Regional Downscaling EXperiment (CORDEX): a diagnostic MIP for CMIP6, Geosci. Model Dev., 9, 4087–4095, https://doi.org/10.5194/gmd-9-4087-2016, 2016. a
Hassell, D., Gregory, J., Blower, J., Lawrence, B. N., and Taylor, K. E.: A data model of the Climate and Forecast metadata conventions (CF-1.6) with a software implementation (cf-python v2.1), Geosci. Model Dev., 10, 4619–4646, https://doi.org/10.5194/gmd-10-4619-2017, 2017. a, b
Hoyer, S. and Hamman, J.: xarray: N-D labeled Arrays and Datasets in Python, Journal of Open Research Software, 5, 10, https://doi.org/10.5334/jors.148, 2017. a
IPCC: Climate Change 2021 – The Physical Science Basis: Working Group I Contribution to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, 1 edn., ISBN 978-1-00-915789-6, https://doi.org/10.1017/9781009157896, 2023. a
Juckes, M., Taylor, K. E., Durack, P. J., Lawrence, B., Mizielinski, M. S., Pamment, A., Peterschmitt, J.-Y., Rixen, M., and Sénési, S.: The CMIP6 Data Request (DREQ, version 01.00.31), Geosci. Model Dev., 13, 201–224, https://doi.org/10.5194/gmd-13-201-2020, 2020. a, b
Mahecha, M. D., Gans, F., Brandt, G., Christiansen, R., Cornell, S. E., Fomferra, N., Kraemer, G., Peters, J., Bodesheim, P., Camps-Valls, G., Donges, J. F., Dorigo, W., Estupinan-Suarez, L. M., Gutierrez-Velez, V. H., Gutwin, M., Jung, M., Londoño, M. C., Miralles, D. G., Papastefanou, P., and Reichstein, M.: Earth system data cubes unravel global multivariate dynamics, Earth Syst. Dynam., 11, 201–234, https://doi.org/10.5194/esd-11-201-2020, 2020. a
Nativi, S., Mazzetti, P., and Craglia, M.: A view-based model of data-cube to support big earth data systems interoperability, Big Earth Data, 1, 75–99, https://doi.org/10.1080/20964471.2017.1404232, 2017. a
OGC: WPS 2.0.2 Interface Standard, http://docs.opengeospatial.org/is/14-065/14-065.html (last access: 17 April 2024), 2015. a
Petrie, R., Denvil, S., Ames, S., Levavasseur, G., Fiore, S., Allen, C., Antonio, F., Berger, K., Bretonnière, P.-A., Cinquini, L., Dart, E., Dwarakanath, P., Druken, K., Evans, B., Franchistéguy, L., Gardoll, S., Gerbier, E., Greenslade, M., Hassell, D., Iwi, A., Juckes, M., Kindermann, S., Lacinski, L., Mirto, M., Nasser, A. B., Nassisi, P., Nienhouse, E., Nikonov, S., Nuzzo, A., Richards, C., Ridzwan, S., Rixen, M., Serradell, K., Snow, K., Stephens, A., Stockhause, M., Vahlenkamp, H., and Wagner, R.: Coordinating an operational data distribution network for CMIP6 data, Geosci. Model Dev., 14, 629–644, https://doi.org/10.5194/gmd-14-629-2021, 2021. a
Rew, R., Davis, G., Emmerson, S., Cormack, C., Caron, J., Pincus, R., Hartnett, E., Heimbigner, D., Appel, L., and Fisher, W.: Unidata NetCDF, Unidata [data set], https://doi.org/10.5065/D6H70CW6, 1989. a
Schnase, J. L., Lee, T. J., Mattmann, C. A., Lynnes, C. S., Cinquini, L., Ramirez, P. M., Hart, A. F., Williams, D. N., Waliser, D., Rinsland, P., Webster, W. P., Duffy, D. Q., McInerney, M. A., Tamkin, G. S., Potter, G. L., and Carriere, L.: Big Data Challenges in Climate Science: Improving the next-generation cyberinfrastructure, IEEE Geosci. Remote S., 4, 10–22, https://doi.org/10.1109/MGRS.2015.2514192, 2016. a
Stern, C., Abernathey, R., Hamman, J., Wegener, R., Lepore, C., Harkins, S., and Merose, A.: Pangeo Forge: Crowdsourcing Analysis-Ready, Cloud Optimized Data Production, Frontiers in Climate, 3, 782909, https://doi.org/10.3389/fclim.2021.782909, 2022. a
Swart, N. C., Cole, J. N. S., Kharin, V. V., Lazare, M., Scinocca, J. F., Gillett, N. P., Anstey, J., Arora, V., Christian, J. R., Hanna, S., Jiao, Y., Lee, W. G., Majaess, F., Saenko, O. A., Seiler, C., Seinen, C., Shao, A., Sigmond, M., Solheim, L., von Salzen, K., Yang, D., and Winter, B.: The Canadian Earth System Model version 5 (CanESM5.0.3), Geosci. Model Dev., 12, 4823–4873, https://doi.org/10.5194/gmd-12-4823-2019, 2019. a
Taylor, K. E., Juckes, M., Balaji, V., Cinquini, L., Denvil, S., Durack, P. J., Elkington, M., Guilyardi, E., Kharin, S., Lautenschlager, M., and others: CMIP6 Global Attributes, DRS, Filenames, Directory Structure, and CV's, https://wcrp-cmip.github.io/WGCM_Infrastructure_Panel/Papers/CMIP6_global_attributes_filenames_CVs_v6.2.7.pdf (last access: 27 May 2024), 2018. a
The HDF Group: Hierarchical Data Format, version 5, GitHub [code], https://github.com/HDFGroup/hdf5 (last access: 16 October 2024), 2024. a
Venturini, T., De Pryck, K., and Ackland, R.: Bridging in network organisations. The case of the Intergovernmental Panel on Climate Change (IPCC), Soc. Networks, 75, 137–147, https://doi.org/10.1016/j.socnet.2022.01.015, 2023. a
Williams, D. N., Balaji, V., Cinquini, L., Denvil, S., Duffy, D., Evans, B., Ferraro, R., Hansen, R., Lautenschlager, M., and Trenham, C.: A Global Repository for Planet-Sized Experiments and Observations, B. Am. Meteorol. Soc., 97, 803–816, https://doi.org/10.1175/BAMS-D-15-00132.1, 2016. a
Short summary
The Earth System Grid Federation (ESGF) stores an enormous amount of climate data spread across millions of files in data centres all over the world. Accessing and working with this scientific information is quite complex. This work presents ESGF Virtual Aggregation, an approach that combines data from different sources into a format that is ready for analysis straightaway.
The Earth System Grid Federation (ESGF) stores an enormous amount of climate data spread across...