Preprints
https://doi.org/10.5194/gmd-2023-46
https://doi.org/10.5194/gmd-2023-46
Submitted as: methods for assessment of models
 | 
04 May 2023
Submitted as: methods for assessment of models |  | 04 May 2023
Status: this preprint is currently under review for the journal GMD.

The analysis of large-volume multi-institute climate model output at a Central Analysis Facility (PRIMAVERA Data Management Tool V2.10)

Jon Seddon, Ag Stephens, Matthew S. Mizielinski, Pier Luigi Vidale, and Malcolm J. Roberts

Abstract. The PRIMAVERA project aimed to develop a new generation of advanced and well-evaluated high-resolution global climate models. As part of PRIMAVERA, seven different climate models were run in both standard and higher resolution configurations, with common initial conditions and forcings to form a multi-model ensemble. The ensemble simulations were run on high performance computers across Europe and generated approximately 1.6 pebibytes of output. To allow the data from all models to be analysed at this scale, PRIMAVERA scientists were encouraged to bring their analysis to the data. All data was transferred to a Central Analysis Facility (CAF), in this case the JASMIN super-data-cluster, where it was catalogued and details made available to users using the PRIMAVERA Data Management Tool's (DMT's) web interface. Users from across the project were able to query the available data using the DMT and then access it at the CAF. Here we describe how the PRIMAVERA project used the CAF's facilities to enable users to analyse this multi-model data set. We believe that PRIMAVERA's experience using a CAF demonstrates how similar, multi institute, big-data projects can efficiently share, organise and analyse large volumes of data.

Jon Seddon et al.

Status: open (until 02 Jul 2023)

Comment types: AC – author | RC – referee | CC – community | EC – editor | CEC – chief editor | : Report abuse

Jon Seddon et al.

Jon Seddon et al.

Viewed

Total article views: 181 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
153 24 4 181 1 2
  • HTML: 153
  • PDF: 24
  • XML: 4
  • Total: 181
  • BibTeX: 1
  • EndNote: 2
Views and downloads (calculated since 04 May 2023)
Cumulative views and downloads (calculated since 04 May 2023)

Viewed (geographical distribution)

Total article views: 175 (including HTML, PDF, and XML) Thereof 175 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Latest update: 05 Jun 2023
Download
Short summary
The PRIMAVERA project aimed to develop a new generation of advanced global climate models. The large volume of data generated was uploaded to a Central Analysis Facility (CAF) and was analysed by 100 PRIMAVERA scientists there. We describe how the PRIMAVERA project used the CAF's facilities to enable users to analyse this large data set. We believe that similar, multi institute, big-data projects could also use a CAF to efficiently share, organise and analyse large volumes of data.